• Welcome to Web Hosting Community Forum for Webmasters - Web hosting Forum.
 

Recommended Providers


Jetpack
Fully Managed WordPress Hosting
lc_banner_leadgen_3
Fully Managed WordPress Hosting

WordPress Theme

WordPress Hosting Divi WordPress Theme
WPZOOM

Forum Membership

Forum Membership

need search engine crawler

Started by Ayen Mahinay, November 16, 2011, 12:31:05 PM


WordPress Hosting


Ayen Mahinay

hello to all respected members I need information about search engine crawler working because I want to open a search engine to visitor helps
The Game is Over

Steve Smith

A search engine crawler is a program or automated script that browses the World Wide Web in a methodical manner in order to provide up to date data to the particular search engine. While search engine crawlers go by many different names, such as web spiders and automatic indexers, the job of the search engine crawler is still the same. The process of web crawling involves a set of website URLs that need to be visited, called seeds, and then the search engine crawler visits each web page and identifies all the hyperlinks on the page, adding them to the list of places to crawl. URLs from this list are re-visited occasionally according to the policies in place for the search engine.


sankalppatil

The process of web crawling involves a set of website URLs that need to be visited, called seeds, and then the search engine crawler visits each web page and identifies all the hyperlinks on the page, adding them to the list of places to crawl.