• Welcome to Web Hosting Community Forum for Webmasters - Web hosting Forum.
 

need search engine crawler

Started by Ayen Mahinay, November 16, 2011, 12:31:05 PM

Divi WordPress Theme


Ayen Mahinay

hello to all respected members I need information about search engine crawler working because I want to open a search engine to visitor helps
The Game is Over

Steve Smith

A search engine crawler is a program or automated script that browses the World Wide Web in a methodical manner in order to provide up to date data to the particular search engine. While search engine crawlers go by many different names, such as web spiders and automatic indexers, the job of the search engine crawler is still the same. The process of web crawling involves a set of website URLs that need to be visited, called seeds, and then the search engine crawler visits each web page and identifies all the hyperlinks on the page, adding them to the list of places to crawl. URLs from this list are re-visited occasionally according to the policies in place for the search engine.


sankalppatil

The process of web crawling involves a set of website URLs that need to be visited, called seeds, and then the search engine crawler visits each web page and identifies all the hyperlinks on the page, adding them to the list of places to crawl.

Divi WordPress Theme