Divi WordPress Theme

Topic: need search engine crawler  (Read 2373 times)

Offline Ayen Mahinay

  • New Member
  • *
  • Posts: 33
  • iTrader: +0/-0
    • View Profile
need search engine crawler
« on: November 16, 2011, 12:31:05 PM »
hello to all respected members I need information about search engine crawler working because I want to open a search engine to visitor helps
VPS Hosting - AccuWebHosting

Offline Steve Smith

  • Junior Guru
  • **
  • Posts: 233
  • iTrader: +0/-0
    • View Profile
Re: need search engine crawler
« Reply #1 on: March 25, 2013, 02:59:59 PM »
A search engine crawler is a program or automated script that browses the World Wide Web in a methodical manner in order to provide up to date data to the particular search engine. While search engine crawlers go by many different names, such as web spiders and automatic indexers, the job of the search engine crawler is still the same. The process of web crawling involves a set of website URLs that need to be visited, called seeds, and then the search engine crawler visits each web page and identifies all the hyperlinks on the page, adding them to the list of places to crawl. URLs from this list are re-visited occasionally according to the policies in place for the search engine.

Offline Thechipper

  • Junior Guru
  • **
  • Posts: 150
  • iTrader: +0/-0
    • View Profile
Re: need search engine crawler
« Reply #2 on: September 02, 2015, 12:29:11 PM »

Offline sankalppatil

  • Junior Guru
  • **
  • Posts: 153
  • iTrader: +0/-0
    • View Profile
Re: need search engine crawler
« Reply #3 on: October 08, 2018, 01:22:33 PM »
The process of web crawling involves a set of website URLs that need to be visited, called seeds, and then the search engine crawler visits each web page and identifies all the hyperlinks on the page, adding them to the list of places to crawl.
Shopify Divi WordPress Theme
Tags: