Divi WordPress Theme

Topic: Robots.txt  (Read 153 times)

Offline AradyaGD

  • Newbie
  • *
  • Posts: 7
  • iTrader: +0/-0
    • View Profile
Robots.txt
« on: August 08, 2018, 06:58:57 PM »
Hi, I am new to SEO
I want to know about robots.txt, What is robots.txt?
VPS Hosting - AccuWebHosting

Offline JesicaJoh

  • New Member
  • *
  • Posts: 31
  • iTrader: +0/-0
    • View Profile
Re: Robots.txt
« Reply #1 on: August 23, 2018, 07:25:14 AM »
It is the text file, You can represent your webpage allow or not allow for a bots or crawler.

Offline RH-Calvin

  • New Member
  • *
  • Posts: 42
  • iTrader: +0/-0
    • View Profile
Re: Robots.txt
« Reply #2 on: August 27, 2018, 10:36:58 AM »
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.

Offline marrysmith

  • Newbie
  • *
  • Posts: 5
  • iTrader: +0/-0
    • View Profile
Re: Robots.txt
« Reply #3 on: September 05, 2018, 05:36:40 AM »
With robots.txt you can tell search engine which directories to index and which to not, in simple words, when google bot comes to your website, it check robots.txt files, and follows the instructions to index the allowed directories.

Offline sankalppatil

  • Junior Guru
  • **
  • Posts: 125
  • iTrader: +0/-0
    • View Profile
Re: Robots.txt
« Reply #4 on: October 17, 2018, 11:30:16 AM »
Hello friends,
Robots.txt is a text file that lists web pages which contain instructions for search engines robots. The file lists web pages that are allowed and disallowed from search engine crawling.
Divi WordPress Theme
Tags: