1. Web Hosting community for Webmasters
  2. Search Engines
  3. Search Engine optimization and Link Exchange
  4. What? If Robot.txt blocked?

Divi WordPress Theme
« previous next »
Pages: [1]   Go Down

Topic: What? If Robot.txt blocked?  (Read 400 times)

Offline AradyaGD

  • New Member
  • *
  • Posts: 25
  • iTrader: +0/-0
    • View Profile
What? If Robot.txt blocked?
« on: February 06, 2019, 04:48:46 PM »
Anyone explains to me What? If Robot.txt tester blocked a website or webpage?
VPS Hosting - AccuWebHosting

Offline RH-Calvin

  • Junior Guru
  • **
  • Posts: 71
  • iTrader: +0/-0
    • View Profile
Re: What? If Robot.txt blocked?
« Reply #1 on: March 11, 2019, 10:40:52 AM »
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
Shopify Divi WordPress Theme
Pages: [1]   Go Up
« previous next »
Tags:
  1. Web Hosting community for Webmasters
  2. Search Engines
  3. Search Engine optimization and Link Exchange
  4. What? If Robot.txt blocked?