Indexed, though blocked by robots.txt


#1

I was checking my webmaster tool. Got below message:

Search Console has identified that your site is affected by 1 new Index coverage related issue. This means that Index coverage may be negatively affected in Google Search results. We encourage you to fix this issue.

New issue found:
Indexed, though blocked by robots.txt

What is this and how to solve it? I do not see any issue in my robots.txt tester link at Search console.


#2
  1. Check your robots.txt file
  2. As per Search Console, make sure useful path shouldn’t be blocked by the Disallow rule.

Please keep remember: The robots.txt disallow rule doesn’t prevent indexing. If disallowed, Google cannot noindex tag, and they will keep in index. As possible as, avoid using robots.txt file unless so important.


#3

@Laalsalam Were you able to fix the problem?

Google offers a tool called robots.txt checker.
screenshot

After fixing the robots.txt issue, you can validate your URL to enures it’s not blocked by Googlebot.

Once fixed, after few days you will see

:white_check_mark: No blocked resources detected (be sure to check all site variants).

Good luck :slight_smile:


#4

@Laalsalam Were you able to resolve the problem?


#5

Yes stored


#6

HI All,

I need help.
I have a blog hosted in blogspot.

What is the meaning of below-mentioned text? (I received it through google team via email)

Google has validated your fix of Coverage issues on site. The specific issue validated was ‘Indexed, though blocked by robots.txt’.

2 pages on your site were validated as fixed.

To examine full details on the validation progress and to learn if there are other Coverage related issues to fix, please follow this link.


#7

Dear, it clearly means that the page which was earlier blocked due to robots.txt now it is allowed for crawling.


#18

I blocked search labels in robots.txt like this:

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap: https://www.domain.com/sitemap.xml

I’m using blogger custom domain. GWT showing Indexed, though blocked by robots.txt. Should I validate fix without removing this tag in robots.txt?


#19

No …

Well, Search results pages generally shouldn’t be indexed. If you can make it noindex, it would be better, or leave search result path blocked in the robots.txt


#20

But by mistake, I’ve clicked the validate fix before posting my question. What should I do now? How to stop the process?


#21

No worries. It is not going to harm either way.


#22

Thanks @GulshanKumar


#23

You’re welcome!


#24

Search console coverage issues > Indexed, though blocked by robots.txt

URL - https://XYZ.com/wp-admin/widget.php

What should i do ? or leave it as is.


#25

/wp-admin/ path shouldn’t be indexed. You can safely ignore it.


#26

iska solution batao koi please. screen shot ke saath/ ye thik kaise hoga