Indexed, though blocked by robots.txt


#1

I was checking my webmaster tool. Got below message:

Search Console has identified that your site is affected by 1 new Index coverage related issue. This means that Index coverage may be negatively affected in Google Search results. We encourage you to fix this issue.

New issue found:
Indexed, though blocked by robots.txt

What is this and how to solve it? I do not see any issue in my robots.txt tester link at Search console.


#2
  1. Check your robots.txt file
  2. As per Search Console, make sure useful path shouldn’t be blocked by the Disallow rule.

Please keep remember: The robots.txt disallow rule doesn’t prevent indexing. If disallowed, Google cannot noindex tag, and they will keep in index. As possible as, avoid using robots.txt file unless so important.


#3

@laalsalaam Were you able to fix the problem?

Google offers a tool called robots.txt checker.
screenshot

After fixing the robots.txt issue, you can validate your URL to enures it’s not blocked by Googlebot.

Once fixed, after few days you will see

:white_check_mark: No blocked resources detected (be sure to check all site variants).

Good luck :slight_smile:


#4

@laalsalaam Were you able to resolve the problem?


#5

Yes stored


#6

HI All,

I need help.
I have a blog hosted in blogspot.

What is the meaning of below-mentioned text? (I received it through google team via email)

Google has validated your fix of Coverage issues on site. The specific issue validated was ‘Indexed, though blocked by robots.txt’.

2 pages on your site were validated as fixed.

To examine full details on the validation progress and to learn if there are other Coverage related issues to fix, please follow this link.


#7

Dear, it clearly means that the page which was earlier blocked due to robots.txt now it is allowed for crawling.


#8

yeah @gulshankumar

Then why my blog is not running now?

When I open my blog then it shows below mentioned messages. (Also attached screenshot). Please help me if u know.

This site can’t provide a secure connection

www.happynewyearwisheslist.com uses an unsupported protocol.

ERR_SSL_VERSION_OR_CIPHER_MISMATCH

HIDE DETAILS

Unsupported protocol

The client and server don’t support a common SSL protocol version or cipher suite.


#9

This is possible in two cases

  1. You have recently enabled Cloudflare but its SSL is not active yet.

  2. You are using win XP

Fix

  • Login to CF > check Crypto menu > SSL either it is active or not

  • Try accessing on different devices.


#10

@gulshankumar Could u explain how to do?

I’m using window 7

I m using Blogspot platform so not familiar how to do it. so please help me to resolve it


#11

Dear, you were using Bigrock by default DNS earlier.

Solution: :arrow_right: Please keep patience for 24 hours.

As I can see, DNS propagation is still pending: https://dnschecker.org/#NS/happynewyearwisheslist.com

Cloudflare SSL supports Windows 7, as tested on Chrome. Can’t say about IE.


#12

thanx @gulshankumar

Hope after 24 hours it will work.


#18

I blocked search labels in robots.txt like this:

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap: https://www.domain.com/sitemap.xml

I’m using blogger custom domain. GWT showing Indexed, though blocked by robots.txt. Should I validate fix without removing this tag in robots.txt?


#19

No …

Well, Search results pages generally shouldn’t be indexed. If you can make it noindex, it would be better, or leave search result path blocked in the robots.txt


#20

But by mistake, I’ve clicked the validate fix before posting my question. What should I do now? How to stop the process?


#21

No worries. It is not going to harm either way.


#22

Thanks @gulshankumar


#23

You’re welcome!