How to Noindex OneSignal WordPress Plugin Directory?

(Amit Tiwari) #21

What will be the benefit of blocking the search result page from crawler,?

(Ultra Noob) #22

To prevent unnecessary crawling, spam links. Ref: labnol


@GulshanKumar Is there some issue with this code? It is not adding the “X-Robots-Tag” header. I added using code snippets. Thanks!

(Ultra Noob) #25

Try this method


Thanks for the help!

It is working! I have added the following in my Nginx configuration.

location ~ .*/(?:onesignal-free-web-push-notifications)/.* {
      add_header X-Robots-Tag "noindex, follow" always;  }

“X-Robots-Tag” header is not showing for this URL. I believe as we have already blocked the main folder, so that should not be an issue!

“X-Robots-Tag” header is showing for following URLs,

(Ultra Noob) #27

Have you purged cache? (If using any CDN)


Well, I have not integrated any CDN as of now, will be adding it soon. Though, I am using Cloudflare with HTTP Proxy, with the suggestions that Bhanu gave by adding* with page rule “cache level: bypass”, “disable performance” etc. so that I get an advantage of hiding my cloud VPS IP, DDoS, firewall for blocking visitors IP etc.

Now, the interesting thing is, when I am using , it does not show “X-Robots-Tag” header.

Though in another incognito tab, when I use Developer tools, it shows “X-Robots-Tag” header. LOL! :yum: I think a few hours back, I checked using Developer tools also. “X-Robots-Tag” header was not showing! Though as you mentioned about CDN, I just did “Purge everything” in Cloudflare!