bonsaiko Posted January 14, 2019 Share Posted January 14, 2019 Hello, A company scanned my site for optimization and they found that my robots.txt file has some URL that are not to index, while these URLs are listed inside the site either like follow links or sometimes canonical URL. So there seems to have a conflict between what the robots.txt "says" and what the URL is. One example of URL given in the report is : https://mysite.com/product?search_query=searchterm When I look at my robots.txt file, there is no such URL in it. But I have this rule: Disallow: /*?search_query= So my question is : what can I do ? I can't remove this rule from my robots.txt so how can I get rid of this error ? Thanks for your help. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now