Jump to content

Crawl Errors


Andy1

Recommended Posts

Hi,

 

I have just logged into my Google Webmaster Account and I notice I have the following crawl errors:

 

 

In Sitemaps = 1

Not Found = 33

Restricted by robots.txt = 21

 

The 'Not Found' errors I can deal with as these are pages I have removed from the site, but need to use the URL removal tool to removed them from Google.

 

I don't know why I have an error in the sitemap, and I am very concerened about the 21 pages restricted by the robots.txt file.

 

I thought you only had a robot.txt file if you wanted google to ignore certain pages. I haven't set the robots.txt file so not sure why this has happened. Within the 21 pages are some products and also the manufactures list etc

 

Does anyone know how I can remove the crawl errors from 'Restricted by robots.txt' and the 'In Sitemaps'

 

Many Thanks

Andy

The 'Not Found' errors I can deal with as these are pages I have removed from the site, but need to use the URL removal tool to removed them from Google.

 

I don't know why I have an error in the sitemap, and I am very concerened about the 21 pages restricted by the robots.txt file.

 

I thought you only had a robot.txt file if you wanted google to ignore certain pages. I haven't set the robots.txt file so not sure why this has happened. Within the 21 pages are some products and also the manufactures list etc

 

Does anyone know how I can remove the crawl errors from 'Restricted by robots.txt' and the 'In Sitemaps'

 

Many Thanks

Andy

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...