Sitemap contains urls which are blocked by robots.txt

[emil1001]

Salut!

 

Am un magazin pe PS 1.6.0.9 si am observat in webmastertools ca primesc aceasta eroare la inregistrarea sitemap-ului:

 

Sitemap contains urls which are blocked by robots.txt

 

Am sters si regenerat robots.txt dar degeaba...

 

Aveti idee despre ce poate fi vorba?

 

Multumesc anticipat!

[MihaiAlex]

Daca ne poti da share si la fisier..sau un screen. Informatile sunt prea putine pentru a putea fi ajutat cu un sfat

[emil1001]

# robots.txt automaticaly generated by PrestaShop e-commerce open-source solution

# http://www.prestashop.com - http://www.prestashop.com/forums

# This file is to prevent the crawling and indexing of certain parts

# of your site by web crawlers and spiders run by sites like Yahoo!

# and Google. By telling these "robots" where not to go on your site,

# you save bandwidth and server resources.

# For more information about the robots.txt standard, see:

# http://www.robotstxt.org/wc/robots.html

User-agent: *

# Private pages

Disallow: /*orderby=

Disallow: /*orderway=

Disallow: /*tag=

Disallow: /*id_currency=

Disallow: /*search_query=

Disallow: /*back=

Disallow: /*n=

Disallow: /*controller=addresses

Disallow: /*controller=address

Disallow: /*controller=authentication

Disallow: /*controller=cart

Disallow: /*controller=discount

Disallow: /*controller=footer

Disallow: /*controller=get-file

Disallow: /*controller=header

Disallow: /*controller=history

Disallow: /*controller=identity

Disallow: /*controller=images.inc

Disallow: /*controller=init

Disallow: /*controller=my-account

Disallow: /*controller=order

Disallow: /*controller=order-opc

Disallow: /*controller=order-slip

Disallow: /*controller=order-detail

Disallow: /*controller=order-follow

Disallow: /*controller=order-return

Disallow: /*controller=order-confirmation

Disallow: /*controller=pagination

Disallow: /*controller=password

Disallow: /*controller=pdf-invoice

Disallow: /*controller=pdf-order-return

Disallow: /*controller=pdf-order-slip

Disallow: /*controller=product-sort

Disallow: /*controller=search

Disallow: /*controller=statistics

Disallow: /*controller=attachment

Disallow: /*controller=guest-tracking

# Directories

Disallow: */classes/

Disallow: */config/

Disallow: */download/

Disallow: */mails/

Disallow: */modules/

Disallow: */translations/

Disallow: */tools/

# Files

Disallow: /password-recovery

Disallow: /address

Disallow: /addresses

Disallow: /login

Disallow: /cart

Disallow: /discount

Disallow: /order-history

Disallow: /identity

Disallow: /my-account

Disallow: /order-follow

Disallow: /order-slip

Disallow: /order

Disallow: /search

Disallow: /quick-order

Disallow: /guest-tracking

Disallow: /order-confirmation

[emil1001]

site-ul este frosty.ro

[Coloranti]

Cred ca este vorba de o noutate in google webmaster tools. Vor ca vizitatorii sa aiba acces la toate modulele si css (cel putin asta am dedus eu).

Si eu am avut aceasta eroare si am observat ca apare in special pe module.

Am sters din robots.txt "Disallow: */modules/" si eroarea a disparut.

Se pot vedea in webmaster tools care sunt resursele blocate si se pot sterge din robots.txt cele care apar ca blocate.

Daca folosesti modulul gsitemap, verifica sa corespunda ceea ce este bifat cu ceea ce apare in robots.txt.

 

Succes!

Edited June 28, 2015 by Coloranti (see edit history)