duda sobre robots.txt

[jonysi_d]

Hola...

el generador de robots,txt de prestashop ya integrado me genera este archibo pero no entiendo porque bloquea a los crawels de totos los archibos y carpetas siguientes.... y ademas no especifica que archibos o carpetas o fotos deja indexar a los crawels.

 

# robots.txt automaticaly generated by PrestaShop e-commerce open-source solution
# http://www.prestashop.com - http://www.prestashop.com/forums
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/wc/robots.html
# GoogleBot specific
User-agent: Googlebot
Disallow: /carpeta/*orderby=
Disallow: /carpeta/*orderway=
Disallow: /carpeta/*tag=
Disallow: /carpeta/*id_currency=
Disallow: /carpeta/*search_query=
Disallow: /carpeta/*id_lang=
Disallow: /carpeta/*back=
Disallow: /carpeta/*utm_source=
Disallow: /carpeta/*utm_medium=
Disallow: /carpeta/*utm_campaign=
Disallow: /carpeta/*n=
# All bots
User-agent: *
# Directories
Disallow: /carpeta/classes/
Disallow: /carpeta/config/
Disallow: /carpeta/download/
Disallow: /carpeta/mails/
Disallow: /carpeta/modules/
Disallow: /carpeta/translations/
Disallow: /carpeta/tools/
# Files
Disallow: /carpeta/addresses.php
Disallow: /carpeta/address.php
Disallow: /carpeta/authentication.php
Disallow: /carpeta/cart.php
Disallow: /carpeta/discount.php
Disallow: /carpeta/footer.php
Disallow: /carpeta/get-file.php
Disallow: /carpeta/header.php
Disallow: /carpeta/history.php
Disallow: /carpeta/identity.php
Disallow: /carpeta/images.inc.php
Disallow: /carpeta/init.php
Disallow: /carpeta/my-account.php
Disallow: /carpeta/order.php
Disallow: /carpeta/order-opc.php
Disallow: /carpeta/order-slip.php
Disallow: /carpeta/order-detail.php
Disallow: /carpeta/order-follow.php
Disallow: /carpeta/order-return.php
Disallow: /carpeta/order-confirmation.php
Disallow: /carpeta/pagination.php
Disallow: /carpeta/password.php
Disallow: /carpeta/pdf-invoice.php
Disallow: /carpeta/pdf-order-return.php
Disallow: /carpeta/pdf-order-slip.php
Disallow: /carpeta/product-sort.php
Disallow: /carpeta/search.php
Disallow: /carpeta/statistics.php
Disallow: /carpeta/attachment.php
Disallow: /carpeta/guest-tracking.php
# Sitemap
Sitemap: http://www.miweb.es/sitemap.xml

[nadie]

Hola...

el generador de robots,txt de prestashop ya integrado me genera este archibo pero no entiendo porque bloquea a los crawels de totos los archibos y carpetas siguientes.... y ademas no especifica que archibos o carpetas o fotos deja indexar a los crawels.

 

# robots.txt automaticaly generated by PrestaShop e-commerce open-source solution
# http://www.prestashop.com - http://www.prestashop.com/forums
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/wc/robots.html
# GoogleBot specific
User-agent: Googlebot
Disallow: /carpeta/*orderby=
Disallow: /carpeta/*orderway=
Disallow: /carpeta/*tag=
Disallow: /carpeta/*id_currency=
Disallow: /carpeta/*search_query=
Disallow: /carpeta/*id_lang=
Disallow: /carpeta/*back=
Disallow: /carpeta/*utm_source=
Disallow: /carpeta/*utm_medium=
Disallow: /carpeta/*utm_campaign=
Disallow: /carpeta/*n=
# All bots
User-agent: *
# Directories
Disallow: /carpeta/classes/
Disallow: /carpeta/config/
Disallow: /carpeta/download/
Disallow: /carpeta/mails/
Disallow: /carpeta/modules/
Disallow: /carpeta/translations/
Disallow: /carpeta/tools/
# Files
Disallow: /carpeta/addresses.php
Disallow: /carpeta/address.php
Disallow: /carpeta/authentication.php
Disallow: /carpeta/cart.php
Disallow: /carpeta/discount.php
Disallow: /carpeta/footer.php
Disallow: /carpeta/get-file.php
Disallow: /carpeta/header.php
Disallow: /carpeta/history.php
Disallow: /carpeta/identity.php
Disallow: /carpeta/images.inc.php
Disallow: /carpeta/init.php
Disallow: /carpeta/my-account.php
Disallow: /carpeta/order.php
Disallow: /carpeta/order-opc.php
Disallow: /carpeta/order-slip.php
Disallow: /carpeta/order-detail.php
Disallow: /carpeta/order-follow.php
Disallow: /carpeta/order-return.php
Disallow: /carpeta/order-confirmation.php
Disallow: /carpeta/pagination.php
Disallow: /carpeta/password.php
Disallow: /carpeta/pdf-invoice.php
Disallow: /carpeta/pdf-order-return.php
Disallow: /carpeta/pdf-order-slip.php
Disallow: /carpeta/product-sort.php
Disallow: /carpeta/search.php
Disallow: /carpeta/statistics.php
Disallow: /carpeta/attachment.php
Disallow: /carpeta/guest-tracking.php
# Sitemap
Sitemap: http://www.miweb.es/sitemap.xml

El código es correcto, es importante bloquear lo que tienes puesto en el robots.txt.