Allowing single module in Robots.txt

[ccw0103]

Hello,

 

I tried adding the code but it doesnt seem to work, any idea what went wrong?

 

 

# Directories

 

Disallow: */modules/

Allow : /modules/nameofthemodule/

 

 

 

[selectshop.at]

Please generate your robots.txt from back-office and add lines if needed. Modules are not catched by Google with native Prestashop robots.txt. Which problem are you having ? Can you please give more details on it ?

 

PS: the line you added is wrong. It should be:

 

# Directories
Disallow: /*modules/

[selectshop.at]

If you want to allow only one module you should add (and please read my topic before, your line added is wrong).

 

Allow: /*whatever/other

 

OR for non-Google bots: Allow: /whatever/other

 

Directives for robots.txt: https://en.wikipedia.org/wiki/Robots_exclusion_standard

 

Perhaps your bot is a fake-bot and these do not follow any rules. Please give more details on your problem.

 

 

PS: * is a placeholder/wildcard, some of the search-bots do not recognize this, better is to use:

 

Disallow: /modules/ instead of Disallow: /*modules/ (native Prestashop)

 

Architecture for pages is:

Disallow: /controller=guest-tracking

Disallow: /tag= 

 

Architecture for files is:

Disallow: /yourfile.php  

[ccw0103]

If you want to allow only one module you should add (and please read my topic before, your line added is wrong).

 

Allow: /*whatever/other

 

OR for non-Google bots: Allow: /whatever/other

 

Directives for robots.txt: https://en.wikipedia.org/wiki/Robots_exclusion_standard

 

Perhaps your bot is a fake-bot and these do not follow any rules. Please give more details on your problem.

 

 

PS: * is a placeholder/wildcard, some of the search-bots do not recognize this, better is to use:

 

Disallow: /modules/ instead of Disallow: /*modules/ (native Prestashop)

 

Architecture for pages is:

Disallow: /controller=guest-tracking

Disallow: /tag= 

 

Architecture for files is:

Disallow: /yourfile.php  

 

Hi,

 

Thanks for the reply.

 

But the robots is from back office generation, even i generate a new robots, it still looks like this:

 

 

Disallow: /*controller=search

Disallow: /*controller=statistics

Disallow: /*controller=attachment

Disallow: /*controller=guest-tracking

# Directories

Disallow: */classes/

Disallow: */config/

Disallow: */download/

Disallow: */mails/

Disallow: */modules/

 

So if I want to allow a single module, my code should be 

 

 

 

 

# Directories

Disallow: */classes/

Disallow: */config/

Disallow: */download/

Disallow: */mails/

Disallow: */modules/

Allow: /*modules/modulename

 

is this correct??

[ccw0103]

I tried Allow: /*modules/modulename 

 

and tested on google robots tester and it works!

 

and I also tried testing modules and it was blocked with

 

Disallow: */modules/

 

 

any idea?

[selectshop.at]

Yes. better is to omit the * which is a placeholder, cause only Google can read this.

 

Allow: /modules/modulename

 

is the correct line FOR ALL Search engines (excl. spam-bots, which do not follow any rules in robot.txt).

 

Which Prestashop version are you using ?

# Directories
Disallow: */classes/
Disallow: */config/
Disallow: */download/
Disallow: */mails/
Disallow: */modules/

Don't seems to be correct.

[selectshop.at]

You are using more than one language ?

 

Than correct line for directories are correct.

[ccw0103]

You are using more than one language ?

 

Than correct line for directories are correct.

 

Yes, I am using three languages in fact.... so it should be alright now???

 

Should I remove the *??