Hello, I did upgraded my application from Prestashop 1.6 on PHP 7.0 to Prestashop 1.7 PHP 7.4. Atter changing from developer webspace to production one, appeared a google crawler error wchich can't crawl items from data feeds.
Reported error is:
Update your robots.txt file to allow user-agents "Googlebot" and "Googlebot-Image" to crawl your site
But robots.txt is "wide open" (attachement).
I've tried many methods, rebuilding robots.txt, leaving it "wide open" for crawler, different nginx rules, plesk repair etc. There are still 404 errors in the log and list of rejected url at google is raising