On 11/28/2022 at 4:02 AM, Rajeshx said:Seems like google is not able to crawl robots.txt and connectivity with the server is also not possible.
Google gave a note out that they do not follow the robots txt anymore in some features. So there could be a new method. I think for Europeans the merchant center is pretty useless. Even for US its hard to find a statistic nothing more than so many businesses have a merchant account. Which is quite meaningless.
https://developers.google.com/search/docs/crawling-indexing/robots/intro
and https://www.searchenginejournal.com/google-robots-txt-noindex/314961/#close