Multi-languages will make pages content duplicated?

[Guest]

Hi,

There are 3 languages in the default themes.

When upload products only in english,without adding french or spanish , will duplicated content appear?

Metag tags are the same in the same page but different languages , will it be penalize by google?

If yes, should I remove the other languages from BO?


Sample: [Fake website - Moderated by Patric]

[mrFox]

Your site should not really get penalised by search engines as they understand multi-lingual sites and the need for them.

If you don't need the languages though just remove them as the "lorem ipsum" stuff looks bad.

If you do need languages but haven't got around to it yet, use your robots.txt file to prevent search engines reaching your multi-lingual sections. Such as:

User-agent: *
Disallow: /lang-fr/
Disallow: /lang-es/

[tomerg3]

See http://www.presto-changeo.com/content/6-prestashopseotips for a fix for that and some other tips.

[SITOLOG - F Bugnet]

Hi,

Tomerg3, as I do struggle to define the right strategy with Presta and multilanguage on my own site ( www.myphotorescue.com ) , I looked on how your own site manages duplicate content (should be close to perfect as you have the knowledge and even sell modules for this).

Still found a risk:
http://www.presto-changeo.com/prestashop-free-modules/35-quantity-discounts.html . I got there from the link you provided yourself above and this is providing the same content than
http://www.presto-changeo.com/lang-fr/prestashop-free-modules/35-quantity-discounts.html, which is the URL I got by clicking on the French flag, after having clicked on the US flag to switch to english once.

Both displayed the same page in French

Isn't this a risk of duplicate content and if so, what is the the good way to prevent it. Is it to redirect 301 all french pages to /lang-fr/ ?

Unless, as Google does not read cookies, when pointing to http://www.presto-changeo.com/prestashop-free-modules/35-quantity-discounts.html, it always got the page in english ?

I checked this , when click on the US flag, you get "http://www.presto-changeo.com/prestashop-free-modules/35-quantity-discounts.html", without /lang-en/ which is good. At least not that sort of duplicate content risk in english, right ?

Your tips are welcome.
Rgds
Franck Bugnet
Webmaster www.myphotorescue.com

[tomerg3]

You saw http://www.presto-changeo.com/prestashop-free-modules/35-quantity-discounts.html in french because I have a module (Location & Currency Detection) that redirects you to your language based on your location.
Prestashop will automatically remove the /lang-xx/ when cookies are set, this will not affect search engines, if the crawl a page without /lang-xx/ it will appear in your default language, and with lang-xx, the pages will be displayed in that language.

As for /lang-en/ redirecting to the same page but without the /lang-en/, that is something I added to my Duplicate URL Redirect module.

[SITOLOG - F Bugnet]

Hi, thanks for your quick response. I think I have understood now.

I have installed your location detection module today. Looks great and usefull. I think I found a small bug with it, about the currencies change and will document lately for you, when I have fully checked the issue is not on my side. Testing is not obvious as I have only one french IP address.
At least I can see the language auto change works with your site as effectively it detects my location and displays your pages in French (may be not a so good ideas btw, as the Google auto transaltion is so so and I immediatly changed it back to english :-) )

You gave me THE info I was missing: Presta removes the /lang-xx/ when the language cookie is detected and managed. I did not know this and thaught Google saw the same than me. As this is not the case when Gooblebot crawls the site, everypage seen without the /lang-xx/ will be indexed in the default language (en in my case) ? correct ?
So, there is no risk that presta is showing to Google my domain.com/pagex in French, right? hé, what a relieve for me as this was the most struggling point, wondering how to rewrite all French pages to /lang-fr/
To be 200% on the safe site, will do a few check, looking for a way to display th pages excatly like google bot saw them.

About the re-routing or re-writting of .com/lang-en/ to .com/, this makes full sence to me and will try to do the same. As I have quit my job and I'm starting my own business, I'm counting every penny, so will try to make it without purchasing the module. Please forgive me ...

Big Thanks for your help, much appreciated.
Franck
www.myphotorescue.com

[tomerg3]

the best way to check is to google site:www.yoursite.com and see how they index your site.
As for purchasing my module, it's obviously your call, but I think $25 is well worth it, and you will most likely end up spending much longer to recreate everything that module does.

[SITOLOG - F Bugnet]

Hi Tomerg,

After having spend some time creating some code myself, to go faster I finally purchased and installed your module.
Looks like it is working as planned, but...

Still, I think (may be I'm wrong), there are still a number of content duplicate cases to work on:

1st cases: I have set up specific friendly URL per language, for each category and each product (to have my strategic keywords in each URLs, in the right language). My site default language is en.

As Presta removes automatically the /lang-fr/, Google see all french pages in english (I made a check to confirm this), while their URL contains a french friendly URL . So they will be indexed in english and seen as duplicates with the same english page, with URL using english friendly url.

I'm quite frustrated, convinced your modules would manage this.

Is the only way being to change all my pages secondary language friendly URL similar than my primary language friendly URL ?



2nd cases: When looking in Google Webmaster tool HTML suggestions made for my site, it does report a number of warnings about duplicate meta titles. In fact, it does index many pages with many parameters, like this one:
/12-autres?isolang=fr&n=10&id_lang=2&orderby=quantity&orderway=desc‎
or this one
/new-products.php?isolang=fr&id_lang=2&orderby=name&orderway=desc
I have tested them after your module installation, they are still working, un-redirected and my cause duplicate content. Let's take this exemple:
if I enter:
http://www.myphotorescue.com/category.php?id_category=5isolang=en&id_lang=1&orderby=name&orderway=desc
it is redirected by your module to
http://www.myphotorescue.com/5-restoration?id_category=5isolang=en&id_lang=1&orderby=name&orderway=desc'>http://www.myphotorescue.com/5-restoration?id_category=5isolang=en&id_lang=1&orderby=name&orderway=desc
which is in fact absolutly the same than
http://www.myphotorescue.com/5-restoration

I understand there is probably nothing your can do about it, those parameters being needed by Presta in many cases.

In Google Webmaster tools parameters, I have asked Google to ignore id-category, id_lang, id_product, isolang,...
but not sure this is taken into account. There are others parameters probably I missed (cms ?), and what about other robots ?
Any suggestion or recommendation ?


3rd case: I do have external links pointing to mydomain.com/lang-fr/index.php , this is a duplicate with mydomain.com/lang-fr/, not corrected with your moduel.
Try this one on your own site:
http://www.presto-changeo.com/lang-fr/index.php still working as well, not good.
I thought your module was redirecting all index.php url, not only the one without /lang-xx/

This last case worries me a lot. Is there something you can do about it ?


Last but not least, just a question: Without a sitemap, containing all french URL with /lang-xx/ for each secondary languages, no robots will find those pages by themselves. So without a good sitemap, only the default language pages are indexed. Correct ?
This is because Presta hide the /lang-xx/ when cookies are detected you said.
But I just de-activated the cookies to make some tries. While the site in set in French (one of the secondary language), all the navigation is done with/lang-fr/ in the urls. So, same comment as before, Google will see all pages (probably in english), with French Friendly URL and no /lang-fr/
To my view, this is messy. Why no simply using systematically the /lang-xx/ in all url when a site is set to a secondary language ?

Regards
Franck

[tomerg3]

If you don't use /lang-xx/ then your site will be displayed in the default language of the shop, if you click on a flag, and have cookies enabled, then prestashop automatically knows to display it in your language, but without the /lang-xx (this does not apply to search engines, since they don't have cookies enabled).

Your sitemap should have /lang-xx URLs for all the languages, except your default one, which will appear without it.

The duplicate meta tag is not a URL issue, the content of the pages are not the same (for the most part), so it's not something that my module will handle, I have however posted a solution to it (altering the meta info based on the variables (sort order, pagination and number of results per page), see http://www.prestashop.com/forums/viewthread/52665/

I will look into the /lang-xx/index.php issue.

You should have a complete and updated sitemap at all times, you can download the free sitemap module from my site, it fixes a few bugs in the original version.

[Mallee Boy]

Frank, Did you solve your duplicate meta titles and the orderby/orderway duplicates? i am having teh same difficulty and would like to be sure i have the proper solution.

Thanks.

[tomerg3]

Frank (and others), see http://www.prestashop.com/forums/viewthread/55895/

[Sanis]

If you don't use /lang-xx/ then your site will be displayed in the default language of the shop, if you click on a flag, and have cookies enabled, then prestashop automatically knows to display it in your language, but without the /lang-xx (this does not apply to search engines, since they don't have cookies enabled).

Your sitemap should have /lang-xx URLs for all the languages, except your default one, which will appear without it.

If i understand correctly, you are saying that all multilang urls should be the same to avoid:


shop.com/4-portables-apple
shop.com/lang-fr/4-portables-apple


And the sitemap should look like this:

/laptops/ - for english
/lang-fr/laptops/ - for french

[tomerg3]

Yes, that's correct.

[djmars]

Dear Tomerg3,

I bought the seo module on your site to generate different language's urls on my eshop.

I have translate my eshop into 40 languages.

I'm using your sitemap generator as well.

My problem now is that my sitemap is too big (308629 lines corresponding to 7000 products translated into 40 languages)

http://www.oxygenez-vous.com/shop/sitemap.xml (don't open it into your navigator, you will crashed it)

Mise à jour Monday 01 November 2010 13:23:29
Taille du fichier 70.681mo
Nombre de pages indexées 308629

Google only accepted sitemap of 50 000 lines.

Do you have a way to divide the sitemap into different sitemap's files by languages ?

best regards
Martial

[tomerg3]

The quickest and easiest solution would be to manually split the sitemap file.

Google will find your pages even without the sitemap, so it's not really a problem, but I highly recommend also applying the fix from http://www.prestashop.com/forums/viewthread/74380/ to make sure google don't over-crawl unnecessary pages.

[Smileone]

Our major clients website are multilingual. Our client or our webmasters never makes complaint or messages on duplicate this. Because the clever Google and other search engines knows, what are the languages you are using in your site.

If your site on English and you are using French language, absolutely the content will be different. isnt so?
If your site on French and you are using Spanish language, again the content will be different

so comparing each language drives different content and metas..;hopefully it will not drive duplicate...

[Guest kumon]

Our major clients website are multilingual. Our client or our webmasters never makes complaint or messages on duplicate this. Because the clever Google and other search engines knows, what are the languages you are using in your site.

If your site on English and you are using French language, absolutely the content will be different. isnt so?
If your site on French and you are using Spanish language, again the content will be different

so comparing each language drives different content and metas..;hopefully it will not drive duplicate...

The question, Smileone, is whether there are two different URL pointing to the same content, regardless of the language.
If there are, Google will not like it.

[Guest kumon]

I am going a bit crazy trying to decide what to do.
I have a live shop with Prestashop 1.3.2. at www.queoriginal.es
I have setup two test shops:

- Prestashop 1.3.2 --> www.mindposition.com/shop02

- Prestashop 1.3.6 --> www.lailai.es

OBSERVATION 1
On Prestashop 1.3.2 (www.mindposition.com/shop02)
If I do
1. Click on US flag
2. Click on Prestashop logo
3. Click on ipod-shuffle
I get: http://mindposition.com/presta02/music-ipods/2-ipod-shuffle.html
and I can see the English product page for ipod shuffle.

If I then continue and
4. click on the French flag
I get: http://mindposition.com/presta02/lang-fr/music-ipods/2-ipod-shuffle.htm
and I can see the French product page for ipod shuffle.

Note the category name has not changed in the URL. It is music-ipods, which is the English name of the category (I ignore the product name for now)

OBSERVATION 2
If I do
1. Click on the French flag
2. Click on the Prestashop logo
3. Click on the i-pod-shuffle
I get: http://mindposition.com/presta02/lang-fr/musique-ipods/2-ipod-shuffle.html
and I can see the French product page for ipod shuffle.


So, two different URL pointing to one page, the French product page for ipod shuffle. This is duplicate content for Google.

----------------------------------------------------------------------------------------

OBSERVATION 3
Now, in the other test site that I have setup with Prestashop 1.3.6. This is not happening.

Try the same with with www.lailai.es [sORRY THIS HAS BEEN PASSWORD PROTECTED NOW]
I only get one URL pointing to the French version of the product page for ipod-shuffle:
http://lailai.es/lang-fr/musique-ipods/2-ipod-shuffle.html

----------------------------------------------------------------------------------------

So, I see an improvement in Prestashop 1.3.6 from Prestashop 1.3.2.

I have been reading the Multilingual SEO Fix here: http://www.presto-changeo.com/content/12-prestashop-multilingual-seo-fix, but it does not seem to refer to exactly the same issue. I think that fix is focused on the problem of the "disappearing lang-xx" for non-default languages due to excessive trust in cookies.

SO THREE QUESTIONS


QUESTION 1
Is the problem I have described the same that tomerg3's Multilingual SEO Fix addresses?


QUESTION 2
Prestashop 1.3.6. seems to have solved at least some of the multilingual URL issue.
Does anybody know whether all the issues have been solved. In other words, that one need tomerg3's Multilingual SEO Fix when using Prestashop 1.3.6?

QUESTION 3
I am thinking of upgrading my live shop to Prestashop 1.3.6 to solve this problem.
Is this an overkill?

Thanks,
Juan

[Guest kumon]

It is important to note that the overlap occurs only in the same language. The same page in English, Spanish and French - even if it expresses the same ideas - will never be perceived as identical. Duplication is the most common challenges in fields that make extensive use of global languages like English, Spanish, French and German.

OK, just to leave no room for further missunderstanding

URL1 --> Page Z in French
URL2 --> Page Z in French

That is two links pointing to same content, duplicate content.

[Abaya Dubai]

Is manually splitting a sitemap as easy as it sounds?

yes, it can be easily done if its html format even if it was in xml there is a way around it.