Jump to content

Maxmind GeoLite IP Database: end of free updates


antoine@ultro

Recommended Posts

As mentioned on their website, Maxmind will no longer offer upate of their City database recommended in the Prestashop documentation.

Quote

As of April 1, 2018, updated versions of the GeoLite Legacy databases will be available to redistribution license customers only, although anyone can continue to download the March 2018 GeoLite Legacy builds. On January 2, 2019, the last build will be removed from our website. GeoLite Legacy database users will need to switch to the GeoLite2 or commercial GeoIP databases and update their integrations by January 2, 2019.

3 questions:

-  does someone know another free alternative with ipv4 and ipv6 data? 

- does someone use the https://www.ip2location.com databases with success?

- can we switch from a cities database to a countries database which is less expensive? I've seen this Forge ticket PSCSX-5643, but it has been closed.

Link to comment
Share on other sites

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Note: as native Geo Location denies shopping for non-resolved IP's then removing 'free/slight accurate' db is best thing for geo location in PrestaShop.

PrestaHeroes Geo Targeting Pro module supports two commercial maxmind db's and the geo insights tab shows 99.4-99.9% resolve rate and includes ipv6 support.  This is by far the best geo targeting module in PS module eco...I know this to be true. :)

Happy selling

 

Link to comment
Share on other sites

This is known problem, but Prestashop didn't yet replayed to it: http://forge.prestashop.com/browse/PSCSX-9710

I've added all informations they need, but seems to be not a priority task for to debug ;) Perhaps somebody can up this ticket by writing there ? My report is dated January 2018 !!

The module will continue to work, but the database will be old in some months...

16 hours ago, El Patron said:

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Not really that what I'm observing. For me accuracy is 99,99% with GeoLite.

Link to comment
Share on other sites

3 hours ago, selectshop.at said:

This is known problem, but Prestashop didn't yet replayed to it: http://forge.prestashop.com/browse/PSCSX-9710

I've added all informations they need, but seems to be not a priority task for to debug ;) Perhaps somebody can up this ticket by writing there ? My report is dated January 2018 !!

The module will continue to work, but the database will be old in some months...

Not really that what I'm observing. For me accuracy is 99,99% with GeoLite.

 

we saw much lower rates since we added geo insights tab to module/%of resolve rate.  It really depended on what part of the world your visitors were from.  here is  screen shot of 'very busy' client shop using commercial maxmind db

https://www.screencast.com/t/XuVHP0geft

we see much lower resolve rate for free db, which is moot since it's going away.....

happy localizing

Link to comment
Share on other sites

GeoLite has country reported as "EU" if it is not sure which country in Europe its located, same as AP. It is causing issues in some of our reporting features.

We are trying to avoid it by using IP2Location LITE which has the results by actual country. Wish we can have the IP2Location LITE solution ready for free.

Link to comment
Share on other sites

  • 4 weeks later...

just to give a feedback about the module

I installed it on my shop running on prestashop 1.5.6 and it works just fine. It also comes with a cron job to download the bases automaticly

The support was awesome. They managed to adapt a module that uses the geolocation feature.

 

  • Like 2
Link to comment
Share on other sites

  • 3 months later...
  • 2 months later...

I disagree above that geolitecity is as accurate as claimed unless you have built in IP resolve rate like Geo Targeting Pro

 

the big mistake people make because they do not understand is that  if you enable PS Geo Location,  non-resolved IP addresses are not allowed to shop.  PS native Geo Location does support IP country but is built to 'restrict' visitors from excluded countries. 

another problem with other solutions is that while they may localize the country via mmdb for example, when the visitor logs in their country is changed to 'registered country' which will affect specific price by country, i.e. you showed  prices for IP country but they will change if registered country is different.

Understanding how native Geo Location works (if you want to restrict shopping) is very important when considering how you implement a geo strategy.

so 75-85% resolve for geolitecity in US, and maxmind is US company....I have no confidence when declared 99% in EU.  I can tell you for sure that  is not true again because we have the stats in our module to prove otherwise.

be careful when picking a geo solution and understand how native works before implementing.

 

happy geo'ing, el

 

 

 

Link to comment
Share on other sites

What should I do here?

I currently have a GeoLiteCity.dat file in tools/geoip/.

Maxmind currently offers the lite verision with a file ending as .mmdb, GeoLite2-City.mmdb. Should I delete the old file and replace it with this?

There are 3 databases GeoLite2 City, GeoLite2 Country and GeoLite2 ASN, should I just use City?

Edited by nicoX (see edit history)
Link to comment
Share on other sites

On 2018-03-14 at 3:58 PM, El Patron said:

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Note: as native Geo Location denies shopping for non-resolved IP's then removing 'free/slight accurate' db is best thing for geo location in PrestaShop.

PrestaHeroes Geo Targeting Pro module supports two commercial maxmind db's and the geo insights tab shows 99.4-99.9% resolve rate and includes ipv6 support.  This is by far the best geo targeting module in PS module eco...I know this to be true. :)

Happy selling

 

 

What does that mean, if you run this module, does it include free lifetime updates of the commercial db's. And would the user need to download the db's?

Link to comment
Share on other sites

hi el Patron your module indicates 99.99% this for visitors coming mainly from europe 

I'm with attention to the statistics of your module,
because the free maxmind file will quickly be outdated

advise everyone to buy your module

I was also waiting for you to adapt your module to the free databases of maxmind
was good for everyone, because I see the prestashop team, very apathetic regarding this subject and the end of prestashop 1.6

I'm looking at thirtybees
It's just my opinion.
 

Link to comment
Share on other sites

On 2018-10-03 at 6:59 PM, nuno.note said:

hi el Patron your module indicates 99.99% this for visitors coming mainly from europe 

I'm with attention to the statistics of your module,
because the free maxmind file will quickly be outdated

advise everyone to buy your module

I was also waiting for you to adapt your module to the free databases of maxmind
was good for everyone, because I see the prestashop team, very apathetic regarding this subject and the end of prestashop 1.6

I'm looking at thirtybees
It's just my opinion.
 

 

Thank's for the headsup on the PrestaShop fork https://thirtybees.com/

I'll definitely will be looking that up for my next project.

EDIT:

Tested it and can say it runs awesome, speed improvements at least. 2500 commits since their fork from 1.6.1.11.

Edited by nicoX (see edit history)
Link to comment
Share on other sites

  • 3 months later...

remember, native geo location logic will disallow shopping when IP does not resolve.  this is why geo targeting pro was born,  it allows you to set a country for non-resolved IP's so they can shop.  Understanding this important fact when implementing native geo location should be considered when choosing the IP lookup source.  Free has 75-85% resolve rate, a bit better in US.  Happy geo'ing,  el

Link to comment
Share on other sites

4 minutes ago, El Patron said:

There is no command, we override it without geo targetig pro.  You would need  to locate the geo localization and hack that to set non-resolved IP to a default country and allow shopping.  

I was referring to the github tool:

$ ./geolite2legacy.py -i GeoLite2-Country-CSV.zip -f geoname2fips.csv -o GeoIP.dat
Database type Country - Blocks IPv4
wrote 306385-node trie with 300679 networks (251 distinct labels) in 8 seconds

# ./geolite2legacy.py -i GeoLite2-ASN-CSV.zip -o GeoIPASNum.dat 
Database type ASN - Blocks IPv4
wrote 518484-node trie with 417952 networks (62896 distinct labels) in 15 seconds

 

 

 

Link to comment
Share on other sites

1 hour ago, NSN said:

If you want to use the GeoLite2 data base without transforming it into the old format you can use the module Support for MaxMind GeoLite2 Databases (MMDB) which not only updates the geolocation to the new database but also provides a cron task for keeping the database up to date.

BTW, it also allows non resolved IPs to access the shop.

Thanks, I will probably check this out, good price, and automation!

20 minutes ago, toplakd said:

python geolite2legacy.py -i GeoLite2-City-CSV.zip -f geoname2fips.csv -o GeoLiteCity.dat

 

I did it in Windows bash. I downloaded GeoLite2 City csv zip file. I don't know what geoname2fips.csv is. There is no file like that. I tried without that as well, but ended up with:

anonX@anonx:~/geolite2legacy$ ./geolite2legacy.py -i GeoLite2-City-CSV.zip -f geoname2fips.csv -o GeoIP.dat
Traceback (most recent call last):
  File "./geolite2legacy.py", line 32, in <module>
    import ipaddr
ImportError: No module named ipaddr
anonX@anonx:~/geolite2legacy$ ./geolite2legacy.py -i GeoLite2-City-CSV.zip -o GeoIP.dat
Traceback (most recent call last):
  File "./geolite2legacy.py", line 32, in <module>
    import ipaddr
ImportError: No module named ipaddr

 

Link to comment
Share on other sites

3 hours ago, toplakd said:

:)
You have to install the module which is described by ImportError: ipaddr

and after that all for which you will get error.

Ok, I downloaded both ipaddr and pageoip.

Which commands do I now have to run? There is Country-CSV.zip, City-CSV.zip and ASN-CSV.zip. Which ones do I need?

Edited by nicoX (see edit history)
Link to comment
Share on other sites

14 hours ago, El Patron said:

...I don't care about free maxmind or fixing so it works.....I just provide feedback of how native geolocation works and to use at your own risk.

happy geoing.....

Have one question about your Pro module.
How does it handle visitors IP's and resolving their country when shop is on cloudflare?

Link to comment
Share on other sites

4 hours ago, toplakd said:

Download database on bottom link

https://geolite.maxmind.com/download/geoip/database/GeoLite2-City-CSV.zip

Rename it to GeoLite2-City-CSV.zip (when downloaded date is added to its name)

than run.

python geolite2legacy.py -i GeoLite2-City-CSV.zip -f geoname2fips.csv -o GeoLiteCity.dat

So not the CSV format database file?

What's the difference between City, Country and ASN?

Link to comment
Share on other sites

6 hours ago, toplakd said:

Nico. Read the post again. The link is to CSV Database which is than with command converted to old legacy format.

For the info about base difference check maxmind.com

The GeoLite2 City database returns location data down to the postal code level when available, the GeoLite2 Country database just returns the continent and country, and the GeoLite ASN returns information about who is operating the IP address.

Link to comment
Share on other sites

On 1/30/2019 at 1:56 PM, toplakd said:

Nico. Read the post again. The link is to CSV Database which is than with command converted to old legacy format.

For the info about base difference check maxmind.com

Do I need to do anything else when owerwriting the .dat file? Disable/enable the geolocation, clear the cache? 

Some customer complain they can't access my site from a specific location and country. By getting a 503 server overloaded status. Is this related to geolocation? 

Link to comment
Share on other sites

On 1/29/2019 at 4:28 PM, NSN said:

If you want to use the GeoLite2 data base without transforming it into the old format you can use the module Support for MaxMind GeoLite2 Databases (MMDB) which not only updates the geolocation to the new database but also provides a cron task for keeping the database up to date.

BTW, it also allows non resolved IPs to access the shop.

How does it allow non resolved IPs? With a default country you select? 

Link to comment
Share on other sites

vor 3 Stunden schrieb nicoX:

It should, or it will? Have you tested it? 

I'm not a programmer nor such a geek that I now how to test it. I can just rely on the developer who told me that the non resolved IPs are granted access.
I thought it would be the default as it is set in the admin and I have no idea what else should be used if the IP does not have a link to any country, but if you know how to check tell me how and I will try.

Link to comment
Share on other sites

One thing I know is that the module uses the same database (exactly the same copy) as my matomo installation and it sometimes happens that I come across a user that has no country flag and says "country unknown".

Just happened:
matomo.png.886c72745d92f38313168fcd841ca186.png

Link to comment
Share on other sites

On 3/14/2018 at 2:58 PM, El Patron said:

The geolitecity.dat file is 'best case' accurate 75-85% of time.

Note: as native Geo Location denies shopping for non-resolved IP's then removing 'free/slight accurate' db is best thing for geo location in PrestaShop.

PrestaHeroes Geo Targeting Pro module supports two commercial maxmind db's and the geo insights tab shows 99.4-99.9% resolve rate and includes ipv6 support.  This is by far the best geo targeting module in PS module eco...I know this to be true. :)

Happy selling

 

How is the module updating the databases? And which databas are in use, and why two? 

Link to comment
Share on other sites

@El Patron Did you read my question?

I'm asking everyone, I disabled geolocation and now my TTFB has got an huge improvement. Site loads within a half second, while when geolocation was active it took 1-2 seconds. 

Anyone else tested this and expirenced similar results?

If geolocation is adding a delay, which I think it is because it needs time to figure out the visitors location, I need another solution as web browser location or anything else. 

Link to comment
Share on other sites

I can confirm after testing with my other site that geolocation is having a huge impact on the TTFB. 

My sites now load within some ms with geolocation disabled. I can understand now since geolocation uses a database and queries the connection, adding extra load and delaying the initial connection. 

I don't know what other solutions there are to geolocation, any way for me it's not that important. 

Link to comment
Share on other sites

On 2/2/2019 at 9:34 PM, nuno.note said:

Hi Nico

Slow in ps 1.6??

Enable geolocation core or module??

Slow only ms not big slow 

I enabled core in 1.6.1.x. But as I said it has a noticable impact on the TTFB connection. I disabled it. I will use geolocation API built in web browsers HTML5 instead.

Edited by nicoX (see edit history)
Link to comment
Share on other sites

Am 1.2.2019 um 8:10 PM schrieb nicoX:

How does it allow non resolved IPs? With a default country you select? 

Update to your question.

I just received the preview of an updated Version which has now the function of setting a default country by your choice for non resolved IPs.

 

Link to comment
Share on other sites

the important thing about geolocation is
- detect the country or region
(obviously that the accurate country is the most real)
- in case of not detecting, how to proceed
(the perfect thing is to consider a country by default, this country should be the country where most visitors come from)
- after login
what should be considered, cookie country or adress county

if we verify the module geolocation pro, consider these scenarios

a customer that accesses the store sees the prices and buys the products with those same prices.

do not forget that some payment modules when returning to the store the successful payment come from different ips of the customer, and will certainly create purchases with wrong prices

Attention this is not to advertise the module, I hope that the module of prestablog also consider what I said, because as it supports the new free databases of maxmind is a good alternative

I'm actually considering buying for testing.

Link to comment
Share on other sites

On 2/4/2019 at 1:25 AM, NSN said:

Update to your question.

I just received the preview of an updated Version which has now the function of setting a default country by your choice for non resolved IPs.

 

It's because I asked for it.

I spoke with a Maxmind agent and he said the CSV databases are much slower than the binaries. This module uses the binaries, so I will go for it.

@El Patron is quite unresponsive so I couldn't show any interest in a module where the dev has a hard time answering.

Link to comment
Share on other sites

vor 6 Stunden schrieb nicoX:

It's because I asked for it.

I spoke with a Maxmind agent and he said the CSV databases are much slower than the binaries. This module uses the binaries, so I will go for it.

I too think the module is the right way to go. Not only because of the original database instead of a converted one, but also because of the possibility to force a country of choice.
Another good point that you can use the free version of maxmind's databases which I think are more than enough for most users.
The updated version (1.2.0) is already available on addons, as I saw recently.

Link to comment
Share on other sites

36 minutes ago, NSN said:

I too think the module is the right way to go. Not only because of the original database instead of a converted one, but also because of the possibility to force a country of choice.
Another good point that you can use the free version of maxmind's databases which I think are more than enough for most users.
The updated version (1.2.0) is already available on addons, as I saw recently.

I have also been looking at https://www.prestaheroes.com/en-us/modules/geo-localization/geo-targeting-pro-price-by-country

MaxMind GeoIP2 Country 99.8% accurate on a country level

MaxMind GeoIP2 Precision Country 

I think those two are commercial databases. But it lets you select the free db as well. 

Link to comment
Share on other sites

vor einer Stunde schrieb nicoX:

I have also been looking at https://www.prestaheroes.com/en-us/modules/geo-localization/geo-targeting-pro-price-by-country

MaxMind GeoIP2 Country 99.8% accurate on a country level

MaxMind GeoIP2 Precision Country 

I think those two are commercial databases. But it lets you select the free db as well. 

I did look at it too. To be honest, I did look at it for a long time because the module looks to be very functional and well developed, but I could not make up my mind to buy it. The various functions a very useful, but nothing I need in particular so that I would be willing to pay so much for it (the module is one part but you also need the database, so the total amount increases). If you then buy the database only once, the idea of the various functions and accuracy is gone, and if you pay for regularly updates it costs quite a lot. (From the module description and documentation it's not visible that the new free database is also supported)
If it is worth paying so much for all together has everybody to decide for himself, but for me it wasn't. Sure the accuracy of the pro database is great, but I have no problem if occasionally someone from Canada gets USD instead of CAD, or a European sees JPY instead of EUR, so the free one is more than enough for me.

In a nutshell, if you need a lot of functionality and high accuracy you should go with the one from PrestaHeroes and than also pay for regularly updates of the pro database.
If you just want to upgrade the basic function of native geolocation to work with the new free database and extend the functionality to deal with non resolved IPs I think the one from PrestaBlog is more than enough.

Link to comment
Share on other sites

  • 2 weeks later...

I'm little confused with PS Localization> Localization> Configuration

  • Set language from browser (Set browser language as default language)
  • Set default country from browser language (Set country corresponding to browser language) YES/NO

Is this some sort of geolocation acting on the browser language. Is this good to use and will it conflict with geolocation if both are enabled?

Edited by nicoX (see edit history)
Link to comment
Share on other sites

  • 2 weeks later...
On 2/19/2019 at 12:28 PM, nuno.note said:

if you use geolocation, with undetected country override (country by default), you should not use the default country in browser

What about the Set language from browser?

If I use Geolocation should this be on or off?

I'm just interested about the currency. We only use English language, but would like EURO countries to see EURO, UK to see Pounds.

Edited by nicoX (see edit history)
Link to comment
Share on other sites

  • 1 year later...

I am still using the Maxmind GeoIP file, which I download and extract every now and then.

I have found 2 different instructions on where the mmdb file should be extracted.

/app/Resources/geoip/GeoLite2-City.mmdb

And

/tools/geoip/GeoLite2-City.mmdb

Can someone tell which on eis correct or whether it doesn't matter?

Is there a logs/way to see how many IP's do not get resolved??

 

I just noticed that the currency is not automatically set per country as it used to (gopure.shop), GBP for UK namely at the moment.

Link to comment
Share on other sites

  • 1 year later...
On 2/28/2021 at 11:14 AM, Surffari said:

I am still using the Maxmind GeoIP file, which I download and extract every now and then.

I have found 2 different instructions on where the mmdb file should be extracted.

/app/Resources/geoip/GeoLite2-City.mmdb

And

/tools/geoip/GeoLite2-City.mmdb

Can someone tell which on eis correct or whether it doesn't matter?

Is there a logs/way to see how many IP's do not get resolved??

 

I just noticed that the currency is not automatically set per country as it used to (gopure.shop), GBP for UK namely at the moment.

This is the location of the geoip since 2019. 

 

image.thumb.png.53874cedf0f55836ca64950d28947944.png

Link to comment
Share on other sites

for anyone using built in PrestaShop geolocation, be WARNED:  when IP does not resolve PS disallows customer shopping.

this module on addons allows you to set a default country when IP does not resolve, also supports Free (not recommended), precision city and country maxmind db look up.

https://addons.prestashop.com/en/international-localization/27368-geo-targeting-pro-by-country-prices-taxes-currency-.html

 

 

 

 

Link to comment
Share on other sites

  • 2 weeks later...
On 8/16/2022 at 4:18 AM, El Patron said:

Stop recommending free maxmind, you will lose plenty of sales ulwith this dataset.  

 

 ps disallows unresolved ips, using free data you will lose a lot of sales.

 

 

What is the best practice for determining the country by IP then?
I've seen 1 module using 1p2location.com.

I've also seen Cloudways/Cloudfare offering GeoLocation support (apparently sent int he HTTP headers).
Does Prestashop (or any modules) support that and how reliable is it?

I am trying to achieve something like this to serve user the correct multishop and language/currency.

The shop would be serving customers in Europe (multishop 1) and USA (multishop 2) and I'd like to automatically do following:

  • If user from USA: Set Multishop 2, Currency USD, Lang EN-US
  • If user from UK: Set Multishop 1, Currency GBP, Lang EN-GB
  • If user from France: Set Multishop 1, Currency EUR, Lang FR-FR
  • If user from Germany: Set Multishop 1, Currency EUR, Lang DE-DE

Question for @El Patron, does this module enable the above: https://addons.prestashop.com/en/international-localization/27368-geo-targeting-pro-by-country-prices-taxes-currency-.html
 

Link to comment
Share on other sites

7 hours ago, Surffari said:

What is the best practice for determining the country by IP then?
I've seen 1 module using 1p2location.com.

I've also seen Cloudways/Cloudfare offering GeoLocation support (apparently sent int he HTTP headers).
Does Prestashop (or any modules) support that and how reliable is it?

I am trying to achieve something like this to serve user the correct multishop and language/currency.

The shop would be serving customers in Europe (multishop 1) and USA (multishop 2) and I'd like to automatically do following:

  • If user from USA: Set Multishop 2, Currency USD, Lang EN-US
  • If user from UK: Set Multishop 1, Currency GBP, Lang EN-GB
  • If user from France: Set Multishop 1, Currency EUR, Lang FR-FR
  • If user from Germany: Set Multishop 1, Currency EUR, Lang DE-DE

Question for @El Patron, does this module enable the above: https://addons.prestashop.com/en/international-localization/27368-geo-targeting-pro-by-country-prices-taxes-currency-.html
 

Geo Target pro supports maxmind precision city and country, both highly accurate and cheap.

Link to comment
Share on other sites

5 hours ago, El Patron said:

Geo Target pro supports maxmind precision city and country, both highly accurate and cheap.

So this would mean subscribing to the GeoIP2 City Database?
Or Country if that is enough, it seems to be $76 cheaper per month than the CIty.

Further question regards the module, can it handle redirection of user to the correct multishop as decsribed?

I need the US users to be directed to the correct multishop if they happen to land on the Euro shop (and vice versa).
 

Link to comment
Share on other sites

Hi,

I have 3 modules for geolocation, please check if it is suitable for your requirements. I have demo videos for each module on my YouTube channel to see a demo of how the module works. I can easily do customization as per your requirements.

[1]. Prestashop Currency & Language by Locations (GeoIP) Auto Redirect

https://www.presta-addons-modules.com/en/prestashop-currency-language-by-locations-geoip-auto-redirect.html

[2]. Prestashop Ban IP Address and Geolocation Redirect Visitor Country

https://addons.prestashop.com/en/website-security-access/28817-ban-ip-address-and-geolocation-redirect-visitor.html

[3]. Prestashop Show Popup Message and Redirect Links Based on Country

https://addons.prestashop.com/en/pop-up/29547-show-popup-message-and-redirect-links-based-on-country.html

  • Like 1
Link to comment
Share on other sites

On 8/28/2022 at 5:19 PM, Zohaib-fk said:

Hi,

I have 3 modules for geolocation, please check if it is suitable for your requirements. I have demo videos for each module on my YouTube channel to see a demo of how the module works. I can easily do customization as per your requirements.

[1]. Prestashop Currency & Language by Locations (GeoIP) Auto Redirect

https://www.presta-addons-modules.com/en/prestashop-currency-language-by-locations-geoip-auto-redirect.html

[2]. Prestashop Ban IP Address and Geolocation Redirect Visitor Country

https://addons.prestashop.com/en/website-security-access/28817-ban-ip-address-and-geolocation-redirect-visitor.html

[3]. Prestashop Show Popup Message and Redirect Links Based on Country

https://addons.prestashop.com/en/pop-up/29547-show-popup-message-and-redirect-links-based-on-country.html

Thanks for the tip!

I'll DM you...

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...