My module "One million of products"

[MiPP]

Hello,

Here is a link to the demo of a module allowing to have 1 million (or more) products on a prestashop.

http://millionpresta.wamania.com/fr/recherche?controller=search&search_query=produit

• 1M products
• 8 languages therefore 8M language lines
• 3M lines of category
• 4.5M lines of features
• 2.5M images and therefore 20 million of image captions (8 languages) and 15 million files
• 5.6M combinaisons, for a total of 13.6 million lines of attribute (a combinaison can have multiple attributes)
• 50M lines of tag (to fully fill SOLR)
• 6.6M lines of stock
• 1000 suppliers
• 1000 manufacturers
• 100 categories (not a lot, to test very large numbers of products by category, with an average of 30 000 products per category)

A total of 125M rows in the database.

All search is based on a Solr server.
In addition of Solr, many optimizations were made to speed up loading, and to make it manageable in the back-office.

It all comes in the form of a traditional PrestaShop module that contains:

• The interface between PrestaShop and Solr.
  I took the party to override the normal operation of PrestaShop. Thus, all search and indexing functions have been diverted by the override mechanisme offered by PrestaShop. The advantage is that the module is fully compatible with other modules provided that these modules do not conflict with this overload.
   
• back-office side, no solr, only for SQL optimization and some straight cuts. For each laborious display, all that was consuming too much compared to the real utility has been removed. For example, the display in the list of products of the default category for each product cost much resource, and its usefulness is quite limited.
   
• A module for the left column of the facets, inspired by the module blocklayered of PrestaShop.

PrestaShop Config

• Compiling smarty templates "Do not recompile"
• No smarty cache because on such a volume, caching costs more resources than it saves in use, especially since every call, cache verifies its validity in the database, already busy .
• Obviously, no memcached or other cache.
• A php apc cahe for opcode cache only.

Server side:

• 2 virtual servers (1 for Presta and 1 for solr), based on a physical server OVH: 2016-HOST-32L (Intel 4c D-1520 / 8t Ram and 32GB)
• web server with 2 thread and 8go ram
• solr server with 4 thread and 16go ram

Feel free to post your comments, questions or requests.

Edited April 7, 2016 by MiPP (see edit history)

[Inform-All]

Sounds promising but your link is not working

[MiPP]

Very sorry, i was working on the theme and i left an error.

It's now OK