PrestaShop increases the stability of version 1.7

The latest version of PrestaShop was launched in December 2018. Indeed, PrestaShop 1.7.5.0 is the most recent minor release of the PrestaShop project, available for download and largely based on feedback from our community. This version brings new features and improvements for both Back Office and Front Office, such as:  SEO optimizations, dynamic preview of Google search results for your product, optimized page loading times and much more! 

Since we launched PrestaShop 1.7.5.0, we have received a lot of questions about the stability of this new version.  Claudia Rodriguez, PrestaShop Ambassador in Mexico, raised the following question:

Version 1.7.5 has worked well for me. Do you have information to eliminate doubts concerning stability? 

Through this article, we took advantage of Claudia’s question in order to clarify what’s the true definition of “software stability” and how PrestaShop’s teams have improved the software testing processes to make it more stable.

 

Stable as "no bugs"

PrestaShop v1.7.5.0 solves 114 reported bugs (source: http://build.prestashop.com/news/prestashop-1-7-5-0-available/). This is twice as many as the previous minor version PrestaShop 1.7.4.0 (source: http://build.prestashop.com/news/prestashop-1-7-4-0-available/). Moreover, quality issues found shortly after public release of PS 1.7.4.0 required us to release 3 patch versions in quick succession. This did not happen with PS 1.7.5.0. In fact,  since no critical issues have been found after the initial release, the first patch version for 1.7.5 (1.7.5.1) was released two months after 1.7.5.0. PS 1.7.5.0 has consequently been proved very reliable as it did not need any urgent patches.

What did we do to improve between 1.7.4.0 and 1.7.5.0? We realized that it was extremely important to provide the community with enough time to test new versions, so that any regressions can be fixed before, and not after public release. This is why, starting with 1.7.5.0, we began to provide an extended full month-long public beta period (the second phase of software testing in which a sample of the intended audience tries the product out) for new minor versions. With 1.7.5.0, we used this extra time to gather feedback and bug reports from contributors and partner agencies. This is the longest beta period we ever did. The fact that we did not need to release a patch version quickly after public release proves that this period helped a lot to stabilize and consolidate this new version.

While this beta was available for PS users to find bugs, during a 1-month period, developers and QA team were also busy at beta-testing the release as well.

 

Stable as "well tested"

PS 1.7 has introduced the use of automated tests for the PrestaShop project:

• unit tests verify selected pieces of code to make sure they behave correctly in isolation.
• integration tests check that the codebase, as a whole, works correctly.
• end-to-end tests automate browser actions to ensure that from a user point of view  the application works as intended to.

These tests allow the development team to quickly detect when some parts of the system stops working, and are executed for every Pull Request (code contribution) on GitHub.

The number of tests increases with each new version of PrestaShop, and the development team is continuously improving the system that executes them. All of this is visible on GitHub (and all the community can contribute as well!).

Also, the quality assurance (QA) process is now very accurate and robust.

Whenever a developer wants to modify the source code (to fix something or to add a new behaviour that would be useful for users), he or she makes a Pull Request on GitHub.

This Pull Request will at first be automatically analyzed to ensure that:

• code style is correct
• unit, integration and end-to-end tests do not break with this new version of the code
• the code can be merged into the codebase without creating issues

Note: The full suite of end-to-end tests is not run for every PR because it takes a very long time (8 hours or more!) to cover the whole application. However the full suite is being run every night during the “nightly build” process, where we build and test the source code automatically every night. So if something is broken by change, then the next morning we will get a report telling us what got broken so it can be fixed. Nightly builds and test reports are available here: https://nightly.prestashop.com/

If automated tests pass successfully, then the Pull Request will be reviewed by a Core Developer in order to make sure that the submitted change is correct and useful. If issues are found, a conversation begins between the reviewer and the contributor to fix them. In some cases, the Core Developer may ask for help from other people relevant to the issue: the Design Team (if the Pull Request modifies PrestaShop design), the Product Team (if the Pull Request modifies PrestaShop behaviour), or other developers or community members.

When the code review has been done and approved (maybe after a few changes), the Pull Request is then tested by the QA team, who will verify that the submitted code behaves correctly from a user point of view AND that it does not break other features that were working well previously.

In the future, most of this work will be automated, but right now we need the combination of automatic QA and human QA. If issues are found, and a discussion begins between the QA analyst and the contributor in order to get the Pull Request fixed. Other verifications are performed as well, such as approving new wordings for example. Because localisation is an important feature of PrestaShop, if a Pull Request modifies texts or content in the application (this will impact its translation), it must be approved by the Content Manager too.

If all of this is approved, the Pull Request can be merged into the codebase!

As you can see, a lot of work is dedicated to minimising the risk that code changes introduces new bugs. Of course, bugs are unavoidable, but most of the time, when  a regression is introduced, it is spotted by the QA Team and corrected before release. The QA team checks Pull Requests very thoroughly, testing special use cases such as multi-shop, multi-language, right-to-left languages...

We have received some feedback from contributors who preferred the "old way", where Pull Requests would be merged with little, or no review. That era won’t come back. We have chosen the path of ever-increasing quality and stability, and even though this process is heavy and time-consuming, we plan on keeping it as it is, because it has been proved very reliable.

 

Stable as "on solid ground"

PrestaShop 1.7 is about migrating PrestaShop from a custom web framework to the industry-standard Symfony framework. This is a very long process, but the benefits will be huge, stability included. Symfony is widely used in PHP applications (PHP is the language used to develop PrestaShop), it is maintained by some of the best PHP developers in the world, has a very thorough quality and stability process (which inspired our own) and is one of the top two most popular PHP web frameworks, alongside Laravel.

Introducing Symfony allows us to get rid of the maintenance workload for a custom framework, which consumes a large amount of time that could be better spent working on what PrestaShop actually does best: managing a cart, handling orders, calculating prices and taxes, generating invoices, etc. Using a proven and popular open-source framework allows us to focus on our core business code with greater efficiency, while enjoying the stability of a globally recognized framework.

By using Symfony to power PrestaShop instead of our own framework, we rely on the quality of their work, which is widely recognized. When bugs are found in Symfony, they are fixed by the Symfony team. When there are performance issues found in Symfony, they are fixed by the Symfony team. By using this framework, we benefit from all the knowledge and expertise of these people to boost PrestaShop, instead of performing the same work ourselves. We follow here the "dwarfs standing on the shoulders of giants" and “don’t reinvent the wheel” approach, which is proven to work very well.

We could also talk about our security issues management process, but that is not really about stability. In short: we care a lot about PrestaShop software security. Whenever a security vulnerability is found, it becomes the top 1 priority of the whole Core team (developers, product, QA) to get it fixed and released as fast as possible.

 

Conclusion

We hope this article has given you a better understanding of what a stable software is, and how the PrestaShop teams invest on Automatic testing and Symfony migration in order to offer our users a better, more stable software.

You can download PrestaShop 1.7.5 now and access all the new features, saving time when it comes to recurring store management activities for example. This new version features significant improvements in productivity and performance, and is a real step forward in terms of mobile and social networks.