It's generally good not to use too much memory, as it can lead to high server use during a potential attack.
With that said, PrestaShop can sometimes use a lot of memory, especially in backoffice functions.
You can set one limit for the main site, and another to the /admin folder you use, which is higher if you need.
Did you run into any memory issues?
For most front ends, 256, and definitely 512 should be enough.