Ok, now I understand what the problem was. Thank you for that link.
https://github.com/PrestaShop/PrestaShop/issues/24207#issuecomment-1044662820
As this says pressing the "send a test email" button doesn't send the email using the stored credentials in the database. Instead, it uses the configuration that is being showed currently in the form above, even if you didn't submit the changes.
Chrome was, because of the password manager feature, "helpfully" changing the SMTP password in that form to the admin panel password. I had noticed that (since the password field turns blue when it's automatically completed) but I didn't think it was important because I had saved the correct password previously using the same form. Turns out I was wrong!