Orders not created when billing address is not PayPal's or when paying credit card without PP acct.

[ErickQC]

Hello, 

 

I have been stydying hundreds of posts in the past few days, related to Prestashop not creating orders after paying with PayPal, seemingly in a random way and starting not long ago. I have that problem and I think I finally found a very good clue of what is going on but still can't find the solution. All hints to solve this problem will be deeply appreciated.   

 

I am using Prestashop 1.5.6.0 and PayPal USA, Canada version 1.3.8. I use PayPal Standard. I noticed that sometimes orders were not created after successful payment in PayPal. I thought it was a random error, but I finally found a pattern: it is related to choosing a billing adress other than PayPal's. I noticed that initially in the live account and then moved to the sandbox to investigate.

 

When checking out in the store, the client has two choices for shipping and for billing: "My Address" (the one provided at the website)  and "PayPal".

 

Here is the error scenario: If he chooses "My address" as billing address, the client is forwarded to the PayPal website, he chooses whether logging into his account or paying with a Credit Card without logging in. He pays and gets confirmation of payment on screen by PayPal (and he is actually charged). Then he/she can click on a link to return to the merchant's site. Back there:

 

• The customer lands in the order history page, rather than in the one of order confirmation.
• No confirmation of payment is displayed by Prestashop or sent to the customer via email by Prestashop.
• No order is created in the backoffice
• The cart is not released: the items continue there as if they were not paid for yet.
• The cart appears as abandoned

 

In the other hand, when the customer chooses "PayPal" as billing address and log into PayPal for paying, the process is consistetly smooth. The client is charged, the landing page is a confirmation page notifying a reference number, the cart is released, the order is created, the cart appears as closed with the corresponding order number.   

 

Intiriguing finding in the PayPal logs:

 

Looking at the logs in the sandbox business account I found that:

 

1. In both cases, if the payment is completed in PayPal, PayPal sends a message with the payment status "Completed". No matter whether I had chosen PayPal's billing adress or not. Or whether I payed with a credit card without logging in or if I actually logged into the PP account. 
2. Although the status is completed, looking at the transaction details, the message sent is slightly different. (see images)
   • For the cases where Prestashop created the order successfully, PayPal returnded a message with the payment status "Completed", and the text  is "OK to complete the transaction"
   • For the cases where Prestashop did not create the order, PayPal returnded a message with the payment status "Completed" too, but the text is "OK to ship"

 

This "OK to ship" seems to be the problem and accordingly to the article below, looks to me as part of  a relatively new PayPal security feature called "Payment Review", which would be consistent with people reporting that this started happening recently: 

 

https://www.paypal.com/c2/cgi-bin/webscr?cmd=xpt/Marketing/securitycenter/sell/PaymentReview-outside

 

If you are wondering: 

• I have IPN enabled
• The "notify URL" is provided from Prestashop but is also configured in the PayPal account. 
• The store is put out of maintenance mode for the tests.
• In the PS_CONFIGURATION table PS_OS_PAYMENT value is 2, which is the same as I have defined in the statuses for "Payment Accepted"   

 

Thank you in advance for any ideas.

 

ErickQC 

 

 

 

 

 

 

[bellini13]

So since Paypal is accepting the payment, then we know the issue does not seem to be caused by Paypal itself.  I think what you need to do is debug the IPN.  I'm assuming that the IPN is being sent by Paypal in both cases, but in the 'failure' case, the module is likely not reacting properly.

 

What I usually do in this scenario is open the validation controller (/paypalusa/controllers/front/validation.php).  This is what receives the IPN from Paypal.

 

You need to add some control statements/debug statements to this file to confirm the following

1) That the IPN is being received.  It probably is, but confirm that.  You can check your access logs, but what I do is use the php function called error_log.  You would create a text file in the folder and just use the error_log function like so

error_log('message', 3, '/path/to/log_file.log');

2) Now assuming you are getting the IPN, then you need to add additional error_log statements throughout the code until you find where the code is stopping.  Since the IPN is a server to server transaction, you can't do this via a browser, this is why you need to use error_log.

 

3) Now the hard part is deciding what to do once you find the issue.  Feel free to do steps 1 and 2 above, and then report back where in the code it stops processing.

[ErickQC]

Hello bellini13, 

 

Thank you so much for the recommendation. I don't know PHP very well but I tried to do it (without much success). I created a file called log_file.log in the path below and added the following line to the code after it checks if the payment had been received: 

 

error_log(Tools::strtoupper(Tools::getValue('payment_status')), 3, 'http://mydomainxxx.com/modules/paypalusa/controllers/front/log/log_file.log');

 

That part of the code looked now like:

 

/* Step 4 - Determine the order status in accordance with the response from PayPal */

 

 

if (Tools::strtoupper(Tools::getValue('payment_status')) == 'COMPLETED')

$order_status = (int)Configuration::get('PS_OS_PAYMENT');

error_log(Tools::strtoupper(Tools::getValue('payment_status')), 3, 'http://mydomainxxx.com/modules/paypalusa/controllers/front/log/log_file.log');

elseif (Tools::strtoupper(Tools::getValue('payment_status')) == 'PENDING')

$order_status = (int)Configuration::get('PS_OS_PAYPAL');

elseif (Tools::strtoupper(Tools::getValue('payment_status')) == 'REFUNDED')

$order_status = (int)Configuration::get('PS_OS_REFUND');

else

$order_status = (int)Configuration::get('PS_OS_ERROR');

 

 

The result was that the orders stopped being created in all cases and nothing was logged in the file. I am sure is my poor code, but I think I understood your point and tried to find out in a different way whether Prestashop is receiving the message. I found that for each message sent, PayPal records a HTTP response code. When my store is in maintenance and PayPal cannot reach it, for example, the response code is 500 and the status of the message is "Retrying". However, when the store is not in maintenance and I do a transaction, the code is 200 and apprears as "Sent" no matter if the order was created or no in Prestashop. Could this mean that for sure Prestashop is receiving the message (although is not processing them as it should)? If not, I would be glad to try again the method you proposed, although I would appreciate an exemple if possible that I can basically use as is, as much as possible.

 

In the meantime, and under the assumption that Prestashop is receiving both messages successfully, and therefore replying with a code "200" in both cases, I analysed the message that is being sent for both cases accordingly to the PayPal logs. I attach my findings.

 

In a nutshell, I see that the message is substantially different but "payment_status" is sent with the same value and the validation.php program seems to use that to define what to do. 

 

This makes me think that it is probably other kind of validation done by prestashop elsewhere and not in the PayPal module. In the Orders module, for exemple. Is that possible? What are your thoughts?

 

Thank you very much!

 

ErickQC

 

See also attached table.

 

 

Message when order is successfully created. 

 

HTTP REPONSE CODE : 200

 

 IPN MESSAGE:

 

 mc_gross=0.09&

 protection_eligibility=Ineligible&

 item_number1=&

 payer_id=SC4D9YSHWYXPG&

 tax=0.00&

 payment_date=20:33:21 Sep 02, 2014 PDT&

 payment_status=Completed&

 charset=windows-1252&

 mc_shipping=0.00&

 mc_handling=0.00&

 first_name=John&

 mc_fee=0.09&

 notify_version=3.8&

 custom=90;1&

 payer_status=verified&

 [email protected]&

 num_cart_items=1&mc_handling1=0.00&

 verify_sign=AmydMwaMHzmxBimnFMnKy3o9n-ElAWzNqmijoANGyvF1JYvhtgRpo4xW&

 [email protected]&

 mc_shipping1=0.00&tax1=0.00&

 txn_id=8FL572676K5097648&

 payment_type=instant&

 last_name=Buyer&

 item_name1=Cuisse de poulet creme de pleurotes du Quebec  sa poire rotie aux epices  datte et noisette&

 [email protected]&

 payment_fee=&quantity1=3&

 receiver_id=E4M89M976NY4G&

 txn_type=cart&

 mc_gross_1=0.09&

 mc_currency=CAD&

 residence_country=CA&

 test_ipn=1&

 transaction_subject=90;1&

 payment_gross=&ipn_track_id=2b758837e608

 

 

 

 Message when order is not created. 

 

HTTP REPONSE CODE : 200

 

IPN MESSAGE:

 

mc_gross=0.09&

protection_eligibility=Eligible&

address_status=confirmed&

item_number1=&

payer_id=UMT55AKPA5Q7N&tax=0.00&

address_street=CanadianAddress&

payment_date=11:57:44 Aug 29, 2014 PDT&

payment_status=Completed&

charset=windows-1252&

address_zip=CandianZIPCODE&

mc_shipping=0.00&

mc_handling=0.00&

first_name=xxx&

mc_fee=0.09&

address_country_code=CA&

address_name=xxxx Prueba&

notify_version=3.8&custom=70;1&

payer_status=unverified&

[email protected]&

address_country=Canada&

num_cart_items=1&

mc_handling1=0.00&

address_city=CanadianCity&

verify_sign=AOh0tu.5JUQyG2Aao4MpntBA2sFjAnrNoTrN5FzSckNxLHbN8WSVTGGD&

[email protected]&

mc_shipping1=0.00&

tax1=0.00&

txn_id=236678920Y5515206&

payment_type=instant&last_name=Prueba&

address_state=Quebec&

item_name1=Cuisse de poulet creme de pleurotes du Quebec  sa poire rotie aux epices  datte et noisette&

[email protected]&

payment_fee=&

quantity1=3&

receiver_id=E4M89M976NY4G&txn_type=cart&

mc_gross_1=0.09&

mc_currency=CAD&

residence_country=CA&test_ipn=1&

receipt_id=5012-3495-7153-5737&

transaction_subject=70;1&

payment_gross=&

ipn_track_id=274e98d8daadb 

Edited September 3, 2014 by ErickQC (see edit history)

[bellini13]

You should use a path on the filesystem, not an HTTP URL

error_log(Tools::strtoupper(Tools::getValue('payment_status')), 3, 'http://mydomainxxx.c...og/log_file.log');

So it should look something like this...

error_log(Tools::strtoupper(Tools::getValue('payment_status')), 3, dirname(__FILE__).'/log_file.log');

This will automatically create the log_file.log in the same directory as the validation.php

 

You cannot test in maintenance mode, since you are basically telling Prestashop to ignore all requests it receives.  You would have to add Paypals IPN server IP addresses to bypass on the maintenance page, and even then sometimes it does not work properly.

 

Now the first thing that the validation does, is post back to the paypal server to get the payment status.  And only if the returned status is 'VERIFIED' will it move forward. 

if ($response == 'VERIFIED')

So you should confirm this by recording the $response value into the log file.

[ErickQC]

Hello bellini13, 

 

Thanks for the code. I tried it but the validation stops working. The behaviour in Prestashop is in all cases failing to create the order (no matter what address I choose) and no logs are created. I checked on the Paypal side and those transactions show during several minutes a "Retrying" status instead of "Sent". The HTTP response code is 500, as if the store was unavailable or under maintenance. (However, every time I have been testing, including this time, the store is out of maintenance mode).

 

I trippled checked for typos and even used your code in little script to test it by "echo"ing the __FILE__, with no problems at all. I also put the log_file.log file in 666 to ensure it could be written to.

 

After I erased my modified version of validation.php and restored the original, all pending transactions started to go through in PayPal and show the usual "Sent", except those that exceeded the 8 retries and show "Failed".

 

I am wondering if somehow, paypal is able to detect that the module was modified and the process stops for security reasons. You mentioned that maybe I could see in the logs whether the messages were arriving. Could you provide some light on where to look or to propose another alternative please?

 

Thank you very much,

 

ErickQC 

[bellini13]

So you likely introduced a syntax issue when you modified the validation.php file.  And yes, Paypal does have logic so that IPN's are stopped after several have failed.  It resumes once IPN's start processing normally.

 

So I'm not sure what you have changed to introduce the issue, but this is up to you.  Either you need to debug using proper php coding, or perhaps you should hire a developer to do this for you

[ErickQC]

Thanks for trying.

[joecoolio66]

We've been having this same issue since we upgraded to Prestashop 1.6 and used the USA/Canada/Mexico module.  Still waiting for a fix, but in the meantime if you use PayPal 3.7.2 instead of 1.3.8 there are no problems. 

[ErickQC]

Hello joecoolio66, 

 

Thank you so much for sharing your solution. I have been trying to find the 3.7.2 version. Just to be sure, is this the PayPal Europe module? I will be operating in Canadian Dollars. So, in your experience it also works with USD, CAN or MXPs?

 

Thank you!

 

Erick