During a scheduled language version upgrade of the Spreedly transaction processing service, Apple Pay payment methods could not be created. The error message was “The encrypted portion of the payment data could not be decrypted, most likely due to other validation errors.”
On February 5, 2020 at 12:02pm EST, Spreedly Engineers released a new version of the application, which included a major programming language version update. This programming language update included changes to the way decryption parameters must be specified. The Apple Pay decryption library had not been updated to use this new parameter. Automated and manual testing did not catch the issue, because both types of tests used data that did not exercise the actual decryption path. In the production environment, this caused decryption of all Apple Pay
payment_data tokens to fail, which occurs when creating payment methods.
Spreedly Engineers determined there was a systemic issue with Apple Pay payment methods, and began investigating at 5:50pm EST. Spreedly Engineers updated the decryption library used for Apple Pay to work with the new language version, and deployed a fix at 7:13pm EST.
We have completed the following activities to prevent this situation from happening in the future:
We will evaluate the following activities to prevent this situation from happening in the future:
We apologize for any disruption this incident may have caused and are taking steps to ensure that our systems continue to be resilient for our customers.