Spreedly API Errors
Incident Report for Spreedly
Postmortem

July 21, 2021 — Spreedly API Errors: ID 503 Service Unavailable

On July 21 2021, a set of infrastructure configuration updates produced unexpected results from internal testing. This impacted our internal authentication application to become unavailable (client requests were responded with a 503 Service Unavailable error). All Spreedly APIs and services that rely on the authentication service were also impacted.

Spreedly engineers corrected the automated configuration deployment errors manually. This incident lasted for ~18 minutes from outage to implementing a fix.

Next Steps

Spreedly is investigating approaches to improve its infrastructure configuration testing and deployment processes.

Posted Jul 30, 2021 - 11:16 EDT

Resolved
After deploying the fix, all systems appear to be stabilized and functioning. The incident is being considered resolved.

We are still investigating to understand the specific causes of the incident and any residual impact. A post incident review will be published with further details.

We apologize for any inconvenience and disruption to service.
Posted Jul 21, 2021 - 17:07 EDT
Update
We are continuing to monitor for further issues.
Posted Jul 21, 2021 - 16:56 EDT
Monitoring
A fix has been implemented addressing the API errors.

We are actively monitoring the results.
Posted Jul 21, 2021 - 16:21 EDT
Investigating
We have identified an issue causing 500 errors on Spreedly's Core API.

This is impacting all transactions and requests to Spreedly's API.

Updates will be provided as they become available.
Posted Jul 21, 2021 - 16:09 EDT
This incident affected: Core Transactional API and Supporting Services (Dashboard).