Early Alert: Potential Customer-Impacting Issue Detected
Incident
Report for Spreedly
Postmortem
March 15th, 2025 — Payment Method Storage Errors
On March 15th, 2025 at 06:12 UTC Spreedly systems encountered a system resource constraint which led to intermittent API call failures resulting in 404 and 500 errors being returned to customers. The unresponsive application containers were restarted allowing the system to return to normal at 06:47 UTC
What Happened
On 2025-03-15 between 06:12 and 06:47 UTC, calls to Spreedly’s payment method tokenization endpoints intermittently failed resulting in HTTP 500 errors. Intermittent elevated rates of HTTP 404 errors when attempting to use a payment method to transact were also encountered. These failures were the result of internal system resource limitations.
Next Steps
- Spreedly has implemented additional resource monitoring in our automated alerting system to prevent future overruns.
- System resources have been adjusted to prevent similar situations in the future.
Resolved
IMPACT STARTED AT:
2:12 AM EST
IMPACT ENDED AT:
2:40 AM EST
After closely monitoring Spreedly Core Services and confirming that all systems are stabilized and functioning as expected, this incident is considered resolved. No further customer impact is expected.
We are completing our investigation concerning the causes of the incident and any residual impact. A post-mortem will be published.
We apologize for any inconvenience or disruption.
2:12 AM EST
IMPACT ENDED AT:
2:40 AM EST
After closely monitoring Spreedly Core Services and confirming that all systems are stabilized and functioning as expected, this incident is considered resolved. No further customer impact is expected.
We are completing our investigation concerning the causes of the incident and any residual impact. A post-mortem will be published.
We apologize for any inconvenience or disruption.
Update
As of 3:10 AM EST, our system availability has been fully restored and we are seeing stability return. We have implemented a fix and stabilized Spreedly Core Services. Our team is closely monitoring the situation to ensure no further impact.
Monitoring
As of 3:10 AM EST, our system availability has been fully restored and we are seeing stability return. We have implemented a fix and stabilized Spreedly Core Services. Our team is closely monitoring the situation to ensure no further impact.
Investigating
We are investigating an elevated rate of 5xx errors affecting Spreedly core services starting at 01:36AM EST. While we are providing an early notification in an effort to alert you as quickly as possible, we are still investigating the actual scope and impact and will provide an update as soon as more details are available.
Our team is actively working to identify the root cause and mitigate the impact. We will provide further updates as soon as we have more information.
Thank you for your patience.
Our team is actively working to identify the root cause and mitigate the impact. We will provide further updates as soon as we have more information.
Thank you for your patience.
This incident affected: Core Transactional API.