During a routine failover between Spreedly’s active and passive Riak clusters, an error occurred which turned off realtime replication between the two data stores. This meant that data being written to one cluster was not being synced to the other in real time. Because of this discrepancy, some customers received a
not found error when attempted to access stored info.
On 11/23, Spreedly deployed a patch that allowed customers to receive data that had been temporarily unreachable after the failover.
Spreedly will complete an internal review of documentation and processes that led to the realtime replication failure between clusters. Spreedly has also built tooling that will allow for real-time patching in the future should a similar incident occur.