Intermittent message send failure and trouble connecting
Incident Report for Discord
Resolved
Things look calm, all alarms have resolved on our end. We are going to close this out and continue our march towards Python 3 tomorrow... thanks for your patience, and our apologies for the interruptions!
Posted 9 days ago. Jul 10, 2019 - 22:19 PDT
Monitoring
We've deployed fixes to the relevant systems and are monitoring. At this point, we believe everything is resolved and will not recur.
Posted 9 days ago. Jul 10, 2019 - 22:03 PDT
Update
We have rolled back a problematic change to our workers (machines that run periodic/on-demand tasks) that were getting duplicated and causing stress to our underlying databases. We are still working through some downstream problems this has caused in our search system.
Posted 9 days ago. Jul 10, 2019 - 21:46 PDT
Identified
We are having some intermittent issues with sending messages and logging in. We know what the cause is and we're working to remediate the issue now.
Posted 9 days ago. Jul 10, 2019 - 21:27 PDT