Things look calm, all alarms have resolved on our end. We are going to close this out and continue our march towards Python 3 tomorrow... thanks for your patience, and our apologies for the interruptions!
Posted 2 months ago. Jul 10, 2019 - 22:19 PDT
We've deployed fixes to the relevant systems and are monitoring. At this point, we believe everything is resolved and will not recur.
Posted 2 months ago. Jul 10, 2019 - 22:03 PDT
We have rolled back a problematic change to our workers (machines that run periodic/on-demand tasks) that were getting duplicated and causing stress to our underlying databases. We are still working through some downstream problems this has caused in our search system.
Posted 2 months ago. Jul 10, 2019 - 21:46 PDT
We are having some intermittent issues with sending messages and logging in. We know what the cause is and we're working to remediate the issue now.