The cluster expansion has resolved the issues. Everything looks stable now. The root cause was just our excellent, continued growth. We hit some limitations on one of our clusters that we knew was coming, so we expanded it and things look good now. Thanks for your patience!
Posted Mar 04, 2018 - 15:08 PST
Monitoring
We've finished the cluster expansion and are monitoring performance of the system at this time. Things should be recovering, but it will take a few minutes for everything to stabilize.
Posted Mar 04, 2018 - 14:05 PST
Update
We're attempting to expand the cluster that handles these operations. This is a disruptive update: we expect a small interruption in presence updates, direct messages, and push notifications.
Posted Mar 04, 2018 - 13:56 PST
Identified
We are aware of some ongoing issues that some users are seeing with viewing the online status for your friends, loading direct messages, and push notifications. These issues seem to be coming in bursts and are affecting a subset of users. We believe this is related to some capacity issues with one of our systems and we're looking for potential mitigations.