At 4:19am PDT, Down alerts for applications hosted at the datacenter began firing. It was quickly determined the datacenter lost internet. A few hours later the connection was restored. While the connection was offline all the sensors were tagged as offline and notifications where queue up. When connectivity was restored the backlog of notifications were sent.
We have take steps to track when data is not longer being injected and preventing offline notifications from being sent. All sensor data sent during the outage was lost. We are considering buffering unsent data on the bridge to avoid dropping data during future outages.