Powered by Invision Power Board


  Reply to this topicStart new topicStart Poll

> Partial network outage, resolved and explained
andy
Posted: Jun 7 2019, 09:09 AM
Quote Post





Group: Advantagecom Staff
Posts: 4,340
Member No.: 9
Joined: 12-July 02



This morning between approximately 6:40AM and 9:40AM (all times PDT, GMT -7:00) we experienced a partial network outage that affected many different machines on our network.

It was traced to a network switch connected to a failed rack power outlet.

Plugging the switch into a different power outlet and reloading the network stack resolved the issue.

Normally, we resolve these types of issues much more quickly, but we were unaware of the issue since our internal monitoring system was connected to the down switch, as was our phone system. Our external network monitoring did not see the problem because the system being monitored on our network from outside our network was up and reachable.

We became aware of the issue by a report from a customer to our off-hours support phone.

We'll be adding additional monitoring targets to our external network monitoring to be sure we catch such issues in an automated fashion in the future.


--------------------
Sincerely,
Andrew Kinney
CTO, Advantagecom Networks

Please do not private message me. My regular management duties preclude responding to every customer that sends me a support issue. Instead, post on the forum or contact tech support.
PMUsers Website
Top
0 User(s) are reading this topic (0 Guests and 0 Anonymous Users)
0 Members:

Topic Options Reply to this topicStart new topicStart Poll