911 call center’s outage caused by ‘almost a perfect storm’

It was failure upon failure, four times over, that shut down Spokane County’s 911 system for nearly a half-hour in November.

Documents obtained through a public records request show that 130 calls were missed during the 911 outage Nov. 26.

“It was almost a perfect storm,” said Lorlee Mizell, the county’s 911 emergency communications director.

The saving grace – after four portions of the system failed – was the county’s backup 911 center north of city limits. Soon after operators got on site, they began taking calls again.

Officials say they never received a complaint from anyone who couldn’t get through and are unaware of anyone harmed by the inability to call for help. CenturyLink, which has the contract to maintain the county’s 911 equipment, said while they could count the number of missed calls, they were unable to recover phone numbers.

It took 30 minutes from the time the county’s main emergency communication center’s phone system lost dial tone at 6:34 p.m. until service was restored, according to county records. That conflicts slightly with CenturyLink’s data, which shows that the system was down for 23 minutes.

Spokane County’s 911 system also experienced outages in 2008 and 2010, Mizell said. At least one of those times, operators also were sent to the backup center to restore service.

Emergency communications experts say the industry goal is for 911 systems to be operational 99.999 percent of the time. That’s all but six minutes a year.

“It is rare for a system that is designed to have that many redundancies to have an outage at all,” said Trey Forgety, the government affairs director for the National Emergency Number Association.

But, he added, thanks to having a backup center, a longer outage was avoided.

“It sounds like Spokane just experienced a very rare event and handled it gracefully,” he said. “Obviously, you never want to rely on your fourth backup, but that’s why it’s there, to handle that very rare event.”

The failure started when one system that handles 911 calls shut down and a piece of equipment that was supposed to automatically switch the calls to a backup system didn’t work. A second link also failed to transfer calls to the backup system. Emergency communications workers then tried to activate another backup system, but that effort also failed.

Operators worked out of the backup center for 58 hours until systems were restored at the main communications center.

In a letter to CenturyLink officials dated Dec. 3, Mizell called the length of the outage and the time needed to fix it “unacceptable.” CenturyLink responded with answers to more than 20 questions she asked about the outage.

In an interview last week, Mizell said despite frustration in the aftermath of the outage, the phone company has generally provided good service to the county’s 911 system.

“It’s our intent to have the equipment we provide working 100 percent of the time,” said Kerry Zimmer, CenturyLink spokeswoman. “Citizen safety is our main concern.”