FAA outage reveals lack of redundancy

WASHINGTON – When a computer glitch at a Federal Aviation Administration center caused widespread airline delays this week, it served as a reminder that the U.S. flight system is waiting for a modernizing overhaul. But it also appears the FAA’s management of its existing technologies falls short of standards in other vital sectors.
By using computing practices that would be considered poor in credit card networks or power plant operators, for example, the FAA was vulnerable to a problem caused when new software was loaded at the Atlanta center that distributes flight plans.
Because the FAA relies on just two computing systems, one in Atlanta and one in Salt Lake City, to handle that chore for the entire nation, the software glitch all but sank the system Tuesday. The Salt Lake center remained up and served as a backup, but it became overloaded by information coming from airlines. More than 600 flights were delayed from Atlanta all the way to Boston and Chicago.
A failure at the same Atlanta center caused major delays across the East Coast in June 2007.
Such breakdowns often can be prevented with sufficient redundancy, or enough different computers and communication channels to handle the same workload in an emergency.
Redundancy is so critical for power and water utilities that they can be fined hundreds of thousands of dollars a day if they’re found insufficiently prepared – and $1 million per day if they’re found to be willfully negligent.
“In the industries I work in, if you have something that critical, you generally build more redundancy,” said Jason Larsen, a security researcher with consultancy IOActive Inc. who previously spent five years at Idaho National Laboratory examining electrical plants’ control systems. “If this (FAA outage) happened at a power plant, I’d be telling them to open up their checkbook and expect to be fined.”
FAA spokeswoman Tammy Jones stressed that these types of problems “don’t happen on a mass scale or a regular basis,” and noted that the FAA handles 50,000 to 60,000 fights a day. And flying on U.S. airlines has never been safer.
“The system is working,” she said. “We are making sure people are getting from one place to another.”
Basil Barimo, vice president of operations and safety for the Air Transport Association of America, a trade association that represents the nation’s largest carriers, says the fundamental problem is that the FAA still relies on outdated technology, including a radar-based control system designed in the 1940s and ’50s. Barimo is optimistic that the FAA’s NextGen modernization program – a $15 billion-plus upgrade to satellite-based technology that will take nearly 20 years to complete – will help make more efficient use of the nation’s airspace and safely allow more planes in the sky.