1. Review of Power Blackout on Telecom P. J
Review of Power Blackout on Telecom P. J. Aduskevicz, NRSC Kathryn Condello, CTIA Capt. Katharine Burton, NCS/NCC

2. Power Related Outage Data
Voluntary Outage Reporting Trial Data
January – July 2003 Power Related Outage Reports
7 of Final Scrubbed Outage Reports
24% of the Total Final Scrubbed Outage Reports
Examples of Causes Included:
Back-up Generator Failure, Failed Equipment Power Supplies, and Configuration of Uninterruptible Power Supplies
Examples of Applicable Best Practices Included:
on Routine Exercising of Engines and  on Planning for Providing Portable Generators
Outage Reports Associated with the August 14-15, 2003 Power Failure
4 Initial Outage Reports received by the NCS/NCC
Mandatory Outage Reporting Data
8 Initial Outage Reports received by the FCC
2 Final Outage Reports received by the FCC

3. ~250 Cyber Security + Disaster Recovery + Public Safety
Quarterly Meeting
August 27, 2003
Review of Effectiveness of NRIC Power Best Practices
Summary Statistics
~750 NRIC Best Practices
~250 Network Reliability
~250 Physical Security
~250 Cyber Security + Disaster Recovery + Public Safety
98 (13%) with “Power” keyword association
72 from NRIC V or earlier
26 new or edited in NRIC VI

4. California Rolling Blackouts Finding for Attention
Quarterly Meeting
August 27, 2003
August 2003 Power Outage Event
History
NRSC Power Studies
August 2002
2001
1996
California Rolling Blackouts
Finding for Attention
Less power expertise identified in NRIC VI December 2002 Report as “Area for Attention”

5. Quarterly Meeting
August 27, 2003
Review of Effectiveness of NRIC Power Best Practices
Possible areas for Best Practices
Evaluate fuel contractors ability to pump fuel (reports that some pumps at refueling locations did not have back-up power (work into contracts)
Assess risk for need to enhancing back-up power at cell towers, cable networks, CLECs, others (implement existing BPs; consider fuel cell technology)
Consider outreach of sharing best practices to electric power sector
Pre-established plans for coordination between telecommunications service providers and city, state, and local emergency services
Consumer education on the benefits of having a corded phone powered from a Central Office
Overall: the traditional wired telecommunications performed well

6. Wireless Performance During Outage
The power outage impacted numerous wireless markets.
Responding carriers operate nearly 100 switches / 18,500 sites in blackout area.
During first three hours, carriers report demand spikes of % over normal busy hour.
All Switches were operational throughout the incident.
More than 2/3 of the network was uninterrupted throughout the incident.
Networks were re-configured centrally to provide continued operation while power was replaced and/or augmented.
Backhaul outages (Landline T-1 and DS-3 circuit faults) have been identified as root cause for some service interruptions.
Restoration activities were coordinated through the Telecom-ISAC.
Regular Electrical-ISAC updates allowed strategic deployment of replacement batteries and generators to impacted sites.
77% of Sites were operational at 24 hour mark, and 97% at 36 hours.

7. 8/14
1530hrs
8/15
0330hrs
8/16
0730hrs

8. National Coordinating Center (NCC) & Telecom ISAC
Established and maintained a conference bridge for wireline and wireless partners for the duration of power outage
Coordinated problem resolution across telecommunications industry including generator and fueling strategies for New York and Detroit
Coordinated with the Electrical ISAC to identify areas impacted by the outage and provided updates on restoration efforts
Manned the ESF 2 position at FEMA HQ
CWIN used as a transport media to coordinate with Canada
SHARES set Operational Level 2 (emergency potential exists)
Stations active in the affected areas
TSP not required (circuits were not damaged)
GETS: 1,813 GETS calls into or out of affected areas
93% Success rate
GETS designed for 85% call completion success

9. Wireless Priority Service (WPS) Performance
Many cell towers had only 2-4 hours of backup battery power
48 WPS calls completed
12 in the Washington DC area
36 in New York/New Jersey
Operational measurement of WPS call attempts not yet implemented.
Only call completion information available, based on billing information
Cell tower access was the major cause of blockage for localized congestion
WPS IOC capability provides priority to cell tower only offering little improvement during periods of widespread congestion.
Unaware of current limitations, users from FRB New York, New York City government (NYPD), and GSA New York expressed concern over WPS performance.

10. Lessons Learned
Power management and restoral practices at the tactical level under review by carriers
Need to understand root cause of outage
Appropriate mitigating or prevention techniques can be developed addressing both cause and expected probability of this event occurring again.
May need modifications to the TESP program to mitigate additional risks
Need to NS/EP services across ALL wireless Carriers and ALL platforms.
WPS worked, but would have worked better if the NS/EP user could have had access to the top 5-6 carriers, both GSM and CDMA.

11. Lessons Learned
Frequent Electrical ISAC teleconference participation provided excellent situational awareness to the NCC Telecom-ISAC
Development of TESP program for cellular networks to address priority restoration of power for critical cellular communications facilities is needed
Additional carriers and WPS FOC capabilities will address widespread outage problem
Potential training issues on use and capabilities of WPS
Review WPS training to better education users on current and projected program capabilities