1. April 7, 2015 Washington, D.C. Area Low-Voltage Event Operations Training Seminar 2016 Erik Johnson System Operations Training (ERCOT)

2. PUBLIC Objectives 1.Identify the cause of the April 7, 2015 Washington, D.C. low voltage event. 2.Recognize how a local breaker failure scheme works. 3.Recognize the reason for delayed load restoration in an area-wide low voltage event. 4.Identify the three recommendations to include for periodic functional testing of protection circuits from the NERC Lessons Learned report. 5.Identify the program and documentation requirements for Transmission and Generation Protection System Maintenance and Testing per PRC-005-1.1b.

3. PUBLIC Objective 1 Identify the cause of the April 7, 2015 Washington, D.C. low voltage event. 3

4. PUBLIC Initial conditions Pre-Disturbance voltage: Normal (1.00-1.05 pu) Weather forecast: Clear and Temperate System abnormalities: Scheduled maintenance on several circuits, transformers, and buses. Initiating event: Failed 230 kV surge arrester creates a single-phase- to-ground fault. Washington, D.C. Low Voltage NERC Report

5. PUBLIC Protection scheme fails Protection scheme initially works as designed to isolate the fault, however… A breaker fails to re-open after reclosing to check the fault’s persistence, thereby re-energizing the fault. The re-energized fault evolves to two-phase-to- ground, and eventually to three-phase-to-ground. The sustained three-phase fault migrates to an adjacent 230 kV circuit, which relays out of service. Washington, D.C. Low Voltage NERC Report

6. PUBLIC Evolution of the event ~2,000 MW of generation trips offline, further depressing voltage area- wide. Time-delayed ground overcurrent protection on the 500 kV system eventually isolates the entire substation with the stuck breaker 58 seconds after the initiating event. 532 MW of load trips due to the low voltage, including many government and customer facilities in the Washington, D.C. metropolitan area. Washington, D.C. Low Voltage NERC Report

7. PUBLIC Subsequent damage C – phase arrester base pitting B – phase arrester base

8. PUBLIC Why did the fault persist for so long? The failure of two independent protection systems prevented the stuck breaker from re- opening and did not trigger the local breaker failure scheme. Relay system No. 1 failed due to a loose connection between the auxiliary relay coil cutoff trip and the breaker contact string. Relay system No. 2 failed due to an intermittent electrical discontinuity in the auxiliary relay coil circuit, which occurred when the breaker reclosed into the fault.

9. PUBLIC Breaker-and-a-Half Scheme – Normal Aux. Relay Operation

10. PUBLIC Breaker-and-a-Half Scheme – Normal Aux. Relay Operation

11. PUBLIC Breaker-and-a-Half Scheme – Open Circuit in Aux. Relay Auxiliary Relay contacts never energize. Backup trip signal and BFI are never sent.

12. PUBLIC Original design Series configuration 52a contacts redundancy

13. PUBLIC Recommended redesign Parallel configuration52a contacts removed

14. PUBLIC Objective 2 Recognize how a local breaker failure scheme works. 14

15. PUBLIC Local Breaker Failure Protection Scheme Local Breaker Failure Protection Description: In the basic scheme for breaker failure protection, as soon as the relay issues a trip to its breaker, a breaker failure timer is started. If the fault still persists after the timer times out, then a breaker failure condition is declared, and the breakers [required to de-energize the one that failed to open] are tripped. Guide For Breaker Failure Protection – Published, by Roger Hedding, ABB, on www.pacw.org

16. PUBLIC Local Breaker Failure Protection Scheme Local Breaker Failure Protection Description: Breaker Failure Initiate (BFI) is the signal coming from the primary protection to trip the breaker. An overcurrent fault detector (50BF) is employed to determine if the fault is still present. 50BF will drop out if the breaker clears the fault. tttttttttt Guide For Breaker Failure Protection – Published, by Roger Hedding, ABB, on www.pacw.org

17. PUBLIC Local Breaker Failure Protection Scheme Local Breaker Failure Protection Description: Breaker Failure Initiate (BFI) is the signal coming from the primary protection to trip the breaker. An overcurrent fault detector (50BF) is employed to determine if the fault is still present. 50BF will drop out if the breaker clears the fault. tttttttttt Guide For Breaker Failure Protection – Published, by Roger Hedding, ABB, on www.pacw.org

18. PUBLIC Generic System One-Line GGG WW GG 345 kV 230 kV 138 kV

19. PUBLIC Example #1 Single Line Fault & Normal System Protection Response

20. PUBLIC Initial Fault GGG WW GG 345 kV 230 kV 138 kV X 230 kV

21. PUBLIC Isolation – Expected Response GGG WW GG 345 kV 230 kV 138 kV X 230 kV

22. PUBLIC Isolation – Expected Response GGG WW GG 345 kV 230 kV 138 kV X 230 kV

23. PUBLIC Example #2 Single Line Fault & System Protection Response with Stuck Breaker

24. PUBLIC Initial Fault GGG WW GG 345 kV 230 kV 138 kV X 230 kV

25. PUBLIC Isolation GGG WW GG 345 kV 230 kV 138 kV X 230 kV

26. PUBLIC Isolation GGG WW GG 345 kV 230 kV 138 kV X 230 kV

27. PUBLIC Isolation – Stuck Breaker GGG WW GG 345 kV 230 kV 138 kV X 230 kV

28. PUBLIC Isolation GGG WW GG 345 kV 230 kV 138 kV X – Local Breaker Failure Protection Scheme 230 kV

29. PUBLIC Isolation GGG WW GG 345 kV 230 kV 138 kV X – Local Breaker Failure Protection Scheme 230 kV

30. PUBLIC Example #3 Single Line Fault & Stuck Breaker, Untriggered Local Breaker Failure, Fault Migration & Sustained Fault Energization

31. PUBLIC Initial Fault GGG WW GG 345 kV 230 kV 138 kV X 230 kV

32. PUBLIC Isolation GGG WW GG 345 kV 230 kV 138 kV X 230 kV

33. PUBLIC Isolation GGG WW GG 345 kV 230 kV 138 kV X 230 kV

34. PUBLIC Untriggered Local Breaker Failure Scheme GGG WW GG 345 kV 230 kV 138 kV X 230 kV

35. PUBLIC Untriggered Local Breaker Failure Scheme GGG WW GG 345 kV 230 kV 138 kV X 230 kV

36. PUBLIC GGG WW GG 345 kV 230 kV 138 kV X X 230 kV Sustained Energization and Fault Migration

37. PUBLIC GGG WW GG 345 kV 230 kV 138 kV X X 230 kV Second Fault Clears

38. PUBLIC Second Fault Clears GGG WW GG 345 kV 230 kV 138 kV X X 230 kV

39. PUBLIC Sustained Energization and Voltage Depression GGG WW GG 345 kV 230 kV 138 kV X X 230 kV

40. PUBLIC Sustained Energization and Voltage Depression GGG WW GG 311 kV 207 kV 124 kV X X 173 kV

41. PUBLIC Sustained Energization and Voltage Depression GGG WW GG 311 kV 207 kV 124 kV X X (.75 pu) (.90 pu) 173 kV

42. PUBLIC Offline Units’ Output Breakers Trip on Phase Imbalance GGG WW GG X X (.75 pu) 173 kV 311 kV (.90 pu) 207 kV (.90 pu) 124 kV (.90 pu)

43. PUBLIC Offline Units’ Output Breakers Trip on Phase Imbalance GGG WW GG X X (.75 pu) 173 kV 311 kV (.90 pu) 207 kV (.90 pu) 124 kV (.90 pu)

44. PUBLIC Online Units Trip on Auxiliary Bus Low Voltage Protection GGG WW GG X X (.75 pu) 173 kV 311 kV (.90 pu) 207 kV (.90 pu) 124 kV (.90 pu)

45. PUBLIC Online Units Trip on Auxiliary Bus Low Voltage Protection GGG WW GG X X (.75 pu) 173 kV 293 kV (.85 pu) 196 kV (.85 pu) 117 kV (.85 pu)

46. PUBLIC Output Breakers Trip Immediately After Units Trip GGG WW GG X X (.75 pu) 173 kV 293 kV (.85 pu) 196 kV (.85 pu) 117 kV (.85 pu)

47. PUBLIC Output Breakers Trip Immediately After Units Trip GGG WW GG X X 0 kV (.75 pu) 173 kV 293 kV (.85 pu) 196 kV (.85 pu)

48. PUBLIC High Voltage Bus Isolates on Timed Overcurrent Relay GGG WW GG X X 0 kV (.75 pu) 173 kV 293 kV (.85 pu) 196 kV (.85 pu)

49. PUBLIC High Voltage Bus Isolates on Timed Overcurrent Relay GGG WW GG 0 kV X X

50. PUBLIC Objective 3 Recognize the reason for delayed load restoration in an area-wide low voltage event. 50

51. PUBLIC Load Impact This disturbance impacted approximately 532 MW of load, 445 MW of which was native Pepco load, and 87 MW of which was SMECO load. Numerous federal, state, and local government facilities and commercial customers were impacted. Following the disturbance, PJM, Pepco, and SMECO executed a deliberate and orderly process to evaluate system conditions and restore the system to a more reliable state. This process included dispatching field personnel to involved substations to inspect facilities and verify the condition of equipment prior to re ‐ energization. This event primarily impacted Pepco customers at commercial facilities and government buildings whose customer ‐ owner internal electrical protective equipment tripped, by design, in response to the severe low ‐ voltage conditions. Washington, D.C., Area Low Voltage Disturbance Event of April 7, 2015 (pgs. 13-14)

52. PUBLIC Load Impact During a protracted low ‐ voltage condition, building electrical systems will generally transfer to their backup systems where such systems are installed and functional. In order to restore service to its normal configuration, each individual facility had to transfer from its backup system to Pepco’s electric distribution system. In some cases, this is done automatically and was performed within minutes of the event. In other instances, transferring load back to Pepco’s electric distribution system required a manual process. Washington, D.C., Area Low Voltage Disturbance Event of April 7, 2015 (pgs. 13-14)

53. PUBLIC Load Impact Although manual transfers are not complicated, they are dependent on trained and authorized electricians or electrical service personnel being available to respond and perform the necessary switching operations on the customer equipment. The location and activity of customer facility electricians at the time of the disturbance affected the speed of manual restoration switching, which could have extended the outage for those customers while customer facility electricians responded. Washington, D.C., Area Low Voltage Disturbance Event of April 7, 2015 (pgs. 13-14)

54. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = 0 min Net load = -532.0 MW A total of 532 MW of load trips from low voltage protection on the distribution system.

55. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +5 min Net load = -532.0 MW 75 MW of load is restored through automatic transfer back to Pepco’s system.

56. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +5 min Net load = -457.0 MW 75 MW of load is restored through automatic transfer back to Pepco’s system.

57. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +20 min Net load = -457.0 MW 53.5 MW of SMECO load is restored through SCADA switching.

58. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +20 min Net load = -403.5 MW 53.5 MW of SMECO load is restored through SCADA switching.

59. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +42 min Net load = -403.5 MW SMECO restores an additional 32.1 MW of load through SCADA switching.

60. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +42 min Net load = -371.4 MW SMECO restores an additional 32.1 MW of load through SCADA switching.

61. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +106 min Net load = -371.4 MW Pepco restores an additional 300 MW of load through manual switching at customer and government vaults.

62. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t = +106 min Net load = -71.4 MW Pepco restores an additional 300 MW of load through manual switching at customer and government vaults.

63. PUBLIC 12:39 – 532 MW total load loss from the grid 12:44 – 75 MW of Pepco load returned 12:59 – SMECO restores 53.5 MW of load 13:21– SMECO restores 32.1 MW of load 13:00-13:25 – 300 MW of Pepco load returned Load Restoration Sequence of Events t > +106 min Net load = -71.4 MW The remaining Pepco load does not return to the system. This was most likely due to organizational decisions to send people home after the power outage.

64. PUBLIC A Quick Recap – What happened? A single piece of equipment failed, creating a faulted line condition. A breaker did not re-open after reclosing on the faulted line due to a failure of two separate and independent auxiliary relays. An area-wide voltage depression occurred for 58 seconds, triggering low- voltage protection schemes on customer and government vaults. Due to the requirement for manual switching at various locations, a delay in restoration was inevitable, impacting area commerce and government functions.

65. PUBLIC A Quick Recap – What could have been different? How could this have been avoided? –Better operator response? –Better inspection of substation equipment? –Better relay testing and maintenance? –Better load restoration process? –All of these? –None of these?

66. PUBLIC Objective 4 Identify the three recommendations to include for periodic functional testing of protection circuits from the NERC Lessons Learned report. 66

67. PUBLIC Protection System Functional Testing Lessons Learned Primary Interest Groups Transmission Owners (TOs), Transmission Operators (TOPs) 1.If functional tests fail to detect open circuits or other defects in any portion of the protection circuit, these defects could prevent either the tripping of the required local remote breakers (as applicable) or the initiation of breaker failure. 2.Verification that primary elements of the protection scheme generate trip output(s), as applicable, to the associated trip auxiliary relay, directly to the breakers, or both. 3.Verification that the required breakers do, in fact, trip as a result of a trip output(s). This verification should be performed during the functional testing if possible.

68. PUBLIC Objective 5 Identify the program and documentation requirements for Transmission and Generation Protection System Maintenance and Testing per PRC-005-1.1b. 68

69. PUBLIC NERC Standard PRC-005-1.1b Requirements Applicable Parties: Transmission Owner, Distribution Provider, Generator Owner R1. Each Transmission Owner and any Distribution Provider that owns a transmission Protection System and each Generator Owner that owns a generation or generator interconnection Facility Protection System shall have a Protection System maintenance and testing program for Protection Systems that affect the reliability of the BES. The program shall include: R1.1. Maintenance and testing intervals and their basis. R1.2. Summary of maintenance and testing procedures.

70. PUBLIC NERC Standard PRC-005-1.1b Requirements Applicable Parties: Transmission Owner, Distribution Provider, Generator Owner R2. Each Transmission Owner and any Distribution Provider that owns a transmission Protection System and each Generator Owner that owns a generation or generator interconnection Facility Protection System shall provide documentation of its Protection System maintenance and testing program and the implementation of that program to its Regional Entity on request (within 30 calendar days). The documentation of the program implementation shall include: R2.1. Evidence Protection System devices were maintained and tested within the defined intervals. R2.2. Date each Protection System device was last tested/maintained.

71. PUBLIC Review Objectives 1.Identify the cause of the April 7, 2015 Washington, D.C. low voltage event. 2.Recognize how a local breaker failure scheme works. 3.Recognize the reason for delayed load restoration in an area-wide low voltage event. 4.Identify the three recommendations to include for periodic functional testing of protection circuits from the NERC Lessons Learned report. 5.Identify the program and documentation requirements for Transmission and Generation Protection System Maintenance and Testing per PRC-005-1.1b. Questions? Thank you!