1. Integrated Reliability Measures Heide Caswell, Team Lead, Reliability Metrics Working Group (RMWG) Director of Network Performance, PacifiCorp Compliance and Certification Committee Meeting September 21-22, 2011 Atlanta, GA

2. 2RELIABILITY | ACCOUNTABILITY 2 Topics Conceptual Model Initial Developments Continuing Our Understanding of the Risk Universe Conceptual Indicators –Event Driven Index –Standards/Statute Driven Index –Condition Driven Index Continuing Dialogue with CCC Discussion and Feedback

3. 3RELIABILITY | ACCOUNTABILITY Integrated Reliability Indicators (IRI) Develop the conceptual model of reliability risks Establish quantitative measures for evaluating the performance of the indicators Support risk-informed decision making May be able to correlate over the long term to predictive measures of reliability risks and if so… Estimate the effectiveness of reliability risk reduction and/or mitigation

4. 4RELIABILITY | ACCOUNTABILITY Severity Risk Index (SRI)

5. 5RELIABILITY | ACCOUNTABILITY Key Concepts

6. 6RELIABILITY | ACCOUNTABILITY Integrated Risk Indicators Event Driven Index (EDI) Measures Risk from Major System Events Standards/Statute Driven Index (SDI) Measures Risks from Severe Impact Violations Condition Driven Index (CDI) Monitors Risk from Key Reliability Metrics Based on event severity risk index (SRI) values and turns into an availability index Uses the roster of reliability metrics developed by RMWG and approved by OC/PC Identify a subset of standards that have highest impact to reliability and create an index of standards violations

7. 7RELIABILITY | ACCOUNTABILITY Severity Risk Index (SRI) Focus on significant events and quantify the historic system performance using daily outage data –GADS and TADS daily outages, forced only –MW load loss and restoration duration from disturbance event reports –Could be considered similar to a daily, quarterly or yearly SAIDI metric for distribution systems –If everything was out of service, all load was unserved, the day’s score would be 1000 –If no lines or units were out of service and no load was lost the day’s score would be 0 –Normal “good days” will measure higher than 0

8. 8RELIABILITY | ACCOUNTABILITY Severity Risk Index (SRI)

9. 9RELIABILITY | ACCOUNTABILITY EDI Trends by Quarter

10. 10RELIABILITY | ACCOUNTABILITY Proposed Approach to Standard Driven Index (SDI)

11. 11RELIABILITY | ACCOUNTABILITY Proposed Approach for Standards/Statute Driven Index (SDI) Segment all Standards and Requirements by their Violation Risk Factors; Lower, Medium or High are the VRFs (as many as 1,427 standard requirements accounted for) Select only Standards and Requirements with High VRF (430 standard requirements relevant) Evaluate the range of possible violation severity levels (VSLs), which could range from Lower, Moderate, High to Severe (198 are severe VSL and high VRF) –VSLs are guidelines established jointly by the compliance and standard drafting team and are detailed as to criteria which identifies the level of severity. These levels are approved by the BoT. Consider Risk Impact Statements…select only those with Severe RIS There are 26 requirements with High VRFs and Severe Risk Impact Statements (see list of these requirements following) NERC used the almost four year history (June 2007-December 2010) to validate the proposed 26 standards requirements There is potential to assign to weight requirements by VRFs or VSLs if appropriate The Standard Applicability matrix enables us to derive a normalization factor (denominator) for all entities exposed to a requirement with a High VRF

12. 12RELIABILITY | ACCOUNTABILITY Uses High Violation Risk Factor (VRF) (Approved standard requirements only) Severe Reliability Impact Statement (RIS) –RIS indicates significance of impact on BPS Considers Violation Severity Level (VSL) Validated against almost 4 year history of violations for most significant standards risks to reliability Violation Risk Factor Violatio n Severity Level HighMediumLower Severe 198256162 High 9611292 Moderate 7710990 Lower 599680 Proposed Approach for Standards/Statute Driven Index (SDI)

13. 13RELIABILITY | ACCOUNTABILITY Violation Severity Level Violati on Risk Factor Risk Impact Statement Value? SevereHighModerateLower High Severe 251626 High Non-Severe 31216186384 Medium Severe 22524 Medium Non-Severe 397106105373 Lower Severe 11332 Lower Non-Severe 2186167306 Severe Reliability Impact Statement (RIS) –RIS indicates significance of impact on BPS Uses Violation Risk Factor (VRF) Demonstrates Violation Severity Level (VSL) Similar to daily SRI calculation Almost 4 year history violation history of 4,410 confirmed violations Proposed Approach for Standards/Statute Driven Index (SDI)

14. 14RELIABILITY | ACCOUNTABILITY * All 26 requirements have high Violation Risk Factor (VRF) and Severe Risk Impact Statements Proposed Approach for Standards/Statute Driven Index (SDI)

15. 15RELIABILITY | ACCOUNTABILITY SDI Calculation 15 SDI = (Total Compliance - ∑(w V *NV/NR))/(Total Compliance) Range from 0 to 100 Total Compliance = 1.0* Number of Days in a specific period w V = weighting of a particular high risk req. violation NV = number of violations for the selected high risk req. NR = number of registered entities required to comply with the high risk req.

16. 16RELIABILITY | ACCOUNTABILITY SDI Calculation Example SDI = (Total Compliance - ∑(w V *NV/NR))/(Total Compliance) SDI = 100*(1.0*Days in a Quarter - ∑(w V *NV/NR)) / (1.0*Days in a Quarter) 1Q2009: –40 violations (PRC-005 R1) –RIS = Severe, VRF=High, VSL=Severe –A total of 335 TOs, 828 GOs registered functions –SDI = 100*(1.0*90 - 0.038*(40)/(335+828))/(1.0*90) = 99.99

17. 17RELIABILITY | ACCOUNTABILITY Possible SDI Trend by Quarter

18. 18RELIABILITY | ACCOUNTABILITY Condition Driven Index (CDI) Began by narrowing down to key Condition Metrics Assumed CDI = 99.80 Have since moved toward measuring improvement in approved metrics published by RMWG

19. 19RELIABILITY | ACCOUNTABILITY Beginning Approach to Condition Driven Index (CDI)

20. 20RELIABILITY | ACCOUNTABILITY Condition Driven Index (CDI) 18 reliability metrics Use SMART criteria to rank the relative importance 5 trend ratings are determined per metric performance –Significant improvement –Slight improvement –Inconclusive –Slight Deterioration –Significant Deterioration

21. 21RELIABILITY | ACCOUNTABILITY ALR Metrics Relative to Standard Objectives

22. 22RELIABILITY | ACCOUNTABILITY Reliability Metrics ALR 1-3Reserve Margin ALR 1-4 BPS Transmission Related Events Resulting in Loss of Load ALR 2-4Disturbance Control Standard Failures (DCS Failures) ALR 2-5 Disturbance Control Events Greater than Most Severe Single Contingency (MSSC) ALR 3-5IROL/SOL Exceedance ALR 4-1Correct Protection System Operations ALR 6-1Transmission Constraint Mitigation ALR 6-2Energy Emergency Alert 3 (EEA 3) ALR 6-3Energy Emergency Alert 2 (EEA 2)

23. 23RELIABILITY | ACCOUNTABILITY Reliability Metrics (cont’d) ALR1-5System Voltage Performance ALR1-12Interconnection Frequency Response ALR2-3UFLS and UVLS Usage ALR6-11 Automatic AC Transmission Outages Initiated by Failed Protection System Equipment ALR6-12Automatic AC Transmission Outages Initiated by Human Error ALR6-13 Automatic AC Transmission Outages Initiated by Failed AC Substation Equipment ALR6-14 Automatic AC Circuit Outages Initiated by Failed AC Circuit Equipment ALR 6-15Element Availability Percentage ALR 6-16Transmission System Unavailability due to Automatic Outages

24. 24RELIABILITY | ACCOUNTABILITY IRI Concept BPS Integrated Reliability Index (IRI): –Event Driven Index (EDI): Based on event severity risk index (SRI) values –Condition Driven Index (CDI): Use a subset of metrics based on selection criteria –Standards/Statute Driven Index (SDI): Identify a subset of standards that have highest impact to reliability and create an index of standards violations

25. 25RELIABILITY | ACCOUNTABILITY IRI Trend (2008-2010 by Quarter) 99.70 99.75 99.80 99.85 99.90 99.95 100.00 Reliability Indices EDI SDI CDI IRI

26. 26RELIABILITY | ACCOUNTABILITY Possible IRI Attributes Scale from 0 to 100 100 would indicate a perfect performance Weighting should be developed and can be adjusted Aggregation at NERC, Interconnection and Region levels Reporting period – targeting quarterly, reported at year end Trial indicators until sufficient history is experienced

27. 27RELIABILITY | ACCOUNTABILITY 27 Reference Materials Quarterly Update is available at Reliability Indicators Display –http://www.nerc.com/page.php?cid=4|331http://www.nerc.com/page.php?cid=4|331 Reliability Metrics Working Group webpage –http://www.nerc.com/filez/rmwg.htmlhttp://www.nerc.com/filez/rmwg.html

28. 28RELIABILITY | ACCOUNTABILITY Coordination/Conversation with CCC Gregory Pierce has been engaged to act as liaison with RMWG and participate in continuing IRI developments CCC Member Thoughts –Desire discussion regarding concepts about SDI –Interested in group’s concerns about overlap of standards versus events versus condition metrics –Is selection of top 26 appropriate? –How can this approach be resilient with time? –Who votes a standard requirement on or off the top list?

29. 29RELIABILITY | ACCOUNTABILITY 29 Discussion and Feedback

30. 30RELIABILITY | ACCOUNTABILITY Standards/ Statute- Driven Metrics System/Event -Driven Metrics Reliability Condition- Driven Indicators Organization Effectiveness Metrics 30 Reliability Improvement Organizational Effectiveness ? ? ? ? ? ? ? ? ? ? ? Standards/ Statute- Driven Metrics System/Event -Driven Metrics Reliability Condition- Driven Indicators Organization Effectiveness Metrics Reliability Performance Measures