Integrated Reliability Measures Heide Caswell, Team Lead, Reliability Metrics Working Group (RMWG) Director of Network Performance, PacifiCorp Compliance and Certification Committee Meeting September 21-22, 2011 Atlanta, GA

2RELIABILITY | ACCOUNTABILITY 2 Topics Conceptual Model Initial Developments Continuing Our Understanding of the Risk Universe Conceptual Indicators –Event Driven Index –Standards/Statute Driven Index –Condition Driven Index Continuing Dialogue with CCC Discussion and Feedback

3RELIABILITY | ACCOUNTABILITY Integrated Reliability Indicators (IRI) Develop the conceptual model of reliability risks Establish quantitative measures for evaluating the performance of the indicators Support risk-informed decision making May be able to correlate over the long term to predictive measures of reliability risks and if so… Estimate the effectiveness of reliability risk reduction and/or mitigation

4RELIABILITY | ACCOUNTABILITY Severity Risk Index (SRI)

5RELIABILITY | ACCOUNTABILITY Key Concepts

6RELIABILITY | ACCOUNTABILITY Integrated Risk Indicators Event Driven Index (EDI) Measures Risk from Major System Events Standards/Statute Driven Index (SDI) Measures Risks from Severe Impact Violations Condition Driven Index (CDI) Monitors Risk from Key Reliability Metrics Based on event severity risk index (SRI) values and turns into an availability index Uses the roster of reliability metrics developed by RMWG and approved by OC/PC Identify a subset of standards that have highest impact to reliability and create an index of standards violations

7RELIABILITY | ACCOUNTABILITY Severity Risk Index (SRI) Focus on significant events and quantify the historic system performance using daily outage data –GADS and TADS daily outages, forced only –MW load loss and restoration duration from disturbance event reports –Could be considered similar to a daily, quarterly or yearly SAIDI metric for distribution systems –If everything was out of service, all load was unserved, the day’s score would be 1000 –If no lines or units were out of service and no load was lost the day’s score would be 0 –Normal “good days” will measure higher than 0

8RELIABILITY | ACCOUNTABILITY Severity Risk Index (SRI)

9RELIABILITY | ACCOUNTABILITY EDI Trends by Quarter

10RELIABILITY | ACCOUNTABILITY Proposed Approach to Standard Driven Index (SDI)

11RELIABILITY | ACCOUNTABILITY Proposed Approach for Standards/Statute Driven Index (SDI) Segment all Standards and Requirements by their Violation Risk Factors; Lower, Medium or High are the VRFs (as many as 1,427 standard requirements accounted for) Select only Standards and Requirements with High VRF (430 standard requirements relevant) Evaluate the range of possible violation severity levels (VSLs), which could range from Lower, Moderate, High to Severe (198 are severe VSL and high VRF) –VSLs are guidelines established jointly by the compliance and standard drafting team and are detailed as to criteria which identifies the level of severity. These levels are approved by the BoT. Consider Risk Impact Statements…select only those with Severe RIS There are 26 requirements with High VRFs and Severe Risk Impact Statements (see list of these requirements following) NERC used the almost four year history (June 2007-December 2010) to validate the proposed 26 standards requirements There is potential to assign to weight requirements by VRFs or VSLs if appropriate The Standard Applicability matrix enables us to derive a normalization factor (denominator) for all entities exposed to a requirement with a High VRF

12RELIABILITY | ACCOUNTABILITY Uses High Violation Risk Factor (VRF) (Approved standard requirements only) Severe Reliability Impact Statement (RIS) –RIS indicates significance of impact on BPS Considers Violation Severity Level (VSL) Validated against almost 4 year history of violations for most significant standards risks to reliability Violation Risk Factor Violatio n Severity Level HighMediumLower Severe High Moderate Lower Proposed Approach for Standards/Statute Driven Index (SDI)

13RELIABILITY | ACCOUNTABILITY Violation Severity Level Violati on Risk Factor Risk Impact Statement Value? SevereHighModerateLower High Severe High Non-Severe Medium Severe Medium Non-Severe Lower Severe Lower Non-Severe Severe Reliability Impact Statement (RIS) –RIS indicates significance of impact on BPS Uses Violation Risk Factor (VRF) Demonstrates Violation Severity Level (VSL) Similar to daily SRI calculation Almost 4 year history violation history of 4,410 confirmed violations Proposed Approach for Standards/Statute Driven Index (SDI)

14RELIABILITY | ACCOUNTABILITY * All 26 requirements have high Violation Risk Factor (VRF) and Severe Risk Impact Statements Proposed Approach for Standards/Statute Driven Index (SDI)

15RELIABILITY | ACCOUNTABILITY SDI Calculation 15 SDI = (Total Compliance - ∑(w V *NV/NR))/(Total Compliance) Range from 0 to 100 Total Compliance = 1.0* Number of Days in a specific period w V = weighting of a particular high risk req. violation NV = number of violations for the selected high risk req. NR = number of registered entities required to comply with the high risk req.

16RELIABILITY | ACCOUNTABILITY SDI Calculation Example SDI = (Total Compliance - ∑(w V *NV/NR))/(Total Compliance) SDI = 100*(1.0*Days in a Quarter - ∑(w V *NV/NR)) / (1.0*Days in a Quarter) 1Q2009: –40 violations (PRC-005 R1) –RIS = Severe, VRF=High, VSL=Severe –A total of 335 TOs, 828 GOs registered functions –SDI = 100*(1.0* *(40)/( ))/(1.0*90) = 99.99

17RELIABILITY | ACCOUNTABILITY Possible SDI Trend by Quarter

18RELIABILITY | ACCOUNTABILITY Condition Driven Index (CDI) Began by narrowing down to key Condition Metrics Assumed CDI = Have since moved toward measuring improvement in approved metrics published by RMWG

19RELIABILITY | ACCOUNTABILITY Beginning Approach to Condition Driven Index (CDI)

20RELIABILITY | ACCOUNTABILITY Condition Driven Index (CDI) 18 reliability metrics Use SMART criteria to rank the relative importance 5 trend ratings are determined per metric performance –Significant improvement –Slight improvement –Inconclusive –Slight Deterioration –Significant Deterioration

21RELIABILITY | ACCOUNTABILITY ALR Metrics Relative to Standard Objectives

22RELIABILITY | ACCOUNTABILITY Reliability Metrics ALR 1-3Reserve Margin ALR 1-4 BPS Transmission Related Events Resulting in Loss of Load ALR 2-4Disturbance Control Standard Failures (DCS Failures) ALR 2-5 Disturbance Control Events Greater than Most Severe Single Contingency (MSSC) ALR 3-5IROL/SOL Exceedance ALR 4-1Correct Protection System Operations ALR 6-1Transmission Constraint Mitigation ALR 6-2Energy Emergency Alert 3 (EEA 3) ALR 6-3Energy Emergency Alert 2 (EEA 2)

23RELIABILITY | ACCOUNTABILITY Reliability Metrics (cont’d) ALR1-5System Voltage Performance ALR1-12Interconnection Frequency Response ALR2-3UFLS and UVLS Usage ALR6-11 Automatic AC Transmission Outages Initiated by Failed Protection System Equipment ALR6-12Automatic AC Transmission Outages Initiated by Human Error ALR6-13 Automatic AC Transmission Outages Initiated by Failed AC Substation Equipment ALR6-14 Automatic AC Circuit Outages Initiated by Failed AC Circuit Equipment ALR 6-15Element Availability Percentage ALR 6-16Transmission System Unavailability due to Automatic Outages

24RELIABILITY | ACCOUNTABILITY IRI Concept BPS Integrated Reliability Index (IRI): –Event Driven Index (EDI): Based on event severity risk index (SRI) values –Condition Driven Index (CDI): Use a subset of metrics based on selection criteria –Standards/Statute Driven Index (SDI): Identify a subset of standards that have highest impact to reliability and create an index of standards violations

25RELIABILITY | ACCOUNTABILITY IRI Trend ( by Quarter) Reliability Indices EDI SDI CDI IRI

26RELIABILITY | ACCOUNTABILITY Possible IRI Attributes Scale from 0 to would indicate a perfect performance Weighting should be developed and can be adjusted Aggregation at NERC, Interconnection and Region levels Reporting period – targeting quarterly, reported at year end Trial indicators until sufficient history is experienced

27RELIABILITY | ACCOUNTABILITY 27 Reference Materials Quarterly Update is available at Reliability Indicators Display – Reliability Metrics Working Group webpage –

28RELIABILITY | ACCOUNTABILITY Coordination/Conversation with CCC Gregory Pierce has been engaged to act as liaison with RMWG and participate in continuing IRI developments CCC Member Thoughts –Desire discussion regarding concepts about SDI –Interested in group’s concerns about overlap of standards versus events versus condition metrics –Is selection of top 26 appropriate? –How can this approach be resilient with time? –Who votes a standard requirement on or off the top list?

29RELIABILITY | ACCOUNTABILITY 29 Discussion and Feedback

30RELIABILITY | ACCOUNTABILITY Standards/ Statute- Driven Metrics System/Event -Driven Metrics Reliability Condition- Driven Indicators Organization Effectiveness Metrics 30 Reliability Improvement Organizational Effectiveness ? ? ? ? ? ? ? ? ? ? ? Standards/ Statute- Driven Metrics System/Event -Driven Metrics Reliability Condition- Driven Indicators Organization Effectiveness Metrics Reliability Performance Measures