Compliance Performance Measure Proposals and Preliminary Trends Greg Pierce – Chair, Performance Measures Task Force Compliance and Certification Committee Meeting March 10-11, 2010

Topics PMTF Tasks and Approach Performance Measure Proposals Preliminary Trends Outreach Effort Next Steps

PMTF Tasks Establish an open, consistent and repeatable metric development process to identify and evaluate metrics Develop compliance improvement measures at the highest levels to provide insight in terms of a set of compliance principles Coordinate and work with Regional Entities, RMWG, OC, PC and CIPC to ensure agreement Recommend how to benefit from the compliance improvement measures

Approach Adopt RMWG’s open metric development process Review data source on compliance and other key input factors Propose a set of performance measures at high levels Gather data and validate trends Solicit feedback and other ideas from stakeholders Prepare a draft report starting April 2010

Proposed Performance Measures Compliance Culture Timeliness of CVI/CIQ Average Duration of Mitigation Plan Completion Violation Risk Index Average Duration of Audit Report Completion w/ No Violations Average Duration of Audit Report Completion w/ Potential Violations Availability/timeliness of 'lessons-learned’ guidance from audits, operating experience, violations and disturbances Feedback to standards development The detailed individual metric definitions and specifications are available at http://www.nerc.com/filez/pmtf.html.

Compliance Culture Measure Metric – Compare Self-Disclosed Violations to the Total Number of Discovered Violations Self-Disclosed = Self-report + Self-certification Indicates the level of trust and cooperation within the “NERC Enterprise” as well as the culture of compliance of the stakeholder community

Trend By Discovery Method

Compliance Culture Trend

Timeliness of CVI/CIQ Average Duration – for CVI/CIQ with Open status Total number of CVIs/CIQs for Open and Closed status Number of lessons learned and trends issued to industry

CVI/CIQ Timeliness Measure Duration (Days) CVI CIQ Oldest Average NERC 644 317 361 242 Region 823 444 126 107

Triggers for NERC/Regional CVIs/CIQs

Average Duration of Mitigation Plan Completion Duration – from Date plan received to Mitigation close date Sum of all durations divided by total number of violations discovered in a particular quarter Remove zero duration days

Average Number of Days for Completing Mitigation Plans

Violation Risk Index (VRI) Violation Risk Index is defined as a risk factor and severity level weighted average based on unmitigated violations Assess potential consequences of a particular unmitigated violation. Unmitigated violations of higher risk factor requirements have more weighting value than of lower risk factors. Determine the change in reliability levels due to Standard requirement violations. Weighting factors and an example are included in Appendix A.

NERC Unmitigated Violations (6/18/07 to 12/31/2009)

Violation Risk Trend

Average Duration of Audit Report Completion w/ No Violations Measure audit process efficiency and ability to promptly advise industry of audit results  These reports do not have potential violations Starting point: audit ending point Ending point: report finalized with entity and received by NERC

Average Duration of Audit Report Completion w/ No Violations

Average Duration of Audit Report Completion w/ Potential Violations Measure enforcement process efficiency and ability to promptly advise industry of audit results  These reports have potential violations Starting point: audit ending point Ending point: report finalized with entity and received by NERC Data is not available until second quarter of 2010

Timeliness of Lessons-Learned Establish a common and central platform to share lessons-learned Timeliness of actionable lesson-learned guidance from operating experience, violations and disturbance events Lessons-Learned guidance Recommendations from event analysis Compliance analysis Advisories

Timeliness of compliance results/feedback to standards # of SARs (e.g., refine a particular requirement) # of suggestion and comment submittals The Standards input form is available at http://www.nerc.com/files/Standards_Input_Form_Final_2008June30.doc Direct input to “results-based” standards initiative Focus on risk-based standards, using first assessment triage sheet

Outreach Effort Chair met with and presented to the RMWG on February 18, 2010 Chair presented to the NERC/Regional Entity Compliance Group at its conference call on February 26, 2010 Chair presented to the OC/PC/CIPC at their joint WebEx conference call on March 9, 2010

Next Steps Continue reaching out to seek guidance Members will meet with OC and PC on March 16-17, 2010 Recommend the proposals in June, desiring CCC approval in September 2010 Comments and suggestions are due April 2, 2010 Send to PMTF Chair, Greg Pierce at GPIERC2@entergy.com and NERC at jessica.bian@nerc.net

Question & Answer

Appendix A Violation Weighting Factor Table Note: 1. The weighting values are derived by applying similar ratios developed in the Base Penalty Amount Table described in section 4 of the ERO Sanction Guidelines, Appendix 4B to the NERC Rules of Procedure. 2. Reference materials are available in a NERC white paper “Toward Ensuring Reliability: Reliability Performance Metrics”. It can be viewed at: http://www.nerc.com/docs/pc/rmwg/Reliability_Metrics_white_paper.pdf.

VRI Example