▪ ▪ CLARITY ▪ ASSURANCE ▪ RESULTS MIDWEST RELIABILITY ORGANIZATION Improving RELIABILITY and mitigating RISKS to the Bulk Power System Thomas P. Tierney Director of Compliance, MRO SPP Compliance Forum May 23, 2013 Reliability Assurance Initiative

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS May 23, Common Mission Improve the Reliability of the Bulk Power System

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Improvement Is the Goal We have very reliable systems within MRO/SPP, but we can still improve by identifying problems and fixing them – no weak links There is always opportunity for improvement within the design criteria of an interconnected system May 23,

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS May 23, Demystifying Internal Controls No, Really… What Is an Internal Control?

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Nothing New Registered Entities have been managing reliability for decades – they have management practices (i.e. controls) around reliability Existing practices have been translated into the Reliability Standards and documented – “operationalizing compliance” Don’t overthink “internal controls” May 23,

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Risk Possibility that something undesirable will happen Measured as a combination of likelihood and impact Control/Control Activity Policy, procedure, checklist, etc. designed to minimize the opportunity for a risk to be realized Internal Control Control activity performed internally, not by a third party Management practices that include control activities performed internally (“self monitoring”) May 23, Definitions

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Inherent Risk Risks “built-in” to a given entity, based on geography, what facilities it operates, “interconnectedness,” etc. Reliability Standards are designed to mitigate inherent risk in a broad sense Control Risk Risk that management practices or control activities are not achieving their reliability or compliance objectives Detection Risk Risk that possible violations are going unnoticed Residual Risk Risk that remains after application of a control and other mitigating factors Difficult and expensive to eliminate 100% of risk – we must live with some risk May 23, Types of Risk

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Preventive Controls designed to stop something from occurring Detective Controls designed to identify when a possible violation has occurred and facilitate timely remediation Also known as “Monitoring” controls May 23, Types of Controls

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Multiple, complementary controls that work together to reduce risk (“Defense in depth”) Primary Secondary Tertiary Secondary and Tertiary controls serve as a “safety net” in case the Primary control does not function as expected Each subsequent tier of controls further reduces residual risk May 23, Control Hierarchies

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Protection System Maintenance and Testing Relay technicians complete work orders according to a pre-defined checklist to prevent steps being skipped or performed incorrectly Supervisors review and approve completed work orders to verify technicians’ proper use of the checklist A sample of work orders is reviewed by Internal Audit to verify accuracy and completeness May 23, Examples

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS May 23, Program Documents (Procedures) Standard Work Order Supervisory Review Management Oversight Checklist followed and completed, exceptions noted, follow-up notes signed Review for completeness and accuracy, follow-up actions closed or scheduled to be completed, signed Periodic sampling of work orders to determine program is being completed and properly reviewed Procedure/ Process Control Control ActivityControl Type Primary Control Secondary Control Tertiary ControlExamples

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Training Management establishes training objectives and reviews training materials to confirm objectives are met Individuals are tested after completion of training to ensure effectiveness of delivery Supervisors conduct performance observations to verify past training has been effective and to identify additional training needs May 23, Examples

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS May 23, Program Documents (Procedures) Training Objectives Training Evaluation Performance Observations Management establishes training objectives and reviews training materials to confirm objectives are met Individuals are tested after completion of training to ensure effectiveness of delivery Supervisors conduct performance observations to verify past training has been effective and to identify additional training needs Procedure/ Process Control Control ActivityControl Type Primary Control Secondary Control Tertiary ControlExamples

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Cybersecurity Systems are configured to require passwords to prevent unauthorized access All changes to systems are reviewed, approved, and tested to ensure that unauthorized changes do not occur Periodic reviews are conducted to ensure that password controls adhere to corporate security policies May 23, Examples

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS May 23, Security Policies Password Controls Configuration Management Security Assessments Systems are configured to require passwords to prevent unauthorized access All changes to systems are reviewed, approved, and tested to ensure that unauthorized changes do not occur Periodic reviews are conducted to ensure that password controls adhere to corporate security policies Procedure/ Process Control Control ActivityControl Type Primary Control Secondary Control Tertiary ControlExamples Configuration Management Procedures

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS May 23, Reliability Assurance Initiative Focusing on Risk

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS “One size fits all” compliance model NERC Actively Monitored Standards do not change based on regional differences, entity size, etc. No consideration of management practices (i.e. controls) around reliability standards Zero-defect approach to enforcement is burdensome Every violation requires a regulatory filing regardless of severity Self-reports require significant effort Administrative Citation Process (ACP) & Find, Fix, Track (FFT) are not sufficient Expediting enforcement won’t solve the problem May 23, Current State

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Shape compliance monitoring and mitigation based on risk Reserve enforcement for most significant risks May 23, Key Elements of RAI

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Assessment of each entity’s inherent risk Some factors influencing assessment Facilities Special Protection Systems IROLs Geographic location Functions performed Connectivity (physical and cyber) EMS/SCADA system Compliance history May 23, Scoping of Work

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS What does a risk assessment look like? Not a letter grade or single rating Entities will not be compared and ranked Assessment will look more like a matrix Certain families of standards may be higher risk for one entity, less risky for another May 23, Scoping of Work

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Internal controls established by each entity must be identified Evaluation of select controls to determine effectiveness Design – Is the control, as documented, adequate to address the risk? Operational – Is the control implemented as designed? Effective controls reduce residual risk to an acceptable level MRO staff can rely on effective controls Regulatory scope can be adjusted – less auditing and testing (or none) where strong controls exist May 23, Scoping of Work

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Risk assessments and internal controls will be leveraged across all compliance monitoring activities Internal emphasis should shift over time toward maintaining effective controls around Reliability Standards Continue to identify and correct issues in a timely fashion Focus on reliable operations first Compliance should be a natural outcome of strong operations May 23, Scoping of Work

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS “Compliance Exceptions” represent lower risk violations Do not represent significant risk to the BES Identified by an entity itself or by regional staff Initially tracked at the regional level No enforcement proceedings, no penalties Mitigation will always be important What was done to address the problem itself? What is being done to prevent recurrence? May 23, Compliance Exceptions

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Enforcement will focus on most significant or high-risk issues Violation poses significant risk to reliable operation of the BES, e.g. cause or contributing factor in a cascading event Multiple smaller issues may aggregate into a bigger problem or are indicative of a poor control environment Willful misconduct May 23, Compliance Exceptions

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Compliance audit Tools being developed with input from industry, the Regions, and NERC Currently developing risk assessment Internal controls evaluation to occur during June & July Scope will reflect risk and presence of effective controls Audit completion in Q4 of 2013 May 23, MRO Pilots

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Self-certification Transition from blanket, “check the box” approach to narrowly focused self-certifications Scope limited to FAC R6 based on problems identified on recent audits Focus on self-assessment process and on controls to identify and correct problems May 23, MRO Pilots

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTS Contact Information Thomas P. Tierney, Director of Compliance Midwest Reliability Organization (651) May 23,

▪▪ CLARITY ▪ ASSURANCE ▪ RESULTSQuestions? May 23,