1. Reliability Assurance Initiative
NERC Reliability Working Group
July 25, 2013

2. What is RAI?
A collaborative effort between NERC, the Regional Entities, and registered entities to identify and implement changes that enhance the effectiveness of the Compliance Monitoring and Enforcement Program
Represents risk-based compliance monitoring
Focuses on risks to reliability
Enforcement will be reserved for significant matters
It is a customized compliance approach
Individualized scoping for each registered entity
Reduces administrative burdens and distractions

3. How will we know it’s successful?
If the end state compliance monitoring and enforcement program is effective* at providing reasonable assurance through compliance monitoring, appropriate deterrence through enforcement and a feedback loop to continuously improve reliability standards.
*resources expended to achieve and monitor compliance and carry out enforcement are sufficient on the larger risk areas and not necessarily over applied on the lower risk areas.

4. What are the components of the RAI?
The four components of the RAI are:
Assessing Reliability Risk
Scoping Compliance Monitoring
Processing Possible Violations in Accordance with Risk
Strengthening the Feedback Loop to the Standards Development Process

5. In the context of RAI, what is meant by risk?
Definition of risk to the BES
Instability, uncontrolled separation, or cascading failures
System-wide risks to the BES
Entity’s Risk to the BES
Inherent risk is a function of registrations and other relevant factors like system design, configuration, size, etc.
Control risk is a function of the entity’s internal controls established to reduce risk of violation or system event.
These two components will be considered in determining an entity’s risk profile or risk assessment.
Project currently underway to determine a regional approach to develop a prototype for risk assessment.

6. Risk Considerations
Analysis of risk assists an entity to deploy controls more effectively.
Review should focus on greatest threats to reliability based on impact and likelihood of occurrence.
Cost of a control should not exceed benefits.
Reliability Standards are dynamic and methodology should be flexible enough to adapt with changes.
There is no “one size fits all” model.

7. How do I do an internal risk assessment?
One size does not fit all!!!
Entity BA DP
LSE TO GO
GOP IA PA
PSE RC RP
RSG TP
TOP
TSP
Entity A (Co-Op) X
Entity B (Gen)
Entity C
Entity D
Entity E (SoCo)

8. What is a risk assessment process?
Assess Risks
Dev
Assmnt
Criteria
Assess
Risk
Interaction
Identify
Risks
Assess
Risks
Prioritize
Risks
Respond
To Risks
AKA
Internal
Controls

9. Questions to Consider
What are risks to reliability of the bulk electric system?
Consider registered functions.
Review event analysis of the entity.
Review operational issues in the industry.
What keeps me up at night relative to reliability?
What are compliance risks for the Standards?
Are there stumbling blocks to compliance for the entity?
Review self-reports for the entity (are there problematic standards?).
Review frequently violated standards.
What keeps me up at night relative to compliance?
Risk Interactions
Interactions between other events/conditions that could increase risk.
How do risks rank relative to each other?
Formal method to calculate risk
Likelihood scale, impact scale
“Pin the tail on the donkey”

10. Internal Control Program
An internal control program helps provide a Registered Entity with reasonable
assurance of compliance with the requirements of the Standards.

11. Functional Overlap of the Standards
Future - Functions Based
Current – Standards Based
Change Management
& Testing
CIP-002
CIP-003
Device Management
CIP-004
Info. Classification
& Handling / Doc Control
CIP-005
CIP-006
Access Control
CIP-007
Physical Security
CIP-008
CIP-009
Recovery & Incident
Response

12. 693 Standards

13. Management Controls
Policies and procedures ensure management’s directives are carried out.
Elements of controls work together and collectively reduce risk of not achieving objectives.
Should not be considered discretely (defense in depth).

14. Types of Control Activities
Internal control is a process, effected by an entity’s board of directors, management and other personnel (people), designed to provide reasonable assurance regarding the achievement of objectives
Continuous Improvement Cycle

15. Associated NERC standard (s) Detective Internal Controls*
Internal Controls Analysis
Review existing processes, procedures and policies to determine if they facilitate compliance with the Reliability Standards
Control
Associated NERC standard (s)
Frequency
Detective Internal Controls*
Compliance Program Management Controls
Self-Assessments prior to Self-Certification
All Standards
Annual
Targeted Compliance Site Assessments
NYPA Internal Event Analysis Plan
NERC EA process, EOP-004
Operations, Maintenance, and Cyber Security Controls
Protection Control & Engr. (PC&E) Quarterly work order review and compliance attestations
PRC-005, PRC-006, PRC-007, PRC-008, PRC-009, PRC-010, PRC-011, PRC-015, PRC-017, PRC-018, PRC-021
PC&E peer review of Relay Operation Analysis
PRC-001, PRC-004
PC&E tracking Maintenance & Testing Exceptions
Operator logging review
COM-002, PRC-001, VAR-002, TOP-001, TOP-002, TOP-003, TOP-006
Incident Response Program
CIP-008
Ongoing
A ‘central’ logging mechanism and transmission to a third party service for the aggregation and analysis of security logs
CIP-007
Operator Shift turn-over compliance check lists
COM-002, PRC-001, VAR-002, TOP-001, TOP-002, TOP-003, , TOP-006

16. ERO RAI Program Conceptual White Papers ERO & Industry Documents
RAI Q&A
Internal Controls Working Guide
Initial Phase Plan/Deliverables
Audit Handbook
ERO & Industry Collaborative Guides
Benefits & Impacts
Internal Control Library
RAI Pilots
MRO - ATC
RFC – PJM, PPL
SERC – integrating into audits
Self-Reporting Process Enhancement
Self-Report Guide
Mitigation Plan Guide
Violation vs Deficiency Pilots
FFT Enhancements
Regional Entity Triage Process

17. References Controls Framework Documents Auditing Guidance Documents
Committee of Sponsoring Organizations of the Treadway Commission (COSO): Internal Control - Integrated Framework
The Institute of Internal Auditors – International Professional Practices Framework – Standard 2210 – Engagement Objectives
Information Systems Audit and Control Association – Control Objectives for Information and Related Technology
Auditing Guidance Documents
American Institute of Certified Public Accountants – Professional Standards, vol. 1 – AU Section 314
United States Government Accounting Office - Government Auditing Standards – Chapter 7 – Reporting Standards for Performance Audits
NERC RAI Documents
Intiative.aspx

18. Questions