Compliance Monitoring and Enforcement Audit Program Background Information

Introduction This presentation is intended as an introduction to the audit process for employees of entities being audited by MACD. Please refer to the manual for more detail

Why is MACD Performing Compliance Audits? Electricity is critical to our society The industry has established standards to ensure reliable operation of the power system Under the Electricity Act (Section 32.1) , it is the IESOs role to monitor and enforce compliance with these standards IESO delegates these responsibilities to MACD

Reliability Standards Reliability Standards are created to improve the reliable operation of the bulk electricity system Minimum standards for operation in key areas Market Participants must continuously follow all applicable Reliability Standards in effect in Ontario Depending on your company role, different standards may apply e.g. Generator Operator Standards compared to Transmission Operator standards

Reliability Standards NERC – North American Electric Reliability Corporation establishes reliability standards. http://www.nerc.com/page.php?cid=2%7C20 NPCC –North East Power Coordinating Council has delegated authority from NERC. Ontario is part of the NPCC, IESO participates in the development and approval of standard. Stakeholder involvement through the Reliability Standards Sub Committee of the IESO The Ontario Energy Board has the right to reject NERC reliability standards

Introduction to Compliance Audits Audits must provide reasonable assurance that evidence is sufficient and appropriate to support the auditors findings and conclusions. Sufficient quantity of evidence, spans the time period being audited Evidence must be of sufficient quality – authenticated Typically involves multiple types of evidence related to the same thing e.g. Approved internal documentation Physical examination/inspection Analytical procedures with adequate data Observations and inquiries

MACD Audit Program A ‘risk based’ audit program, not a pre set audit cycle MACD considers industry trends, Ontario specific inputs when developing the program MACD makes an assessment of the risk to reliability for various standards and participants The Audit Program begins with an implementation plan issued in the fall

MACD Implementation Plan Issued in the fall – after NERC issues their plan Provides a list of reliability standards which MACD will focus on during the upcoming year The implementation plan may be revised during the year depending on industry developments Individual audits will include standards from the implementation plan but may also include additional standards. Individual audits are initiated with an Audit Notification, are not listed in the Implementation Plan

Introduction to Compliance Audits Part Two of this presentation will deal with the audit process once an Audit Notification has been sent to a Market Participant. Audit Notification Pre Audit Meetings Data Submission The Audit Exit Meeting Audit Report

Compliance Monitoring and Enforcement Audit Program - Background Information End Of Presentation

Compliance Monitoring and Enforcement Audit Program - The Audit Process

Introduction This presentation is intended as an introduction to the audit process for employees of entities being audited by MACD. Please refer to the manual for more detail

Introduction to Compliance Audits Audit Notification Pre Audit Meetings Data Submission The Audit Exit Meeting Audit Report

Introduction to Compliance Audits Audits are not “out to get you” “Trust but Verify” is the philosophy for the process The Audit team must, in their professional judgment, be satisfied that the evidence demonstrates compliance If the Audit team cannot confirm compliance to a standard, the report will indicate potential non compliance which MAY trigger an investigation

Key Roles Your company’s Audit Coordinator The Single Point of Contact for the audit Will do all coordination with MACD The person you should talk to with any concerns or questions The only person who should identify concerns or issues to the MACD Audit Team The person responsible for all logistics with the MACD Audit Team

Key Roles Subject Matter Expert The person your company has identified as being able to discuss audit information with the audit team Essential to your company’s ability to demonstrate compliance with standards May be interviewed by the Audit Team to discuss reliability standards Should discuss any questions about the audit in advance with your Audit Coordinator

Key Roles Audit Team Lead An employee of MACD Coordinates all audit efforts for MACD with your Audit Coordinator Discusses any issues or concerns with your Audit Coordinator The local decision maker for MACD

Audit Notification At least 90 days before your audit, you will receive a notification Audit Team Scope Off Site vs. On Site Date of Audit Pre Audit Survey Initial Data Requests Pre Audit Schedule of Activities

Notification - Audit Team Audit Team Members: are responsible for producing an opinion on adherence to standards have experience in the industry and/or auditing have completed NERC Audit Training have confidentiality agreements in place are independent, have not worked in your company in the recent past issues? Discuss with MACD in advance

Notification - Audit Scope Scope - Standards, Audit Period, Sample Size Standards: Responsible for all applicable Reliability Standards Audit will focus on a sub set chosen by MACD Subset reflects industry trends, impact on reliability Period is a rolling six years, audit team can examine compliance during entire six years Data requested is usually for previous 12 months

Pre Audit Activities Pre Audit Survey A tool for the audit team to learn more about your organization. Generic, some questions may not apply to your organization. Questions about assets, organization structure, physical location and neighbours Some logistical information and contact information Helps the audit progress more smoothly

Pre Audit Activities RSAWs – Reliability Standard Audit Worksheet Tool to help you demonstrate your compliance Guides you through providing evidence to demonstrate your compliance Assists you in organizing and packaging the information Clearly identify appropriate sections in the files for evidence, reference the standard Submited to MACD through the Portal

RSAWs

Pre Audit Activities It is important to annotate documents If using the same document for multiple requirements, put a copy of the document in each folder!

Pre Audit Activities The IESO Portal is a secure system already in use for providing this information to the audit team Portal training material is available on the IESO website http://www.ieso.ca/imoweb/marketplaceTraining/showCourses.asp?id=15

Pre Audit Activities It is important to organize information to show where it ties to the standard (to show compliance) Include the RSAW for each standard in the top folder, and then evidence documentation with annotation for each requirement.

Pre Audit Meetings Open dialogue is important during the audit process. Opportunities for 3 pre audit meetings. Pre submission meeting – date agreed to, allows for general questions and discussion Pre Audit meeting, optional, done after MACD has reviewed the data submission Pre Audit Logistics – At least one week prior to audit to discuss the logistics of the audit week

On Site Logistics Audit Team will require: Rooms to work in Lunch/coffee (working lunch delivered to room) MACD to be invoiced Access to projector, internet, printers, if possible via our laptops – All evidence should be electronic unless otherwise agreed by the MACD audit team Access to site as needed, with appropriate escorts

Audit Presentations The Audit team will start the audit with a brief opening presentation. Introducing the team Expectations for the process It can be helpful if the MP makes a brief opening presentation to Introduce the MP audit team Provide some background on the company and the facility Explain any safety/security requirements

Interviews The Audit Team may want to talk to SMEs (Subject Matter Experts) and other staff Your Audit Coordinator will arrange interviews The Audit Team may want to ask questions about procedures, their implementation, data

Exit Meeting At the end of the audit period, the team will give an exit presentation. This presentation will give an overall summary: Recap of the audit review The Audit Team observations, preliminary findings and recommendations Schedule for remainder of process, including when the final report will be issued

Final Report The audit report would identify if the team was unable to confirm compliance with a reliability standard The potential non compliance is turned over to the investigation unit in MACD for follow up. There is no “penalty” from the audit itself

Potential Non Compliance Findings If the Audit Team identifies an immediate risk to reliability, the Team Lead will immediately communicate this to your Audit Coordinator and then to MACD for investigation For other potential non compliance observations that do not have an immediate impact on system reliability, these will be reported at the conclusion of the audit

Escalation Process Any questions or concerns during the audit: Discuss with your company Audit Coordinator The Audit Coordinator from your company should discuss with the Audit Team Lead as soon as possible. If not satisfied after discussing with the Team Lead, your Audit Coordinator can escalate the issue to the MACD supervisor/director If still not satisfied, your company can take issues to the Dispute Resolution Panel (very formal process) – OEB process

Summary A minimum of 90 days notice Initial Information Request - before the audit begins, submit information to demonstrate compliance Subsequent Information Requests may come during the audit SMEs and other staff may be interviewed The Audit Team will provide an exit interview with preliminary results The Audit Report will follow with details

Compliance Monitoring and Enforcement Audit Program - The Audit Process End Of Presentation