1. Compliance Monitoring and Enforcement Audit Program Background Information

2. Introduction
This presentation is intended as an introduction to the audit process for employees of entities being audited by MACD. Please refer to the manual for more detail

3. Why is MACD Performing Compliance Audits?
Electricity is critical to our society
The industry has established standards to ensure reliable operation of the power system
Under the Electricity Act (Section 32.1) , it is the IESOs role to monitor and enforce compliance with these standards
IESO delegates these responsibilities to MACD

4. Reliability Standards
Reliability Standards are created to improve the reliable operation of the bulk electricity system
Minimum standards for operation in key areas
Market Participants must continuously follow all applicable Reliability Standards in effect in Ontario
Depending on your company role, different standards may apply e.g. Generator Operator Standards compared to Transmission Operator standards

5. Reliability Standards
NERC – North American Electric Reliability Corporation establishes reliability standards.
NPCC –North East Power Coordinating Council has delegated authority from NERC.
Ontario is part of the NPCC, IESO participates in the development and approval of standard. Stakeholder involvement through the Reliability Standards Sub Committee of the IESO
The Ontario Energy Board has the right to reject NERC reliability standards

6. Introduction to Compliance Audits
Audits must provide reasonable assurance that evidence is sufficient and appropriate to support the auditors findings and conclusions.
Sufficient quantity of evidence, spans the time period being audited
Evidence must be of sufficient quality – authenticated
Typically involves multiple types of evidence related to the same thing e.g.
Approved internal documentation
Physical examination/inspection
Analytical procedures with adequate data
Observations and inquiries

7. MACD Audit Program
A ‘risk based’ audit program, not a pre set audit cycle
MACD considers industry trends, Ontario specific inputs when developing the program
MACD makes an assessment of the risk to reliability for various standards and participants
The Audit Program begins with an implementation plan issued in the fall

8. MACD Implementation Plan
Issued in the fall – after NERC issues their plan
Provides a list of reliability standards which MACD will focus on during the upcoming year
The implementation plan may be revised during the year depending on industry developments
Individual audits will include standards from the implementation plan but may also include additional standards.
Individual audits are initiated with an Audit Notification, are not listed in the Implementation Plan

9. Introduction to Compliance Audits
Part Two of this presentation will deal with the audit process once an Audit Notification has been sent to a Market Participant.
Audit Notification
Pre Audit Meetings
Data Submission
The Audit
Exit Meeting
Audit Report

10. Compliance Monitoring and Enforcement Audit Program - Background Information
End Of Presentation

11. Compliance Monitoring and Enforcement Audit Program - The Audit Process

12. Introduction
This presentation is intended as an introduction to the audit process for employees of entities being audited by MACD. Please refer to the manual for more detail

13. Introduction to Compliance Audits
Audit Notification
Pre Audit Meetings
Data Submission
The Audit
Exit Meeting
Audit Report

14. Introduction to Compliance Audits
Audits are not “out to get you”
“Trust but Verify” is the philosophy for the process
The Audit team must, in their professional judgment, be satisfied that the evidence demonstrates compliance
If the Audit team cannot confirm compliance to a standard, the report will indicate potential non compliance which MAY trigger an investigation

15. Key Roles Your company’s Audit Coordinator
The Single Point of Contact for the audit
Will do all coordination with MACD
The person you should talk to with any concerns or questions
The only person who should identify concerns or issues to the MACD Audit Team
The person responsible for all logistics with the MACD Audit Team

16. Key Roles Subject Matter Expert
The person your company has identified as being able to discuss audit information with the audit team
Essential to your company’s ability to demonstrate compliance with standards
May be interviewed by the Audit Team to discuss reliability standards
Should discuss any questions about the audit in advance with your Audit Coordinator

17. Key Roles Audit Team Lead An employee of MACD
Coordinates all audit efforts for MACD with your Audit Coordinator
Discusses any issues or concerns with your Audit Coordinator
The local decision maker for MACD

18. Audit Notification
At least 90 days before your audit, you will receive a notification
Audit Team
Scope
Off Site vs. On Site
Date of Audit
Pre Audit Survey
Initial Data Requests
Pre Audit Schedule of Activities

19. Notification - Audit Team
Audit Team Members:
are responsible for producing an opinion on adherence to standards
have experience in the industry and/or auditing
have completed NERC Audit Training
have confidentiality agreements in place
are independent, have not worked in your company in the recent past
issues? Discuss with MACD in advance

20. Notification - Audit Scope
Scope - Standards, Audit Period, Sample Size
Standards:
Responsible for all applicable Reliability Standards
Audit will focus on a sub set chosen by MACD
Subset reflects industry trends, impact on reliability
Period is a rolling six years, audit team can examine compliance during entire six years
Data requested is usually for previous 12 months

21. Pre Audit Activities Pre Audit Survey
A tool for the audit team to learn more about your organization.
Generic, some questions may not apply to your organization.
Questions about assets, organization structure, physical location and neighbours
Some logistical information and contact information
Helps the audit progress more smoothly

22. Pre Audit Activities RSAWs – Reliability Standard Audit Worksheet
Tool to help you demonstrate your compliance
Guides you through providing evidence to demonstrate your compliance
Assists you in organizing and packaging the information
Clearly identify appropriate sections in the files for evidence, reference the standard
Submited to MACD through the Portal

23. RSAWs

24. Pre Audit Activities It is important to annotate documents
If using the same document for multiple requirements, put a copy of the document in each folder!

25. Pre Audit Activities
The IESO Portal is a secure system already in use for providing this information to the audit team
Portal training material is available on the IESO website

26. Pre Audit Activities
It is important to organize information to show where it ties to the standard (to show compliance)
Include the RSAW for each standard in the top folder, and then evidence documentation with annotation for each requirement.

27. Pre Audit Meetings
Open dialogue is important during the audit process. Opportunities for 3 pre audit meetings.
Pre submission meeting – date agreed to, allows for general questions and discussion
Pre Audit meeting, optional, done after MACD has reviewed the data submission
Pre Audit Logistics – At least one week prior to audit to discuss the logistics of the audit week

28. On Site Logistics Audit Team will require: Rooms to work in
Lunch/coffee (working lunch delivered to room)
MACD to be invoiced
Access to projector, internet, printers, if possible via our laptops – All evidence should be electronic unless otherwise agreed by the MACD audit team
Access to site as needed, with appropriate escorts

29. Audit Presentations
The Audit team will start the audit with a brief opening presentation.
Introducing the team
Expectations for the process
It can be helpful if the MP makes a brief opening presentation to
Introduce the MP audit team
Provide some background on the company and the facility
Explain any safety/security requirements

30. Interviews
The Audit Team may want to talk to SMEs (Subject Matter Experts) and other staff
Your Audit Coordinator will arrange interviews
The Audit Team may want to ask questions about procedures, their implementation, data

31. Exit Meeting
At the end of the audit period, the team will give an exit presentation. This presentation will give an overall summary:
Recap of the audit review
The Audit Team observations, preliminary findings and recommendations
Schedule for remainder of process, including when the final report will be issued

32. Final Report
The audit report would identify if the team was unable to confirm compliance with a reliability standard
The potential non compliance is turned over to the investigation unit in MACD for follow up.
There is no “penalty” from the audit itself

33. Potential Non Compliance Findings
If the Audit Team identifies an immediate risk to reliability, the Team Lead will immediately communicate this to your Audit Coordinator and then to MACD for investigation
For other potential non compliance observations that do not have an immediate impact on system reliability, these will be reported at the conclusion of the audit

34. Escalation Process Any questions or concerns during the audit:
Discuss with your company Audit Coordinator
The Audit Coordinator from your company should discuss with the Audit Team Lead as soon as possible.
If not satisfied after discussing with the Team Lead, your Audit Coordinator can escalate the issue to the MACD supervisor/director
If still not satisfied, your company can take issues to the Dispute Resolution Panel (very formal process) – OEB process

35. Summary A minimum of 90 days notice
Initial Information Request - before the audit begins, submit information to demonstrate compliance
Subsequent Information Requests may come during the audit
SMEs and other staff may be interviewed
The Audit Team will provide an exit interview with preliminary results
The Audit Report will follow with details

36. Compliance Monitoring and Enforcement Audit Program - The Audit Process
End Of Presentation