1. Verification, Validation and Internal Audits - Jose Jimenez Federal Facilities Program Manager – EPA Region III

2. May 3, 20062 What is an Environmental Management System? A set of interrelated elements to develop and implement an organization's environmental policy A set of interrelated elements to develop and implement an organization's environmental policy

3. May 3, 20063 What is an EMS Internal Audit? A systematic and documented process for objectively obtaining and evaluating evidence to determine whether an organization’s Environmental Management System (EMS) conforms to the EMS audit criteria set by the organization, and for communication of the results of this process to management. A systematic and documented process for objectively obtaining and evaluating evidence to determine whether an organization’s Environmental Management System (EMS) conforms to the EMS audit criteria set by the organization, and for communication of the results of this process to management.

4. May 3, 20064 What is an EMS Internal Audit? Snapshot in time Snapshot in time Systematic verification process Systematic verification process Documented procedure Documented procedure Management tool Management tool Not a compliance audit Not a compliance audit

5. May 3, 20065 What is an EMS Internal Audit? Verification process Verification process –Does the system conform to ISO 14001? –Does the system conform to your agency's metrics? –Does the system conform to your facility/organization's procedures ? –Has the EMS been effectively implemented? –Has the EMS been effectively maintained? Basis for continual improvement Basis for continual improvement

6. May 3, 20066 What do you need to do? Develop audit procedure(s) and programs that describe the “who, what, when, and how” of your organization's EMS internal audit Develop audit procedure(s) and programs that describe the “who, what, when, and how” of your organization's EMS internal audit Conduct an internal audit Conduct an internal audit Keep proper records Keep proper records Initiate corrective action Initiate corrective action Communicate findings to senior leadership Communicate findings to senior leadership Continually improve the EMS Continually improve the EMS

7. May 3, 20067 Why do you need to conduct an EMS Internal Audit? EO 13148 EO 13148 ISO 14001 ISO 14001 Agency self-declaration protocols Agency self-declaration protocols

8. Audit Process Planning the Audit

9. May 3, 20069 Audit Planning Addresses the specifics of who, what, where, when, and how Addresses the specifics of who, what, where, when, and how Must align with site procedure for auditing Must align with site procedure for auditing

10. May 3, 200610 Audit Planning Audit Team Audit Team Resources Resources Scope Scope Protocol Protocol Schedule Schedule Management approval Management approval Notify appropriate managers and staff Notify appropriate managers and staff

11. May 3, 200611 Audit Team Selection Must collectively have knowledge of: Must collectively have knowledge of: –EMS –Auditing –Processes or activities –Environmental legal and other requirements –Any unique, site-specific needs

12. Exercise No. 1

13. May 3, 200613 Potential Additional Audit Team Requirements Written appointments Written appointments Training Training Temporary release from duties Temporary release from duties Travel Travel

14. May 3, 200614 Potential Audit Resource Requirements Personnel Personnel Time Time Equipment Equipment Other? Other?

15. May 3, 200615 Scope/Objectives/Protocol/Schedule Documented Documented Approved by senior leadership Approved by senior leadership Communicated Communicated

16. Conducting the Audit

17. May 3, 200617 Meeting with Management Communicate audit process to management Communicate audit process to management Topics: Topics: –Audit objectives and protocol –Audit timeframe –Operations targeted –Personnel targeted –Audit schedule (where, when) –Close out meeting to review audit findings and discuss corrective action Advise and Consent Advise and Consent

18. Audit Step 1: Document Review

19. May 3, 200619 What Auditors Look for in Terms of Documentation Does your documented system include all the required elements? Does your documented system include all the required elements? Do the procedures describe what’s actually happening? Do the procedures describe what’s actually happening? Is there objective evidence that the procedures are being followed? Is there objective evidence that the procedures are being followed? Is the documentation controlled? Is the documentation controlled? Are related documents and records referenced? Are related documents and records referenced?

20. May 3, 200620 EMS Procedures Say what you do/Do what you say Say what you do/Do what you say Paper or electronic Paper or electronic Must be: –Consistent with significant aspects and objectives and targets –Usable and appropriate –Controlled –Maintained

21. May 3, 200621 Documentation Hierarchy Level I Level I Level II Level II Level III Level III –Work Instructions –Reference documents = Regulations, etc. –Records - Verify the system

22. May 3, 200622 Level 1 Description of scope Description of scope Policy Policy Aspects, significant aspects Aspects, significant aspects Objectives, targets, programs Objectives, targets, programs References to legal requirements References to legal requirements

23. May 3, 200623 Level 2 EMS Procedures EMS Procedures –Relatively short (1 to 3 pages) procedure for each element describing how this EMS addresses each element or requirement of the Standard –Certain specified records Training Records Training Records Progress toward achieving objectives and targets Progress toward achieving objectives and targets

24. May 3, 200624 Operational Controls Operational Controls –Functional instructions and specific how-to procedures for activities associated with significant aspects. Typical SOP’s, e.g., equipment operation, hazardous waste management etc. Reference Documents Reference Documents –Supporting documents procedures, e.g., regulations, plans, permits. Records Records –Required documentation including tracking of performance indicators, checklists, manifests, DMRs, senior leadership meeting minutes. Level 3

25. May 3, 200625 Typical Documents and Records to Review (by Element) Aspects Aspects –Aspects list –Aspects identification and Significance determination protocols –Significant aspects/impacts list Legal and Other Requirements Legal and Other Requirements –Access procedure –Register of applicable legal and other requirements –Appropriate instructions for compliance

26. May 3, 200626 Documents and Records (continued) Objectives, targets and programs Objectives, targets and programs –Procedure for or minutes/notes of meeting in which objectives and targets were developed –List of objectives and targets –Related programs/action plans Resources, roles, responsibility and authority Resources, roles, responsibility and authority –Assigned roles and responsibilities n Job descriptions n Organizational charts

27. May 3, 200627 Documents and Records (continued) Competence, Training and Awareness Competence, Training and Awareness –Training needs listings/matrix –Manuals, course materials –Sign-in sheets –Training records, certificates etc. Communication Communication –Procedure for and records of internal communication –Procedure for and records of external communication –Documented decision on external communication of significant aspects

28. May 3, 200628 Documents and records continued Documentation Documentation –All required elements and procedures Control of Documents Control of Documents –Instructions for document approvals –Revision status identified –Available at point of use –External documents controlled –Obsolete documents clearly identified

29. May 3, 200629 Documents and records continued Operational Control Operational Control –Specific environmental controls in work instructions for activities associated with significant aspects –Contractor policies, work orders, etc. –Supplier requirements

30. May 3, 200630 Documents and records (continued) Emergency Preparedness and Response Emergency Preparedness and Response –Emergency plans and protocols –Documented review and corrective action following events –Results of exercises and drills Monitoring and Measurement Monitoring and Measurement –Progress toward achieving objectives and targets –Specific procedures and work instructions –Data for performance indicators –Records of equipment calibration and maintenance

31. May 3, 200631 Documents and records (continued) Evaluation of compliance Evaluation of compliance –Findings from annual internal assessments of compliance with legal and other requirements Non-conformance, Corrective and Preventive Action Non-conformance, Corrective and Preventive Action –Evidence of assignment of responsibility for corrective/preventive action –Reports on progress in implementing corrective and preventive action –Evidence of reviews of effectiveness of corrective and preventive action (meeting notes etc.)

32. May 3, 200632 Documents and records (continued) Records Records –Required records documented –Required records legible and accessible EMS Audit EMS Audit –Specific audit procedures, checklists, forms –EMS audit findings and notes –EMS audit reports

33. May 3, 200633 Documents and records (continued) □ Management Review nMeeting agendas, minutes and action items nEvidence of follow-up actions, reports etc.

34. Audit Step 2: Conducting Interviews

35. May 3, 200635 Typical Interviewees: EMS Manager EMS Manager Cross-functional Team members Cross-functional Team members Director of Environmental Program Director of Environmental Program Environmental Program staff Environmental Program staff Commander/Director Commander/Director Director/Manager of each organization associated with significant aspects Director/Manager of each organization associated with significant aspects Emergency Services Emergency Services Education and Training Education and Training Construction and Facilities Management Construction and Facilities Management Safety and Occupational Health Safety and Occupational Health Contracting and Procurement Contracting and Procurement Public Affairs Public Affairs Shop Chiefs representing each activity associated with significant aspects Shop Chiefs representing each activity associated with significant aspects Personnel representing each activity Personnel representing each activity

36. May 3, 200636 How to Interview Explain the reason for the interview Explain the reason for the interview Ask the person to describe their work Ask the person to describe their work Address EMS concerns Address EMS concerns Summarize interview results Summarize interview results Respond to any questions or concerns Respond to any questions or concerns Thank the interviewee for his/her cooperation Thank the interviewee for his/her cooperation

37. May 3, 200637 Ask good questions Avoid any question that can be answered simply "yes" or "no" Avoid any question that can be answered simply "yes" or "no" Research first Research first Use the "why" approach Use the "why" approach Two-way communication Two-way communication

38. May 3, 200638 Typical EMS Audit Questions 1. How did you become aware of the Environmental Policy? 2. How does the Policy apply to you and your work? 3. What are the environmental aspects/impacts of your work? 4. What EMS objectives/targets apply to your work? 5. Do you follow this procedure? 6. How does your work vary from the procedures? 7. What kind of environmental training have you received? 8. How do you communicate environmental concerns to management? 9. How would you respond if a neighbor asked about environmental problems at this facility?

39. Writing Audit Findings Audit Step 3:

40. May 3, 200640 EMS Audit Findings: Conform or Non-Conform Audit evidence is evaluated against the audit criteria/protocol Audit evidence is evaluated against the audit criteria/protocol Audit findings indicate that the procedure and the EMS either conform or fail to conform to the audit criteria/protocol and/or the Standard Audit findings indicate that the procedure and the EMS either conform or fail to conform to the audit criteria/protocol and/or the Standard Auditors work as a team to develop the findings Auditors work as a team to develop the findings

41. May 3, 200641 Types of EMS Non-conformance Findings Findings –No procedure –Procedure does not meet the requirements of the Standard –Organization fails to meet the requirements of the procedure –Requirements of the procedure are inadequate Observations Observations –Recommendations for improvement

42. May 3, 200642 Finding Management Review Management Review The Management review procedure states that the facility's Management Council will conduct a review of the EMS on an annual basis and record recommendations in the meeting minutes. Following the review, the facility EMS Manager will develop an action plan that defines the courses of action, responsibilities, and timeframes for completion. No records of Management Review are available. Management Council meeting minutes for the past four quarters do not mention a review of the EMS. The audit team could not locate any recommendations or an action plan. No records of Management Review are available. Management Council meeting minutes for the past four quarters do not mention a review of the EMS. The audit team could not locate any recommendations or an action plan.

43. May 3, 200643 Observation Internal EMS Audit Internal EMS Audit Audit procedures have been developed and an EMS audit has been conducted. The audit scope covered all areas of the facility against all criteria of the standard. Auditors have received appropriate training. An audit report outlines the findings. Corrective action for audit findings appear to be in process. –Audit procedures are not detailed enough to demonstrate how the audit was conducted (what records were reviewed, personnel interviewed, departments visited). –Audit records do not show which auditor reviewed which area, to demonstrate independence of auditors.

44. May 3, 200644 Non-Conformance vs. Non- Compliance An EMS non-conformance indicates that the EMS is not doing what it should be doing An EMS non-conformance indicates that the EMS is not doing what it should be doing An organization may have regulatory non- compliances but no EMS non- conformances, and vice versa. An organization may have regulatory non- compliances but no EMS non- conformances, and vice versa.

45. Audit Step 4: Writing an Audit Report

46. May 3, 200646 The Audit Report The audit findings should be communicated in a written report submitted to the EMS Manager The audit findings should be communicated in a written report submitted to the EMS Manager Audit reports may include: Audit reports may include: –Description of the audit objectives and scope –Audit criteria/protocol –The date(s) the audit was conducted and the period covered by the audit –Names and duties of audit team members –Names and duties of facility staff interviewed

47. May 3, 200647 The Audit Report Summary of the audit process, including any obstacles encountered Summary of the audit process, including any obstacles encountered Audit findings Audit findings Audit conclusions Audit conclusions –Whether the system is properly implemented and maintained –Whether the internal management review process is able to ensure the continuing suitability and effectiveness of the EMS –EMS conformance to the EMS audit criteria or the Standard

48. May 3, 200648 Distribution of the Audit Report Determined in accordance with the audit plan Determined in accordance with the audit plan Document control? Document control? Document retention? Document retention?

49. May 3, 200649 Closing Meeting Recap of audit scope, objectives, and plan Recap of audit scope, objectives, and plan Overview of findings Overview of findings Any special issues or circumstances Any special issues or circumstances Initial discussion of corrective action, as appropriate Initial discussion of corrective action, as appropriate