1. A Feedback and Continuous Improvement Tool to ACE Audit Readiness Keith S. Joy Quality Manager September 14, 2004

2. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 2 Title_date DOE OA-40 DOE Office of Independent Oversight and Performance Assurance Integrated Safety Management (ISM) System Evaluation

3. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 3 Title_date ORNL Profile  Conduct basic and applied research that provides innovative solutions to complex problems  3800 staff members  Scientific competencies include  Neutron Science  Energy  High Performance Computing  Complex Biological Systems  Advanced Materials  National Security

4. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 4 Title_date Facilities and Operations Profile  Provides facilities and operating services to support research and development  930 total employees (527 bargaining unit)  Employee types:  Managers  Administrative staff  Engineers  Technicians  Bargaining Unit – Pipefitters, Carpenters, Laborers, Janitors, Painters, Millwrights, etc.

5. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 5 Title_date Facilities and Operations Directorate Chief Engineer Matrix Support F&O Director Reservation Manager Craft Resources Division Fabrication Division Facilities Management Division Facilities Development Division Laboratory Protection Division Logistical Services Division Nonreactor Nuclear Facilities Division Integrated Operations Support Division Facilities Strategic Planning

6. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 6 Title_date The Request Take the lead in preparing the entire organization for the evaluation and help ensure that the group doesn’t look bad.

7. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 7 Title_date Initial Preparation Thought Mimic upcoming audit Obtain Lines of Inquiry Conduct series of self-assessments Submit a Final Report of findings Schedule/Track corrective actions

8. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 8 Title_date Field interviews Often unescorted Unannounced interviews with ANY staff member Interviews focused on ISM on the tasked being conducted Not much time spent on procedure reviews The OA-40 audit method

9. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 9 Title_date The Problem How do you ensure that 930 employees of different job types understand, and can articulate, how ISM controls are implemented for any job task that they conduct? Training in ISM and work control has already occurred In less than 5 months Minimal disruption of productive time

10. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 10 Title_date The Challenge Find an efficient method to simultaneously Assess worker knowledge of actual hazards and hazard controls for the tasks they perform Correct any worker misunderstandings Educate worker on the “how” and the “why” controls are in place

11. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 11 Title_date The Solution Mimic the approach to be used by the upcoming evaluation by conducting field interviews of workers to ASSESS worker knowledge And for readiness… CORRECT any misunderstandings and EDUCATE workers on hazard controls The ACE Concept

12. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 12 Title_date To deploy our ACE concept, we  Developed a tool to guide the assessment  Conducted orientation sessions to acquaint managers with the procedure  Programmed a database to capture results  Created a performance metric to illustrate areas of opportunity

13. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 13 Title_date To develop the tool, we obtained questions from  Other site’s escorts  Our staff members who were previous OA-40 auditors  An analysis of our internal controls  Previous OA-40 reports

14. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 14 Title_date Our ACE Card

15. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 15 Title_date ACE Procedure  Be acquainted with questions on checklist  Set aside time to conduct field assessment  Select worker, unannounced, while conducting work  Enter into a non-threatening, two way dialog guided by the questions

16. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 16 Title_date ACE Procedure (con’t)  Immediately correct wrong responses  Educate worker on the reasons for controls  Thank worker and encourage worker to continue safe performance  Complete online ACE Card

17. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 17 Title_date ACE Results  Performance metric  Ad Hoc Reports  Corrective Actions

18. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 18 Title_date OA-40 Feedback Quote from OA-40 final report

19. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 19 Title_date ACE Concept Benefits  Adapts to different areas  Demonstrates management presence in field  Promotes worker feedback  Assesses actual worker knowledge  Allows trending of opportunities for improvement  Provides a quick, effective self assessment

20. O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 20 Title_date