1. Click to edit Master title style Click to edit Master title 1 1 ISM Best Practices Workshop September 13, 2006 John L. McCabe, P.E., CSP Office of Corporate Safety Analysis DOE/NNSA Corrective Action Effectiveness Review

2. Click to edit Master title style Click to edit Master title 2 2 Effectiveness Review Effectiveness Review (ER) is most frequently cited weakness of feedback and improvement cycle due to lack of followup and recurrence of identified findings Columbia Shuttle and Davis-Besse reactor corrosion issues are 2 of most well-known of identified and reported problems lacking ER followup

3. Click to edit Master title style Click to edit Master title 3 3 Discussion Effectiveness Review process is 4 th step of feedback and improvement core safety function, discussed as part of Corrective Action Management Program (CAMP) Feedback and Improvement Cycle CLOSE FINDING (Verification) EVALAUTE FINDING (Disposition) RESOLVE FINDING (Implementation) IDENTIFY FINDING (Review) Review /Report Effectiveness of Corrective Actions Corrective Action Tracking and Implementation Assessment /Event Report Corrective Action Plan

4. Click to edit Master title style Click to edit Master title 4 4 CAMP Background Systematic process to effectively resolve: SP-40 ES&H and Emergency Management findings Type A Accident Investigation Judgments of Need Office of Aviation Management findings Findings directed by the Secretary or Deputy Secretary CAMP initiated in response to DNFSB Recommendation 98-1 CAMP responsibilities and requirements in DOE O 414.1C Resources: CA Program Guide (DOE G414.1-5); website: www.eh.doe.gov/camp

5. Click to edit Master title style Click to edit Master title 5 5 Effectiveness Review Objectives verify successful finding closure. determine if CA resolved issue and prevented recurrence if CA was not effective, why not prepare revised/additional CA to improve effectiveness & resolve finding collect followup data for subsequent analysis and/or lessons learned

6. Click to edit Master title style Click to edit Master title 6 6 Examples of Effectiveness Reviews Documentation Review Performance Indicators Walkaround/Facility Tours Performance Test/Field Ex Interviews

7. Click to edit Master title style Click to edit Master title 7 7 Possible Causes for Ineffective CA Causal factors incorrectly identified. Causal factors correctly identified, but CA inappropriate CA not fully implemented as stated in the CAP CA implementation not timely CA creating new/different problems Org/personnel reject ownership

8. Click to edit Master title style Click to edit Master title 8 8 Effectiveness Review Ratings Effective: CA closed and findings are resolved Partially Effective: CA closed and findings partially resolved w/additional or revised CA needed Ineffective: CA did not resolve finding and/or will not prevent recurrence of finding

9. Click to edit Master title style Click to edit Master title 9 9 Fixing Unresolved Findings Re-analyze causal factors Re-examine CA and determine why finding was unresolved Develop/implement revised or additional CA Conduct followup effectiveness review

10. Click to edit Master title style Click to edit Master title 10 CAMP Effectiveness Review Requirements Field Element Manager (FEM) reviews CA effectiveness for each finding FEM decides how, when, and by whom review conducted Report approved by FEM 6 months after CAP completion Report describes review activities, results, conclusions, rating, and recommendations for each finding Approved report recommendations implemented and followed up

11. Click to edit Master title style Click to edit Master title 11 CAMP Effectiveness Review Results 10 completed since old QA Order (O 414.1-B) revised in APR 2004 7 open (3 late and 4 on schedule) Managers are complying and experiencing results Feedback from field has helped improve ER process (As of 8/22/06)

12. Click to edit Master title style Click to edit Master title 12 Conclusions Purpose and objectives of Effectiveness Reviews (ER) are being better understood Effective CA improves safety, optimizes operational performance, and reduces costs Must maintain management involvement and ER emphasis