Presented by: Rick Kendall, NA-17 Integrated Safety Management Best Practices Workshop September 2006; Denver Co. Assessment Criteria and Guidelines for Performing Assessments of the Effectiveness of Incorporation of Integrated Safety Management (ISM) and Quality Assurance (QA) Principles Into Activity Level Work Planning and Control at NNSA Sites Known as the “Appendix B CRAD” (Criteria, Review, and Approach Document) because it is Appendix B to the NNSA Document: “Activity Level Work Planning and Control Processes - Attributes, Best Practices, and Guidance for Incorporation of Integrated Safety Management and Quality Assurance” (dated January 2006) NNSA CRAD for Assessing Activity Level Work 1

Activity Level Work Planning and Control Processes Attributes, Best Practices, and Guidance for Effective Incorporation of Integrated Safety Management and Quality Assurance NNSA Document January 2006 Attributes There are 40 attributes listed among the following seven headings: 1. Activity Level Work Planning and Control Process; 2. Identify, Prioritize, and Approve Work; 3. Scheduling; 4. Planning Work; 5. Readiness; 6. Performing Work; and 7. Post Work Review and Closeout. Incorporation of the attributes into work planning and control processes will help to ensure that ISM and QA requirements are met at the activity level. Best Practices and Guidance Best practices and guidance information is provided under each of the attributes to help implement them, and thus to meet associated requirements. The best practices/guidance provides suggested approaches/acceptable methods for meeting DOE/NNSA requirements and expectations. CRAD (Appendix B) focused at the work activity level; will complement existing site ISM processes that flowdown nfrequirements from the site and facility levels; interact with workers, work planners, and work supervisors; not resource intensive (people, time, or documentation) compared to mother assessments; use standard forms (part of the CRAD); and results can be used to fulfill other assessment requirements where mappropriate. 2

3 NNSA Document with Appendix B CRAD

4 Appendix B CRAD Scope Focused at the Activity Level - where the problems are! Used for Assessing All Types of Activity Level Work Operations Research and Development Maintenance Construction Modifications Used to Assess a Single/Specific Work Activity

5 Purpose/Goals Ensure reliable, consistent work planning and control performance nfconsistent with requirements/expectations so that mission work is nfaccomplished safely and efficiently Interact with Workers, Work Planners, and Work Supervisors - ensure requirements/expectations appreciated/understood - use their knowledge/experience to improve work planning and nfcontrol process Requirements: - 48 CFR and 48 CFR and CFR DOE O414.1C - 10 CFR 851 Guidance: - DOE G B Vol.

6 Different Assessment Approach No pre-visit, plan of action, entrance meeting, or large team of people on-site nffor a couple of weeks Thorough and Focused (specific work activity) - Generally does not look at ISMS, QAP, or site level or facility level nfpolicies/procedures not germane to work activity being assessed Can be performed by one or two people - Site Office, Contractor, or combination Non-Intrusive - limited/minor impact on mission/facility Results documented using standard forms Documentation requirements are minimal - provide objective evidence where criteria are met - sufficient detail to understand and correct issues - additional explanation where needed Needed flexibility built in (graded approach) Typically short duration (roughly equivalent to work activity) Can be used as input to other assessments/oversight activities Can be used as training tool

7 Assessment Method Comparison Assessment Methodology/Approach Typical Team Assessment (ORR, ISM, SSA/SP, Phase II VSS, etc.) Appendix B CRAD Plan of Action RequiredYesNo Entrance Meeting/Daily Meetings Required YesNo Use of Criteria, Review, and Approach Document (CRAD) Yes Conduct document reviews, interviews, and observations Yes Identified Issues tracked in Site- wide Integrated Issues Management System Yes Preparation and use of qualified and independent assessors Yes Requires Team Leader and large team (e.g., 5-15 people) YesNo Detailed documentation of assessment required Yes Only if criteria are not met, and where explanation is needed Performance FrequencyInfrequent/As Necessary Several times per year at least/As Necessary

8 Approach Benefits Allows assessments to be performed more frequently Can assess different organizations, departments, divisions, crews, teams, nfsubcontractors, etc. Can interact with more workers, planners, and supervisors over shorter nfperiod of time Provides consistent reinforced set of expectations across site and nfcomplex (same rules when move to different job) Works well with other Site Office Action Plan ( #23) actions (e.g., nftraining and qualification of work planners; management on the floor nfinteraction with workers to reinforce ISM; etc) Leads to improvements in work processes (ISM/QA more effectively nfimplemented on the floor) and activity level work (repeat work and nfpersonnel efficiency) Repeated use over time ensures/promotes continued excellence

9 Assessment Form Sample Pages

10 Conclusion The Department’s Implementation Plan for DNFSB Recommendation , Oversight of Complex, High-Hazard Nuclear Operations, states that improvement in reliability and consistency of work planning and work control performance at the activity level is needed, and that that the current ISM system contains minimal expectations, and no explicit requirements, at any level to routinely assess the implementation of work planning and work control processes at the activity level. Institutionalization of the NNSA Appendix B CRAD ( #23 Site Office Action Plans) is intended to address these concerns by helping to achieve needed improvements and ensure that all future work is performed in a safe, efficient, quality manner.

11 Future Work Revise the January 2006 Activity Level Work Planning and Control Processes - Attributes, Best Practices, and Guidance for Incorporation of Integrated Safety Management and Quality Assurance document, including the Appendix B CRAD “Assessment Criteria and Guidelines for Performing Assessments of the Effectiveness of Incorporation of Integrated Safety Management and Quality Assurance (QA) Principles Into Activity Level Work Planning and Control at NNSA Sites” based on: Feedback (comments from reviewers and users - both the main nfdocument and the Appendix B CRAD) DNFSB Tech-36 Human Performance Improvement Principles 10 CFR 851 Worker Safety and Health Rule ISM Champions Workshops (April 2006 and September 2006) Please provide feedback/comments/suggestions for improving these documents to: or )

12 Final Thought - Leading Indicators Assessments conducted using the NNSA Appendix B CRAD can be used to identify real-time leading performance measures/indicators of safety culture problems: Failure to implement ISM - Failure to adequately define the scope of work - Working outside the approved envelope (scope creep) - Failure to identify all hazards (work and site) - Failure to work within approved controls Failure to implement Conduct of Operations - Lockout/Tagout violations - Failure to follow procedures - Failure to properly perform independent verifications - Failure to record field conditions, errors, near misses, etc. - Failure to ensure unexpected conditions are analyzed and understood before nfproceeding As compared to conventional reporting mechanisms (e.g., ORPS), collecting and analyzing this information may provide a more accurate and leading indication of pending safety problems, and lead to taking corrective actions that prevent future unacceptable outcomes/events