Document Evaluation Process May 2005 Revision 8T8L May 2005 Revision

Upon successful completion of this topic, you should be able to: Describe the preparation for onsite document review Discuss the ORR/RA principle of sampling versus auditing Discuss the complete trail sampling technique Discuss good Team Member record / note keeping practices related to records review 8T8L May 2005 Revision

Review Preparation Flow Chart Requirements Objective Core Orders, Codes, Criteria Requirements Standards, Regulations Identification of Required Program & Processes Identification of important implementing Procedures / Policies Identification of important Systems, Structures, Components Identification of important Job Factors 8T8L May 2005 Revision

Review of Procedures What procedures* are required? * All documented, implementing mechanisms . . . Implement Regulatory Drivers . . . Programs and Processes . . . Implement Commitments . . . DSA/SAR / TSR / Accident assumptions; BIO / JCO / FHA, and EIS; List A/B, WSS, S/RIDs; Company / Operations Office procedures & requirements . . . Control Procedures, those not specifically required, but needed because of complexity, variability, risk, quality: Control and guide ops and activities to achieve desired outcome 8T8L May 2005 Revision

Review of Procedures (continued) Document Reviews do not have to be line by line copies of Order or regulatory driver, but must: Detail the mechanisms to fulfill the intent and requirements of the regulatory/contract driver . . . Detail the mechanisms and requirements to adequately control and document the activity . . . Focus on implementation . . . 8T8L May 2005 Revision

Review of Procedures (continued) Define the responsible individual(s) for each specified requirement . . . Be adequately communicated to and understood by those identified individuals . . . Comply with form and content guidance consistent with department and local expectations . . . 8T8L May 2005 Revision

Programs to Control Procedure Adequacy Review system: Procedures are adequate? V&V in the field? Configuration control: Procedures relate to systems; commitments identified? Formal change control? Adequate supervisory control: Ensure adequate implementation & performance of procedures? Issue and use procedures ensure only current revision in use? Training / Qualification and Assessment programs: Ensure procedures are adequately understood? 8T8L May 2005 Revision

Performance Review Flow Top Tier Requirement Performance Review Flow DOE / Operations Office Requirement Requirement Specific Contract (what) Requirements Company Policy / Procedure Manual Implementation Specific Project Procedures (how) Line Instructions Result Project Records

Sampling Not a complete document audit . . . Sample and expand if needed . . . Sampling may include complete vertical slice: Engineered change; USQ screening; Work package; 8T8L May 2005 Revision

Your Notes Detailed enough to reconstruct . . . Be able to describe the hierarchy and flow of requirements from highest directive to implementation . . . 8T8L May 2005 Revision

Step 1 -- Preparation At the programmatic or activity level select the target document(s) for review: Which are the best indicators of the program? Which are important to: Environment, safety and health? Safeguards and security? Mission? Prioritize documents to review . . . 8T8L May 2005 Revision

Step 1 -- Preparation (continued) Communicate with site counterpart to determine: appropriate site documents; custodians & control mechanisms of documents. Communicate with Team: avoid duplication of review; ensure that team members are aware of documents that may affect their review, and don't assume they know. Records of Performance comply with Requirements 8T8L May 2005 Revision

Step 2 --Document Review Read and understand the documents . . . Determine if document contains required information, from developed criteria . . . Determine how document is controlled . . . 8T8L May 2005 Revision

Step 2 -- Document Review (continued) Determine if document is consistent with facility and activities: Equipment/hardware/software; Personnel; Higher tier commitment; Related programmatic and performance requirements. Records of performance comply with Requirements? 8T8L May 2005 Revision

Step 3 -- Follow-up Check beyond the document evaluation: Training; Controls to ensure procedures are followed; Document configuration control. Check compliance of base level standards to codes, standards, regulations . . . Consider for interview topics . . . 8T8L May 2005 Revision

Step 4 -- Reporting In addition to the basic process reporting: Record the information relating to the review criteria; Check reference information; Integrate information gained into observation and interview processes; Discuss conclusions with Team Members, as appropriate; Meet expected Team Report and Forms 1 and 2 format. 8T8L May 2005 Revision

Summary… Does the documentation accurately and completely implement the requirements: Contractual? Authorization Basis? Programmatic? Performance? Is the documentation being implemented and requirements met? Is “Flow-Down” consistent, complete, and accurate? 8T8L May 2005 Revision