DOECAP Pilot Audit – Phase I The Saga of a Thorough, but Inefficient Audit September 2016

September 2015 Annual QA Planning Session 2016 objectives and plans set for QA organization Including audits by Larry Penfold, one scheduled for Denver lab Audit to be done using TestAmerica (TA) SOP and checklist developed over last 15 years

January-February 2016 Received procedure no. AD-1-2A “Conducting DOECAP Laboratory and TSDF Pilot Audits” Denver lab informed selected to participate in a “Phase I” pilot audit Received DOECAP checklists Happy New Year

March 2016 Plan for Denver Lab Audit Objective: Dual audit covering both TA QA program requirements and all DOECAP requirements Misfit? Shouldn’t be: Based on same standard (ISO, TNI, QSM/QSAS) Only DOECAP Module 6 is not covered in TA QA audits

March 2016 Compared DOECAP/TA Checklists In most cases 1 line of inquiry satisfies both Biggest difference is organization of checklists TA uses separate checklist per lab section DOECAP Checklist Modules Corresponding TestAmerica Checklists 1.Quality Assurance 1 - QA, Management Systems, & General Lab Practices 2. Project Management & Archives   3. Purchasing 4. Bottle Prep 5. Sample Control 6. Organic Extractions 7. TCLP 8. Semivolatiles – GC 2 - Organic Analyses 9. Semivolatiles - GC/MS 10. HPLC/LCMS 11. Volatiles – GC 12. Volatiles - GC/MS

March 2016 Each Line of Inquiry Compared 2016 DOECAP Module 2 - Organic Analyses 2015 TA Audit Checklists   1 checklist 5 organic dept., 5 checklists Section # Areas of Review 1.0 SOPs In each department cklist 1-14, 39-43 2.0 Sample Preparation/Extraction Organic Extractions All sections 3.0 Cleanup Methods 81 4.0 Initial and Continuing Calibration ~45-72 5.0 Standards and Reference Materials ~28-46 6.0 Method Proficiency ~44 or 50 7.0 Limit of Detection (LOD) and Limit of Quantitation (LOQ) Quality Assurance 99-105 8.0 Precision and Accuracy 92-97 9.0 Instrument Run Logs ~68-74 10.0 Carryover Minimization ~74 11.0 Spike Recovery Requirements ~75-80 12.0 Second-Column Confirmation 65, 83 13.0 Method Performance 84-90 or 92-97 14.0 Analytical Worksheets not explicitly covered n/a 15.0 Instrument Operation ~52-56 16.0 Contamination Control 109 17.0 Data Review ~98-109 Here only 1 DOECAP line of inquiry not explicitly covered in TA checklists Easy to modify audit to include the few extra lines of inquiry

March 2016 Audit Conducted On-site effort was similar to typical time for the on-site by a DOECAP audit team: 5 people involved, covering 10 lab areas Total time approximately 90 hours One important difference as to what had to be done to document the audit…

Size of Completed Checklists Guess which stack is DOECAP’s and which is TA’s?

Completed Checklists In My Usual Fashion Handwritten notes for observations and findings But not all details entered when requirements were met and I knew where the supporting evidence could be found Sufficient notes to conduct de-brief and prepare final report Audit completed April 7, 2016 Audit report issued April 25, 2016 In May I was pulled to other projects

The Big Crunch Crash effort to type the DOECAP checklists in early July For each question, explained exactly how the lab is compliant or not - Then list source of supporting evidence, such as SOP#, log ID, section# QA Manual… 20-30 hrs of work

The Bad Part Made it unnecessarily tough for DOECAP team! Left them only 1 week to review a 3.85 lb package

The Best Part July 12-14 on-site audit by DOECAP August 2016 received letter that the Denver lab (TAA) successfully completed Phase I, and approved for a Phase II limited scope audit in 2017

Observations TestAmerica’s audit checklists cover almost all the same ground, but DOECAP’s checklists are long and have more detail, particularly for systems requirements, with some redundant questions DOECAP’s checklists are in-depth in some technical areas, such as radiochemistry, but also Light in some technical areas, such as TCLP

Make It Easy for the DOECAP Auditors Main Lesson Learned Make It Easy for the DOECAP Auditors Don’t be like me Budget enough time to do a good job on the checklists

Postscript & Additional Lessons Learned Phase I pilot audit conducted last week at another TA lab This time only used 1 checklist TA’s internal audit procedure now recognizes audit conducted with DOECAP checklist + some additions Checklist to be completely written prior to final report

Questions?