1. DOECAP Pilot Audit – Phase I The Saga of a Thorough, but Inefficient Audit
September 2016

2. September 2015 Annual QA Planning Session
2016 objectives and plans set for QA organization
Including audits by Larry Penfold, one scheduled for Denver lab
Audit to be done using TestAmerica (TA) SOP and checklist developed over last 15 years

3. January-February 2016
Received procedure no. AD-1-2A “Conducting DOECAP Laboratory and TSDF Pilot Audits”
Denver lab informed selected to participate in a “Phase I” pilot audit
Received DOECAP checklists
Happy New Year

4. March 2016 Plan for Denver Lab Audit
Objective: Dual audit covering both TA QA program requirements and all DOECAP requirements
Misfit?
Shouldn’t be: Based on same standard (ISO, TNI, QSM/QSAS)
Only DOECAP Module 6 is not covered in TA QA audits

5. March 2016 Compared DOECAP/TA Checklists
In most cases 1 line of inquiry satisfies both
Biggest difference is organization of checklists
TA uses separate checklist per lab section
DOECAP Checklist Modules
Corresponding TestAmerica Checklists
1.Quality Assurance
1 - QA, Management Systems, & General Lab Practices
2. Project Management & Archives
3. Purchasing
4. Bottle Prep
5. Sample Control
6. Organic Extractions
7. TCLP
8. Semivolatiles – GC
2 - Organic Analyses
9. Semivolatiles - GC/MS
10. HPLC/LCMS
11. Volatiles – GC
12. Volatiles - GC/MS

6. March 2016 Each Line of Inquiry Compared
2016 DOECAP Module 2 - Organic Analyses
2015 TA Audit Checklists
  checklist
5 organic dept., 5 checklists
Section #
Areas of Review
1.0
SOPs
In each department cklist
1-14, 39-43
2.0
Sample Preparation/Extraction
Organic Extractions
All sections
3.0
Cleanup Methods 81
4.0
Initial and Continuing Calibration
~45-72
5.0
Standards and Reference Materials
~28-46
6.0
Method Proficiency
~44 or 50
7.0
Limit of Detection (LOD) and Limit of Quantitation (LOQ)
Quality Assurance
99-105
8.0
Precision and Accuracy
92-97
9.0
Instrument Run Logs
~68-74
10.0
Carryover Minimization
~74
11.0
Spike Recovery Requirements
~75-80
12.0
Second-Column Confirmation
65, 83
13.0
Method Performance
84-90 or 92-97
14.0
Analytical Worksheets
not explicitly covered
n/a
15.0
Instrument Operation
~52-56
16.0
Contamination Control
109
17.0
Data Review
~98-109
Here only 1 DOECAP line of inquiry not explicitly covered in TA checklists
Easy to modify audit to include the few extra lines of inquiry

7. March 2016 Audit Conducted
On-site effort was similar to typical time for the on-site by a DOECAP audit team:
5 people involved, covering 10 lab areas
Total time approximately 90 hours
One important difference as to what had to be done to document the audit…

8. Size of Completed Checklists
Guess which stack is DOECAP’s and which is TA’s?

9. Completed Checklists In My Usual Fashion
Handwritten notes for observations and findings
But not all details entered when requirements were met and I knew where the supporting evidence could be found
Sufficient notes to conduct de-brief
and prepare final report
Audit completed April 7, 2016
Audit report issued April 25, 2016
In May I was pulled to other projects

10. The Big Crunch
Crash effort to type the DOECAP checklists in early July
For each question, explained exactly how the lab is compliant or not
- Then list source of
supporting evidence,
such as SOP#, log ID,
section# QA Manual…
20-30 hrs of work

11. The Bad Part Made it unnecessarily tough for DOECAP team!
Left them only week to review a lb package

12. The Best Part July 12-14 on-site audit by DOECAP
August 2016 received letter that the Denver lab (TAA) successfully completed Phase I, and approved for a Phase II limited scope audit in 2017

13. Observations
TestAmerica’s audit checklists cover almost all the same ground, but
DOECAP’s checklists are long and have more detail, particularly for systems requirements, with some redundant questions
DOECAP’s checklists are in-depth in some technical areas, such as radiochemistry, but also
Light in some technical areas, such as TCLP

14. Make It Easy for the DOECAP Auditors
Main Lesson Learned
Make It Easy for the DOECAP Auditors
Don’t be like me
Budget enough time to do a good job on the checklists

15. Postscript & Additional Lessons Learned
Phase I pilot audit conducted last week at another TA lab
This time only used 1 checklist
TA’s internal audit procedure now recognizes audit conducted with DOECAP checklist + some additions
Checklist to be completely written prior to final report

16. Questions?