Quality Control/ Quality Assurance Annabelle Allison ITEP/TAMS Center

2 Quality Assurance PLAN DO CHECK FIX KEEP DOING IT!

3 Quality Assurance Definitions Common sense written down “Integrated system of management activities involving planning, implementation, assessment, reporting, and quality improvement to ensure that a process, item, or service is type and quality needed and expected by client”

4 Quality System Structured, documented management system Describes policies, principles, organizational authority for ensuring quality needed in product Provides framework for planning, doing, documenting, & assessing work and for carrying out QA/QC activities

5 Quality Control (QC) Technical activities Measures performance of the process Compares against defined standards “Customer” sets standards & requirements QC is part of QA

6 Quality Control Check (Measure… compare with limits… graph it… document and report) Fix

7 Quality System: management (common sense) Assurance: policies (serve our community to produce needed result) Control: assessments (we check up on our work)

8 American National Standard ANSI/ASQC E Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs National consensus standard authorized by American National Standards Institute (ANSI) Developed by American Society for Quality Control (ASQC) Consistent with ISO standards EPA Guidance for Quality Assurance Project Plans: EPA QA/G-5 based on E4

9 Why is a QAPP Important? Saves money Saves time Helps communication No later misunderstandings Everything contained in or referenced in one document Public document in case of litigation

10 Using SOPs Easier to write than QAPPs Describe what is done Once SOPs are written, QAPP is much easier Reference and attach SOPs to QAPP

11 Using Existing Materials Plagiarism when writing QAPPs and SOPs is okay! Guidance for the Preparation of Standard Operating Procedures (SOPs) for Quality-related Documents: EPA QA/G-6 Use template QAPPs for BAM and TEOM in TAMS diskette

12 6 Elements of Data Management (1) Data processing and transmission (WHAT) (2) Data end use and integrity protection (WHY) (3) Data access (WHO)

13 6 Elements of Data Management (cont.) (4) Data dissemination (WHERE) (5) Data storage and retrieval (HOW) (6) Data disposal (WHEN) Document everything!

14 Error …the difference between your answer and the “truth”

15 Continuous PM QC Instrument stability checks (EPA calls this general QA in their 1995 memo; see Attachment A in the TAMS template QAPPs) Precision Bias

16 Instrument Stability Internal diagnostics conducted with each measurement (no error codes) TEOM uses standard filter kit available from dealer BAM uses standard foil or internal source Use these stability test kits more frequently at beginning of program, thereafter as necessary and at least twice a year

17 Results of Stability Checks Use dealer’s guidelines Within 2.5% for TEOM standard filter Within 7% for BAM foil Plot results on control chart (see Excel table and linked chart as figure 14-1 in TAMS template QAPPs)

18 Precision Error Unavoidable Sometimes up, sometimes down– “random” Estimated by measuring the same thing several times Minimized by carefully following procedures

19 Precision Collocated analyzers is too expensive, so use repeated measurements of the same thing to estimate precision Flow rate is the most important parameter for PM Flow rate changes with time—how much it varies is estimate of precision error

20 Two Ways To Estimate Precision of Flow Rate At beginning of program, use external flow rate transfer standard in field and compare flow rate transfer standard’s result with that shown (or set) on analyzer (one-point check) Conduct this check at least every 2 weeks

21 During the First Months Ensure external flow rate transfer standard is checked against another flow rate transfer standard at least once (every six months if you want to use second, easier way to assess precision) Record every check of analyzer’s flow rate, verify it’s within 4% of external flow rate transfer standard’s flow rate for 3 consecutive biweekly checks

22 Easier Way To Estimate Precision after Several Months Use “alternative procedure:” Record analyzer’s set point flow rate, assume same flow rate if measured by external flow rate transfer standard Record analyzer’s indicated flow rate Calculate relative percent difference, use value as precision error of flow rate (use relative percent difference calculation; see cells B5 – G28 in the Tribal Data Analysis spreadsheet, pg “PM 10 Automated”)

23 Bias Error Minimized by calibrating equipment against a standard Make sure standard is certified against NIST-traceable standard (keep certificate!) Compare transfer standard to field equipment Measures any bias in your equipment

24 Definitions EPA calls “accuracy” what others call “total error” Accuracy, or total error, combines both precision and bias errors

25 Continuous PM Accuracy Can estimate only accuracy of flow rate Most important parameter, so that’s okay Use external flow rate transfer standard to compare indicated (or set) flow rate of analyzer with that shown by transfer standard

26 Accuracy (cont.) Flow rate transfer standard must not be same used to calibrate analyzer Audit may be conducted by tribal staff, but not routine site operator Conduct flow rate audit at least once per year for each analyzer

27 Accuracy Calculations Relative percent difference between audit flow rate transfer standard and your analyzer’s flow rate is estimated accuracy of your flow rate No other (inexpensive) way to estimate accuracy, so this is your estimated accuracy, or total error