QA Strategy Workgroup Ambient Air Quality System Training Ambient Air Monitoring Quality Systems Requirements Mike Papp- OAQPS

QA Strategy Workgroup Ambient Air Quality System Training 2 CAA- Clean Air Act “(2) Establishment of a national network to monitor, collect, and compile data with quantification of uncertainty in the status and trends of air emissions, deposition, air quality, surface water quality, forest condition, and visibility impairment and to ensure the comparability of air quality data collected in different States and obtained from different nations.”

QA Strategy Workgroup Ambient Air Quality System Training 3 Major 40 CFR Pt. 58 App. A QA Requirements Section 1 –General Info –Applicability –Definitions –Differences SLAMs/PSD Section 2 –QMPs/QAPPs* –Independent quality management function* –DQOs- (EPA) PM 2.5, O 3, PM –NPAP/PEP –TSAs-(EPA Regions) –NIST Traceable Standards Section 3 –PQAO definition –Quality Control and Assessment 1-point QC Annual PE Flow Rate Verification Flow Rate Audit Collocated Sampling PEP Section 4 –QA Stats Section 5- – Reporting Req. List- Site ID/PQAO by pollutant Quarterly submission of QA per reporting requirements Calculation of QA (EPA) once annual submission occurs } Gaseous pollutants } PM/Pb * EPA QA Policy There is so much more to QA beyond Appendix A

QA Strategy Workgroup Ambient Air Quality System Training 4 The Important CFR QA Requirements QMPs/QAPPs/SOPs a must Independent quality management Establishment of a Primary Quality Assurance Organization Use of NIST Traceable Standards Use of FRM/FEMs at the appropriate operating ranges

QA Strategy Workgroup Ambient Air Quality System Training 5 More Important CFR Gaseous QA Requirements One Point QC Checks- Biweekly Annual Performance Evaluation at 3 consecutive audit ranges NPAP Through the Probe Audits- 20% of Sites Precision and Bias data reporting to AQS Data assessments with the correct statistics –Data Toobox, DASC, AMP255

QA Strategy Workgroup Ambient Air Quality System Training 6 Planning QMPs, QAPPS and SOPs QMP- All programs QAPP SOP QMP - Organization Specific Describes organizations quality system Establishes capability QAPP - Project Specific Identifies the reasons for collecting data and for collecting it in a specific way Documents how the data are collected and how quality is maintained SOP - Instrument/Method Specific Ensures consistency - From day to day - From one person to the next

QA Strategy Workgroup Ambient Air Quality System Training 7 All QAPPs Are Not Created Equal- the Graded Approach The level of detail in each QA Project Plan will vary according to the nature of the work being performed and the intended use of the data. High Road- Big important decisions with little uncertainty Low Road – Small studies without major consequences or an allowance of greater uncertainty OAQPS has developed a graded approach for ambient air monitoring

QA Strategy Workgroup Ambient Air Quality System Training 8 TURBO-QAPP

QA Strategy Workgroup Ambient Air Quality System Training 9

QA Strategy Workgroup Ambient Air Quality System Training 10 SOPs They can never be too detailed –Remember those coming after you They are easy to write! Look at the reference methods in CFR for the minimum requirements Don’t reinvent the wheel –Share good SOPs- Ask vendors for electronic versions Use document control Use quality bulletins for changes Army Field Manual

QA Strategy Workgroup Ambient Air Quality System Training 11 Independent Quality Management who functions independently of direct environmental data generation, who reports on quality issues to the senior manager having executive leadership authority for the organization; and who has sufficient technical and management expertise and authority to conduct independent oversight of and assure the implementation of the organization’s quality system in the environmental programs of the organization. EPA QA Policy Requires: A quality assurance manager (QAM), or person/persons assigned to an equivalent position:

QA Strategy Workgroup Ambient Air Quality System Training 12 Establishment of Primary Quality Assurance Organizations Common factors of PQAOs Operation by a common team of field operators according to a common set of procedures; Use of a common QAPP or standard operating procedures; Common calibration facilities and standards; Oversight by a common quality assurance organization; and Support by a common management, laboratory or headquarters.

QA Strategy Workgroup Ambient Air Quality System Training 13 Possible PQAO Consolidations Consolidation of Tribes within a State Consolidation of Tribes across States within an EPA Region Consolidation of Tribes with State PQAO Consolidation of Tribes across EPA Regions

QA Strategy Workgroup Ambient Air Quality System Training 14 Use of NIST Traceable Standards (CFR Part 58 App A Sec 2.6.1) “Gaseous pollutant concentration standards must be traceable to either a National Institute of Standards and Technology (NIST) Traceable Reference Material (NTRM) or a NIST-certified Gas Manufacturer’s Internal Standard (GMIS), … Vendors advertising certification…. must participate in the EPA Protocol Gas Verification Program or not use ‘‘EPA’’ in any form of advertising.”

QA Strategy Workgroup Ambient Air Quality System Training 15 Certifications Testifies to the accuracy of an instrument/standard/gas as it compares to a NIST traceable standard. Make sure you are aware of how long a cylinder is certified for. It’s different for different gasses and concentrations Create a tracking mechanism Follow guidance for your MQOs in QAPP

QA Strategy Workgroup Ambient Air Quality System Training 16 Use of Federal Reference Methods (FRM) anmd Federal Equivalent Methods (FEMs) All monitors used for comparison to NAAQS must be FRM/FEM (40 CFR Part 53) Non-NAAQS monitoring has more flexibility EPA National Environmental Research Laboratory (NERL) develops an FRM for each NAAQS NERL approves each FRM/FEM –Posted on AMTIC Each has acceptable operating range(s) –Pick the range that matches your need.

QA Strategy Workgroup Ambient Air Quality System Training 17 Performance Characteristics of Gaseous FRM/FEM Methods Acceptable levels of –detection –precision –bias –drift Relatively unaffected by interfering compounds Sensitive at typical ambient concentrations Commercially viable Supports reliable and relatively maintenance-free operation Easily “QC-able”

QA Strategy Workgroup Ambient Air Quality System Training 18 Quality Control and Evaluation Quality Control Checks Performance Evaluations Technical Systems Audits

QA Strategy Workgroup Ambient Air Quality System Training 19 Understanding and ControllingUncertainty in Order to MinimizeDecision Errors Uncertainty = Natural Variability + Measurement 2.Precision* 3.Bias* 4. Completeness* 5. Comparability 6. Detectability MQOs Preparation Field Laboratory DQO DQA } 1. Representativeness Data Quality Indicators Spatial/Temporal

QA Strategy Workgroup Ambient Air Quality System Training 20 Data Quality Indicators & MQOs CO/ NO2 / SO2 O 3 Representativeness # sites/freq # sites/freq Comparability FRM/FEM/ FRM/FEM l Completeness 75% 75% Sensitivity (Detectability)1.0 / 0.01 / ppm l Precision10%* 7%* l Bias+10%* +7%* * Based on 3 years of data so any individual value can be higher than MQO gaseous P & B estimates at site level

QA Strategy Workgroup Ambient Air Quality System Training 21 QC Checks QC Checks have a variety of purposes To estimate our errors and keep within limits (internal program use, stay ahead of auditors) To troubleshoot performance of equipment (lamp intensity, pump efficiency) To identify drift over time To keep system stable (instrument checks, shelter conditions) To report to EPA for their use A QC Check is essentially an audit on yourself...

QA Strategy Workgroup Ambient Air Quality System Training 22 QC Checks One Point QC Check –Biweekly- Used to estimate precision and bias at the site level. –Select QC Check concentration at around routine concentrations. Annual Performance –More thorough independent check –Independent standards (and possibly the person) –3 consecutive ranges (some flexibility to concentration levels in CFR

QA Strategy Workgroup Ambient Air Quality System Training 23 NPAP-TTP PEP NATTS PT & ORIA Round Robins SRPPAMS Cylinders Protocol Gas Performance Evaluation-What and Why Performance evaluations (PEs) are a type of audit in which the quantitative data generated in a measurement system are obtained independently and compared with routinely obtained data to evaluate the proficiency of an analyst, or a laboratory

QA Strategy Workgroup Ambient Air Quality System Training 24 3 NPAP TTP Mobile Platforms

QA Strategy Workgroup Ambient Air Quality System Training 25

QA Strategy Workgroup Ambient Air Quality System Training 26 Technical Systems Audits Two types of Audits: Internal: Conducted by your own agency QA group. This group is independent of the monitoring group External: Conducted by your Regional EPA staff or another agency. This group is independent of your agency completely

QA Strategy Workgroup Ambient Air Quality System Training 27 Acceptance Criteria for Calibrations, QC Checks, and Instrument Audits Look for these tables All the ranges and parameters you will need are neatly laid out here

QA Strategy Workgroup Ambient Air Quality System Training Statistical Data Assessments CV +

QA Strategy Workgroup Ambient Air Quality System Training 29 Another Way of Looking at It… # of Observations % Difference Ideal: Precise w/o Bias Precise w Negative BiasPrecise w Positive Bias Low Precision w/o Bias Low Precision w Negative Bias Low Precision w Positive Bias

QA Strategy Workgroup Ambient Air Quality System Training 30 Precision Precision is more conservative when evaluated at the 90% one-sided upper confidence level Defines the “Width” of the Bell Curve Gaseous Estimate truth 90 th Percentile of a Chi- Squared Distribution  More Conservative

QA Strategy Workgroup Ambient Air Quality System Training 31 truth Bias Central Tendency 95 th Quartile of a Student-t Distribution w/ n-1 df Standard Error

QA Strategy Workgroup Ambient Air Quality System Training 32 DASC Tool Holds Up to 500 Pairs of Values Graphs & Stats Automatically Updated References Equations in Appendix A

QA Strategy Workgroup Ambient Air Quality System Training 33 DASC Tool (Cont’d) Easy to Use Navigation to Find the Stats You Need

QA Strategy Workgroup Ambient Air Quality System Training 34 AMP255 Features Statistics Shown on a per Site and Per PQAO Level as Appropriate

QA Strategy Workgroup Ambient Air Quality System Training 35 Box and Whisker Plots * * * + Lower Fence Q1 – (1.5*IQR) Upper Fence Q3 + (1.5*IQR) Interquartile Range (IQR) Outliers Whisker min Whisker max Q1 (25 th Percentile) Q2 (50 th Percentile) Q3 (75 th Percentile) Mean

QA Strategy Workgroup Ambient Air Quality System Training 36 Box and Whisker Plots Posted Box and Whisker Plots are Available at: Remember Me?

QA Strategy Workgroup Ambient Air Quality System Training Don’t Forget The It contains a number of the QC Checks Also!!!

QA Strategy Workgroup Ambient Air Quality System Training 38 What Data is in AQS? Collocated Data Flow Data Annual Performance Audits NPAP Data Technical Systems Audit Information???

QA Strategy Workgroup Ambient Air Quality System Training Guidance On All This!

QA Strategy Workgroup Ambient Air Quality System Training 40 G-4 R5-G-5 EPA Guidance for planning, implementation, and assessment activities. G-6 G-9R G-9S G-8 G-3 G-7

QA Strategy Workgroup Ambient Air Quality System Training 41

QA Strategy Workgroup Ambient Air Quality System Training 42 Excellent Air Monitoring Resource REDBOOK! Light reading for the air monitoring newbies and grizzled veterans