Guest Speaker: Jeremy Howe Little River Band Of Ottawa Indians The Measuring Sticks of Air Monitoring

 April 7: QA Systems, EPA definitions, PQAOs and common sense – Mike Papp  April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks) – Travis Maki  April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) – Jeremy Howe  April 28: Calculating Bias and Precision and AQS reports – Bill Frietsche  May 5: 40 CFR 58 App. A- Gaseous Pollutants – Glenn Gehring  May 12: 40 CFR 58 App. A- Ozone – Brenda Jarrell  May 19: 40 CFR 58 App. A- PM filter and continuous methods –Brandy Toft 2

 Audit – “a systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives”  Independent means using a device other than the one you use routinely at a minimum, and in some cases a completely independent system of instrument and operator 3

 Basically compare ◦ What is actually being done in the field and the office ◦ Against what is stated in the QAPP and SOPs ◦ Are done by you or someone outside your office, depending on the type of audit 4

 Number and types of assessments  People or organizations doing the assessments  Schedule  Criteria for assessments (we will compare what we do against the SOP or QAPP)  Reporting and responsibility for follow-up (e.g., we will revise the SOP in pen for now, and retype it with changes within a month) 5

Program: CLOSE THE LOOP (fix problems and take action to make sure they do not happen again) And make sure it is documented 6

 1. Six-month flow rate audits you make with non-routine standards (PM)  2. Annual Performance Evaluations (gas)  3. NPAP and PEP audits conducted by certified auditors  4. Technical Systems Audits conducted by EPA regions (every 3 years)  To see how many/often of the 1 st three types you need for NAAQS-comparable data, see Calculations for Tribes Interested in Joining a Tribal PQAO A.xls 7

MethodPollutantsFrequencyMQO One-Point QCSO 2, NO 2, O 3, COEvery 2 Weeks O 3 : Precision = 7% Bias = 7% SO 2, NO 2, CO : Precision = 10% Bias = 10% Annual Performance Evaluation (Audit) SO 2, NO 2, O 3, COOnce per Year<= 15% for each audit concentration Flow Rate Verification (QC check) PM 10 (lo-Vol), PM 2.5 Once every 4 weeks <= 4% of Standard & 5% of Design Value Flow Rate Verification (QC check) PM 10 (high-Vol), TSP Once per quarter <= 7% of Standard & 10% of Design Value Semi-Annual Flow Rate Audit PM 10, PM 2.5 Every 6 Months <= 4% of Standard & 5% of Design Value Semi-Annual Flow Rate Audit PM 10 (high-Vol), TSP Every 6 Months <= 10% of Standard & 10% of Design Value Collocated SamplingPM 10, TSP, PM % of Network Every 12 Days PM 2.5, TSP, PM 10 : Precision = 10% PM 2.5 PEP Program NPAP PM 2.5 SO 2, NO 2, O 3, CO Quarter *! Year (see Calculations… spreadsheet) Bias = 10% 8

MethodPollutantsFrequencyWho? With What? One-Point QCSO 2, NO 2, O 3, CO Every 2 WeeksSite operatorAnnual calibration standard, owned by tribe Annual Performance Evaluation SO 2, NO 2, O 3, CO Once per YearPreferably not site operator, no national certification required, not the same standard that is used for routine QC checks Flow Rate Verification (not required to be reported to AQS) PM 10, PM 2.5 Every 4 weeksCan be site operator, no national certification required, not the same standards that are used to calibrate your analyzer Semi-Annual Flow Rate Audit PM10, PM2.5Every 6 MonthsPreferably not site operator, no national certification required, not the same standards that are used for monthly QC checks PM PEP, NPAPPM 2.5, All gaseous See Calculations…sprea dsheet OAQPS/TAMS certified auditor OAQPS/TAMS equipment 9

Flow Rate Verification PM 10, PM 2.5 automated and manual Every 4 weeksCan be site operator, no national certification required, the flow rate standard used must be certified annually Flow Rate Verification PM10 (high-Vol), TSP manual Once per quarter Semi-Annual Flow Rate Audit PM 10, PM 2.5 automated and manual Every 6 MonthsPreferably not site operator, no national certification required, not the same standards that are used to calibrate the analyzer, but can be calibrated to the same primary stnd as your cal stnd 10

11

12

13

 Discussed in more detail in PM webinar  Rely on owner’s manuals  EPA guidance available:  2.12 "Monitoring PM2.5 in Ambient Air Using Designated Reference or Class I Equivalent Methods" /pm25/qa/m212covd.pdf /pm25/qa/m212covd.pdf  See docs in course webpage and AMTIC and CARB SOPs 14

 check the operational flow rate of the analyzer, before any flow rate adjustment  Randomize the time and day  Use a flow rate transfer standard certified to be traceable to an authoritative volume or other applicable standard (within 2% of NIST – Traceable Standard Part 50, App L Sec 9.2)  Reported to AQS as RA (bias) because it is made with a non-routine standard (the ones with your routine stnd are precision (RP)  % diff =[ (meas-audit)/audit ]*

 Annual performance evaluation for SO 2, NO 2, O 3, or CO  …The gas standards and equipment used for evaluations must not be the same as the standards and equipment used for calibration or calibration span adjustments…the auditor should not be the operator or analyst who conducts the routine monitoring, calibration, and analysis. 16

 40 CFR 58 App A sec 3.2.2:  If there are fewer than four analyzers for a pollutant within a PQAO, P.E. annually (preferably in different quarters)  The evaluation should be conducted by a trained experienced technician other than the routine site operator  Current CFR guidance will change soon 17

18

19

20

 use…audit gas from at least three consecutive audit levels. The audit levels selected should represent or bracket 80 percent of ambient concentrations measured by the analyzer being evaluated: 21

 New guidance for audit levels:  Ten levels, the lowest as low as 4-6 ppb for O3  Consecutive audit levels no longer required  See memo:  A101_Resources/EPA%20guidance%20and%20 other%20reference%20material/TechMemofor PEAuditLevels.pdf A101_Resources/EPA%20guidance%20and%20 other%20reference%20material/TechMemofor PEAuditLevels.pdf 22

 3.2.2…Report both the audit and measured concentration  The % differences ( d i ) between these concentrations are used to assess the quality of the monitoring data, and to confirm your biweekly QC check results: 23

 Allow…the audit gas to pass through all filters, scrubbers, conditioners, and other sample inlet components used during normal ambient sampling and as much of the ambient air inlet system as is practicable  < 15 % for each audit concentration, and  EPA uses these (RA) values you report to AQS to generate probability limits for your bias 24

2.4 National Performance Evaluation Programs. ….the consent of the monitoring organization for EPA to apply an appropriate portion of the grant funds, which EPA would otherwise award to the monitoring organization for monitoring activities, will be deemed by EPA to meet this requirement. For clarification and to participate, monitoring organizations should contact either the appropriate EPA Regional Quality Assurance (QA) Coordinator… 25

 PEP- $2000/audit ◦ 5 audits/yr for PQAO with < 5 sites = 10K/year ◦ All sites audited at least once every 6 years  NPAP- $2200/audit ◦ Each site audited at least once every 5 yrs ◦ At least 1 NPAP audit per PQAO network that includes even 1 gas analyzer ◦ Would need 10 sites for 2 audits a year ◦ Each NPAP audit covers all gas analyzers at a site  These (rough) costs are for EPA contractors doing the audits 26

 Auditors (including tribal folks) can take training and exam  Equipment must be approved/provided as audit-specific equipment  SOPs available on AMTIC,

Separate from Cradle to Grave Separate from Cradle to Grave For Gaseous Pollutants For Gaseous Pollutants O 3 + CO + SO 2 + NO X O 3 + CO + SO 2 + NO X 28

Separate from Cradle to Grave Separate from Cradle to Grave For Filter Based Pollutants For Filter Based Pollutants PM Lead PM Lead PEP Collocation PEP Audit 29

 2.5 Technical Systems Audit Program. Technical systems audits of each ambient air monitoring organization shall be conducted at least every 3 years by your EPA regional office  Review of siting, documentation, procedures’ implementation  Assessment of whether the “QA function is independent”  Checklists that the region will use are available as QA Handbook Volume II, Appendix H 30

 At least 3 months notice, with written audit plan  Checklist sent ahead of time, see NAAM Technical Systems Audit Form in: oads/QA101_Resources/EPA%20guida nce%20and%20other%20reference%20 material/ oads/QA101_Resources/EPA%20guida nce%20and%20other%20reference%20 material/ 31

 All audits are “friendly” and intended to improve the quality of the data  Findings will always include insufficient documentation  Site visits are included  Some AQS data are pulled, then records of all the QC and original data are looked for in office  Important for small organizations: the “independency of the QA function” 32

 CFR App. A: 2.2 …a quality assurance management function…includes strategic planning, allocation of resources and …assessing and reporting…  must have sufficient technical expertise and management authority to conduct independent oversight…  should be organizationally independent of environmental data generation activities (e.g., NOT the site operator or their manager)  Ideal QA manager is described at: nitorstrat/qamroles2.pdf nitorstrat/qamroles2.pdf 33

 Adds necessary quantitative and qualitative (TSAs) yardsticks for data defensibility  Adds value by exchanging information  Course website: QA101_Resources/ QA101_Resources/  Consider joining the TAMS Steering Committee to help develop national tribal training to address these complicated issues ( us) ◦ Jeremy Howe: ◦ ◦ 34