Dirk Felton RTP, NC February 12-13, 2008 Air Quality Data Summit: Session: Inventory of Data Systems Data provider perspectives

The NY State Dept of Environmental Conservation Submits Air Quality data to the following Databases: AQS: Nearly 80 sites, hundreds of parameters, since the 1970’s - Submit, Valid, Certified, Flagged or null AirNow: selected Ozone, PM-2.5 and PM-10 - Submit selected un-validated, modified data (NY’s Network is too dense in urban area: NYC) VIEWS: 2 urban IMPROVE sites - IMPROVE submits and validates NADP/MDN: 2 urban wet deposition and Continuous Hg sites - NADP submits and validates NARSTO: 5 Yr collaboration with NY Supersite - PI/NYSDEC submit Valid and Flagged data

AQS: Input Issues: Difficult format, parameter and method codes limit detail, not suitable for short term studies, no ability to calculate: flows (std vs vol) Retrieval issues: Flags not included automatically (should be), Public, EPA Regional offices and many consultants have difficulty finding data Meta data issues: site data not updated, system does not accept accessory and diagnostic data, scale data not by parameter or complete, no analysis detail or history of changes Correction/certification issues: Should have a method to inform users of changes to datasets or at least institute a version or customer tracking system. Data certification not necessarily meaningful, Blanks not incl. Frustration: downloaded data by parameter creates data sets that are not internally compatible (FRM, TEOM, STN mass), 120 days not long enough for data requiring lab analysis, FEMs not always equal.

AQS Frustration Continued: (This is what normally happens) Data Provider Public & Consultants

AirNow: Input Issues: “FRM-like” data not consistently modified. (Should there be a test?) Need hourly connectivity to monitoring site, Some older data systems limited to Ozone and PM-2.5. Retrieval issues: Limited, Access to un-validated data should be limited. Meta data issues: Access to FRM like adjustments is provided but is rarely used by data users. (may be password protected) Correction/certification issues: Some “error catching” can be implemented by provider but this is difficult in a live system. Buddy system checking should be provided by database. Data not likely to match AQS data. Frustration: Invalid data gets out to the public more often than we would like. Max data thresholds can limit bad data but in extreme conditions (fires) can also limit good data when the public really needs it.

VIEWS : Input Issues: Data input by administrator not data providers. Retrieval issues: Very long delay before data is available, Data sometimes partially available or in draft before it is finalized Meta data issues: Helpful documents provided but cannot be uploaded by data provider. QA data and QA program not sufficient. Correction/certification issues: No data provider input Frustration: Data delays of more than 1 year are common.

NADP/MDN Input Issues: Data input by administrator not provider. Retrieval issues: Limited to public, More availability to providers. Meta data issues: Limited but not usually a problem for a consistent network. Correction/certification issues: Tekran data will be a challenge Frustration: We’ll see, The Mercury program is still being designed.

NARSTO Input Issues: File format is archaic, cumbersome and does not provide enough options. Retrieval issues: ?? I don’t know how. Meta data issues: Some information is included in the input file format. Correction/certification issues: Utilizes Data QC flags (0,1,2) Frustration: Seems like a black hole.

NARSTO: Format for a single data entry

What Data Providers Need: Accurate: verified datasets: State and Locals have to provide very accurate data for use in attainment designations. Ozone PM-2.5 Annual and Daily (98%tile) PM-10 New Lead Standard? (Monthly)

2 -What Data Providers Need: Long term datasets utilizing consistent methods and locations Health Studies SIP/Trend Analysis Haze monitoring

3-What Data Providers Need: Real-Time Public Access: The public, the press and Government officials demand information. This data does not have to be verified. In the past data turnaround was so slow, monitoring Agencies were not even asked unless the inquiry was retrospective.

STN Mass is not the same as mass from the FRM 4- What Data Providers Need: A method to explain data inconsistencies that do not make the Public distrust the provider

This is harder to explain when it effects individual components preferentially: The lower flow rate of the SuperSass sampler makes it more efficient for the collection of volatile species.

The FDMS data indicates that the FRM and the 50 Deg C TEOM did not capture a substantial fraction of PM-2.5 during this pollution episode.

5- What Data Providers Need: An easier method to communicate dataset issues that compromise data quality: Sodium Filter Contamination (After more than 2 Yrs of complaining the Sodium Data was Flagged in AQS)

This data is compromised but it has not been flagged because the Lab Denies that Ammonium concentrations were effected by Na contamination (This data is still in AQS)

Summary: Needs of Data Providers: Access to accurate, validated data (including from other States) Access to long data records (20+ years) Access for the Public for near real-time reports and warnings Easy input of short term special purpose, special study data Method to communicate meta data and nuances to users: flags, inconsistent methods, different sampler type, different lab analyses, errors Method to communicate changes or updates to data sets. NY is currently changing 2006 FRM data to fix error in null reporting

End of Presentation