1. NCAI Exchange Network Tribal Users Meeting April 9, 2008 Tribal Water Quality Data Management

2. Overview Tribal 106 Grants Requirements for Monitoring and Assessment Tribal 106 Grants Requirements for Monitoring and Assessment Essential concepts of Water Quality Data Management Essential concepts of Water Quality Data Management Understanding WQX Understanding WQX Steps to a WQX Submittal Steps to a WQX Submittal Accessing Data Accessing Data Upcoming Developments Upcoming Developments How to decide where to begin How to decide where to begin

3. Tribal 106 Guidance Requirement All tribes must submit a TAR (Tribal Assessment Report) to continue eligibility for 106 funds (8-3) All tribes must submit a TAR (Tribal Assessment Report) to continue eligibility for 106 funds (8-3) Monitoring Strategy Monitoring Strategy Water Quality Assessment Water Quality Assessment STORET-compatible electronic data STORET-compatible electronic data

4. 106 Guidance parameters Fundamental Programs: - Dissolved Oxygen (DO) - Temperature - pH - Turbidity Intermediate Programs: - Nitrogen - Phosphorus Mature Programs: - Pathogens - Macroinvertebrates - Physical Habitat indicators

5. I. Monitoring Strategy II. Monitoring Objectives III. Monitoring Design IV. Core Indicators V. Quality Assurance VI. Data Management VII. Data Analysis/Assessment VIII. Reporting IX. Programmatic Evaluation X. General Support and Infrastructure 10 Elements of a Tribal Monitoring Strategy

6. Essentials of Data Management Monitoring data consists of: Monitoring data consists of: Where sample/measurement was collected Where sample/measurement was collected When it was collected When it was collected What was collected What was collected How it was collected How it was collected Why it was collected Why it was collected Who collected it Who collected it It does not take a lot of information to create a good data set. It does not take a lot of information to create a good data set.

7. How does data management fit in? The life of a sample begins here Labs generate data Field Sheets capture data Now what to do with all this data?

8. First, make sense of what you have Organize data electronically Use the tools you’re familiar with: Access Excel Other higher-powered database management system (i.e. Oracle) Have a plan for growth Not only will your skill grow, but your data volume will grow. Be prepared for it. Find out what’s already available

9. Data are valuable, plan for re-use Electronic data are more valuable than data in file cabinets The more data are re-used, the more valuable they become Shared data are of even higher value Provide for better planning decisions Incentivize collaborative efforts Make the most use of the monitoring $$s being invested

10. How EPA Describes Monitoring Data Projects (Why data were collected) Projects (Why data were collected) - Brief summary of monitoring plan Monitoring Locations (Where data were collected) Monitoring Locations (Where data were collected) - Describe where monitoring takes place Results (When, How and What were collected) Results (When, How and What were collected) - Measurements of what were monitored Three categories of data :

11. WQX is designed around these concepts ORGANIZATION ORG Description ORG Electronic Address ORG Phone ORG Address MONITORING LOCATION Monitoring Location Identity Monitoring Location Geospatial Monitoring Location Binary Object RESULT Result Description Result Binary Object Result Analytical Method Result Lab Info Result Detection Quantitation Level Lab Sample Prep MONITORING ACTIVITY Activity Description Activity Location Sample Description Sample Prep Subsample Description Activity Binary Object PROJECT Project Description Project Binary Object ACTIVITY GROUP Activity Group Name Activity Group Type Activity IDs

12. The WQX Schema The WQX schema is a definition of the structure and format that data must have to be submitted through WQX. The WQX schema is a definition of the structure and format that data must have to be submitted through WQX. Primary components: Organization, Project, Monitoring Location, Monitoring Activity, Samples, and Results Primary components: Organization, Project, Monitoring Location, Monitoring Activity, Samples, and Results Data elements currently accommodate physical, chemical, and fish tissue data Data elements currently accommodate physical, chemical, and fish tissue data Biological monitoring and habitat data flow will be incorporated in the very near future Biological monitoring and habitat data flow will be incorporated in the very near future

13. Submitting data using WQX Water Quality Exchange (WQX) uses the Exchange Network to flow physical, chemical and fish tissue, and soon biological and habitat data from an tribe’s own database to the existing STORET Warehouse Water Quality Exchange (WQX) uses the Exchange Network to flow physical, chemical and fish tissue, and soon biological and habitat data from an tribe’s own database to the existing STORET Warehouse Tribes that have been awarded Exchange Network Grants will be able to submit data through WQX Tribes that have been awarded Exchange Network Grants will be able to submit data through WQX

14. CDX WQX Parse and Load Software WQX ODS Data Submitter XML Submission Schema Validation Receive Error Responses Approved XML Submission Data Context Error Report Data Load Confirmation and Summary How WQX Works

15. WQX in the big picture

16. Steps to Submitting to WQX 1. Review the resources that are available 2. Getting involved with the Exchange Network 3. Getting a NAAS Account 4. Getting an Organization ID 5. Understanding XML (How does my data relate to an XML file?) 6. Submitting a file 7. Understanding errors 8. Understanding the WQX structure 9. Interacting with WQX

17. Step 1. Resources Available The STORET/WQX team has several resources available to help you: The STORET/WQX team has several resources available to help you: Available at the STORET web site: www.epa.gov/storet Available at the STORET web site: www.epa.gov/storetwww.epa.gov/storet Visit the WQX section: http://www.epa.gov/storet/wqx.html Visit the WQX section: http://www.epa.gov/storet/wqx.htmlhttp://www.epa.gov/storet/wqx.html Diagrams and presentations about how WQX works Diagrams and presentations about how WQX works Presentations from others on how they’ve implemented WQX (including tribes) Presentations from others on how they’ve implemented WQX (including tribes) Lists of allowable values for WQX Lists of allowable values for WQX Available at the Exchange Network Site: http://www.exchangenetwork.net/exchanges/water/wqx.htm Available at the Exchange Network Site: http://www.exchangenetwork.net/exchanges/water/wqx.htm http://www.exchangenetwork.net/exchanges/water/wqx.htm Read the Flow Configuration Document Read the Flow Configuration Document See example XML Files See example XML Files Examine the Data Exchange Template Examine the Data Exchange Template Last but not least, don’t be afraid to ask for help: Last but not least, don’t be afraid to ask for help: Join the STORET list serve: www.epa.gov/storet/listserv.html Join the STORET list serve: www.epa.gov/storet/listserv.html www.epa.gov/storet/listserv.html Call the STORET hot line: 1-800-424-9067 Call the STORET hot line: 1-800-424-9067 end an email to storet@epa.gov end an email to storet@epa.govstoret@epa.gov

18. Step 1. Remember this All of this information is available at: www.epa.gov/storet www.epa.gov/storet

19. Step 2. Getting involved in the Exchange Network Ask yourself the following questions: Ask yourself the following questions: 1. Does my organization have a node? 2. Have we submitted data for another flow? 3. Did we use a node? Node client? Or CDX Web? 4. Who is my Node Administrator? Do we have one? If the answer to these questions is ‘No’ or ‘I don’t know’ visit: www.exchangenetwork.net If the answer to these questions is ‘No’ or ‘I don’t know’ visit: www.exchangenetwork.netwww.exchangenetwork.net

20. Step 3. Getting a NAAS Account The NAAS or Network Authentication Authorization Service is a user account that you can use to access the Exchange Network. They can be specific for an individual node or for a particular user The NAAS or Network Authentication Authorization Service is a user account that you can use to access the Exchange Network. They can be specific for an individual node or for a particular user NAAS accounts are assigned by Node Administrators. If you don’t have a Node Administrator, contact the CDX help desk at: 888-890-1995 Or epacdx@csc.com NAAS accounts are assigned by Node Administrators. If you don’t have a Node Administrator, contact the CDX help desk at: 888-890-1995 Or epacdx@csc.comepacdx@csc.com

21. Step 4. Getting an Org ID An Organization ID (Org ID) is a unique identifier assigned to your organization (i.e. YUROK for the Yurok Tribe). An Organization ID (Org ID) is a unique identifier assigned to your organization (i.e. YUROK for the Yurok Tribe). Org IDs are assigned by EPA upon the request of the user. Org IDs are assigned by EPA upon the request of the user. You must have an Org ID to be able to submit to WQX You must have an Org ID to be able to submit to WQX To request an Org ID, send an email to storet@epa.gov To request an Org ID, send an email to storet@epa.gov storet@epa.gov Provide the NAAS accounts that you want to have access to your Organization Provide the NAAS accounts that you want to have access to your Organization

22. Step 5. Understanding XML XML can be intimidating at first XML can be intimidating at first There are many ways to create XML, some things to do: There are many ways to create XML, some things to do: 1. The sample files and Data Exchange Template are key to understanding how to create a WQX XML file. 2. Find a programmer who can spend 20-40 hours on designing an XML file 3. Have some test data that are representative of the data you’ll be submitting 4. Test the procedures that the programmer developed (plan for an additional 20-40 hrs testing) 5. Start small, it takes time to create an XML file, don’t waste development time waiting for a computer to process a file. ***You should only have to do this once, after that, you’re just running the procedure.

23. Step 6. Submitting a File Once you have a NAAS account and an Org ID, and have successfully created an XML file, you’re ready to submit. Once you have a NAAS account and an Org ID, and have successfully created an XML file, you’re ready to submit. The original test file should be small, but representative of the data you’ll be submitting The original test file should be small, but representative of the data you’ll be submitting Submit to CDX test first Submit to CDX test first Once you get a successful submission, keep track of the transaction ID Once you get a successful submission, keep track of the transaction ID Send an email to the CDX help desk:…. Requesting access to WQX production. Provide them the transaction ID of the successful submission. Send an email to the CDX help desk:…. Requesting access to WQX production. Provide them the transaction ID of the successful submission. Go ahead an begin submitting to production Go ahead an begin submitting to production

24. Step 7. Understanding Errors There are two types of errors in WQX: There are two types of errors in WQX: 1.Schema Errors: the submitted file does not conform to the WQX Schema. (Report is available in the file ‘Validation Results’ which is an xml file). 2.Processing Errors: the submitted file does not pass all of the WQX Business Rules (Report is available in the file ‘ProcessingReport.zip’)

25. Step 8. Understanding the WQX Structure WQX files should be organized as follows: WQX files should be organized as follows: Header (with processing instructions) Header (with processing instructions) Organization Organization Organizational Information Organizational Information Projects Projects Monitoring Locations Monitoring Locations Activities Activities Linkage to Projects Linkage to Projects Linkage to Monitoring Locations Linkage to Monitoring Locations Results Results Activity Groups Activity Groups

26. Step 9. Interacting with WQX WQX is designed to function in a Node-to- Node environment (i.e. you manage your data, let your computer and Node keep EPA in sync). WQX is designed to function in a Node-to- Node environment (i.e. you manage your data, let your computer and Node keep EPA in sync). WQX allows Inserts, Updates, and Deletes via a batch process WQX allows Inserts, Updates, and Deletes via a batch process WQX has Outbound services via the Query and Solicit method data visualization and system synchronization purposes. WQX has Outbound services via the Query and Solicit method data visualization and system synchronization purposes.

27. Accessing Data The National STORET Data Warehouse serves as the national repository for Modern STORET data as well as WQX data. Users may obtain data from the warehouse through the STORET home page : www.epa.gov/storet Web services providing direct access to warehouse data will also provide enhanced accessibility to the data.

28. STORET Warehouse Updates The National STORET Data Warehouse enhancements for improved data accessibility The National STORET Data Warehouse enhancements for improved data accessibility New Watershed Summary – Available Now New Watershed Summary – Available Now New Web Services – Available Now New Web Services – Available Now Please visit the STORET website for more information : www.epa.gov/storet Please visit the STORET website for more information : www.epa.gov/storet

30. Where do Web Services fit? Data in/out services via WQX Data out services through direct access to the WH WH services available via Exch. Net.

31. All Done with Web Services!!

32. An example using Monitoring Data

33. What’s coming next? WQX 2.0 is near completion WQX 2.0 is near completion Adds Biological data and Habitat data to the schema Adds Biological data and Habitat data to the schema Supports Bio/Habitat metrics and indexes Supports Bio/Habitat metrics and indexes EPA will continue to support WQX 1.0 for the foreseeable future EPA will continue to support WQX 1.0 for the foreseeable future XML Generation Tool XML Generation Tool Tool that will support organizations that don’t have the ability to create XML Tool that will support organizations that don’t have the ability to create XML Batch Geo-location tool Batch Geo-location tool Tie Monitoring Locations to the National Hydrography Dataset Tie Monitoring Locations to the National Hydrography Dataset

34. Dates to Remember Dates to Remember WQX Version 1.0 data flow in production now WQX Version 1.0 data flow in production now Physical, chemical, microbiological, and fish tissue data only Physical, chemical, microbiological, and fish tissue data only Add Biological/Habitat data to WQX schema Add Biological/Habitat data to WQX schema Bio/Habitat WQX in production (May 08) Bio/Habitat WQX in production (May 08) Develop Web tool for XML Generation Develop Web tool for XML Generation Tool available for general use (September 08) Tool available for general use (September 08) Batch Geo-location Tool Batch Geo-location Tool Under development available January 2009 Under development available January 2009

35. What is the XML Generation Tool? A web-based application that assists a data provider with the creation of a valid submission file for the Water Quality Exchange. A web-based application that assists a data provider with the creation of a valid submission file for the Water Quality Exchange. The next generation of SIM and WebSIM, redesigned to target WQX. The next generation of SIM and WebSIM, redesigned to target WQX.

36. This tool is meant to allow those who do not have Exchange Network Nodes the ability to share data over the web This tool is meant to allow those who do not have Exchange Network Nodes the ability to share data over the web The tool will be easy to use and accept basic input formats (i.e. Excel Spreadsheets). The tool will be easy to use and accept basic input formats (i.e. Excel Spreadsheets). Data submitted via this tool will be available on the internet within one week of submission. Data submitted via this tool will be available on the internet within one week of submission. Data-sharing for all

37. WQX Database CDX Web Online Registration Log In How the XML Generation Tool fits in CDX Node XML Generation Tool Import Flat File Apply Defaults & Translations Validate & Correct Export XML State/Tribal Node Submit Get Status Download CDX Lite Submit Get Status Download State/Tribal Database State/Tribal System Retrieve Data from Database Export XML WQX System Load XML File Update WQX Database Create Processing Report

38. Demo By the Way, what happened to National WebSIM?

39. Batch Geo-Locating New development that EPA is working on New development that EPA is working on Allow for any new stations submitted via WQX to be automatically georeferenced to the NHD+ Allow for any new stations submitted via WQX to be automatically georeferenced to the NHD+ The Georeferenced reach codes would be returned to the data provider, as well an opportunity to QA the georeferenced locations The Georeferenced reach codes would be returned to the data provider, as well an opportunity to QA the georeferenced locations This will use a concept called ‘Rain Drop’ to ensure that points are tied to the correct NHD reach This will use a concept called ‘Rain Drop’ to ensure that points are tied to the correct NHD reach

40. If the two points represent Accuracy/Precison issues then note how raindrop results in same indexed location Note how upper point with straight snap would go to wrong catchment Note how the path does trace the topo delineation for water

41. Why this is Important? Allows for monitoring data to be easily linked to other types of data (i.e. Impaired waters, assessments, permits, etc.) Allows for monitoring data to be easily linked to other types of data (i.e. Impaired waters, assessments, permits, etc.) Allows for upstream/downstream analysis Allows for upstream/downstream analysis Greatly expands the usability of monitoring data Greatly expands the usability of monitoring data

42. Batch Indexing Timeline This capability should be available January 2009 This capability should be available January 2009 Deployed as a service that any data flow could use (not just for WQX) Deployed as a service that any data flow could use (not just for WQX) Also provide some Geospatial QA services (i.e. is the lat/long in the county/HUC/state that the user specified) Also provide some Geospatial QA services (i.e. is the lat/long in the county/HUC/state that the user specified)

43. Where to Get Started 1. Figure out what data you have, and what format it is in. 2. Find out where you stand with the exchange network 3. Visit: www.epa.gov/storet www.epa.gov/storet 4. Give us a call, we can walk you through any questions.

44. Web Resources www.epa.gov/storet www.epa.gov/storet www.epa.gov/storet www.exchangenetwork.net/exchanges/water/ wqx.htm www.exchangenetwork.net/exchanges/water/ wqx.htm www.exchangenetwork.net/exchanges/water/ wqx.htm www.exchangenetwork.net/exchanges/water/ wqx.htm www.epa.gov/storet/wqx.html www.epa.gov/storet/wqx.html www.epa.gov/storet/wqx.html

45. Question/Comments Contact: Contact: Dwane Young, EPA (202)566-0616 Dwane Young, EPA (202)566-0616 Young.dwane@epa.gov Kristen Gunthardt, EPA (202)566-1194 Kristen Gunthardt, EPA (202)566-1194 gunthardt.kristen@epa.gov STORET Help Desk 800-424-9067 STORET Help Desk 800-424-9067 storet@epa.gov Join the STORET list serve for more information, including monthly conference calls about STORET and WQX: www.epa.gov/storet/listserv.html www.epa.gov/storet/listserv.html