Flowing Data from Texas to EPA: The WQX Experience Nancy Ragland Texas Commission on Environmental Quality STORET Conference November 28, 2007

The Data Systems

TCEQ Regulated Activities and Compliance System (TRACS) Antiquated Antiquated Hard to Maintain Hard to Maintain No longer meeting agency business needs No longer meeting agency business needs

Surface Water Quality Monitoring Information System (SWQMIS) Why not STORET Why not STORET Buy or Build? Buy or Build? Enhancing the Experience Enhancing the Experience

SWQMIS – Assessment Enhancement Needed an integrated way to complete the 305(b) assessment Needed an integrated way to complete the 305(b) assessment Wanted to interface directly with SWQMIS Wanted to interface directly with SWQMIS Wanted to replace all the hand calculations that were being done Wanted to replace all the hand calculations that were being done Built an add-on to SWQMIS that uses SAS ® to compute statistics. Built an add-on to SWQMIS that uses SAS ® to compute statistics.

SWQMIS – Data Flow Enhancement Had not reported data to EPA since 1998 Had not reported data to EPA since 1998 Needed a way to report data to EPA without radically changing business processes Needed a way to report data to EPA without radically changing business processes Participated in WQX pilot project Participated in WQX pilot project

NEIEN Grant

National Environmental Information Exchange Network Grant Easy to apply Easy to apply Application process online Application process online Freed up agency staff to continue agency business Freed up agency staff to continue agency business Still let us have control of the development of the data flow Still let us have control of the development of the data flow There are some commitments to be aware of There are some commitments to be aware of Budget tracking Budget tracking Reports Reports

The Water Quality Exchange (WQX) – Getting Started

THE place to start THE place to start Includes step by step instructions Includes step by step instructions Has many helpful downloads Has many helpful downloads SRS characteristics in electronic format SRS characteristics in electronic format WQX database in MS Access WQX database in MS Access XML training manual XML training manual

Short Hand Steps Step 1 – Get 2 NAAS accounts (one for test, one for production) Step 1 – Get 2 NAAS accounts (one for test, one for production) Step 2 – Get an Org ID Step 2 – Get an Org ID Step 3 – Get a TPA in place Step 3 – Get a TPA in place Step 4 – Map data to the schema Step 4 – Map data to the schema Step 5 – Generate XML Step 5 – Generate XML Step 6 – Create test submission Step 6 – Create test submission

Helpful Hints

Mapping to the Schema If you have not been using STORET, you may be calling the same piece of information something slightly (or totally) different than what EPA has been calling it If you have not been using STORET, you may be calling the same piece of information something slightly (or totally) different than what EPA has been calling it Some of our data has pick lists where EPA had values and vice versa Some of our data has pick lists where EPA had values and vice versa We did not always collect data in a way that meshed with the schema We did not always collect data in a way that meshed with the schema

SRS Characteristics This was the most challenging and time consuming part of the mapping This was the most challenging and time consuming part of the mapping Used MS Access to match as many as possible Used MS Access to match as many as possible Matched the other 200+ by hand Matched the other 200+ by hand

SRS Characteristics Not case sensitive Not case sensitive Spaces have to match exactly Spaces have to match exactly All characters have to match exactly All characters have to match exactly All punctuation has to match exactly All punctuation has to match exactly

Nodes

What is a Node? A node is your presence on the exchange A node is your presence on the exchange A node can talk to other nodes A node can talk to other nodes Can send and receive data packages Can send and receive data packages Can send and receive error logs or other documents Can send and receive error logs or other documents

State Node Texas Integrated Data Exchange Node (TIDEN) Texas Integrated Data Exchange Node (TIDEN) A shell to build the flow in A shell to build the flow in Sends submissions to CDX Sends submissions to CDX Retrieves error logs Retrieves error logs

EPA Node Central Data Exchange (CDX) Central Data Exchange (CDX) Test Test Production Production Houses WQX schema and Schematron validation tool Houses WQX schema and Schematron validation tool

The Submission

The Tool We chose PL/SQL because our contractor was most proficient in this language and because they decided the most efficient and flexible way to go We chose PL/SQL because our contractor was most proficient in this language and because they decided the most efficient and flexible way to go The PL/SQL code carries out the mapping and creates the XML submission The PL/SQL code carries out the mapping and creates the XML submission Makes the error logs more human readable Makes the error logs more human readable

The Data Package Contains the actual data Contains the actual data Must be formatted according to the schema Must be formatted according to the schema Must be zipped Must be zipped

Delivering the Data Must go from state node to CDX Must go from state node to CDX Is transferred using Web Services Is transferred using Web Services Will result in a log file Will result in a log file

Conclusions

Next Steps Biological Data Biological Data Pilot participants Pilot participants Will add to our current flow Will add to our current flow Maintain the current flow Maintain the current flow We will continue to flow data according to our current Trading Partner Agreement (TPA) We will continue to flow data according to our current Trading Partner Agreement (TPA) We plan to continue renewing our TPA We plan to continue renewing our TPA

Lessons Learned Map before you build Map before you build Plan ahead Plan ahead Test, test, test! Test, test, test! Work closely with EPA Work closely with EPA Work closely with your IT people Work closely with your IT people

Contact Info

Nancy Ragland (512)