Data Extraction, Utilization, and Assessment

Data Extraction, Utilization, and Assessment
AWQMS: Data Extraction, Utilization, and Assessment Session C5: Data Soup: Recipes & Secret Spices Chris Adams, Oklahoma Water Resources Board Mark LeBaron, AWQMS / Gold Systems, Inc. Chris Discuss an innovative approach for using WQP data via AWQMS and some of the challenges we've encountered while using multi-agency data.

An innovative approach for using Water Quality Portal data
OKWRB’s implementation of AWQMS Using Water Quality Portal data within AWQMS Challenges encountered while using multi-agency data Solutions and demonstrations Chris Introduce self if not done by the moderator. Today we are going to be discussing an innovative approach for using data from the National Water Quality Portal using the Ambient Water Quality Monitoring System (“Awww Kwums”). Everything we discuss and show is available to any organization that has implemented AWQMS. These are not just features implemented only for the OK Water Resources Board. Why we needed a new system and what it is. An innovative approach for using Water Quality Portal data

OKWRB’s implementation of the Ambient Water Quality Monitoring System (AWQMS)
Various data types Different databases, spreadsheets, or custom data solutions Limited IT support Poor adaptation to changing programatic needs and business rules Large knowledge base to handle the disparate databases QC often external to the data repositries.

A web-based water quality sample and observations database application
Consolidate Validate Analyze Assess Share Chris The system allows for the import/entry, validation, analysis, and export of data to both generic data formats and export files (XML) for the submission of data to the EPA's Water Quality eXchange (WQX). Data submitted to WQX also flows to the National Water Quality Portal (WQP). Originally developed by various tribes, states, and agencies., still being developed. In 2016 OWRB bgan using the system and along with various in and out-of-state partners, are working to upgrade and enhance the AWQMS application in various ways, including improved groundwater tracking, habitat assessment, biological assessment, and overall water quality assessment support; including submission of assessment decisions to the EPA's ATTAINS system. A web-based water quality sample and observations database application

Major AWQMS Capabilities
Data Consolidation – One secure, multi-user place for all types of our water quality data Optimized for both both time series and discrete samples Data Quality & Consistency Checking Data Management / Editing Tools Basic Data Analysis (Statistics, Reporting, and Graphing)Tools Built in Google Mapping Tools Enforces WQX business rules Scaleable knowledge base. Major AWQMS Capabilities

Other Features Worth Mentioning
Easily submit data to WQX and related feedback tools Export to other crosstab and flat file formats Public water quality portal (if needed) Integration with other systems via web services Trip Planning Easily submits BEACH Notifications to PRAWN (if needed) WQX Domain values auto-sync Assessments Meta Data & Exceedance Exports Key point: AWQMS has been improving gradually since inception and the plan is for it to keep improving. Mutiple state and numerous tribes using AWQMS means sustainability. Shout out to Colorado CDSN. Other Features Worth Mentioning

Recent New Features Data Preparation:
Batch updates for Activities data Batch updates for Results data Redundant Data Entry for QC purposes Leverage EPA Water Quality Portal for combined data analysis Mark Recent New Features

Major Features Under Construction
Integration with EPA national solution for Continuous Monitoring (time series sensor) Data (Interoperable Watersheds Network) Capture and submit assessments to ATTAINS Better decision support for assessments Auto calculation of water quality standards for metals Integrated Continuous Monitoring Data Review & QA Processess Automated generation of benthic macroinvertebrate abundance metric scores per associated index type Mark Major Features Under Construction

Using Water Quality Portal data within AWQMS
Chris

Request & Receive WQP Data
National Water Quality Portal Web Services 1. Request data OK Water Resources Board 2. Respond with data Chris – High level conceptual overview Request & Receive WQP Data

Request & Receive WQP Data
OK Water Resources Board 2. Respond with data One-time configuration Configure as many as you need Schedule the period Data is stored in AWQMS but flagged as “External” Can be reviewed / analyzed like any other data in AWQMS Chris High level overview of how it is set up. Request & Receive WQP Data

Challenges encountered while using multi-agency data
Discussion of some of the key challenges

Ensuring consistency for analysis and assessment
Characteristics, Measurement Units, and Monitoring location types Monitoring Locations Assessment Units Designated Uses Water quality criteria Ability to provide filtering criteria Chris Biggest challenge is consistency of data sets Characteristics Synonyms fractions speciation units sampling/analytical methods Monitoring locations – various IDS, lacking metadata, truncated spatial information Need to manually combine them into assessment units – AWQMS then has beneficial uses and appropriate water quality criteria assigned Sort out the assessment uses and the rest falls into place. Examples of challenges include ensuring consistency of Characteristics, Measurement Units, Monitoring Location Types so that data can be used together during analysis. Also, there has to be a way to track and link together Monitoring Locations, Assessment Units, Designated Uses, and associated Water Quality Criteria in order to facilitate the assessment process. Ensuring consistency for analysis and assessment

Solutions and demonstrations
Mark -

Solutions and demonstrations
Ability to set up translations on characteristics , measurement units, and monitoring location types in AWQMS Ability to configure retrieval of data from the WQP using AWQMS Ability to display WQP data in the AWQMS internal Google map Ability to analyze WQP data alongside our own latest data in AWQMS Mark We implemented an ability to set up translations on characteristics, measurement units, and monitoring location types to help standardize the data, which makes it easier to compare internal agency data with external WQP data. Solutions and demonstrations

Ability to set up translations on characteristics in AWQMS
Mark - Menu makes it easy to navigate to and set up your organization’s Characteristics translations. Ability to set up translations on characteristics in AWQMS

Ability to set up translations on characteristics in AWQMS
Mark – This slid is showing a list of existing translations. A good on to look at is the Carbonate (C03) a couple of rows up from the bottom. The first column is the characteristic name that comes from the WQP. The rest of the column shows how AWQMS will interpret that WQP characteristic name. The Carbonate (CO3) from the WQP will be translated to a WQX characteristic name of Carbonate with a Method Speciation of “as CO3”. And you can set up similar translations for Measurement Units and Monitoring Location Types. Ability to set up translations on characteristics in AWQMS

Ability to configure retrieval of data from the WQP using AWQMS
Mark We can configure batch retrieval of data from the WQP using AWQMS. This screen shot is showing how we’ve set up a new organization in AWQMS but are flagging it as an External organization and the source is the Water Quality Portal. Ability to configure retrieval of data from the WQP using AWQMS

Mark Once that is saved we can then add additional organization preferences for this new organization by clicking the Preferences button. Ability to configure retrieval of data from the WQP using AWQMS

Mark This external organization has been configured to download data from the portal every 60 days. That will occur during non-peak hours. You can also determine whether, when using the AWQMS Google Map, to use previously downloaded data or whether to go get new information from the portal every time. Here, we’ve specified to keeip up to 5 years worth of the external organization’s data in AWQMS. Ability to configure retrieval of data from the WQP using AWQMS

Mark And now we’ve saved our setup and can see the new external organization in our list of organizations. Ability to configure retrieval of data from the WQP using AWQMS

Ability to display WQP data in the AWQMS internal Google map
Mark We can display the WQP data using the AWQMS internal Google map and analyze it alongside agency internal AWQMS data. In this case, I’ve specified to use the new externa organization that we configured and an internal organzation. Let’s pretend that I represent that organization and I want to see if the state is also getting similar readings or results as we’ve been gettting. I click on the Create Map button. Ability to display WQP data in the AWQMS internal Google map

Mark Ability to display WQP data in the AWQMS internal Google map

Mark Here are two locations on the same water body right next to each other. One from WI and one from LDF. Talk about mouse over meta data Talk about clicking on a droplet. Ability to display WQP data in the AWQMS internal Google map

Ability to analyze WQP data alongside our own latest data in AWQMS
Mark So, way to see both organizations’ data together is just using the quick and dirty review tools in AWQMS. We could do this by searching for either Activities or by results, metrics, or index scores. Ability to analyze WQP data alongside our own latest data in AWQMS

Ability to analyze WQP data alongside our own latest data in AWQMS
Mark Ability to analyze WQP data alongside our own latest data in AWQMS

Making a good tool into a great tool
Need Ability to Filter WQP Data by Lat/Long Box Need to efficiently process and track partial WQP Downloads Need to address some of the issues preventing WQP data from being saved in AWQMS Need a separate page for managing/viewing WQP Settings and Status - Credit: Brian Boling, ORDEQ Mark Making a good tool into a great tool

Data Extraction, Utilization, and Assessment
Questions? AWQMS: Data Extraction, Utilization, and Assessment Session C5: Data Soup: Recipes & Secret Spices Chris Adams Quality Assurance & Data Manager Water Quality Division Oklahoma Water Resources Board Mark LeBaron Program Manager & Sr. Business Analyst AWQMS / Gold Systems, Inc. Discuss an innovative approach for using WQP data via AWQMS and some of the challenges we've encountered while using multi-agency data. Examples of challenges - Ensuring consistency of Monitoring Location, Assessment Unit, Designated Uses, and water quality criteria to reduce workload and ensure standardization during assessments - Others??

Data Extraction, Utilization, and Assessment

Similar presentations

Presentation on theme: "Data Extraction, Utilization, and Assessment"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Data Extraction, Utilization, and Assessment

Similar presentations

Presentation on theme: "Data Extraction, Utilization, and Assessment"— Presentation transcript:

Similar presentations

About project

Feedback