BioData a new bioassessment database for the USGS Briefing for the CDI

Today  What is BioData?  Why Did We Build It?  Current Capabilities  Future Possibilities  Data Integration/Interoperability Challenges

What is BioData? – in a nutshell A data management, storage, and distribution system for aquatic bioassessment data. data capture data curation data publication

Why We Built It - A Brief History  1992 – National Water-Quality Assessment Program (NAWQA) began collecting bioassessment data (macroinvert, fish, algae, stream habitat)

NAWQA Study Units

Why We Built It - A Brief History  1992 – National Water-Quality Assessment Program (NAWQA) begins collecting bioassessment data (macroinvert, fish, algae, stream habitat)  1992 – 1999: Local data management and national data aggregations  1999 – NAWQA national bioassessment database – (BioTDB)

WRD Needs Assessment (2006)  Surveyed WRD Science Centers to find out:  How much aquatic ecology data is being collected outside the NAWQA Program?  What kinds?  What methods?  Where and how are data being stored?

What We Discovered  Water collaborative projects with other agencies, states, localities, and partners are producing as much data as the NAWQA Program  80 % of WSC’s reported projects collecting aquatic ecology data  120 projects had a macroinvertebrate, fish, algae, or habitat component (2000 – 2005)  Approximately 15,000 samples  The majority of samples are being collected using NAWQA and USEPA national stream bioassessment protocols  Samples are being sent to a variety of taxonomic labs

What We Discovered  The data are stored electronically, but are very difficult to discover, access, and integrate  47% in Excel  13% are in EPA databases  19% in home-grown relational databases 79%

U.S. Department of the Interior U.S. Geological Survey BioData a new bioassessment database for the USGS briefing for the USGS GCMRC 5/9/2011

What Should We Do? 1. Do nothing? 2. Implement a federated system? 3. Incrementally refurbish existing NAWQA database? 4. Redesign and “re-build” using modern, web- enabled, extensible architecture? (BioData)

Biodata - Version 1 Objective A data storage, retrieval, and distribution system for aquatic bioassessment data most commonly produced by USGS WRD projects.

“Most Commonly Produced”  Project Objectives  Setting  Types of Data  Sampling Protocols  Bioassessment and monitoring  Streams and rivers  Macroinvertebrates  Fish  Algae  Study reach habitat  NAWQA  USEPA

Additional Characteristics  An internet application  Available to any USGS ecologist.  Designed to be adapted and extended  Support scientific workflow  Serve as an online data archive  Curate taxonomic nomenclature - map it forward and harmonize it across all the data  Support biologist lab data exchange  Readily add web data services

BioData Retrieval (DWH) project data management BioData Input data distribution field datalab data field data input data exchange with labs data review external data NAWQA legacy data public web site web data services application- specific output

Data Retrieval Features  Real-time feedback on how many samples your query will return  Save the query to your desktop – then to friends for them to run  Variety of file formats  Multiple data sets downloaded in one step

Data Retrieval Demo 

BioData Retrieval (DWH) project data management BioData Input data distribution field datalab data field data input data exchange with labs data review external data NAWQA legacy data public web site web data services application- specific output

Data Input/Management Features  Retrieve restricted (unreleased) data  Manage and organize data by project  Project control over rights to enter and edit data  Built in help and data validation checks  Auto-saving  Data entry screens tailored to field sheets  Send electronic orders to labs

Data Input/Mgt Demo

Data integration – touchpoints  First challenge – find the data  Second challenge - compatible methods?

Data integration – touchpoints  First challenge – find the data  Second challenge - compatible methods?  Third challenge – get the data  We need to pick a data exchange standard

Data integration – touchpoints  First challenge – find the data  Second challenge - compatible methods?  Third challenge – get the data  Fourth challenge – harmonize taxonomy  Does “Thienemannimyia group” = “Thienemannimyia gr.” ??  Does ITIS solve this?

ITIS

 Only handles published names  We have to handle unpublished names  Provisional = new taxon claimed but not “officially” published  Conditional = uncertain or indeterminate identification, e.g. “Thienemannimyia group”  ITIS is not complete for all groups  Fish – good, we can integrate tightly with it  Macroinvertebrates – doable  Algae – ITIS not ready yet

Data integration – touchpoints  First challenge – find the data  Second challenge - compatible methods?  Third challenge – get the data  Fourth challenge – harmonize taxonomy  Does “Thienemannimyia group” = “Thienemannimyia gr.” ??  Fifth challenge – integrate with physio- chemical and ancillary data  Common geospatial framework would help

NHD  Which NHD?  NHD “snap to” service with API’s that developers could use in their application(s)?  Service to translate NHD address to other versions of NHD (and future)

BioData For more information contact: Pete Ruhl

U.S. Department of the Interior U.S. Geological Survey BioData a new bioassessment database for the USGS briefing for the USGS GCMRC 5/9/2011

NAWQA BioTDB Database  NAWQA data from present  2,294 sites  21,689 samples  6,715 macroinvertebrate community samples  2,819 fish community samples  8,749 algae community samples  2,819 reach habitat assessments  > 1,200,000 specimen records