National Center for Supercomputing Applications University of Illinois at Urbana–Champaign Great Lakes to Gulf Virtual Observatory Marcus Slavenas September.

Slides:

Advertisements

Similar presentations

X-SIGMA (An XML based Simple data Integration system for Gathering, Managing and Accessing scientific experimental data in grid environments) Karpjoo

Advertisements

Pentaho Open Source BI Goldwin. Pentaho Overview Pentaho is the commercial open source software for Business Pentaho is the commercial open source software.

Data Management Updates Kristen Gunthardt, EPA OW Nate Booth, USGS CIDA NWQMC February 1, 2010.

GCERC German Coastal Engineering Research Council Rainer Lehfeldt Federal Waterways Engineering and Research Institute - Coastal Division (BAW), Hamburg.

The Visibility Information Exchange Web System is a database system and set of online tools originally designed to support the Regional Haze Rule enacted.

Development of a Community Hydrologic Information System Jeffery S. Horsburgh Utah State University David G. Tarboton Utah State University.

Cushing – gCORE Eco Observatories Ecological Observatories MES gCORE, Fall 2010 Judy Cushing Why Ecological Observatories? Where are they? Where do they.

Robust Tools for Archiving and Preserving Digital Data Joseph JaJa, Mike Smorul, and Mike McGann Institute for Advanced Computer Studies Department of.

16 months…. The Visibility Information Exchange Web System is a database system and set of online tools originally designed to support the Regional Haze.

Deployment and Evaluation of an Observations Data Model Jeffery S Horsburgh David G Tarboton Ilya Zaslavsky David R. Maidment David Valentine

SAN DIEGO SUPERCOMPUTER CENTER Developing a CUAHSI HIS Data Node, as part of Cyberinfrastructure for the Hydrologic Sciences David Valentine Ilya Zaslavsky.

0-1 Team ?? Status Report (1 of 3) Client Contact –Point 1 –Point 2 Team Meetings –Point 1 –Point 2 Team Organization –Point 1 –Point 2 Team 1: Auraria.

An End-to-End System for Publishing Environmental Observations Data Jeffery S. Horsburgh David K. Stevens, David G. Tarboton, Nancy O. Mesner, Amber Spackman.

U.S. Department of the Interior U.S. Geological Survey U.S. National Water Census “Cyber – Platform” Update Progress and challenges to overcome in realizing.

NWS Calibration Workshop, LMRFC March, Historical Data Access Basic Calibration Workshop March 10-13, 2009 LMRFC.

U.S. Department of the Interior U.S. Geological Survey Data Integration Progress and Guiding Principles Disciplines, generalization, and open-access. David.

U.S. Department of the Interior U.S. Geological Survey CDI Data Management Working Group December 12, 2011 Sally Holl, USGS Texas Water Science Center.

U.S. Environmental Protection Agency WATERS Status Update

Spatial Water Data Subcommittee Update to Coordination Group Wendy Blake-Coleman, EPA (Representing Tod Dabolt, EPA & Bob Pierce, USGS)

NIWA National Science Centre for Environmental Information Jochen Schmidt, Chief Scientist Federated Information Infrastructure.

material assembled from the web pages at

Clean Black Sea A Web Portal for the SSA Project Ch.Belias, N.Dimakopoulos, M.Dassenakis University of Athens Laboratory of Environmental Chemistry.

1 Data Integration Community of Practice Meeting September 15, 2009 Science Data Integration.

Steve Dower Software Engineer Python Tools for Visual Studio.

UWG 2013 Meeting PO.DAAC Web Services Demo. What are PO.DAAC Web Services?

BioData a new bioassessment database for the USGS Briefing for the CDI

Transforming Agricultural Ecosystem Management with Web 2.0 Technology Doug Miller Center for Environmental Informatics Earth and Environmental Systems.

BIEN Confederated DB (S) Analytical DB(s) Heterogeneous source database(s) of Plots/Specimens/Occurrences Synonymy Names Reference taxonomy *** *** Feedback.

CUAHSI Hydrologic Information Systems David R. Maidment Center for Research in Water Resources University of Texas at Austin and Ilya Zaslavsky, David.

Users/Historical Data Working Group Update to Coordination Group Butch Lazorchak Library of Congress Chair, U/HDWG September 9, 2014 Coordination.

HYDROLOGY PATHFINDER WITH A FOCUS ON WETLANDS. GOALS After the Eluminate session with Robert Bruce, and assignment 9 called for developing a small pathfinder,

U.S. Department of the Interior U.S. Geological Survey USGS Water Data Exchange Services USGS Office of Water Information June 2009 Nate Booth, Dave Briar.

National Center for Supercomputing Applications Barbara S. Minsker, Ph.D. Associate Professor National Center for Supercomputing Applications and Department.

HYDROLOGIC DATA. BACKGROUND Analysis and synthesis of data is required to perform any hydrologic computation. The engineer needs to: Identify and define.

CUAHSI HIS Features of Observations Data Model. NWIS ArcGIS Excel NCAR Trends NAWQA Storet NCDC Ameriflux Matlab AccessSAS Fortran Visual Basic C/C++

Information Management using Ecological Metadata Language Corinna Gries - CAP Margaret O’Brien - SBC.

1 Web Services USGS/EPA Collaboration. 2 Overview  EPA and USGS are working together to develop a commons suite of web services  These services will.

Mercury – A Service Oriented Web-based system for finding and retrieving Biogeochemical, Ecological and other land- based data National Aeronautics and.

September 2012 Developed by Agricultural and Biological Engineering Department at Purdue University and Department of Regional Infrastructures Engineering.

MDPHnet & ESP Data Partner Participation Overview The following slides describe the necessary steps for a data partner to participate in the MDPHnet Network.

U.S. Department of the Interior U.S. Geological Survey Decision Support Tools and USGS Data Management Best Practices Cassandra Ladino USGS Chesapeake.

Data Model / Database Implementation (continued) Jeffery S. Horsburgh Hydroinformatics Fall 2014 This work was funded by National Science Foundation Grants.

WFM 6202: Remote Sensing and GIS in Water Management © Dr. Akm Saiful IslamDr. Akm Saiful Islam WFM 6202: Remote Sensing and GIS in Water Management Dr.

National Center for Supercomputing Applications University of Illinois at Urbana–Champaign Data Management for the Mississippi River Luigi Marini September.

Larry Theller, Bernie Engel, Youn Shik Park for Purdue University ABE and Office of Indiana State Chemist September 27-28, 2012 “Purdue Load Duration Curve.

Implement Viewing Transactions in Real Time James Payne Managing Director for New Media / Advancement July 27, 2015.

Data analytics and mash-up Real time analytics of employment data Team Shadowfax 1/25/2016 CMPE Class Project 0.

Users/Historical Data Working Group Update to Coordination Group Jacqueline Nolan Congressional Cartography Program Library of Congress March.

Global Change Master Directory (GCMD) Mission “To assist the scientific community in the discovery of Earth science data, related services, and ancillary.

Data and Metadata Archiving: Atlantic Coast Environmental INdicators Consortium (ACE INC) Lexia M. Valdes June 11, 2003 R

Napa Valley Watershed Information Center Presentation to the Napa River Watershed Conservancy and WIC Board December 18, 2002.

U.S. Department of the Interior U.S. Geological Survey Stewardship of the National Hydrography Dataset Elizabeth McCartney National Geospatial Technical.

U.S. Department of the Interior U.S. Geological Survey Water Data Sharing Using OGC Standards and the Internet David Blodgett USGS Center for Integrated.

Data Management: Data Processing Types of Data Processing at USGS There are several ways to classify Data Processing activities at USGS, and here are some.

The Virtual Observatory and Ecological Informatics System (VOEIS): Using RESTful architecture and an extensible data model to provide a unique data management.

Larry Theller, Bernie Engel, Youn Shik Park for Purdue University ABE and Office of Indiana State Chemist September 27-28, 2012 “Purdue Load Duration Curve.

WV DOT Scanning Project

Dwane Young, U.S. EPA Office of Water

Delivery of Science Components to NRCS Business Applications

Jeffery S. Horsburgh Utah State University

Lecture 8 Database Implementation

KISTERS TimeSeries HUB

Corps Water Management System (CWMS) Modernization

GIS Applications in the Water Management Sector

Transit ITS Data Exchange Specification TIDES

The Interoperable Watersheds Network

Project BCHB697.

Astronomy Visualization Service

DIBBs Brown Dog BDFiddle

Presentation transcript:

National Center for Supercomputing Applications University of Illinois at Urbana–Champaign Great Lakes to Gulf Virtual Observatory Marcus Slavenas September 2nd, 2015

Outline Motivation / Funding Architecture Front End Interface Data – Sources and Parsing Questions

Motivation and Funding Hypoxic Zone Aggregate Sources for: Scientists Decision Makers

Software Architecture Long Term Archive Geospatial Database Mongodb: Raw Data Postgres: Geospatial Cache Clowder: Web service (API) Data Manage Geodashboard: Visualize Search Retrieve Parsed DataRaw Data

Explore the Data Explore Data By: Sources Reaches Watershed

Explore Data Time Scale Binning Nitrate Load Cumulative Load

Calculations Gap Filling Load Calculation

Compare

Search Location Time Source Parameter

Search – River Reaches and Watershed

Raw Data Archive

Raw Files - Provenance Trail

CURRENT DATA SOURCES US Geological Survey Nitrate, Discharge -> Load (More Coming) ~Real Time U.S. Army Corps of Engineers Long Term Monitoring Program Quarterly National Oceanic and Atmospheric Administration Temperature ~Real Time Water Quality Portal (USGS, USEPA, USDA) Mostly for historical data Great Rivers Ecological Observation Network Water Quality and Environmental Data ~Real Time MORE…

Extract Transform Load (ETL) Python Scripting Extract 1.Through Web Service 2.From Files Generalization Transform All data parsed to similar format JSON/GEOJSON Load Through Clowder Web Service

Datapoint ISO 8601 With timezone Open JSON document GeoJSON Geolocation

Data Storage Structure/Format DatapointStreamSensor/Site

Provenance Trail

Questions? Contact Michael Brennan Marcus Slavenas Live Site