By Tim Ahern, Rick Benson, Rob Casey, Chad Trabant and Bruce Weertman and many more talented people at the IRIS DMC.

Slides:



Advertisements
Similar presentations
TeraGrid Deployment Test of Grid Software JP Navarro TeraGrid Software Integration University of Chicago OGF 21 October 19, 2007.
Advertisements

Supersites infrastructure: data access services and tools Chuck Meertens, Jeff McWhirter, Fran Boler, Stuart Wier, Susanna Gross, Scott Baker, UNAVCO Supersites.
IRIS Services, Products, Quality Assurance Efforts, and Potential Links to High Performance Computing in the Era of BIG DATA By T. Ahern, M. Bahavar, R.Casey,
Test Case Management and Results Tracking System October 2008 D E L I V E R I N G Q U A L I T Y (Short Version)
ASCR Data Science Centers Infrastructure Demonstration S. Canon, N. Desai, M. Ernst, K. Kleese-Van Dam, G. Shipman, B. Tierney.
03/17/2014 Data Management System -Data Services- Temporary Experiment Data Brief Data Services News MUSTANG update- QC on PASSCAL Data PASSCAL SC Spring.
PREMIS in Thought: Data Center for LC Digital Holdings Ardys Kozbial, Arwen Hutt, David Minor February 11, 2008.
IRIS Services Initiative Improving Data Access and Integration for the GeoSciences Linus Kamb, Joanna Muench, Tim Ahern IRIS Data Management Center.
Oct 31, 2000Database Management -- Fall R. Larson Database Management: Introduction to Terms and Concepts University of California, Berkeley School.
Development of a Community Hydrologic Information System Jeffery S. Horsburgh Utah State University David G. Tarboton Utah State University.
DataGrid Kimmo Soikkeli Ilkka Sormunen. What is DataGrid? DataGrid is a project that aims to enable access to geographically distributed computing power.
Improved Quality Control for Seismic Networks ---MUSTANG.
Chapter 1 Introduction to Databases
Earthquake Seismology waveform data – research resource – data challenge Torild van Eck (ORFEUS, FDSN, NERIES, NERA) ORFEUS NGO (since 1987)
Network developments in Europe Torild van Eck Web site: FDSN General Assembly. July 8, Perugia, Italy.
The GEON Integrated Data Viewer (IDV) and IRIS DMC Services: CyberInfrastructure Support for Seismic Data Visualization and Interpretation Charles Meertens.
August 13-19, 2010Data Management Workshop Foz do Iguassu- Brazil Seismic Quality Assurance Rick Benson IRIS DMC Rick Benson IRIS DMC.
Bob Woodward, Bob Busby, Katrin Hafner, and David Simpson
About CUAHSI The Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) is an organization representing 120+ universities.
MEDIN Data Guidelines. Data Guidelines Documents with tables and Excel versions of tables which are organised on a thematic basis which consider the actual.
By Tim Ahern, Program Manager IRIS DMS A “Short” Introduction to the IRIS Data Management Center Data Holdings, Data Organization, and Data Access.
Array Response Functions with ArrayGUI
Controlled Vocabularies (Term Lists). Controlled Vocabs Literally - A list of terms to choose from Aim is to promote the use of common vocabularies so.
By Tim Ahern and Chad Trabant.  Federated Services in Seismology (COOPEUS)  Standardized access to seismological Data Centers resulting in federated.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES High Performance Computing applications in GEON: From Design to Production Dogan Seber.
September 11, 2015 The IRIS Data Management System Operations Overview & Data Curation Rick Benson IRIS DMC Operations Managing Data from Seismic Networks,
Accessing Data Using Web Services. IRIS Services – service.iris.edu FDSN Web services dataselect station event Documentation IRIS web services fedcatalog.
+ IRIS Data Services Shortcourse December 15, 2014 MOMA Room, Palomar Hotel.
 Web Services and Data Products DSSC Fall 2014 Manoch Bahavar Alex Hutko Yazan Suleiman Bruce Weertman Mike Stults Mick Van Fossen Robert Weekly Chad.
EPOS & EarthScope 1 Progress report WP5 Torild van Eck (ORFEUS/EPOS) Tim Ahern (IRIS/EarthScope/SAGE) Chuck Meertens (UNAVCO/EarthScope/GAGE) Fran Boler.
Accessing Data through Web Services. IRIS Services – service.iris.edu  FDSN Web services  dataselect  station  event  Documentation Documentation.
EGY Workshop 2008, Boulder, CO The EarthScope Data Portal Charles Meertens 3, Chaitan Baru 1, Tim Ahern 2, Adrian Borsa 3, Kurt Behrends 4, Robert Casey.
Web Services at IRIS Implementations, Directions, and International Coordination Tim Ahern, Director of Data Services, IRIS Web Services Team: Bruce Weertman,
FDSN WG III Coordination of Products, Tools and Services.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
By Tim Ahern, Program Manager IRIS DMS A “Short” Introduction to the IRIS Data Management Center Data Holdings, Data Organization, and Data Access.
DPC High-resolution multi-disciplinary monitoring of active fault test-site areas in Italy S5S5S5S5 Roma 24/03/ 2010 S5 Project High-resolution.
IT at the IRIS DMC: Synergy with the CIG By Tim Ahern, IRIS DMS Program Manager By Tim Ahern, IRIS DMS Program Manager.
By Tim Ahern, Director of Data Services, IRIS A short introduction to the IRIS Data Management System Data Holdings, Data Organization, and Data Access.
Automated Data Quality Assurance: QUACK and MUSTANG Mary Templeton IRIS Data Management Center Mary Templeton IRIS Data Management Center.
Tim Ahern Non Real Time SEED Request Tools. Workshop Representation Europe – ORFEUS Germany – Geofon United States - IRIS FDSN Centers with Significant.
NQuery: A Network-enabled Data-based Query Tool for Multi-disciplinary Earth-science Datasets John R. Osborne.
The Global Land Cover Facility is sponsored by NASA and the University of Maryland.The GLCF is a founding member of the Federation of Earth Science Information.
06-1L ASTRO-E2 ASTRO-E2 User Group - 14 February, 2005 Astro-E2 Archive Lorella Angelini/HEASARC.
Key foundational element of IRIS NSF funded the construction of the GSN (DoD funds also) Beginning in 1998 USGS line item for GSN established to take.
By Tim Ahern, Director of Data Services, IRIS An Overview of IRIS Incorporated Research Institutions for Seismology.
H. Widmann (M&D) Data Discovery and Processing within C3Grid GO-ESSP/LLNL / June, 19 th 2006 / 1 Data Discovery and Basic Processing within the German.
SEEK Science Environment for Ecological Knowledge l EcoGrid l Ecological, biodiversity and environmental data l Computational access l Standardized, open.
1 ORFEUS board meeting update Torild van Eck G öteborg, Sweden July 25, 2013.
By Tim Ahern, IRIS Director of Data Services Rick Benson, Manoch Bahavar, Alex Hutko, Celso Reyes, Yazan Suleiman Chad Trabant, Bruce Weertman.
GEO Online Detector Characterization System R. Balasubramanian Cardiff University LSC March 2003 LIGO-G Z.
1 Object-Oriented Analysis and Design with the Unified Process Figure 13-1 Implementation discipline activities.
Understanding SEED Headers. SEED is an international standard for the exchange of digital seismological data SEED was designed for use by the earthquake.
By Tim Ahern Program Manager IRIS Data Management System Interoperability Developments at IRIS.
Tim Ahern IRIS Director of Data Services Active Source Data Management Workshop January 2014 – Tucson, Arizona.
© 2012 IBM Corporation IBM Linear Tape File System (LTFS) Overview and Demo.
Thurs Nov 12, 12:45 Nov 8-17, 2009Data Management Workshop Cairo, Egypt.
Data Acquisition, Diagnostics & Controls (DAQ)
Data challenges in Earthquake Seismology
Data Formats & Data Structures
An Overview of IRIS and IRIS Data Services
The FDSN and International Data Exchange
ISPAQ: IRIS System for Portable Assessment of Quality
Requesting a Standardized Data Set for the FDSN Network
Requesting a Standardized Data Set for the FDSN Network
Presented by Munezero Immaculee Joselyne PhD in Software Engineering
GLOBAL BIODIVERSITY INFORMATION FACILITY
The Web Service based approach for data distribution at the IRIS DMC
Growing importance of metadata for synthetics: Calculating and Sharing Synthetic Seismic Data Dogan Seber University of California, San Diego San Diego.
PLANNING A SECURE BASELINE INSTALLATION
Presentation transcript:

By Tim Ahern, Rick Benson, Rob Casey, Chad Trabant and Bruce Weertman and many more talented people at the IRIS DMC

 Reliability of IRIS Data Services  Improvements in Data Quality monitoring  MUSTANG  Vertical Integration in seismology  Federated Services (COOPEUS)  IRIS Federator  Horizontal Integration in Earth Sciences  Deploying web services across geosciences

 Historically IRIS has operated a primary data center in Seattle, Washington  Backup system for redundant copies of data files, database files, software, etc.  Primarily for protecting assets in case of a major catastrophe  IRIS currently operates a second facility in the San Francisco Bay Area near a High Performance Computing installation (LLNL)  (Cycles Close to Data effort)

ADC1 DBMS Wave forms Wave forms Seattle DBMS Wave forms Wave forms Ingestion BUD Real Time System File Ingestion System Web Services - Entire suite Breqfast WILBER3 MUSTANG SeismiQuery Breqfast Requests WebRequest Load Balancer Load Balancer

MUSTANG Metric Estimators Gaps, overlaps, completeness, signal to noise, power density, pdf mode changes, Glitches, (~24 metrics in phase 2) MUSTANG Metric Estimators Gaps, overlaps, completeness, signal to noise, power density, pdf mode changes, Glitches, (~24 metrics in phase 2) PostgreSQL Database Data Quality Technician Domestic & Non-US Network Operators Domestic & Non-US Network Operators Archived and Real Time Data

 FDSN federated web services  dataselect – for time series observations  station – for metadata describing the recording station  event – info related to earthquakes and events  Fully adopted by the FDSN working groups  Description of the payload (XML, text, etc)  Calling convention ( parameter specifications)

 Europe  France – RESIF  Germany – GFZ  Italy – INGV  Netherlands – ORFEUS Data Center  Switzerland – ETH  US  IRIS DMC  NCEDC

IRIS ORFEUShttp:// NCEDChttp://ncedc.org/fdsnws/station/1/ RESIFhttp://ws.resif.fr/fdsnws/station/1/ GEOFONhttp://geofon-open2.gfz-potsdam.de/fdsnws/station/1/ INGVhttp://webservices.rm.ingv.it/fdsnws/station/1/ queryquery?net=IU&sta=ANMO&loc=00&cha=BHZ&starttime= T06:30:00&endtime= T10:30:00&nodata=404net=IU&sta=ANMO&loc=00&cha=BHZ&starttime= T06:30:00&endtime= T10:30:00&nodata=404 Root URL: unique for each data center Query Parameters: standardized More information will be presented on Thursday at 8:45 AM in SM1.3 Room G3 Integrated Research Infrastructures and Services to users: supporting excellence in a science for society

 IRIS is developing a federating web service that will allow a user to make a request to a web service at IRIS. This service will return a set of URLs that will allow the software running on a user’s computer to directly access the appropriate data center to service their request IRIS Federator IRIS Federator USER

 To simplify  Data discovery ▪ Standard and simplified web services supporting space- time (and more) queries  Data access ▪ Simplified services also mean simple clients ▪ PERL, MatLab, R, wget, etc  Data Usability ▪ When possible standard widely used formats will be supported and when reasonable text output formats will be available to aid in interdisciplinary access

Geo WS IRIS UNAVCO CUAHS I SDSC CUAHS I SDSC Columbia IEDA Columbia IEDA Unidata Caltech GPlates Caltech GPlates CINERGY B-Cube GGP UTEP Gravity UTEP Gravity Inter Magnet Inter Magnet Structural Geology Structural Geology NEON NGDC OOI WOVODAT RAMADDA Long Tail Data

 Standardized space-time queries for 14 geosciences data types/centers  Data discovery client  Standardized documentation  URL builders  GUI to URL builders to provide proper URL construction  Development of Simple clients  Standard and Simple cross-domain formats developed

LLNL DBMS Wave forms Wave forms Seattle DBMS Wave forms Wave forms Web Services With Research Readiness Web Services With Research Readiness Scriptable Event Extraction Scriptable Event Extraction Event Products Research Ready Formatted for HPC ADIOS HDF5 other

MUSTANG Metric Estimators Gaps, overlaps, completeness, signal to noise, power density, pdf mode changes, Glitches, (~24 metrics in phase 2) MUSTANG Metric Estimators Gaps, overlaps, completeness, signal to noise, power density, pdf mode changes, Glitches, (~24 metrics in phase 2) PostgreSQL Database Data Quality Technician Domestic & Non-US Network Operators Domestic & Non-US Network Operators Researcher Specifies Required Data Metric Constraints DMC Filters Data Request Using Defined Constraints Filtered Data Request Returned to Researcher Archived and Real Time Data Research Ready Data Sets