Accessing Data through Web Services. IRIS Services – service.iris.edu  FDSN Web services  dataselect  station  event  Documentation Documentation.

Slides:



Advertisements
Similar presentations
IRIS Services, Products, Quality Assurance Efforts, and Potential Links to High Performance Computing in the Era of BIG DATA By T. Ahern, M. Bahavar, R.Casey,
Advertisements

Data Management Workshop
IRIS Services Initiative Improving Data Access and Integration for the GeoSciences Linus Kamb, Joanna Muench, Tim Ahern IRIS Data Management Center.
Multiple Tiers in Action
Tutorial 11: Connecting to External Data
Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System 1 Zaihua Ji Doug Schuster Steven Worley Computational.
Web Interfaces and Data Portals John Porter Department of Environmental Sciences University of Virginia.
Web Programming Language Dr. Ken Cosh Week 1 (Introduction)
The GEON Integrated Data Viewer (IDV) and IRIS DMC Services: CyberInfrastructure Support for Seismic Data Visualization and Interpretation Charles Meertens.
6/1/2001 Supplementing Aleph Reports Using The Crystal Reports Web Component Server Presented by Bob Gerrity Head.
OCLC Online Computer Library Center CONTENTdm Migration Training Craig Yamashita Vice President, Technology and Product Development DiMeMa, Inc. July 2005.
GRAPPA Part of Active Notebook Science Portal project A “notebook” like GRAPPA consists of –Set of ordinary web pages, viewable from any browser –Editable.
By Tim Ahern, Program Manager IRIS DMS A “Short” Introduction to the IRIS Data Management Center Data Holdings, Data Organization, and Data Access.
Chapter 33 CGI Technology for Dynamic Web Documents There are two alternative forms of retrieving web documents. Instead of retrieving static HTML documents,
Server-side Scripting Powering the webs favourite services.
Basics of Web Databases With the advent of Web database technology, Web pages are no longer static, but dynamic with connection to a back-end database.
By Tim Ahern, Rick Benson, Rob Casey, Chad Trabant and Bruce Weertman and many more talented people at the IRIS DMC.
Array Response Functions with ArrayGUI
An introduction to PDCC the Portable Data Collection Center.
By Tim Ahern and Chad Trabant.  Federated Services in Seismology (COOPEUS)  Standardized access to seismological Data Centers resulting in federated.
Dynamic Data Exchanges with the Java Flow Processor Presenter: Scott Bowers Date: April 25, 2007.
AIRNow Web Services Data to Go! Prepared by Steven A. Ludewig, Timothy S. Dye Sonoma Technology, Inc. Petaluma, CA John E. White U.S. Environmental Protection.
Accessing Data Using Web Services. IRIS Services – service.iris.edu FDSN Web services dataselect station event Documentation IRIS web services fedcatalog.
+ IRIS Data Services Shortcourse December 15, 2014 MOMA Room, Palomar Hotel.
EPOS & EarthScope 1 Progress report WP5 Torild van Eck (ORFEUS/EPOS) Tim Ahern (IRIS/EarthScope/SAGE) Chuck Meertens (UNAVCO/EarthScope/GAGE) Fran Boler.
Web Services at IRIS Implementations, Directions, and International Coordination Tim Ahern, Director of Data Services, IRIS Web Services Team: Bruce Weertman,
Dynamic Content On Edge Cache Server (using Microsoft.NET) Name: Aparna Yeddula CS – 522 Semester Project Project URL: cs.uccs.edu/~ayeddula/project.html.
Web services at TRFIC TRFIC has developed the Access Technologies to achieve its goals of interoperability and provide access to data and information on.
FDSN WG III Coordination of Products, Tools and Services.
Management Information Systems MS Access MS Access is an application software that facilitates us to create Database Management Systems (DBMS)
Portable Data Collection Center (PDCC) and the Nominal Response Library (NRL) Tim Ahern.
Putting it all together Dynamic Data Base Access Norman White Stern School of Business.
Chapter 6 Server-side Programming: Java Servlets
MS Access 2007 Management Information Systems 1. Overview 2  What is MS Access?  Access Terminology  Access Window  Database Window  Create New Database.
By Tim Ahern, Program Manager IRIS DMS A “Short” Introduction to the IRIS Data Management Center Data Holdings, Data Organization, and Data Access.
IT at the IRIS DMC: Synergy with the CIG By Tim Ahern, IRIS DMS Program Manager By Tim Ahern, IRIS DMS Program Manager.
By Tim Ahern, Director of Data Services, IRIS A short introduction to the IRIS Data Management System Data Holdings, Data Organization, and Data Access.
Tim Ahern Non Real Time SEED Request Tools. Workshop Representation Europe – ORFEUS Germany – Geofon United States - IRIS FDSN Centers with Significant.
OAI Overview DLESE OAI Workshop April 29-30, 2002 John Weatherley
Web Services The Seismic Data Portal is supported by several web data services, including services providing: event ( waveform (
ECMM6018 Enterprise Networking for Electronic Commerce Tutorial 7
By Tim Ahern, IRIS Director of Data Services Rick Benson, Manoch Bahavar, Alex Hutko, Celso Reyes, Yazan Suleiman Chad Trabant, Bruce Weertman.
Intro to Web Services Dr. John P. Abraham UTPA. What are Web Services? Applications execute across multiple computers on a network.  The machine on which.
K. Harrison CERN, 22nd September 2004 GANGA: ADA USER INTERFACE - Ganga release status - Job-Options Editor - Python support for AJDL - Job Builder - Python.
EarthScope Short Course 2015: Pick Repository Team Picks 4.0 Tiku Ravat (Kentucky) Yenjoe Tan (Columbia) Stefanie Whittaker (Alaska) Yao Yao (Utah)
FDSN Working Group II Data Exchange and Data Centers By Bernard Dost, ORFEUS Data Centre, deBilt Netherland Tim Ahern FDSN Data Center for Continuous Data.
July 27, 2014Metadata Workshop, Bogotå Metadata Help from the DMC Our motivation is to enable networks to perpetually archive their data in open and accessible.
Understanding SEED Headers. SEED is an international standard for the exchange of digital seismological data SEED was designed for use by the earthquake.
Copyright 2007, Information Builders. Slide 1 iWay Web Services and WebFOCUS Consumption Michael Florkowski Information Builders.
Tim Ahern IRIS Director of Data Services Active Source Data Management Workshop January 2014 – Tucson, Arizona.
David Adams ATLAS ATLAS Distributed Analysis and proposal for ATLAS-LHCb system David Adams BNL March 22, 2004 ATLAS-LHCb-GANGA Meeting.
Kai Ruhl, Pablo Beltrami November 2009 Webservices for Distributed Access to Space Weather Models ESWW6, Bruges, Belgium.
Web Services Essentials. What is a web service? web service: software functionality that can be invoked through the internet using common protocols like.
TSDS (HPDE DAP). Objectives (1) develop a standard API for time series-like data, (2) develop a software package, TSDS (Time Series Data Server), that.
Review of PARK Reflectometry Group 10/31/2007. Outline Goal Hardware target Software infrastructure PARK organization Use cases Park Components. GUI /
A brief introduction to NERA – NA3 work package ‘Networking accelerometric networks and strong- motion data users’ Sinan Akkar On behalf of entire NERA-NA3.
Discovery and Metadata March 9, 2004 John Weatherley
June 12, 2004Synthetic Seismogram Exchange Standards But We Have A Standard. Can SEED Support Synthetics? Linus Kamb IRIS DMC.
Station Metadata: What do I Need?
Web Programming Language
Data Formats & Data Structures
Scripting Languages Info derived largely from Programming Language Pragmatics, by Michael Scott.
The FDSN and International Data Exchange
Requesting a Standardized Data Set for the FDSN Network
Requesting a Standardized Data Set for the FDSN Network
The Web Service based approach for data distribution at the IRIS DMC
Coordination of Products, Tools and Services
Introduction to World Wide Web
ExPLORE Complex Oceanographic Data
OPeNDAP/Hyrax Interfaces
Presentation transcript:

Accessing Data through Web Services

IRIS Services – service.iris.edu  FDSN Web services  dataselect  station  event  Documentation Documentation  IRIS web services  timeseries  rotation  sacpz  resp  evalresp  virtualnetwork  traveltime  Flinnengdahl  distaz  products

IRIS Services – easy to use through a variety of tools  IRIS and FDSN Services  Extensive documentation  Builders exist to help understand URL patterns  Key Links  Overall IRIS Services Overall IRIS Services  Documentation Documentation  URL Builders URL Builders

Modern computer languages that include support for basic web services include: Programmatic support is widespread JavaScript R (e.g. Rcurl) C# C/C++ (multiple libraries) Java Perl Python PHP MatLab

Perl Fetch scripts: command line access FetchData FetchEvent FetchMetadata

FetchData retrieves miniSEED, simple metadata, SEED RESP and/or SAC Poles and Zeros using the following selection criteria: Network, Station, Location and Channel  all optional, can contain ‘*’ and ‘?’ wildcards, virtual networks supported Start and end time range Geographic box or circular region Selections: command line, selection list file or BREQ_FAST file FetchData options

Request 1 hour of GSN/ANMO long-period vertical (LHZ) data and simple metadata for M8.8 Chilean earthquake: $ FetchData -N IU –S ‘ANMO’ –L 00 –C ‘LHZ' -s ,06:34:00 -e ,07:34:00 -o /data/Chile-GSN-LHZ.mseed -m /data/Chile-GSN-LHZ.metadata Convert the miniSEED to SAC with metadata $ mseed2sac Chile-GSN-LHZ.mseed –m Chile-GSN- LHZ.metadata -E '2010,058,06:34:11/ / /22.9' FetchData example

2 minutes later… 121 SAC files and a quick-n-dirty record section: FetchData example results

Performance  WS-dataselect has been shown to be able to deliver 5 terabytes of data per day to a single remote user

FetchEvent retrieves event information from ws-event and prints simple ASCII output. Events can be selected using these criteria: Start and end time range Geographic box or circular region Depth range Magnitude range and type Catalog and contributor IRIS event ID Other options: Include secondary origins (default is primary only) Order results by magnitude or time Limit to origins updated after a specific date FetchEvent options

Request events for a 20 minute period including secondary origins: $ FetchEvent -s ,6:30 -e ,6:50 -secondary FetchEvent example

The success of web services

International Coordination  FDSN web services are well coordinated between Europe and the US  Intend to promote them elsewhere  Canada, Japan, China, SE Asia  Many developers producing ws aware clients  ObsPy  SOD  jWeed  WILBER 3

Vertical Integration of Seismological Data Centers  FDSN federated web services  dataselect – for time series observations  station – for metadata describing the recording station  event – info related to earthquakes and events  Fully adopted by the FDSN working groups  Description of the payload (XML, text, etc)  Calling convention ( parameter specifications)

Participants in Federation  Europe  France – RESIF  Germany – GFZ  Italy – INGV  Netherlands – ORFEUS Data Center  Switzerland – ETH  US  IRIS DMC  NCEDC

Standardization of web services IRIS ORFEUShttp:// NCEDChttp://ncedc.org/fdsnws/station/1/ RESIFhttp://ws.resif.fr/fdsnws/station/1/ GEOFONhttp://geofon-open2.gfz-potsdam.de/fdsnws/station/1/ INGVhttp://webservices.rm.ingv.it/fdsnws/station/1/ queryquery?net=IU&sta=ANMO&loc=00&cha=BHZ&starttime= T06:30:00&endtime= T10:30:00&nodata=404net=IU&sta=ANMO&loc=00&cha=BHZ&starttime= T06:30:00&endtime= T10:30:00&nodata=404 Root URL: unique for each data center Query Parameters: standardized

Accessing Information from NCEDC (BK), RESIF (FR), and IRIS (IU)

IRIS Federator  IRIS is developing a federating web service that will allow a user to make a request to a web service at IRIS. This service will return a set of URLs that will allow the software running on a user’s computer to directly access the appropriate data center to service their request IRIS Federator IRIS Federator USER