Data Management Workshop

Slides:



Advertisements
Similar presentations
1 caAdapter Jan 24, caAdapter The caAdapter is an open source tool that facilitates HL7 version 3 message building, parsing and validation based.
Advertisements

Antelope Relational Database Management System A practical introduction.
IRIS Services Initiative Improving Data Access and Integration for the GeoSciences Linus Kamb, Joanna Muench, Tim Ahern IRIS Data Management Center.
An Introduction to Earthquake Analysis Tools (Part 1) Bill Langin 10/2/02.
CONTENT: A model for collaborative database building Trevor Bond Alan Cornish Washington State University Libraries.
Simple Web SQLite Manager/Form/Report
PDS Radio Science Dick Simpson Stanford University Stanford Status PDS4 Status at Stanford o Testing o Mission Support o Migration Proposal o Tools o Issues.
What is Unix Prepared by Dr. Bahjat Qazzaz. What is Unix UNIX is a computer operating system. An operating system is the program that – controls all the.
August 13-19, 2010Data Management Workshop Foz do Iguassu- Brazil Seismic Quality Assurance Rick Benson IRIS DMC Rick Benson IRIS DMC.
SENG521 (Fall SENG 521 Software Reliability & Testing Software Reliability Tools (Part 8a) Department of Electrical & Computer.
PQLX - A Station Assessment & Data Quality Control System
OCLC Online Computer Library Center CONTENTdm Migration Training Craig Yamashita Vice President, Technology and Product Development DiMeMa, Inc. July 2005.
By Tim Ahern, Program Manager IRIS DMS A “Short” Introduction to the IRIS Data Management Center Data Holdings, Data Organization, and Data Access.
By Tim Ahern, Rick Benson, Rob Casey, Chad Trabant and Bruce Weertman and many more talented people at the IRIS DMC.
Mini-SEED Fundamentals
An introduction to PDCC the Portable Data Collection Center.
NetLogo Dr. Feng Gu. NetLogo NetLogo is a programmable modeling environment for simulating natural and social phenomena, authored by Uri Wilensky in 1999.
Accessing Data Using Web Services. IRIS Services – service.iris.edu FDSN Web services dataselect station event Documentation IRIS web services fedcatalog.
Should our academic community be more formal in honoring true SEGY? David Okaya Univ. Southern California IEEE OBS gather read in as IBM (A) Are we overly.
Accessing Data through Web Services. IRIS Services – service.iris.edu  FDSN Web services  dataselect  station  event  Documentation Documentation.
Web Services at IRIS Implementations, Directions, and International Coordination Tim Ahern, Director of Data Services, IRIS Web Services Team: Bruce Weertman,
Portable Data Collection Center (PDCC) and the Nominal Response Library (NRL) Tim Ahern.
Thurs Nov 12, 13:45. PQLX - A Station Assessment & Data Quality Control System Applications and Uses.
Tuesday Nov 10, 13:30 November 8-17, 2009Data Management Workshop Cairo, Egypt.
October 21-26, 2007Data Management Workshop Kuala Lumpur, Malaysia Using SEED Using: Jrdseed PQL SAC RESP JPlotResp Using: Jrdseed PQL SAC RESP JPlotResp.
Portable Broadband Seismic Stations - Checking Data Quality Cairo, Egypt November, 2009 Noel Barstow Bruce Beaudoin Marcos Alvarez.
Facilitate – Collaborate – Educate Tuesday, January 10, 11:05am Data Management Workshop Bangkok Thailand, January 2011.
By Tim Ahern, Program Manager IRIS DMS A “Short” Introduction to the IRIS Data Management Center Data Holdings, Data Organization, and Data Access.
The Statistics New Zealand Prototype PREMIS creation tool Euan Cochrane PREMIS Fair October 2009
Rick Benson Saturday, Aug 14, 14:00 August 13-19, 2010Data Management Workshop Foz do Iguassu- Brazil.
By N.Gopinath AP/CSE Cognos Impromptu. What is Impromptu? Impromptu is an interactive database reporting tool. It allows Power Users to query data without.
By Tim Ahern, Director of Data Services, IRIS A short introduction to the IRIS Data Management System Data Holdings, Data Organization, and Data Access.
Introduction to PQLX Dr. Mary Templeton IRIS Data Management Center Dr. Mary Templeton IRIS Data Management Center.
Automated Data Quality Assurance: QUACK and MUSTANG Mary Templeton IRIS Data Management Center Mary Templeton IRIS Data Management Center.
SIG: Synthetic Seismogram Exchange Standards (formats & metadata) Is it time to establish exchange standards for synthetic seismograms? IRIS Annual Workshop.
Navigation and Ancillary Information Facility NIF Introduction to WebGeocalc October 2014 SPICE components and services are not restricted under ITAR and.
F Drag and Drop Controls Display and Builder (Synoptic Display) Timofei Bolshakov, Andrey Petrov Fermilab Accelerator Controls Department March 26, 2007.
ORFEUS Torild van Eck & Chad Trabant Why is ORFEUS interested in distributed OO computing? Presentation ORFEUS Present status Orfeus Data Center ORFEUS.
August 2003 At A Glance The IRC is a platform independent, extensible, and adaptive framework that provides robust, interactive, and distributed control.
Converting Waveform Data to Mini-SEED. August 2010 Foz do Iguaçu - Brasil IRIS Data Management Workshop Mini-SEED Fundamentals 1 Mini-SEED is a bare time.
Using RSNA’s Teaching File Software (MIRC): A Hands on Course Mary Wyers, MD.
An introduction to CDO, NCL and PRECIS utilities
FISSURES SEED Type = 50 Field 1 = yyy Field 2 = yyy Field 3 = yyy Etc. Type = 10 Field 1 = xxx Field 2 = xxx Field 3 = xxx Etc. Type = data Field 1 = ttt.
Building a Network Dataless SEED Volume with PDCC.
Facilitate – Collaborate – Educate Thursday January 12, 15:30 January 10, 2012 Metadata Workshop, Bangkok, Thailand.
Building Station Metadata with PDCC
Real-time Waveform Data Exchange. November 2009 Cairo, Egypt IRIS - ORFEUS Data Management Workshop Real-time Data Exchange 1 Allows automated integration.
July 27, 2014Metadata Workshop, Bogotå Metadata Help from the DMC Our motivation is to enable networks to perpetually archive their data in open and accessible.
Station Metadata The NRL (Nominal Response Library) Dr. Mary Templeton IRIS Data Management Center Managing Data from Seismic Networks September
Updating SEED Responses in PDCC After Calibration.
Understanding SEED Headers. SEED is an international standard for the exchange of digital seismological data SEED was designed for use by the earthquake.
Min-Hung Shih 1/22/2015 Ver Analyst Review (I)  dbevents – display sesmic events and wavefroms $dbevents demo.
Future: PDCC and Spreadsheets Information About a New “Branch” of PDCC.
Tim Ahern IRIS Director of Data Services Active Source Data Management Workshop January 2014 – Tucson, Arizona.
CAA Database Overview Sinéad McCaffrey. Metadata ObservatoryExperiment Instrument Mission Dataset File.
SAC: An Overview SAC ( “Seismic Analysis Code” ) was originally developed by LLNL SAC ( “Seismic Analysis Code” ) was originally developed by LLNL An interactive.
Thurs Nov 12, 12:45 Nov 8-17, 2009Data Management Workshop Cairo, Egypt.
June 12, 2004Synthetic Seismogram Exchange Standards But We Have A Standard. Can SEED Support Synthetics? Linus Kamb IRIS DMC.
SPS Spotlight Series October 2014
Station Metadata: What do I Need?
Data Formats & Data Structures
ISPAQ: IRIS System for Portable Assessment of Quality
Requesting a Standardized Data Set for the FDSN Network
Requesting a Standardized Data Set for the FDSN Network
What is FITS? FITS = Flexible Image Transport System
Station Metadata: What do I Need?
The Web Service based approach for data distribution at the IRIS DMC
[2001] Delay Time Information Blockette (36 bytes)
Monday January 10, 14:05 Metadata Workshop, Bangkok, Thailand
Coordination of Products, Tools and Services
Presentation transcript:

Data Management Workshop More Data Center Tools Rick Benson IRIS DMC October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

SHAPE Software SEED Header Assembly and Preparation Engine SHAPE - Build dataless SEED files from SEED RESP and other info Controlled with a Header Assembly Template (HAT) file in XML format wherein SHAPE finds the needed details for building a SEED file including names of RESP files and blockette field values Easily used in scripts, good for batch creation of dataless SEED, complimentary to PDCC at this point Good for templating (common components defined once) Java, runs anywhere a JVM is available Available from http://www.isti.com/~sid/shape/ October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

SHAPE HAT Examples SHAPE Header Assembly Template examples Field substitution: Very simple example: October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

Portable SEED File Reading Jrdseed - Java port of the IRIS-DMC’s ‘rdseed’ Reads full SEED, dataless SEED and Mini-SEED+dataless Writes SEED, SAC, AH, CSS, SEGY, SEED RESP Time-window and other output selection possible Runs anywhere a JVM is available Based on generic JavaSeed classes available separately Not as complete as ‘rdseed’ yet October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

libmseed - Mini-SEED Library Allows reading and writing of Mini-SEED records Written in C Supports all structures of SEED 2.4 with a few minor caveats (blockette 2000 and blockette 405) Supports all valid record lengths and big or little endian Supports ASCII, int-16, int-32, float-32, float-64, Steim-1 and Steim-2 encoding formats for reading and writing; supports SRO/ASRO, DWWSSN and GEOSCOPE encoding formats read-only Manages arbitrarily long continuous trace segments and groups of continuous trace segments composed, for example, of many Mini-SEED records Tested on Solaris, Linux, Mac OSX and MS-Windows Available from: http://www.iris.edu/pub/programs/ October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

Example libmseed programs msi - Mini-SEED Inspector Parse and print useful information about Mini-SEED in one or many files. Including continuous segment construction, data sample printing, etc. msstat - Calculate minimum, maximum, arithmetic mean, standard deviation and RMS for Mini-SEED in one or many files. msrouter - Sort Mini-SEED records based on one or many format details (network, station, location, start time, sample rate, quality, etc.), for example “%s-%c.mseed” PQLII/PQLX - Data visualization/analysis program using libmseed. October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

Data Management Workshop Example converters seisan2mseed - SeisAn to Mini-SEED converter mars2mseed - MARS 88/lite to Mini-SEED converter gse2mseed - GSE 2.0/2.1 or IMS 1.0 to Mini-SEED converter, supports both INT and CM6 data encoding sac2mseed - SAC to Mini-SEED converter mseed2sac - Mini-SEED converter to SAC converter All run on Linux, Solaris, Mac OSX and MS-Windows Available from: http://www.iris.edu/pub/programs/converters/ * Many other converters are available at the IRIS-DMC, IRIS-PASSCAL, ORFEUS, etc. October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

Data Management Workshop PQLII/PQLX Software The evolution of the PASSCAL Quick Look software for data viewing and analysis Key new features: - Data display organized as continuous time series (by channel), large data sets - User-defined filters - Integrate and differentiate time series - Remove mean from trace - Read "RESP" format metadata - PDF plots Coming soon: - Compute cross-correlation coefficient between selected traces - PSD plots for selected data - Remove instrument response - Event-orient analysis: predicted arrivals, alignment, sorting PQLII: http://www.passcal.nmt.edu/software/software.html PQLX: http://geohazards.cr.usgs.gov/staffweb/mcnamara/Software/PQLX.html October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

Real-time Data Exchange A number of open-source and commercial data exchange systems exist, the bridges between them are illustrated below.  Earthworm SeedLink NRTS LISS Antelope NAQS Scream! Yes Possible October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia

Data Management Workshop IRIS-DMC Contact Information Administration, cooperative agreements, etc. Tim Ahern, Program Manager, Data Management System tim@iris.washington.edu Operations: submitting and/or requesting data and metadata Rick Benson, Director of Operations rick@iris.washington.edu Software details Robert Casey, Director of Software engineering rob@iris.washington.edu Quality control and related bits Chad Trabant, USArray Lead Data Control Analyst chad@iris.washington.edu October 21-26, 2007 Data Management Workshop Kuala Lumpur, Malaysia