MUSTANG Quality Assurance Gillian Sharer TA Team Meeting November 9-10, 2015 With contributions from Rob Casey, Bruce Weertman, Laura Hutchinson, and Mary.

Slides:



Advertisements
Similar presentations
Mark Servilla & Duane Costa LTER Network Office LTER 2012 All Scientist Meeting LTER Network Office.
Advertisements

Y O U R O N L I N E R E S E A R C H A N D L E A R N I N G C E N T E R
CHAPTER 15 WEBPAGE OPTIMIZATION. LEARNING OBJECTIVES How to test your web-page performance How browser and server interactions impact performance What.
The North American Carbon Program Google Earth Collection Peter C. Griffith, NACP Coordinator; Lisa E. Wilcox; Amy L. Morrell, NACP Web Group Organization:
OVERVIEW TEAM5 SOFTWARE The TEAM5 software manages personnel and test data for personal ESD grounding devices. Test and personnel data may be viewed/reported.
Pacific Northwest National Laboratory Primavera P6 R8 Upgrade Presented by: Brian Fitzsimons 1 Date: 10/18/2011.
IWay Service Manager 6.1 Product Update Scott Hathaway iWay Software Copyright 2010, Information Builders. Slide 1.
Software Delivery. Software Delivery Management  Managing Requirements and Changes  Managing Resources  Managing Configuration  Managing Defects 
WebVERS Introduction The Visitor Estimation and Reporting System (VERS) program was developed as a DOS program. Several years ago the meter reading and.
03/17/2014 Data Management System -Data Services- Temporary Experiment Data Brief Data Services News MUSTANG update- QC on PASSCAL Data PASSCAL SC Spring.
POP QUIZ!!! What kind of software is Medisoft? Name ONE of the 4 things that you can do to data in Medisoft. What is the Medisoft Program Date? What key.
UserSupport Help Desk System at CCIN2P3 Jean-René Rouet IN2P3 Computing Center
Everything but the Kitchen Sink: Building a metadata repository for time series data at the Federal Reserve Board San Cannon and Meredith Krug Federal.
1 Infrastructure Trial Running a Dress Rehearsal.
Improved Quality Control for Seismic Networks ---MUSTANG.
Tutorial 11: Connecting to External Data
Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System 1 Zaihua Ji Doug Schuster Steven Worley Computational.
Intelligent Tutoring System Mobile Communication Team Drew Boatwright Nakul Dureja Richard Liou.
CHAPTER © 2011 The McGraw-Hill Companies, Inc. All rights reserved. 3 Introduction to Medisoft.
Upcoming Enhancements to the HST Archive Mark Kyprianou Operations and Engineering Division Data System Branch.
HTML 5 New Standardization of HTML. I NTRODUCTION HTML5 is The New HTML Standard, New Elements New Attributes Full CSS3 Support Video and Audio 2D/3D.
Research Support Services UCH Facility Use Approval Workflow Management Update Jennifer Holcomb.
Bob Woodward, Bob Busby, Katrin Hafner, and David Simpson
FireRMS SQL Audit, Archiving & Purging Presented by Laura Small FireRMS Quality Assurance.
Channel Archiver Stats & Problems Kay Kasemir, Greg Lawson, Jeff Patton Presented by Xiaosong Geng (ORNL/SNS) March 2008.
Web Page Design I Retest Terms Review. 1. Web pages are created using a language known as ___________. The coding of this language must follow specific.
SOLUTIONS FOR THE EFFICIENT ENTERPRISE Sameer Garde Country GM,India.
© 2004 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice SISP Training Documentation Template.
M1G Introduction to Database Development 6. Building Applications.
Update on a New EPICS Archiver Kay Kasemir and Leo R. Dalesio 09/27/99.
Higher-Level Clients to Leverage MUSTANG Metrics Dr. Mary Templeton IRIS Data Management Center Managing Data from Seismic Networks September
Diagnostic Pathfinder for Instructors. Diagnostic Pathfinder Local File vs. Database Normal operations Expert operations Admin operations.
Automated (meta)data collection – problems and solutions Grete Christina Lingjærde and Andora Sjøgren USIT, University of Oslo.
Automated Data Quality Assurance: QUACK and MUSTANG Mary Templeton IRIS Data Management Center Mary Templeton IRIS Data Management Center.
Resource USL: The Good, The Bad and The Ugly UNIVERSITY LIBRARY JANE BARTON | Coordinator Shared Client Services.
Process Architecture Process Architecture - A portion of a program that can run independently of and concurrently with other portions of the program. Some.
Introduction to Morpho RCN Workshop Samantha Romanello Long Term Ecological Research University of New Mexico.
1 Unit Test in VS 2010 Presented by ZAN ZAN KOE. 2 Types of testing provided by VS  Unit test  Manual test  Web Performance Test  Coded UI Test 
Girish M. Jashnani Sales Consultant Manage your E-Business Suite more effectively.
John Porter Sheng Shan Lu M. Gastil Gastil-Buhl With special thanks to Chau-Chin Lin and Chi-Wen Hsaio.
Unit 17: SDLC. Systems Development Life Cycle Five Major Phases Plus Documentation throughout Plus Evaluation…
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
Building Station Metadata with PDCC
Magda Distributed Data Manager Prototype Torre Wenaus BNL September 2001.
Station Metadata The NRL (Nominal Response Library) Dr. Mary Templeton IRIS Data Management Center Managing Data from Seismic Networks September
LCG Accounting Update John Gordon, CCLRC-RAL WLCG Workshop, CERN 24/1/2007 LCG.
November 1, 2004 ElizabethGallas -- D0 Luminosity Db 1 D0 Luminosity Database: Checklist for Production Elizabeth Gallas Fermilab Computing Division /
SINN01 Technical Workshop Oldenburg 5-7 Dec 2001 Technical Discussion SINN01 Technical Workshop.
Basic Navigation in Oracle R12 BY: Muhammad Irfan.
ONBASE: HOW TO UPLOAD DOCUMENTATION EFFICIENTLY April 2, 2015.
2 Copyright © 2006, Oracle. All rights reserved. Running a Forms Developer Application.
CS177 week 3 scavenger hunt team mini-project start in class finish as part of homework this will include a mixture of things we have and have not covered.
Enhance Your Page Load Speed And Improve Traffic.
XHTML. What Is XHTML? XHTML stands for EXtensible HyperText Markup Language XHTML is almost identical to HTML XHTML is stricter than HTML XHTML is HTML.
Running a Forms Developer Application
Archiving and Document Transfer Utilities
Software for Spectrometer T0 jumps correction
Network Interactions with MUSTANG: MUSTANG Clients and Network Reports
COMMANDseries V3 Roadmap
ISPAQ: IRIS System for Portable Assessment of Quality
Web Caching? Web Caching:.
SharePoint Site Admin Training
PHP / MySQL Introduction
USArray Quality Assurance
DBM 405 Innovative Education- -snaptutorial.com
CodePeer Update Arnaud Charlet CodePeer Update Arnaud Charlet
Operational Dataset Update Functionality Included in the NCAR Research Data Archive Management System Zaihua Ji Doug Schuster Steven Worley Computational.
MUSTANG Training Session for North American Regional Networks
Best Practices in Higher Education Student Data Warehousing Forum
Presentation transcript:

MUSTANG Quality Assurance Gillian Sharer TA Team Meeting November 9-10, 2015 With contributions from Rob Casey, Bruce Weertman, Laura Hutchinson, and Mary Templeton Recent Progress and Development

MUSTANG Modular Utility For Statistical Knowledge Gathering Seismic data quality metrics and PSD,PDF web services ( QUACK was officially retired in January It has been running on an unsupported basis and the oracle database that it depends on will be gone soon, possibly by the end of the year. )

Currently, we have almost full metrics coverage of _US-TA stations going back to In the past, we ran metrics based on virtual network designation. This has resulted in differences in metrics coverage within networks. For example, _CASCADIA-TA station coverage differs _US-TA coverage (10% vs 99% complete for sample_mean). Now, we are running metrics on a network basis. All networks in our archive will have metrics for all their stations for (at least) current data. Channels are still limited to BH and HH..

Metrics Coverage per Month for TA, N4, AK Networks

Process for Automated STatistic Update REponse PASTURE is the system that facilitates the correction of metric values by MUSTANG when data changes. Compares waveform archive dates with MUSTANG load dates and returns: M: measurement is current W: waveform, no measurement S: measurement is stale and needs to be recalculated O: measurement, but no waveform (orphan) For each station-channel-day and metric (does not cover event-based metrics yet)

Using PASTURE to Update Metrics Currently, automatic recalculation of stale metrics occurs: 1.Once a week, covering a time period of 3 weeks to 1 week prior 2.Every 21 days, covering the previous calendar month The recalculation is based on a single representative metric. If this metric is stale, then all metrics for that day are recalculated. In the future, automatic recalculation will occur on a larger time scale and all metrics will be checked for staleness.

Correction of PSD Values When Metadata Changes 1.PSD Response Refresher  compares cached responses with current metadata. This runs every Monday and marks days that need correction as “stale”. 2.PSD Corrector  recognizes what days are stale and applies the current response information to the PSDs. 3. PSD Merger  tiered storage of PSD histograms and modes for each station-channel  day, week, month, year, all  very fast data retrieval

url <- &starttime= &format=plot&plot.interpolation=bicubic&plot.period.min=0.1 &plot.period.max=183&plot.power.min=-200&plot.power.max=-50 &plot.subtitlefont.size=18&plot.labelfont.size=16&plot.titlefont.size=20” system.time(download.file(url,"TA.PDF.BHZ.png",method="curl")) user system elapsed

R code for MUSTANG metrics is now available from CRAN The Comprehensive R Archive Network 1.IRISMustangMetrics – statistics and metrics for seismic data 2.IRISSeismic – classes and methods for seismic data analysis 3.seismicRoll – additional functions operating on a rolling window

Simple text report Created by multiple calls to services.iris.edu/mustang/me asurements in a shell script

Prototype Report for Analyst Review Html report created using R Markdown

MUSTANG Quality Assurance Gillian Sharer TA Team Meeting November 8-10, 2015 With contributions from Rob Casey, Bruce Weertman, Laura Hutchinson, and Mary Templeton Recent Progress and Development