Network Interactions with MUSTANG: MUSTANG Clients and Network Reports Dr. Mary Templeton IRIS Data Management Center Managing Data from Seismic Networks August 20-26, 2017 Pretoria, South Africa

Network Interactions How you can QA your own network using MUSTANG clients LASSO MUSTANG Databrowser How IRIS staff QA networks Writing custom scripts for metrics retrieval Verifying and reporting data problems Tracking problem resolution Clients retrieve metrics from MUSTANG and present them to you

LASSO: A MUSTANG Client http://lasso.iris.edu/ Displays MUSTANG metrics In control-panel format As time series Ranks channels based on metrics values Can display multiple metrics for virtual networks Includes some derived metrics Useful for getting an overview of network health over some time period

LASSO: Basic Retrieval Virtual networks only Choose from 5 combinations of measured or derived metrics describing: Mass position Noise power Signal quality Time series integrity Metadata validity Quantitative ranks channels From 0 (all metrics poor) to 100 (all metrics good). Qualitative lists a count of good, fair and poor metrics. Basic (pre-select metrics combinations for a full network) and Advanced (you choose your metrics and stations) querying Snapshot retrieves a single day; mean and median retrieve longer time spans

LASSO Control Panel View Click column name to sort Qualitative ranking: 2 good 1 fair 0 poor Note: m1-3 are derived metrics that scale the mean amplitude of a mass position channel in V/Count and scales it to Volts (the value shown here) The color is determined by mass position volts divided by a sensor type-dependent voltage threshold as described by “m3 Rules”

LASSO: Advanced Retrieval You choose metrics combinations Network or Virtual Network code You choose the metrics, stations and channels that you’re interested in

Completeness Metrics Metrics combination Table type Start/End times max_gap num_gaps percent_availability Table type Mean for Period Start/End times 2017-07-01 to 2017-07-08 One metrics combination that I find useful

IU Completeness Metrics Data stopped flowing on 2017/07/07 Click cell to graph values: Outage began with a single “gap” We asked for mean metric values over a week. Plots will reveal the daily details

Other Metrics Combinations Amplitude health dead_channel_exp – seismic signal absent low values suspect pct_below_nlnm – low amplitude levels high values suspect pct_above_nhnm – noise non-zero values suspect sample_mean – mass drift very high values suspect sample_rms – amplitude variation very low or very high values suspect Here’s another metric combo that I find useful

Other Metrics Combinations Signal anomalies cross_talk – same signal on multiple channels Absolute values near 1 suspect dc_offset – step difference in sample_mean High values suspect num_spikes – transients lasting one to a few samples sample_unique – count of unique amplitude values Very high or very low values suspect And a third combo

MUSTANG Databrowser: Another Client http://ds.iris.edu/mustang/databrowser/ Displays MUSTANG metrics As network or station “boxplots” As time series Grouped with similar channels Grouped with related metrics Displays seismic traces Displays PDF plots Good for exploring the full story (multiple metrics) behind a single suspicious metric value

Exploring Amplitude Health with Boxplots IU.BILL.00.BHZ has unusually high mean amplitudes! The dark blue or “boxed” area shows where 50% of the values lie Except for “outliers” all values fall between the “whiskers”

Exploring Amplitude Health Seismic trace: drift is confirmed Metric Timeseries: Drift began April 8, 2017 PDF Plot: Seismic energy is absent

How IRIS staff QA networks IRIS staff used to review waveforms to QA some special networks – staff intensive We wanted to develop an automated metrics-based approach – less staff intensive Special thanks to the Global Seismic Network and the Alaska Regional Network for their feedback as we tested this approach using their data We’ve assumed you’ve already registered your network and station names. We will do the next three steps before lunch. After lunch, we’ll do the last step.

How IRIS staff QA networks Custom R scripts retrieve metrics: Metric thresholds suggesting problem data Unlike the interactive clients we seen, we created R scripts so they could be automated We’ve typically QAed a month of network data for each report

How IRIS staff QA networks Verify and report data problems An analyst still needs to verify that “suspicious-looking metric values” are a true data problem The previous problem began on November 19 and was still occurring on the report date of December 20

How IRIS staff QA networks Track Problem Resolution True data problems are recorded in a tracking system. When the problem is resolved, the end data is entered and the issue is marked “Closed” Closed problems are typically no longer reported to network operators.

Can Anyone Write Custom Clients? Yes!!! Here are two tutorials to help you learn more: Seismic Data Quality Assurance Using IRIS MUSTANG Metrics http://ds.iris.edu/ds/nodes/dmc/tutorials/seismic-data-quality-assurance-using-iris-mustang-metrics/ R Class for Seismologists http://ds.iris.edu/ds/nodes/dmc/tutorials/r-class-for-seismologists/ Clients can be written in any language that can send HTTP queries, often with the curl or wget command