03/17/2014 Data Management System -Data Services- Temporary Experiment Data Brief Data Services News MUSTANG update- QC on PASSCAL Data PASSCAL SC Spring

Temporary Data Is 26% of DS Holdings 03/17/2014PASSCAL SC Spring

Temporary Experiment Data Is Large, Important 03/17/2014PASSCAL SC Spring

_PASSCAL Virtual Network 9577 Stations to date 03/17/2014PASSCAL SC Spring

Notable DS Activities: 03/17/2014PASSCAL SC Spring The Livermore Auxiliary Data Center is operational Currently being used by LLNL staff to access data using existing web services, was delivered to them on time This ADC is used like its locally located on the same LAN, since it has 10Gb connectivity: Archiving of data automatically multicasts to both LLNL and local storage in Seattle; no need to cache Can service breq_fast and web service requests, not currently done routinely. (just weekly as function test) We anticipate installing a global load balancer to accommodate traffic to be sent to ADC when load is high in Seattle

Notable DS Activities: (Cont’d) 03/17/2014PASSCAL SC Spring Data Services has begun integrating a 10% budget decrease We will migrate large, write-once read never data like PASSCAL Flex Array “RAW” data to lower-cost tape for example. I am currently undergoing an extensive audit to prioritize storage strategies, perform de-duplication, etc

MUSTANG- QC Across All Data 03/17/ PASSCAL SC Spring 2014

What does MUSTANG Stand For? I’ll only say this once: MUSTANG is an acronym violation that stands for: Modular Utility for Statistical kNowl- edge Gathering 03/17/ PASSCAL SC Spring 2014

Data Coverage NetworkStartEnd # Records Per Metric _GSN M _PASSCAL K _OBSIP K _CASCADIA K TA K II K IU K (representative)  Latency measurements (all networks) are currently working on >36,000 channels!  xt 03/17/ PASSCAL SC Spring 2014

March 2014: Decision to use Livermore (ADC) for MUSTANG Since FDSN web services are currently installed and running at the offsite Auxiliary Data Center at LLNL, we’ll utilize the server and storage VM environment and update the RDBMS in Seattle simultaneously. This will act as a test bed, but in addition will offload Seattle resources which are currently at maximum I/O 03/17/ PASSCAL SC Spring 2014

Accessing MUSTANG Metrics There will be a “live” web service front end soon: s/1 is main landing page, with help s/1 A URL builder to help construct the query and get correct syntax is directly reached using: s/docs/1/builder/ s/docs/1/builder/ 03/17/ PASSCAL SC Spring 2014

Quick Look at the Interface; Similar to all current web services: 03/17/ PASSCAL SC Spring 2014

Available Soon: “Visualizing” Metrics There is a beta version of a web service front end that can access the stored metrics that will be similar to this unreleased version: 03/17/ PASSCAL SC Spring 2014

We are committed to PH5 Format Metadata Exposure We will enable parsing of station/site metadata that is currently stored in PH5 so that utilities like can display locations and enable increased awareness NOTE: This is for PH5 data sets only and will not include “assembled” data sets, as these are in no standardized or parse-able format 03/17/ PASSCAL SC Spring 2014

We have begun the migration out of Oracle RDBMS We have just entered into an agreement to migrate our 2.1Tb Oracle RDBMS to Postgres. (EnterpriseDB) We will not renew FY15 Oracle in October but can still use it, without support, so we are in a hurry We currently have a solid list of known unknowns and workarounds, and Oracle-specific procedures that we’ll have to work on. Consider this a traveler’s advisory, but we intend it to be transparent externally 03/17/ PASSCAL SC Spring 2014

We have begun the migration out of Oracle RDBMS (cont’d) We will have support and professional help We currently have 9 Postgres databases in operation so we have PL/PGSQL tribal knowledge that we can leverage. Not totally new, but PostgreSQL dialects are different. We hope to have it majorly done in 3 months 03/17/ PASSCAL SC Spring 2014

That’s all for now Questions? Comments? Requests? 03/17/2014PASSCAL SC Spring