Managed by UT-Battelle for the Department of Energy Best Ever Archive Utility, Yet (BEAUtY) Kay Kasemir April 2013

2Managed by UT-Battelle for the Department of Energy CSS-based OPI Archive Archive Engine config.xml IOC Binary Data Files Data Server ASCII Config. Channel Access XML-RPC Previous Setup with Channel Archiver History ~2000: Started by Bob Dalesio ~2003: Index Tools, Data Server ~2007: CSS Client

3Managed by UT-Battelle for the Department of Energy Details Older Java archive viewer also available. Archive Engines running on sampling computer, data server on serving machine. Scripts to restart engines, copy data, update indices. Headaches with data maintenance: –Index time grows with data –Stuck when index files reach 2GB –SNS Users faced with ~80 sub-archives –No clue what needs fixing after network/power problems –No idea who contributes how many samples –No way to remove selected channels or time ranges –Improving on this means implementing an RDB

4Managed by UT-Battelle for the Department of Energy CSS-based OPI ArchiveEngine config.xml IOC RDB (Oracle/MySQL) Channel Access New Setup Samples Config. EngineConfig -Import Other tools for config & samples

5Managed by UT-Battelle for the Department of Energy RDB Alternatives Data Browser –read from old Channel Archivers ArchiveDataServer.cgi –Read from RDB –use your own site-specific reader plugins Archive Engine –Pluggable Configuration –Pluggable Write support

6Managed by UT-Battelle for the Department of Energy CSS Data Browser Handles Both New URL Just one RDB sub-archive Old and new data can be combined in one plot

7Managed by UT-Battelle for the Department of Energy View Archive Data in CSS Data Browser Open Data Browser Plot Add PV –If default archive setup lists data source with historic data for the PV, it will be displayed

8Managed by UT-Battelle for the Department of Energy Searching for PVs in Archive Open Archive Search View –Data Browser Perspective, or Plots context menu Search for channels via pattern Send to new Data Browser –Via context menu Send to existing Plot –Via drag & drop

9Managed by UT-Battelle for the Department of Energy Configure Sample Engines Configuration is in RDB –Directly use SQL –EngineConfigImport for legacy config files –View/Edit via web Hierarchical (as before) –Sampling engine (name, where to run, …) Groups –Channels No more duplicate channels!

10Managed by UT-Battelle for the Department of Energy Web Interface to Engine Config in RDB Tomcat/JSP/Servlets to view and edit

11Managed by UT-Battelle for the Department of Energy Web Config View: Channel Stats

12Managed by UT-Battelle for the Department of Energy Web Config View: Sample Stats OK ??

13Managed by UT-Battelle for the Department of Energy Stats 34 sample engines, channels Host that runs sampling engine: –CPU load 45%, zero disk I/O wait, very responsive Oracle performance –8000 samples/sec peak in write tests –Operationally maybe ¼ of that Better configuration would likely have fewer samples/sec

14Managed by UT-Battelle for the Department of Energy RDB Supported Currently supported: MySQL or Oracle –Main table difference: Oracle TIMESTAMP has nanosecs, MySQL has separate nanosecs col. –All tools auto-configure based on JDBC URL PostgreSQL support from ITER

15Managed by UT-Battelle for the Department of Energy Which one to use? Channel Archiver –Fast –Cant be maintained RDB –Slow –Super easy to maintain Worth a look: –SLAC Archiver Appliance –HyperArchiver –Both of them work with CSS Data Browser

16Managed by UT-Battelle for the Department of Energy Thanks!