1. Channel Archiver Overview Jan. 2003

2. Channel Archiver Channel Access client Stores samples in disk files Design target: handle 10000 values/sec Documentation, Snapshots: LANL EPICS Page. From http://www.aps.anl.gov/epics, see Other Sites, LANL, Tools, Archiver.http://www.aps.anl.gov/epics

3. Operation Config. file lists channels to archive # Example: channel_X 1 channel_Y 10 # Monitor: might change every 0.2 second channel_Z 0.2 Monitor Scanned Periodically store most recent value Monitor Store all incoming values – up to buffer limit Original time stamps are stored!

4. 1s Period  Data @ 9.96, 10.98, 11.76 s ?

5. Components CA Data Directory Storage Retrieval LibIO (C++) ArchiveEngine LibIO (C++) WinBrowser (Win32) LibIO (C++) xarr (Unix) LibIO (C++) StripTool (Unix) LibIO (C++) CGIExport WebServer Native: LibIO (C++) SWIG TCL tk,BLT LibIO (C++) SWIG Python tk,BLT,PMW LibIO (C++) SWIG Perl CGI,DBI,DBD Scripting: Online: (also possible via scripting) LibIO (C++) Archive- Export LibIO (C++) Archive- Manager

6. ArchiveEngine Syntax: ArchiveEngine [options] config [dir] Options: -d -p : change HTTPD from 4812 -l : write log file

7. Engine’s HTTPD URL of engine’s HTTPD: http:// : Status & Config., not data! “Client Pull”: Updates on reload Changes (added groups/channels) written to cfg subdirectory, original config. files unchanged

8. “Directory File” File name, not a directory name Specified when running archive engine Default: “freq_directory”. Better: “dir”. Used by retrieval tools Details of “directory” and “data” files are given in documentation Note: Keep directory & data files together, don’t modify them.

9. ArchiveExport Syntax: ArchiveExport [options] [dir] {channel names} Options: -start “mm/dd/yyyy hh:mm:ss” -end “mm/dd/yyyy hh:mm:ss” -fill -interpolate -gnuplot -Matlab …

10. Export Options Original Time Stamps: ill-suited for Spreadsheets “Fill” missing values by repetition Linear Interpolation for given period (initial #N/A until all channels have valid value)

11. “Fill”, “Interpol.”,...

12. MATLAB Export Tools generate MATLAB command file ASCII, portable No MATLAB binaries required Full value info, time & status Big & slow

13. WinBrowser Win. Only

14. CGI Export Data Access from any Web browser Download formats: Spreadsheet, Matlab Data Web Server CGIExport WWW

15. CGI Export Online Plots

16. Scripting Allows writing perl/tcl/python programs for Automated generation of daily beam statistics for Web page Specialized scripts to answer questions like: “How often was XX below 10.0 and for how long?” …

17. ArchiveEngine Details More options, see manual: !write_period “Groups” for getting organized conditional archiving !group Disable

18. Retrieval f. Multiple Archives Run several ArchiveEngines, creating several “small enough” data sets Look at them all at once: master_version=1 # List sub-archives, most recent first /archives/2001/july/dir /archives/2001/june/dir /archives/2001/may/dir # … # Then check Fred's "xyz" archive /home/fred/xyzarchive/dir Sub-archives should not “overlap”

19. Suggested Setup Long Term Archive Sample ~10min, kept forever Middle Term Sample every 60s, restarted after 15 days, switching between two directories Short Term Up to ‘Monitor’, toggling two directories every shift  High-grained data for last shift, good data for last 2 weeks, some log of everything kept forever

20. CAManager (Thomas Birke, BESSY) ArchiveEngine control GUI Start/Stop Status of (multiple) ArchiveEngines Periodic re-start in different directory, updating a MultiArchive File

21. Data Management Do not separate Directory & Data files! Periodically restart Engine in new directory to keep data sets manageable (CAManager helps) ArchiveManager program can copy selected channels & time range, performing basic repairs and compression while doing this

22. Summary “ChannelArchiver” is a Toolset for archiving any ChannelAccess data Some generic retrieval options, scripting and Matlab allow further analysis Future Plans: Port to R3.14 and RH8 CORBA interface for data retrieval (common w/ JLab archiver, replace tcl/perl/python binding, allow Java access to data)