1. © 2013 Paul Scherrer Institut Slides available at http://people.web.psi.ch/ischebeckhttp://people.web.psi.ch/ischebeck Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA ADAQA An Accelerator Data Acquisition & Analysis Framework Bolko Beutner, Helge Brands, Rasmus Ischebeck >The Past >Accelerator Data Acquisition >Design Ideas >The Present >Available Tools >Demo >The Future >Data Acquisition with Large Data Sets at 100 Hz 1 1

2. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA What Came Before… (my personal collection of accelerator data acquisition) >TESLA Test Facility at DESY, Hamburg >Access to accelerator process variables from MATLAB, C++ (read/write) >Easy to write data acquisition software, scan parameters, record variables… >Data stored in.MAT, ASCII & ROOT files >No common data format >Experts required to set up scan, analyze data… >SLAC LINAC at Stanford, California >VAX-based control system >Scan software to determine the dependency of a variable on a parameter >On-line display & fits >Writes MATLAB 4 files for further analysis 2 2

3. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Design Goals >Have a universal scan and data acquisition software >Well-tested scan core >Universal GUI to set up scan easily >Specialized GUIs for easy access for regularly performed scans >Error checking and treatment >Common HDF5-based file format for easy access from any data analysis code >Automatic generation of logbook entries >Link to auxiliary information on accelerator configuration >Synchronized data acquisition 3 3

4. Rasmus Ischebeck Data File Format >Hierarchical format to store Instruments >Based on HDF5 >Readable from Matlab, Root, Mathematica… 4 4 www.nexusformat.org

5. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Accelerator Data Acquisition Many accelerator measurements consist of the following tasks: >Control 0…N actuators >Acquire 1…M measurements >Analyze data Important features >On-line analysis >Reasonable error handling >Robust code for use by other persons than the programmer 5 5

6. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA ADAQA Layers >ADAQA consists of several separate building blocks >Applications, written by different programmers, make use of the same data acquisition code >Instruments gather connected devices >Logbook >Common data storage format >Online model* >Device information server* >Synchronized data acquisition *ongoing work, not yet ready for prime time 6 6

7. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA ADAQA Layers >Tree-like structure: >Many devices (roots) >One data acquisition core (trunk) >Many applications (branches) >Building blocks can be exchanged 7 7 Devices Data Acquisition Applications Wikipedia

8. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Synchronized Data Acquisition >All VME crates for diagnostics are equipped with an Event Receiver >Connected with a star topology network to the Event Generator >Supported for: >BPMs, Bunching monitor, RF, … >Ongoing work: >Laser shutter, oscilloscopes, cameras 8 8 Sven Nordquist

9. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Abstraction Layers 9 9 Machine Interface >EPICS call >Error handling >Image data conversion EPIC S

10. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Abstraction Layers 10 Instruments and Knobs >Assemble several connected values >On-line data processing and actions Machine Interface XML files

11. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Abstraction Layers 11 Instruments and Knobs Scans >Write data file >Make entry in electronic logbook Machine Interface HDF5 files eLog

12. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Abstraction Layers 12 Graphical User Interfaces >Programmed by different people >Include Data Analysis Instruments and Knobs Scan s Machine Interface

13. + SwissFEL SwissFEL Meeting 24.1.2013 ScreenMonitorTool General Purpose Scan Tools Specialised Scan Tools –Slice Emittance –Optics Matching –Slit Scan –… “Do it yourself! - Scan Tools ADAQA Applications

14. + SwissFEL SwissFEL Meeting 24.1.2013 General purpose camera display and analysis tool In every day use for different kinds of cameras and screen monitors ScreenMonitorTool Dione and Saturn http://www.ciclops.org

15. + SwissFEL SwissFEL Meeting 24.1.2013 General Purpose Scan Tool

16. + SwissFEL SwissFEL Meeting 24.1.2013 the future…

17. + SwissFEL SwissFEL Meeting 24.1.2013 Scans can be stored in Template files The last scan is always stored as template –“Repeat Last Scan…”-Button Data files are scan templates as well –All scans can be exactly repeated Templates can handed over to the scan tool at startup –example: extracted charge vs. gun phase scan Scan Templates

18. + SwissFEL SwissFEL Meeting 24.1.2013 Specialised Scan Tools

19. + SwissFEL SwissFEL Meeting 24.1.2013 Specialised Scan Tools

20. + SwissFEL SwissFEL Meeting 24.1.2013 Automatised logbook entries

21. + SwissFEL SwissFEL Meeting 24.1.2013 General and specialised scan tools are possible with ADAQA –a wide variety of tools is available Common data format Templates –data files are templates – for all tools Actions –e.g. automatic screen insertion or background subtraction Analysis functions Automatic logbook entries Scan Tools Summary

22. + SwissFEL SwissFEL Meeting 24.1.2013 knob{1} = LoadKnobDefinition('ManagementKnob');% setup Knob and Instrument knob{1}.SetpointValues = [1, 2.4, 3, 3.78, 92];% scan setpoints instrument{1} = LoadInstrumentDefinition('SwissFEL_Meeting_Demo'); Setup.Function = 'scan';% define scan setup Setup.Repeat = 10;% “shots” per scan step Info.Author = 'Beutner';% human readable info Info.Title = 'SwissFEL Meeting Demo'; Info.Purpose = 'this is a small scan tool for demo...'; [DAQFileName, Setup, knob, instrument, Info] =... daq(Setup,knob,instrument,Info);% execute scan % …insert your analysis code here… Do it yourself! - Scan Tools

23. Image based measurements: today IOC AFS EPICS Control+Data camera workstation Helge Brands control room

24. Image based measurements: today Problems: available buffers on IOC are not used „lost“ of measured data high frequencies can‘t be reached high CPU load on workstations Helge Brands IOC AFS EPICS Control+Data camera workstation control room

25. Image based measurements : future DATA Compute-Node Results Results+Data IOC AFS EPICS Control+Preview camera workstation Helge Brands control room

26. Why? less radiation higher recording speed (10Hz/100Hz) independent of the workstation (CPU and memory) system is similar to SLS (Tomcat, in operation, 1* PCO Edge) Helge Brands

27. Why? less radiation higher recording speed (10Hz/100Hz) independent of the workstation (CPU and memory) system is similar to SLS (Tomcat, in operation, 1* PCO Edge) BUT Helge Brands

28. Technical values example: ~30TB/week more storage needed Helge Brands

29. Specs storage size/speed network (camera IOC) IOC local memory compute nodes WHLA Test-Injector 558 GB/50MB/s 1Gb/s 16GB ~3000 pic 1-2 SwissFEL(?)/(?)10Gb/s(?)min. 5-10 Helge Brands

30. Current situation WHLA: each Camera-IOC has 1GB/s (enough for 10Hz) 3 PCO Edge in operation + 1 PCO Edge in preperation upload to AFS max 50MB/s software is ready for 100Hz needed network speed 10GBit/s for 100 Hz other GIGE cameras can be used too additional compute nodes should be planned storage policies are needed Helge Brands

31. >Project started in 2008, always between 5 and 10 developers >Written mostly in Matlab >89093 lines of code (including code sourced from MathWorks Central and other external sources) >Performed 22775 scans, resulting in 558 GB of data >A joint project between the Controls, Beam Dynamics & Diagnostics Sections at PSI Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA ADAQA 31 ADAQ A Diagnostics Controls Beam Dynamics

32. © 2013 Paul Scherrer Institut Slides available at http://people.web.psi.ch/ischebeckhttp://people.web.psi.ch/ischebeck Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Outlook >We are looking forward to many new applications for SwissFEL! >Thank You to: >all ADAQA programmers and users, in particular Eduard Prat, Marta Divall, Peter Peier, Marc Guetg, Gian Luca Orlandi, Bennie Smit & Thomas Schietinger >support from Jan Chrin & Babak Kalantari 32 Sven Nordquist

33. Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA Thanks to >Fellow programmers >Masamitsu Aiba, Simona Bettoni, Bolko Beutner, Kairat Doshekenov, Natalia Milas, Frédéric Le Pimpec, Eduard Prat, Clemens Ruchert >Former members of the team >Prineah Narang, Bernd Steffen >Future members? >Alexandre Trisorio, Marta Divall >Support from the Controls Section >Damir Anicic, Helge Brands, Jan Chrin, Mirek Dach, Elke Zimoch >SwissFEL High Level Applications organized by Thomas Schietinger >All our users for their bug reports 33