1. Control Systems at DESY
(with an eye on Data Acquisition)

2. First, some general observations
A Control System is more than …
clicking a button “here” and making something happen “there”.
taking data “there” and displaying it “here”.
Every aspect of the machine/facility under control should be part of the control system.
What do you mean “off-line” analysis? (Why isn’t it on-line?)
If the “going gets tough”, does the control system quit?
The Control System can have different flavors
Different platforms, io busses, etc.
Different “control systems”
If TINE and EPICS are used to control a machine, then the control system is “TINE and EPICS” not TINE or EPICS.
The Presentation layer shall not be ignored!
“Ask not what you can do for your control system, ask what your control system can do for you!”

3. Control Systems (one way or another) have to deal with …
Distributed end points and processes
Data Acquisition (front end hardware)
Real-time needs (where necessary)
Process control (automation, feedback)
Central Services (Archive, Alarm, Name Resolution, …)
Security (who’s allowed to do what from where?)
States (Finite State Machines, sequencing, automation…)
Time synchronization (time stamps, cycle ids, etc.)
Databases (configuration, machine data, post-mortem data, …)
Statistics (control system itself, operation, …)
Logging (central, local, application, …)
Data transport (data flow, control system protocol, scalability)
DAQ : focus on Data Acquisition and Central Services !

4. Control Systems Protocols …
How to move information efficiently (does it scale?) ….
Services: DAQ
Automation
Users/Operators
Data Acquisition Layer
current Data timestamp: Sat Feb 3 16:50: msec (UTC: )

5. Some Control Systems at DESY
In house
MKI (legacy)
DOOCS
TINE
ZMP (Zeus Message Passing)
SPECTRA (HASYLAB Beamlines)
Somebody just “did” something
(Private solutions that moved data from there to here”)

6. Some Control Systems at DESY
Outside Commercial
PVSS (H1)
D3 (Cryogenics)
Outside Other:
EPICS (Cryogenics, Utilities)
TANGO (HASYLAB) …

7. “So, why aren’t you using EPICS?”
PR campaign in the ’90s led to EPICS being used ‘somewhere’ at most accelerator facilities.
Good at hardware support.
Good at putting ‘simple’ clients together (EDM, MEDM, etc.)
Channel based
Flat namespace
Everything is a named record
‘no’ rich clients => records such as “r:add:2:things.calc”
Don’t “have to program” !
(scared of programming?)
End up with lots of ‘panels’
JLAB : > 2000 panels (HERA : < 200 rich applications)
Treat EPICS as a layer upon which to build bigger and better things?
a la “Berkley sockets”

8. Also in the ’90s MKI System (legacy) pre-DOOCS => DOOCS
Windows only
IPX datagrams
Asynchronous transfer
Producer-Consumer oriented
pre-DOOCS => DOOCS
Solaris, Linux, Java
SunRPC based
Synchronous transfer
Early collaboration with ESRF (TACO)
Lots of data types
ISOLDE => TINE
Most platforms (even DOS, VMS !)
Socket based (UDP, TCP, IPX)
Synchronous/Asynchronous transfer (events)
Early collaboration with CERN
Lots of data types + User-defined Structures!

9. DOOCS and TINE Developed separately Similar Naming Hierarchy
(but constantly looking over the other guy’s shoulder)
Similar Naming Hierarchy
Similar Data Sets
Similar features
Now merging
TINE embedded in DOOCS
Goal: Your favorite DOOCS tool will work with TINE clients and servers and vice versa !
Both also used outside DESY

10. TANGO
Just as the 2 of the 3 principal control systems are making a serious effort to become one control system, we get number 3 again!
Similar Naming Hierarchy as DOOCS/TINE
Comes from the same community which brought you TACO.
Similar data access possibilities as TINE (except multicast)
CORBA based
Interest in the TANGO community in running TANGO on the TINE protocol (ICALEPCS’07)
Features map well
Jens Meyers coming for Sabbatical in Sept.
Maybe get back to 2 principal control systems after all?

11. How to get all of this to work together ?
Either a supreme dictator forbids all control systems except “System X” or …
Or allow religious freedom (within reason)
Use your favorite tool for what you want
Use “the best” tool for solving your problem
Have to deal with Apples and Oranges one way or another!
Apple to Orange ‘translators’
live on ‘Apple’ servers
Apple to Orange ‘plugs’
live on ‘Orange’ clients
Apple to Orange ‘gateways’
lives as an external process

12. The TINE Approach (translators)
Embed TINE on the ‘foreign’ server
All Clients are TINE Clients and speak “TINE” to it.
All expected server functionality is available
e.g. send data as multicast
DOOCS
TINE already embedded !
EPICS
epics2tine is an add-on process that runs embedded on the EPICS ioc
You run EPICS over TINE instead of Channel Access
EPICS ioc is effectively a TINE Server
TANGO
Tango2tine is a gateway that should run on the TANGO server. (May not be necessary later)

13. The DOOCS Approach (plugs)
TINE is embedded
EPICS:
DOOCS clients can wrap Channel Access calls
EPICS iocs only ever see Channel Access calls
Need to embed the Channel Access library (which version?) at the client
Need to resolve channel access names
TANGO:
DOOCS clients can wrap TANGO calls
Need to embed the TANGO library at the client (we’re talking some Megabytes here!)
Need to resolve TANGO names

14. Translators and plugs …
cdev 1996 (client-side plugs)
abeans 1997 (client-side plugs)
acop 1997 (client-side plugs)
epics2tine 2000 (translator)
1st translator on the market!
TINE embedded in DOOCS 2001 (translator)
CSS (Control System Studio) 2006 (client-side plugs)
epics2tango 2007 (gateway)
tango2tine 2007 (gateway)

15. What’s up, DAQ? DAQ = Archive System !
DAQ is an archive system with special requirements, e.g. :
Store all data all the time
Don’t miss anything
Don’t throw anything away (ever) …
What about the ‘normal’ archive system?
Still need it?
Yes !

16. DAQ Users Want … Control guys want to debug/improve operations
Machine parameters
Experimental analysis guys want to examine the stored data and look for things (off-line?)
Experimental data (and machine parameters)
Easy to add things to the system
Plug-and-play
Need to ‘browse’ the DAQ in quasi-realtime
You may have terabytes of data somewhere, but if it takes tera-seconds to retrieve it, you’ll go crazy.
Browsing the DAQ = Browsing the Archive System

17. Browsing the DAQ (DOOCS)
DOOCS calls to the “LONGARCHIVER”

18. Browsing the DAQ (CSS)
“DAL/DAA” calls to the underlying control system archiver.

19. DAQ-Lite (TINE Archiver)
TINE Central Archive System
Stores everything ‘forever’
Filters stored data (machine state, tolerance)
Archives ‘best effort’ versus ‘lossless’
No explicit retry of ‘misses’
Fast data retrieval!
Should work in parallel with more extensive DAQ Systems (e.g. the FLASH DAQ)
It does not ‘hurt’ anything to archive the same thing twice!
Don’t be concerned that in theory every thing stored by DAQ-Lite is also in the DAQ.
Browsing through the data becomes very fast (regardless of how far back in the past you go).
Operations probably only need DAQ-Lite anyway
If you browse to something interesting, then get all the data with the full DAQ.

20. Browsing the DAQ (TINE)
3 Hera Lumi Runs shortly before the End of Operations
Protons :
LUMI : ??
Pick and Choose other Zeus Parameters :

21. Data Delivery to DAQ
Also have clients who want to monitor the same data live!
Do not want unmanageable load on the servers
Make them get the data 2nd hand from the DAQ?
Introducing latency
Single-point of failure ?
Use Multi-cast where necessary
e.g. video

22. TINE Video (Multicasting + Scheduling)
.5 Mbyte Video 10 Hz multicast (100 Mb ethernet). (also runs 20 Hz)
Uses the CM_NETWORK switch.
Server calls the Scheduler when a new frame is grabbed => as real-time as it gets !!!

23. Conclusions
We are able to mix control system protocols fairly easily these days
Use the best tool for the job
But …
you might be introducing headaches for somebody
So don’t go overboard!
A DAQ system won’t care where the data came from
Central DAQ
Data stored in the same (expected) way from all sources
Parallel DAQ-Lite
Quick data browsing
In some cases is all you need anyway