9/04/2001CHEP'01, Beijing, China On the way to maturity - The CLEO III Data Acquisition and Control System Hubert Schwarthoff Cornell University With V. Frolov, University of Minnesota, USA K. Honscheid, D. Hufnagel, T. Pedlar, J. Thayer, T. Wilksen, The Ohio State University, USA D. Kreinick, C. Strohman, Cornell University, USA E. Lipeles, A. Weinstein, California Institute of Technology, USA

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China1 Contents CLEO III DAQ software Online operation Future plans Purpose of this talk: Give insight in engineering type operations!

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China2 Introduction CLEO until 2001: Y(4S)  Crystal E.M. calorimeter  Tracking drift chamber  Muon chambers  Time of Flight system CESR: superconducting cavities  Have achieved 1.3 x /cm 2 s  Upgrade Luminosity to 3 x /cm 2 s Since 2000: CLEO III detector  RICH detector:  -K separation 4   SVX detector: 4 layers, r  resolution < 20  m

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China3 The CLEO III detector

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China4 CLEO III upgrade

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China5 Introduction Data Acquisition  ~400k channels  Readout: 33 Motorola PowerPC (VxWorks, VME)  17 VME crates, 12 FASTBUS  Standard Ethernet – TCP/IP  Data rate < 9MB/s through EventBuilder (solaris)  Storage on AIT tape robot

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China6 DAQ software C, C++, Java, on  VxWorks 5.3  solaris 2.6  Windows NT CORBA: VisiBroker 3 Objectivity 5 Online database

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China7 DAQ software Object oriented development crucial  Lots of code reuse  Lots of encapsulation  Objects are natural Integration with ODBMS natural  Seamless integration of real objects  Accomodates complex storage structure  Eliminates conversion overhead

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China8 Online operation Readout and controlling is critical to record max. luminosity. System availability Failure recovery Startup performance: 3min Object database critical: MROW - Multilevel caching No database outages or wait times Online master DB exclusive

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China9 Online operation Special issues  VxWorks = realtime OS  Avoid locking of multiple components CLEO III physics data processing  Calibrate (within days)  Reconstruct (whenever ready) Performed on solaris farm with separate databases

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China10 Online operation Need mirror databases for processing  Update regularly and automatically (special application)  Prevent interference with Online operation  Allow easy handling (documentation, troubleshooting)

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China11 Online operation Fault tolerance  Alarm system  Shifter instructions for non-experts  Detect and tolerate dead components  Avoid lockup in CORBA connections with timeouts in separate threads (cannot use CORBA timeout)  High level monitoring through 24h plots of key variables, e.g. hadronic cross section, trigger rates, noise levels, …

9/04/2001Hubert Schwarthoff, Cornell CHEP'01, Beijing, China12 Future plans CLEO III has been operating for 18 months Gathered experience  Reliability comes before versatility System will be used for Y resonance running (2002) and CLEO-C (starting 2003) Upgrades  Gigabit Ethernet for EventBuilder  OS upgrades (solaris 8, VxWorks 5.4)  Middleware: VisiBroker 4, Objectivity 6  Improved Constants database system for better performance and manageability