MICE VC June 2009Jean-Sébastien GraulichSlide 1 Feed back from the DAQ review o Facts o Detector DAQ o Important Comments o Bottom line Jean-Sebastien Graulich, Geneva

Facts  One day review for MICE Data Acquisition AND Controls At RAL on June 4 th 2009  4 external Reviewers Pierre Vande Vyvre (Chair); CERN ALICE Fritz Bartlett; FNAL D0 Tim Nicholls; RAL T2K Emilio Radiccioni; INFN Bari TOTEM  1 Internal Reviewer John Cobb; Oxford MICE VC June 2009Jean-Sébastien GraulichSlide 2

Organization  Presentations on all the aspects followed by discussions The reviewers have been very active in asking questions and giving comments  The committee has been given a charge which included a list of questions  Access these documents at MICE VC June 2009Jean-Sébastien GraulichSlide 3

General Comment  The committee was satisfied by the presentations  They recognize A significant amount of progress has already been achieved  They Identify Critical areas, most notably the control systems, where manpower is extremely limited  They would urge The management of the collaboration to identify additional effort as a matter of priority.  An overall view of the architecture and inter- relationship of the various online systems was lacking MICE VC June 2009Jean-Sébastien GraulichSlide 4

Some Answers to the charges  Does the baseline design meet the experiment’s objectives for all Steps of MICE? The system designed is well adapted to the present needs of the experiment during a phase that can be qualified of test and commissioning. The committee considers that it should evolve for the production phase of the experiment.  Will the systems be fully functional for step III? The system will most probably be ready for step III, contingent to the addition of some manpower to the control system and the addition of some redundant hardware to avoid single points of failure.  Is the interface between the DAQ and the Control system well thought out? The committee considers that the interface between the DAQ and the Controls system is not well thought out. This interface requires a systematic approach to identify all the control flows inside the experiment and to implement these flows with a top-down approach MICE VC June 2009Jean-Sébastien GraulichSlide 5

Important Remarks  Data Flow There was considerable discussion about the impact of the near-online dataflow – for both reconstruction and data archival – on the performance of the DAQ. While the data rate is not particularly high (30 MB/s) consideration must be given to the impact of archiving data files into the Tier-1 storage systems on DAQ. In particular attention should be given to having a single RAID file system where concurrent read and write accesses could lead to contention.  Monitoring Given the relatively limited time needed to reconstruct the data, it would seem appropriate and feasible technically and financially to reconstruct online a significant fraction (if not all) of the data. MICE VC June 2009Jean-Sébastien GraulichSlide 6

Important Remarks  Controls Both of the controls development groups, the MICE collaboration group and the Daresbury group, appear knowledgeable about general control system design principles and the use of EPICS  The MICE controls group is significantly understaffed considering the scale of the effort required to complete the project And - The possible departure of Pierrick Hanlet in the near future - The transfer of responsibility from Daresbury group to the MICE collaboration MICE VC June 2009Jean-Sébastien GraulichSlide 7

Important Remarks (2)  Controls Both of the controls development groups, the MICE collaboration group and the Daresbury group, appear knowledgeable about general control system design principles and the use of EPICS  The MICE controls group is significantly understaffed considering the scale of the effort required to complete the project And - The possible departure of Pierrick Hanlet in the near future - The transfer of responsibility from Daresbury group to the MICE collaboration MICE VC June 2009Jean-Sébastien GraulichSlide 8

Control DAQ Interface The present approach of inserting explicit code fragments in the DAQ to communicate to/from the DCS will not scale to the needed complexity, and adds interdependencies between the State Machines of the two systems which cannot be easily modelled and managed. To avoid unforeseen behaviours, it is recommended that a formal unified state machine is used to coordinate the various components of the online and control/monitoring subsystems. The control system must be able to read this state and other DAQ system parameters that relate to controls actions. It is also possible that the control/monitoring system may need to sense the state of the accelerator and this will require a gateway or data exchange link between the two control systems. MICE VC June 2009Jean-Sébastien GraulichSlide 9

Other remarks on Control  Single-Point Failures The miceecserv node appears to be a single-point failure component for the control/monitor system.  Application Development Environment was not discussed during the review. The committee recommend to consider using interpretive languages (Java, Python, Ruby...) MICE VC June 2009Jean-Sébastien GraulichSlide 10

Configuration database  Structure of the database The committee is concerned it would not easily provide access to online distributions taken at some arbitrary date in the past for comparison with current data  Concurrent accesses A major concern is that the proposed architecture allows multiple concurrent offline accesses to the configuration database, which is a critical element of the online system No authentication, access control or throttling was presented in the design, which presents significant performance and security risks MICE VC June 2009Jean-Sébastien GraulichSlide 11

Bottom line  The DAQ Review is Done !  The DAQ Review is Done !  Very useful comments have been received  Detector DAQ appears to be in shape and should allow data taking for Step III and beyond  Unified Controls, Data Flow and Online Database are more recent projects and the committee has noticed it’s not yet mature Need better integration DAQ/Controls More thinking on Online Data Buffer Disk Access  Serious concern about the man power available on Controls Call for attention from the management  The Control and DAQ Project is not yet over... MICE VC June 2009Jean-Sébastien GraulichSlide 12