Online System Status LHCb Week Beat Jost / Cern 9 June 2015

Beat Jost, Cern Hardware and infrastructure work ❏ LS1 is over ❏ A lot of work has been performed during the last ~30 months ➢ All control PCs have been replaced/virtualized ➢ All PCs upgraded to new OS (W2008S/W7/SLC6) ➢ Configuration Management changed from Quattor to Puppet ➢ Controls infrastructure upgraded to new version of PVSS ❏ Much hardware replaced ➢ ~600 farm nodes extracted ➢ ~800 new farm nodes installed (90% cabled and in production) ➢ ~25 SPECS masters replaced by new hardware ➢ ~30 CAN control PCs replaced by Credit-card PCs 2 LHCb Week, 9 June 2015

Beat Jost, Cern Software ❏ Main challenge was to support the Split HLT ➢ Whole control of the farm had to be adapted/redone ➥ Concurrent running of HLT1 and HLT2 ➥ New task architecture in the farm nodes ➥ New management of conditions ❏ Completely new is the online Alignment and Calibration to provide the HLT with the best Alignment and Calibration constants ➢ HLT1 selected/enriched data for alignment tasks on local disks ➢ Alignment runs on all farm nodes (~10 minutes) ➢ Calibration for Rich and OT done online using either output of online reconstruction or monitoring data coming from HLT1. ➢ Calibration constants available typically within seconds after change of run ❏ To synchronize and sequence all these all these activities automatically a new Scheduler has been developed ➢ For the moment the activities are performed manually, though 3 LHCb Week, 9 June 2015

Beat Jost, Cern Software II ❏ Completely new system for HLT2 and Data Quality monitoring ➢ As HLT2 runs asynchronously and on different runs in different nodes, the normal Data monitoring cannot be used ➥ New system, based on HLT2 output files, has been devised and implemented ➥ Same for the final Data Quality monitoring, running Offline Brunel ➢ Dry tested, but not yet in production 4 LHCb Week, 9 June 2015

Beat Jost, Cern Where are we? ❏ All collision data have been taken without major problems ❏ Most of the data are still spinning around on the local disks ❏ Velo and Tracker alignment works ❏ So far, all actions are performed manually ❏ For a few runs, HLT2 has been run and files transferred to offline for test processing ➢ Some minor problems transferring the conditions to the offline database, will be fixed today or tomorrow 5 LHCb Week, 9 June 2015

Beat Jost, Cern Summary ❏ After all the changes and new development the system is in very good shape ➢ Thanks to extensive testing on the way, especially on data saved end of 2012 ❏ Operationally we still have to learn a lot to run the system in a mainly automatic fashion Many thanks to all involved 6 LHCb Week, 9 June 2015