1. LHCb Software & Computing Status
Stefan Roiser
LHCC WLCG Referees
12 September 2017

2. Stefan Roiser -- LHCC WLCG Referees
Production Status
Very good usage of distributed computing resources
Peak usage with HLT farm fully available
Last two weeks running HLT & MC in parallel
Dominated by Simulation (85 %), then
User analysis (9 %)
Data processing (5 %)
NB: ”Turbo” at % (Run 3 main workflow)
2017 data processing following closely data exported from HLT
HLT
12 Sep '17
Stefan Roiser -- LHCC WLCG Referees

3. Optimizing Resources While LHC performs less efficient:
Execute simulation workloads in the HLT farm in parallel to HLT processing
Envisage to use “fast stop” to reduce simulation payloads when needed
i.e. send signal to payload, tear down a job successfully in O(minute(s))
HLT1
HLT2
Simulation
HLT Farm Running Jobs by Job Type

4. Job Success Rates & Efficiencies
Failed CPU time within Job Types
Data Processing
2.3 %
MC Processing
4.6 %
User Analysis
24.1 %
Working Group Prod
11.2 %
Job Types accounting
for 99,9 % of work spent
5.8 %
CPU/Walltime efficiency continuously at high rate across all job types
Overall job success rate > 94 %
Production jobs (94 % of total CPU) at 95 – 97 %
User analysis only at 75 % with high peaks of single users submitting failing jobs  more training needed
12 Sep '17
Stefan Roiser -- LHCC WLCG Referees

5. Monte Carlo Production Status
Many optimizations for storage done for MC Productions
”Filtered” productions  less events on disk
mDST produces a factor 10 smaller event size
Requests for Simulation are growing rapidly since EYETS, current estimates
~ 1 month to produce most urgent samples
~ 5 ½ months to produce all currently requested events
Danger for negative impact on analysis
Re-visit system for requesting MC samples for better prioritization and forecast
Encourage requestors whenever possible to use new fast simulation options
12 Sep '17
Stefan Roiser -- LHCC WLCG Referees

6. Stefan Roiser -- LHCC WLCG Referees
2017 Pledges & 2018/19 Requests
Because of reduced efficiency of LHC in 2017 forecasts for disk until end of data taking year were reduced as well
Re-use now available disk for
Un-parking of 2016 Charm Turbo data
Re-stripping of 2016 data
Requests for 2018/19 are slightly reduced b/c of lower 2017 baseline
Disk 1.6% / 3.5 % lower than initially requested
Tape 19 % / 18 % lower than initially requested
B/c of suppression of “master DST” format used to eventually regenerate mDST
NB: Importance for LHCb to have balanced T1 resource distributions
12 Sep '17
Stefan Roiser -- LHCC WLCG Referees

7. Analysis Reproducibility & Preservation
Concentrating together with physics working groups on an analysis preservation system
Kickoff at hackathon April 2017
Document with strategy and best practices in preparation
Backed by additional resources stated first time in CRSG 2018 requests
Initially 50 cores & 10 TB disk / working group
12 Sep '17
Stefan Roiser -- LHCC WLCG Referees

8. Stefan Roiser -- LHCC WLCG Referees
Run 3 Upgrade Status
Working on performance optimizations during summer
Several big gains achieved with code optimization, algorithm re-design, etc.
O(10s %) to factors in different algorithms. Final checks needed.
Several optimizations already backported and available in 2017 data processing
Working on TDR, due by the end of 2017
Skeleton is prepared, responsible people for chapters are identified
Hot phase of writing the document is starting now!!
 Dedicated talk in the LHCC/LHCb experiment session
12 Sep '17
Stefan Roiser -- LHCC WLCG Referees

9. Stefan Roiser -- LHCC WLCG Referees
Summary
Very good usage of available resources
High job success rate & CPU efficiency
Simulation running in parallel with HLT processing
Data processing productions following closely data export
Simulation requests above what can be achieved with available resources
Lower LHC efficiency allows disk usage for important tasks
Un-parking of Turbo data, re-stripping of 2016 data
Push towards analysis reproducibility and preservation
Run 3 upgrade work ongoing with TDR at the end of 2017
12 Sep '17
Stefan Roiser -- LHCC WLCG Referees