Future Framework John Baines for the Future Framework Requirements Group 1

Contents Introduction : why a new framework FFReq report Key concepts Timescales Recommendations & Observations Next Steps Prototypes/Demonstrators The HLT use-case & Views Demonstrator Summary 2 Note: More technical discussion in Core Software meeting on Thursday

Why do we need a new framework? To exploit full potential of future computer hardware – Transistor density continuing to rise: Increase in cores, not clock speed Per-core memory a limitation – Rapid developments in co-processors  Need to go parallel: Run algorithms in parallel On different events On different parts of same event Internal parallelisation Out-sourcing to external accelerators Take the opportunity to improve structure & functionality – Support trigger functionality as integral part of framework – Improve configuration – transparent, easily understood, recordable, reproducible – Further Improve memory layout of EDM objects: benefit from SIMD & facilitate outsourcing to accelerators – … 3 Year Moore’s law Clock Speed

Future Framework Requirements Future Frameworks Requirements group – Started work in March ’14 – Regular open meetings => Report in CDS for comments (5 Dec 2014) ATL-COM-SOFT Gives overview of current system – offline & trigger Lists requirements for New Framework Recommendations & Timescales Observations Please read & comment in CDS – So far 3 comments 4

Some Key Concepts Whiteboard: – stores data objects and exchanges them between components View: – same interface as whiteboard. Contains subset of data objects e.g. in RoI Algorithms: – explicitly declare their Data Dependencies – Receive context: either whole event or view Scheduler: – Executes algorithms - in parallel where possible (subject to constraints) – Also handles schedulable resources and tasks: Incidents become tasks Tools: – Configurable component of algorithm - private (no public tools) Services: – shared and global – must be aware of context when called from algs and tools. 5 Retain core design principles of current framework

Timescales: driven by Run-3 readiness 2014 Q3Q4 LS 1 New Framework Delivered and production-ready Basic tests running in initial framework prototype Run 3 Framework Requirements Capture Complete 6 Decision on core technological solution Design, Prototyping, Initial Implement n Continued Development Refinement & further development as needed Complete migration Validate & Commission Migration of a limited set of algos. Migration of extended set of algos. Bug fixes, further refinement as needed HLT software Commissioning Complete Migration of Algorithms and Tools to new framework complete Framework Algs & Tools Framework running limited set of algos.

Recommendations ATLAS should evolve framework to support multi-threaded and multi-process execution of events and algorithms in offline & trigger use cases – Retain core design principles of current framework, but extend to: better support concurrent execution incorporate the HLT requirements Readiness for Run-3 => not so much time => need to quickly decide next steps Development should include real algorithmic code from very early stage 7 Observations Basing New Framework on evolution of Gaudi/Athena framework would be a good choice Continue to benefit from collaboration from LHCb and CERN SFT Development of New Framework requires significant effort from skilled developers Will also need significant effort for algorithmic changes: => investment in training for existing & new developers

Next Steps Under Discussion – to be finalised Produce Final version of FFReq document – Taking into account comments - please read & comment in CDS – Add appendix on Analysis use-case Small subgroup formed Aim to produce appendix on ~short timescale – Take into account comments from ATLAS management: Clearer separation of requirements & design Distinguish “must-have” and “would be nice” requirements Add discussion of migration strategy for non-framework code – algorithms, tools... – Review by small technical review team – Final sign-off from OAB Establish project to design & implement Future Framework – Prioritisation of work – Add effort estimates => Endorsement by EB 8

Prototypes/Demonstrators Continue and expand current prototyping work Focus on answering key design questions Demonstrators: – G4 Simulation – CaloHive – Scheduled Incidents – Views – see next slides 9

Key HLT Concepts 10 Only 1 in 100 L1-accepts selected by HLT and recorded to disk Need to minimize resource cost to reach decision Key design principles: Incremental Reconstruction & Selection  Provides Early Rejection Reconstruction inside geometrical Regions (Regions of Interest) Independence of trigger chains: one trigger does not influence another => can add/remove/pre-scale chain without affecting other triggers  facilitates calculation of trigger efficiencies Ability to import offline tools into online environment

Views Demonstrator 11 In Run-1 & Run-2 the HLT use-case implemented via extensions to current framework: HLT Steering: Schedules HLT algorithms Provides them with a RoI context HLT Navigation: Associates objects in a RoI In New Framework: Scheduler also supports HLT use case Views provide RoI context Views Demonstrator will: Assess options for implementing views Demonstrate that they meet HLT requirements Demonstrate minimal impact to offline use-case Address portability of offline tools to online environment with no/minimal changes HLT in Existing Framework

Summary Finalise FFReq Report – incorporate comments – Small sub-group established to add Appendix on Analysis Use Case – Review by small technical review team => Final sign-off from OAB Next Steps: – Establish project to Design and Implement Future Framework Develop work-plan: Priorities, deliverables, milestones, effort estimates => Endorsement by EB – Recruit effort and start design & prototyping work Happening now – a very good time to join effort! – Kick-off workshop planned for May 12 Note: More technical discussion in Core Software meeting on Thursday Under Discussion. Details to be finalised

CDS Comments Anna: – Request for small corrections and clarifications – addressed in new version Markus: – Clarify relation of New Framework to Gaudi – Mention EDM for raw and PrepRawData, Identifiers, Identifiable containers – Extrapolator tool – example of big complex shared tool – Need to allow deletion of objects from whiteboard to limit memory – Demonstrator needed for views – Framework must support internal parallelism within algorithms order in which the threads are submitted and processed known and deterministic – Clarify the roles of algorithm/tool interface and whiteboard in data exchange Interface specifies data, objects themselves on whiteboard – Persistency: needs to support current and future formats – Support for coprocessors: Framework needs hooks Implementations for specific architectures should follow a cost/benefit analysis – Clarification on configuration – Need for follow-up on Analysis 13

14