Exascale Runtime Systems Summit Plan and Outcomes Sonia R. Sachs 04/09/2014 AGU, Washington D.C.

Generate a roadmap for achieving a unified runtime systems architecture for Exascale systems – Reach consensus on the top six challenges and solutions for them. – Agree on a comprehensive set of questions that must be answered in order to achieve such architecture – Current known answers to posed questions Generate a roadmap for a research program on runtime systems – Consistent with achieving a unified runtime systems architecture Discuss future workshop Prepare for writing a report Summit Goals 1 Summit rule: Participants should not promote their current research agenda

Agree on top six (6) challenges and solutions (1 hour) – Strawman set of challenges: slide 3 For each challenge: discuss current state-of-the-art and how is challenge addressed in existing runtime systems to be leveraged? (1-2 hours) Agree on a set of top questions to be answered (1 hours) – Strawman set of questions: slide 4 For each question: discuss currently known answers and how existing runtime systems answer it? (1-2 hours) Vision (1-2 hours) – what are major components? – programming interfaces and interfaces to the OS – How do we measure success Plan to create the Unified Runtime Systems Architecture Roadmap 2 We need to leverage current investments in runtime systems: : OCR, HPX, ARTS, SEEC, GVR runtime and runtimes to support advance/extended MPI and Global Arrays.

Strawman set of challenges Key abstractions need to be identified – and jointly supported by the runtime system, compilers, and hardware architecture. Runtime support for lightweight tasks and their coordination – capable of dealing with system heterogeneity and with end-to-end asynchrony. Runtime support for locality-aware, dynamic task scheduling – Enabling continuously optimizing code or data movement. Need for task coordination and synchronization primitives Runtime support for dynamic load balancing – To deal with load imbalances created by a large number of sources for non-uniform execution rates 3 What is the current known key abstractions? Which key abstractions are currently supported by runtime systems to be leveraged? For each of these challenges: What is the current state-of-the- art on such runtime support? How is this done in runtime systems to be leveraged?

Runtime system software architecture – What would the principal components be? What are the semantics of these components? What is the role of the different execution models? – What are the mechanisms for managing processes/threads/tasks and data? – What policies and/or mechanisms will your runtime use to schedule code and place data? – How does the runtime dynamically adapt the schedule and placement so that metrics of code-data affinity, power consumption, migration cost and resiliency are improved? – How does the runtime manage resources (compute, memory, power, bandwidth) to meet a power, energy and performance objective? – How does the runtime scale? – What is the role of a global address space or a global name space? – What programming models will be supported by the runtime architecture? – What OS support should be assumed? Community buy-in: – How do we achieve community buy-in to an envisioned runtime architecture and semantics? – We need a process to continuously evaluate refine the envisioned runtime architecture and semantics while keeping focus on achieving an Exascale runtime system. What should this process be? Strawman Set of Questions 4

Current Runtime Investments 2012 X-Stack program [2]: application-driven runtime systems support: – maximizing concurrency efficiency, – dealing with asynchrony of computation and communication, – exploiting data locality, – minimizing data movement, – managing faults, – support for heterogeneous computing elements, – semantics for programmability, – support for novel programming models Runtime systems to be leveraged – OCR, HPX, ARTS, SEEC, GVR runtime and runtimes to support advance/extended MPI and Global Arrays 5

Mapping important questions to projects: – facing) facing Curent Runtime Investments 6

Current Runtime Investments 2013 OS/R Program [4]: systems-driven mechanisms described in the OS/R report: – thread management, – low-level communication services, – resource management, – different runtime service components tightly connected to deal with challenges: resilience, asynchronous computations, and locality of computation. Runtime systems approaches to be leveraged in ARGO, HOBBES, and X-ARCC projects. Mapping important questions to projects: – Not yet available To be completed after upcoming OS/R semi-annual review 7

Summit Outcome: top challenges Growing gap between communication and computation Scalability & Starvation: dominant parameters to optimize, critical path management Locality and data movement: need terminology for inter and intra Power is critical Overhead Resilience: scalability and power problems exacerbates Load balancing: contention, hot spots, Heterogeneity: performance irregularities, static and dynamic, heterogeneity in storage/memory In-situ data analysis and mgmt: new dimension of interoperability 8 Exploitation of runtime information (introspection), feedback control of performance data, managing performance data Resource allocation Scheduling & workflow orchestration Complexity/optimization/tuning Portability Synchronization: event-driven, mutual exclusion, barriers, phasers Computing everywhere Name space: both data and computation, includes location management Support tools Support for migratable computational units Hardware support, tight-coupling Expose some of runtime elements to system managers

Summit Outcome: top challenge classes 1.Locality and data movement: need terminology for inter and intra, dynamic decisions, handling variability, conflict of optimizing locality, data movement and costs of dynamic scheduling- questions of policy. Synchronization: event- driven, mutual exclusion, barriers, phasers. Overhead. Growing gap between communication and computation 2.Resilience: scalability and power problems exacerbates. 3.Variability. Static and Dynamic. Power management. Load balancing: contention, hot spots. Exploitation of runtime information (introspection), feedback control of perfomance data, managing performance data 4.Heterogeneity: performance irregularities, static and dynamic, heterogeneity in storage/memory. Computing everywhere. 9 5.Scalability & Starvation: dominant parameters to optimize, critical path management. Name space: both data and computation, includes location management. Complexity/optimization/tuning 6.Portability and interoperability. In-situ data analysis and mgmt: new dimension of interoperability: runtime systems composability. 7.Resource allocation. Scheduling & workflow orchestration. Cross jobs (apps) scheduling: OS role. Focus scheduling for one job. Support for migratable computational units. Hardware support, tight-coupling. Expose some of runtime elements to system managers 8.Usability. Support tools.

Summit Outcome: Challenge Problems For each challenge problem, we want to give examples in the context of challenge problems – Vivek suggested one multi-physics challenge problem. – Homework: identify and describe challenge problems 10

Summit Outcome: Key Abstractions Unit of computation – attributes: locality, synchronization, resilience, critical path Naming: data, computation, objects that combine both (active objects) Global side-effects: programming model abstraction? Execution Model Machine Model, Resources: memory, computation, storage, network, … Locality and affinity, hierarchy Control State: collective of info distributed across the global system that determines the next state of the machine. Distributed snapshot of the system. Logical abstraction, how to reason about the system. Enclave Scheduler: local scheduler of a single execution stream Execution Stream: something that has hardware associated with Communication data transfer Concurrency patterns, synchronization Resilience, detection, fault model 11

Summit Outcome: runtime services Runtime Services – Schedule and execute threads/tasks/work unit, including code generation – Resource allocation (give me resources dynamically, as needed, release resources): including networks. heterogeneity – Introspection services: info about power, performance, heterogeneity. Variability. – Creation, translation, isolation, security, release: name space, virtualization – Communication of data and code, including synchronization (event- oriented). Migration services. Move work, move data. Is not separate from the communication services, it is composed with. 12 Runtime services – Concurrency control: isolation, atomics (it gets into scheduling?) – Location and Affinity/Locality services: map to some things that are mentioned above. Provides information and does binding. – Express error checking/detection and recovery. Allows to specify resilience properties. Both to computation and data and hardware resources. – Load balancing. Scheduling. – OS requests services from the runtime: give me back resources that I gave you, tell runtime to graceful degradation/shutdown – Services can make requests to other services, e.g., tools Service Attributes – How the service will be provided? – Expected resilience – Expected resources usage – Persistence of memory – Locality attributes

Summit Outcome: runtime services 13 Homework: – Refine key abstractions and their definition – Using refined key abstractions, define runtime services – Create a matrix for mapping on what current investments (including Charm++ runtime) provide these services and a brief description on how the services are provided. Deadlines – Initial draft to be distributed to summit participants: April 22 – Final draft including comments/suggestions from summit participants: April 309 Homework Team: – Wilf, Vivek, Kathy, Vijay

Presentation of solutions by Wilf (slides ) Discussions of the presented ideas More questions than we had time for: – We will post Wilf’s presentation in the xstack wiki – Wilf will present these again at the X-Stack PI meeting – Summit participants are encouraged to send Wilf and I comments/questions/suggestions – We will encourage X-Stack meeting participants to give us comments/questions/suggestions Summit Outcome: community buy-in 14

Community buy-in Ecosystem Creation: How do we achieve community buy-in to an envisioned runtime architecture and semantics? Process: We need a process to continuously evaluate refine the envisioned runtime architecture and semantics while keeping focus on achieving an Exascale runtime system. What should this process be?

Ecosystem Creation Establish an open, transparent environment where the solution is not pre-determined Provide an organic process for community decision- making, ensuring that the best solution wins Avoid a single player or clique dominating Lower the barrier to participation by providing stable, reliable releases of candidate solutions to a broad audience

Process Build an independent, open-source foundation that ensures the different projects can be continuously available, evolved, and supported. The different projects will evolve based on the contributions made. As solutions demonstrate their superiority, they will attract more contributions as well as consensus. The community will organically migrate to the superior solution. DOE can continuously view progress and help fund projects to cover any critical shortfalls.

Who is in the Community 1.Exascale Computing Research Community (us) 2.High Performance Computing User Community (current users) 3.Academic Community (future users) 4.Application Development Community (scientists and engineers) 5.Software Development Community 6.Hardware Vendors

Community Services Project Team Infrastructure - e.g. source code control, tooling, debuggers, collaboration/communication Release Engineering Technical Support IP management Education, instruction and training Community Development

Build on Experience – Community 2.0 Learn from the best Eclipse, Apache, Mozilla Building the community/ecosystem is top priority Support multiple projects and give them autonomy Support Regular Community Interaction Long-term commitment to quality through education and process Avoid the pitfalls Commercial control of the purse strings leads to community breakdown Get to community support quickly and maintain community control

Runtime system software architecture – What are the major services provided by this architecture? – What is the strategy that the runtime system has to embody? What is the role of the different execution models? – What would the principal components be? What are the semantics of these components? – What are the mechanisms for managing and scheduling units of computation and data? – How does the runtime dynamically adapt the schedule and placement so that metrics of code-data affinity, power consumption, migration cost and resiliency are improved? – How does the runtime manage resources (compute, memory, power, bandwidth) to meet a power, energy and performance objective? How are resources exposed? – How does the runtime scale? How does the runtime ensures its scalability? – What is the role of name/address spaces? Are they global or not? What is their scope? – What programming models will be supported by the runtime architecture? – How will composability be enabled? – What OS support is assumed? What can the OS ask/expect from runtimes? – What can compilers ask/expect of runtimes? What runtimes can ask/expect of compilers? – Just in time compilation: what is the runtime support needed? – What hardware support is assumed, can be exploited, can helpful? What is the machine model assumed? – What is the cost model assumed (energy, performance, resilience)? – How does the runtime system enables use of application or system info for resilience? In general, how does the runtime system uses information? – What tools expect from runtimes, and what runtimes expect from tools? Summit Outcome: comprehensive set of questions 21

Unit of computation manager and scheduling Name service for everything that one wants to virtualize. Address allocation/translation. Data distribution and redistribution Locality management Power management Communication interfaces and/or infrastructure. Network I/O Active storage (compute in storage). I/O, locality Load balancers Summit Outcome: runtime systems major components 22 Location managers Prefetcher for explicit memory mgmt Lightweight migratable threads Introspection management. Monitoring/tools interface Reliable data store. I/O is embedded here. Global termination detection Event and synchronization framework Failure detection Failure recovery Adaptive controller Interoperability (with in-situ analysis, visualization, etc.) Composability manager

Summit Outcome: component interfaces 23 Interfaces should: – Support hybrid programming models – Interface to compilers and OS – Ensure progress guarantees (formal methods) Interfaces may be distributed/centralized Homework: – Refine major components and their definition – For each component, describe its interfaces Team: – Sanjay, Vivek, Thomas, Costin, Ron (composability), Vijay, Milind Deadlines: – Initial draft to distribute to summit participants: May 12 – Final draft to be presented at X-Stack meeting: May 27

Enable efficient applications execution on exascale hardware with runtime systems that address the need for massive hierarchical concurrency, data movement minimization, failure tolerance, adaptation to performance variability, and management of energy and system resources. Summit Outcome: Runtime Systems Vision Statement 24

Proposed high-level criteria – Efficiency, scalability, productivity – Reliability, power management – Move from static control to dynamic control; introspection – Move programming burden from programmer to system – Heterogeneity – Strong scaling and greater generality – SLOWER: starvation, latency, overhead, waiting, energy, resilience. – How well is energy conserved? – How do we measure runtime ability to handle heterogeneity and variability? – How do we measure resilience? – How do we measure the ability to handle load imbalances? – How do we measure scalability? Summit Outcome: how do we measure success 25 Micro benchmarks and mini-apps How to measure? Metrics: – Time, work, energy – Idleness – Combined task scheduling and communication metric – Flexibility of the system (doing new things quickly) – Overhead: not orthogonal to starvation, lower bound on the thread that can be explored, reduces concurrency. Sanjay can explain how. – Need to engage performance tools. – What is the key bottleneck? – Ease of programming/productivity metric: what could that be?

New runtime research – runtime mechanisms to extract parallelism – Proof that dynamic adaptive runtime systems are/not needed due to variability: simulation modeling? When can we get this done? Need trends, need to know bounds. – Metrics crosscutting with existing and new research – Programming interfaces for programmer engagement – Composability management – Integration with IO, network, storage – Debugging, debugging, debugging – Compute everywhere: adds challenges for debugging – Distributed algorithms for scheduling that scales – Workflow usage models – Improved micro-benchmarks, mini-apps that exploit runtime systems attributes Plan to create Roadmap for Runtime Research 26 New runtime research – Dynamic, interactive steering – Energy consumption/ power management – Make the machine more useable by sys admin – Learning runtime with observations – How to deal with variability – Interoperability of runtime with workflow and job scheduler and in- situ analytics: model s of use Integration into a open-source community runtime: Modelado Testing/validation for ASCR/NNSA apps running at scale

Major milestones and time-line – Should follow Hardware timeline – P0: petascale node by 2017 – P1: exascale node by 2019 – P2: exascale cabinet prototype by 2022 Requirement: – demonstrated the benefits of dynamic adaptive runtime for regular apps ( ) P0: – 1. evaluation of proxy apps with different runtimes, exercising composability – 2. identifying hardware dependencies and pruning the list – 3. demonstrate that runtimes can scale up to petascale – 4. intermediate representation identified/specified – 5. Models and evaluation methodologies – 6. Model for compute everywhere – 7. Model for debuggability – 8. demonstrate on a multi-note context – 9. demonstrate explicit management of memory, or the other way around. If it can be done by Plan to create Roadmap for Runtime Research 27 P1: – 1. evaluation of larger proxy apps with different runtimes, exercising composability – 2. refining hardware dependencies list – 4. demonstrate benefits of intermediate representation – 5. demonstrate runtime mechanisms to extract parallelism on exascale context – 3. demonstrate that runtimes can scale up to exascale – 10. validation of Models and evaluation methodologies – 6. validation of Model for compute everywhere – 7. validation Model for debuggability – 8. demonstrate on a multi-node context, – 9. demonstrate explicit management of memory, or the other way around

P2: – 1. evaluation of apps running at scale with different runtimes – 2. refining hardware dependencies list – 4. demonstrate benefits of intermediate representation – 5. demonstrate runtime mechanisms to extract parellelism on exascale context – 3. demonstrate that runtimes can scale up to exascale – 10. validation of Models and evaluation methodologies at scale – 6. validation of Model for compute everywhere at scale – 7. validation Model for debuggability at scale – … Plan to create Roadmap for Runtime Research 28 Homework: – Refine the milestones for P0, P1, and P2 – Team: Thomas, Wilf, Ron, Costin Deadlines: – First draft to be distributed to the summit participants: May 12 – Final draft to be shared at the X-Stack PI meeting: May 27

Proposed outline – Top Challenges – Comprehensive set of questions to be answered – State-of-the-art: How challenges and questions are addressed in existing runtime systems that we want to leverage? – Towards a Unified Runtime Systems Architecture Components Interfaces – Conclusion Recommendations towards jointly evolving vision of unified runtime systems architecture Recommendations on the roadmap to Runtime Systems Research Recommendations regarding workshops Schedule. – First draft: May 27 (target 10 pages) – Present at X-Stack meeting and collect feedback from X-Stack community – Coordination calls in June and July. – Final draft: July 31 Assignments plan – Send me proposed changes to the outline and volunteer for report section by April 16 – I will distribute assignments by April 22 Plan for Writing the Report 29