Breakout Group: Debugging David E. Skinner and Wolfgang E. Nagel IESP Workshop 3, October, Tsukuba, Japan

Exascale Debugging Debugging: finding problems in the execution of code. Identifying and dealing with sources of: –incorrectness (application and architecture) –application failure (deadlock, hang, segfault) –critical application bottlenecks (standstill, performance cliff) Exascale issues –Concurrency expense of debugging –Scalability of debugger methodologies (data and interfaces) –Concurrency scaling of the frequency of errors/failures –Heterogeneity and lightweight OS 2009/05/21

Exascale Trends relevant to debugging To which broad exascale trends is debugging related? Concurrency ✓ Reliability ✓ Power Costs Heterogeneity in a node ✓ I/O and memory: ratios and breakthroughs 2009/05/21

What’s different about exascale debugging? Assumption that many things may/will go wrong at the same time will require triage, filtering, and clustering of faults and problems Focus on multi-level debugging, communicating details of faults between software layers Synthesis of fault information into understanding in the context of application and architecture Simulation of concurrency when possible Excision of buggy code snippets to run at lower concurrencies 2009/05/21

Debugging 2009/05/21 Priority Research Direction (use one slide for each) Key challenges Vertical integration of debug and performance information across software layers Layered contexts of debugging (just MPI, just I/O, or framework/application defined ) Scalable clustering of application process states and contexts. Filter/search within debugger Automatically triggered debugging Basic challenge of concurrency (hard & $$) Interoperability with compiler, library, runtime, OS and I/O Debugging without stopping (resilient analysis of victim processes) More eyes on debug information besides the person running the debugger Multi-layered debug histories become available/useful to system-wide monitoring Debugging meets performance analysis Debugging informs system software Lowering overhead and barriers to debugging at large scale Debuggers begin to communicate user level metrics, debugging becomes more meaningful Greater certainty in scientific validity of exascale’s computational results. Trust. Summary of research direction Potential impact on software component Potential impact on usability, capability, and breadth of community

UR Graph 2009/05/21 Roadmap for exascale debugging Planning & Workshops 1e5 cores 1e6 cores Breakthroughs needed for 1e6 core production debug Near-production exascale Scale of debugging

4.x.Debugging narrative 2009/05/21 Technology drivers Alternative R&D strategies Recommended research agenda Crosscutting considerations

Roadmap sections on debugging tools Technology drivers for Debugging Alternative R&D strategies for Debugging Recommended research agenda Debugging + Identify cross-cutting consideration and connections (compilers, resiliency and performance) + Identify key regional interests, expertise, and resources 2009/05/21

State of the art Debuggers scale to 10K procs Vendors are developing solutions for new debugging contexts (memory, communication, etc.) Some progress in clustering and data aggregation 2009/05/21