An Accelerated Strategic Computing Initiative (ASCI) Academic Strategic Alliances Program (ASAP) Center at The University of Chicago The Center for Astrophysical Thermonuclear Flashes Development version of Flash (Flash 2.0) A. Siegel Flash Code Architect Flash Code Group: A. Caceres, A. Calder, T. Dupont, J. Dursi, B. Fryxell, T. Linde, A. Mignone, K. Olson, P. Ricker, R. Rosner, K. Riley, A. Siegel, F. Timmes, H. Tufo, N. Vladimirova, G. Weirs, K. Young, M. Zingale

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGeneral goals for Flash development qApproach to Flash2.0 development qArchitectural changes in Flash2.0 q Flash2.0 performance (M. Zingale) q Flash2.0 compatibility testing (K. Riley) qAdditional tools in Flash2.0 q New FlashTest Implementation (A. Caceres) q FUI (K. Young) qFlash future

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGeneral goals for Flash development

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Flash Clients Developers Application Programmers End Users FLASH Framework Modular Design, Development Tools Extensibility Rules Ease of Use Modular testing Reduce learning curve Reduce debugging cycle Swap meshes Add Modules Add data structures Ease of deployment Portability Ease of use

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Minimal impact on performance!

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGeneral goals for Flash development qApproach to Flash2.0 development

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Flash1.6 Assessment qStrengths q Well tested, working q Easy to use, well documented q Portable q Efficient q Modular algorithmic structure qCompromises q Must be expert to develop q Non-scalable data sharing -> tight coupling q Does not take full advantage of new tools, languages q Overly dependent on Paramesh data management q No explicit extensibility API

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Strategy for migration to Flash2.0 qBuild from previous production version qDo not modify Paramesh qBackward compatibility, incremental testing scheme qWrap existing data structures when possible – minimize untangling of low-level code qMerge ongoing changes to production version qLanguage choices q Python (top level) q Fortran90 (drivers) q Not specified (modules) q Client GUI’s (Java Swing)

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGeneral goals for Flash development qApproach to Flash2.0 development qArchitectural changes in Flash2.0 q Flash2.0 performance q Flash2.0 compatibility testing

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Architectural Improvements in FLASH 2.0 qComplete restructuring of data sharing architecture q Variable database with simple API q Standardized interfaces for physics modules q Extensive use of class scope variables q Elimination of common data qGreater use of F90 features q Data hiding using private module fields q Structures, Intent statements q Modules attributes define extensibility qPython/autoconf top-level glue

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Hydro3d.f90 computeRHS(arglist); eulerStep(arglist); Physics Modules (easily interchanged) Driver.f90 evolve(); timeStep(); AMR Library (PARAMESH) Variable Database Evolve.f90 hydro3D( ); burn(); Burn.f90 burner(arglist); Class accessor methods Global memory Parameter list Data Flow PPM.f77NetInt.f77 EulerStep.c Top-level code glue (Python) Flash2.0 Architecture Standard interfaces Framework

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Flash2.0 Extensibility qImplement documented F90 Module methods q High-level code qApplication developer supplies hydro_1dModule and implements functions q Communication with code only through argList and database Subroutine hydro_sweep(…) use variable_database use runtime_parameters use hydro_1dModule, ONLY : hydro_1d, update_soln, & update_soln_boun ! … call hydro_1d(argList) call update_soln(argList) call update_soln_boun(argList)

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGeneral goals for Flash development qApproach to Flash2.0 development qArchitectural changes in Flash2.0 q Flash2.0 performance q Flash2.0 compatibility testing

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Flash2.0 Performance qPotential bottlenecks q Intuitive, string-based database API with error-checking q Numerous database calls q Sharing of module-scope variables – optimization hindrance?

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Flash2.0 Performance qPotential bottlenecks q Intuitive, string-based database API with error-checking q Numerous database calls q Sharing of module-scope variables – optimization hindrance? qPerformance results to date q 10-20% slowdown (worst case) q Comparable scaling

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGeneral goals for Flash development qApproach to Flash2.0 development qArchitectural changes in Flash2.0 q Flash2.0 performance q Flash2.0 compatibility testing qAdditional tools in Flash2.0 q New FlashTest Implementation (A. Caceres) q FUI (K. Young) qFlash future

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago FlashTest Sample output: comparison test report Features: Runs a user-selected list of tests, each test produces an html report. Current tests: comparison, compilation Written in Python Architecture makes it simple to add new types of tests Future: Run tests on multiple hosts (Globus) More types of tests: scaling, performance... A program to automate FLASH verification

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGeneral goals for Flash development qApproach to Flash2.0 development qArchitectural changes in Flash2.0 q Flash2.0 performance q Flash2.0 compatibility testing qAdditional tools in Flash2.0 q New FlashTest Implementation (A. Caceres) q FUI (K. Young) qFlash future

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago FLASH 2.0 Graphical User Interface (FUI) ASCI Red ASCI Blue CLIENT FUI GUI to setup code o edit Modules/Configs o select compiler flags Copy code to Globus grid machine o select machine to run on o monitor/manage run with GUI Manage Globus commands built with FUI. Transfer output back to FUI for interpretation.

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Outline qGoals for Flash development qCore architectural changes in Flash2.0 qFlash2.0 performance qAdditional tools in Flash2.0 q FlashTest q FUI q Python glue qFlash future

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Flash Future qShort-term q Deploy Flash2.0 q Improve interfaces, enhance database q Complete client/server front-end using Globus q Integrate FlashTest and FUI qLonger term q Incorporate remote visualization tools q Architect for Threading (OpenMP) q Add data structures qOngoing q Build new strategies into framework (Linde, Weirs, Paul Fisher) q Collaboration with CCA Forum to define standard mesh API’s q L. Freitag, R. Loy, R. Lusk q Incorporate new mesh implementations

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Summary qNew version of Flash (nearing deployment) qCompletely overhauled data-flow architecture qFoundation for extensibility API, future Flash development qSpeeds development, debugging qNew tools for ease of use, testing

The ASCI/Alliances Center for Astrophysical Thermonuclear Flashes The University of Chicago Software Architecture Shared Libraries Objects (non-abstract) Components Implementations Frameworks Driver Subdriver config Driver factoryAbstract classes client server