Computational Requirements for NP Robert Edwards Jefferson Lab TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAAAAAAAA

Cold and Hot QCD Generate “snapshots” of gluon fields –Integrate a D=4+1 version of Hamilton’s eqs. –Expensive part: builds in all quantum fluctuations Correlation functions computed over “snapshots”

Nuclear Structure+Reactions “Ab-initio” methods –Green’s Func. MC, No-core Shell Model(s), Coupled-cluster Density-func. method

Nuclear Astrophysics The works! –QCD+QED+Weak(neutrinos)+Gravity (including Gen. Rel.) –Structure, reactions, detailed 3d radiation HD, time-evolution… FLASH: thermodynamic supernovae Core-collapse supernovae

Leadership resources essential

The “wide world” of performance improvement: LQCD as representative Time-consumer: solving Dirac equation (large sparse linear system) Heterogeneous system (cpu+gpu – TitanDev) Increased performance: domain decomp. methods (not bulk synchronous) Example of path forward: collab. with AM/CS & in this case also industry! 100 TF-s on 768 gpus

Trends Architectures going “wide” – not (as) fast A single narrative (NVIDIA, Intel, etc.) – only degree of wideness These new architectures imply/open-up new opportunities for parallelism Bulk-synchronization bad. New programming models needed. Managing large simulations will be a challenge. Might will be application specific.

Available cycles in US LQCD (Cold+Hot) –USQCD facilities: (funded by HEP and NP): 300M core-hrs (clusters - capacity) 4.7M gpu-hrs ! 140M core-hrs Roughly half to NP ! 220M –[NP]ALCC+INCITE +NERSC+NSF+Regional ! 200M [capability] –Total to NP ! 420M core-hrs Nuclear structure+reactions –ALCC+INCITE ! 12+60(+)M ! 75M Nuclear astrophysics –INCITE ! 150M Across NP (lqcd+structure+astro) –~ 650M core-hrs

Computing Requirements Rough conversion factor estimated from LQCD calculations –1 B core-hrs ! 0.1 PF-yr Compare to resource requirements in NP Exascale report Where do we need to be to deliver on NSAC milestones? Where does Moore’s Law get us?

Timelines: Cold QCD

Properties of dense QCD Desired Trajectory Flat Trajectory (Moore’s law) Hot & Dense QCD

Nuclear Reactions Desired Trajectory Flat Trajectory (Moore’s law) Microscopic Theory of Fission Nuclear Structure and Reactions

Solar modelingCore-collapse supernovae Nuclear Astrophysics Desired Trajectory Flat Trajectory (Moore’s law)

How to deliver on science? Clearly, more computational resources essential to deliver on science Moore’s law does not get us there However, unrealistic to expect funding agencies to solve the order of magnitude flop problem Solution: –Algorithms tied to emerging architectures –People Consider a Cold QCD milestone

2018 NSAC milestone: exotic mesons JLab 12 GeV turns on

Delivering on science Increased levels of HPC (cycles) Improve algorithms & extract more performance Partnership with CS & AM: SciDAC Partnerships with Vendors Partnerships with Leadership Computing Facilities Increased level of HPC staffing

Funding profiles for NP Theory computations Computation (cycles) –DOE & NSF Centers [increase ~5x in 2013] –USQCD facilities Software infrastructure, Interdisciplinary support –DOE: SciDAC, Topical Centers –NSF: MRI, PIF, former PetaApps Base funding/leveraging –Leverage off univ. as well as lab positions –Bridge/joint positions with labs – can (have!) be targeted for computations In particular, concern over software/interdisciplinary support –Support for junior positions –Support for work beyond(outside) base funding

SciDAC, topical centers, … SciDAC: – : ~$3M/yr – : ~$3M/yr –2012: ~~ $3M/yr –2013: $1M –2014: ?? DOE Topical centers: – : ~$1M/yr

Answers for Tribble Committee What is the minimum level of support needed to maintain a viable program in computational nuclear physics? At the minimum, a vibrant and healthy program of computational nuclear physics should advance nuclear theory, enhance our understanding of experimental results, and be competitive internationally. A vibrant program will support the achievement of NSAC milestones and deliver a compelling case for continued support of the Nuclear Physics program within the US. What workforce is needed to maintain a viable program? What will it require to take the community to the exascale era? Computational nuclear physics bridges many areas of science, and as such, the expertise of a broad range of individuals including physicists, computer scientists, applied mathematicians, as well as students is vital to the success of the program. As computational environments become more diverse, it is crucial that the workforce include those individuals with skills sufficient to master these challenges. This workforce will follow the emerging trends in these architectures and will develop the algorithms and software infrastructure to enable their exploitation and advance the overall program. Interdisciplinary programs such as SciDAC will become even more crucial in the future to help foster such collaborations.

Will others solve this for us? Japan: –“Origins of the Cosmos” one of 5 major fields for K-computer at Riken [LQCD, Nuclear structure (mostly Cold QCD) and Cosmology] Large HPC efforts in Japan (RIKEN), China and Europe –LQCD Significant efforts in hadron and nuclear spectroscopy as well as nuclear structure. These groups have significant experience in all aspects of the calculations. Research in support of large experimental facilities JParc(Japan), BES(Beijing), GSI(Germany), LHC(Cern) –Nuclear structure + reactions Significant efforts internationally. US well organized. –Nuclear astrophysics Significant efforts internationally. US well organized.