1. C OMPUTATIONAL R ESEARCH D IVISION 1 Defining Software Requirements for Scientific Computing Phillip Colella Applied Numerical Algorithms Group Lawrence Berkeley National Laboratory

2. C OMPUTATIONAL R ESEARCH D IVISION 2 High-end simulation in the physical sciences consists of seven algorithms: Structured Grids (including locally structured grids, e.g. AMR) Unstructured Grids Fast Fourier Transform Dense Linear Algebra Sparse Linear Algebra Particles Monte Carlo Well-defined targets from algorithmic and software standpoint. Remainder of this talk will consider one of them (structured grids) in detail.

3. C OMPUTATIONAL R ESEARCH D IVISION 3 Locally-Structured Grid Calculations Numerical solution represented on a hierarchy of nested rectangular arrays. Three kinds of computational operations: - Local stencil operations on rectangles. Explicit methods, iterative solvers. - Irregular computation on boundaries. - Copying between rectangles.

4. C OMPUTATIONAL R ESEARCH D IVISION 4 Algorithmic Characteristics O(1) flops per memory access (10 - 100’s). Codimension one irregularity. Multiple opportunities for parallelism: within a patch, over patches. Multiphysics complexity: many different operators acting in sequence on the same collection of state variables (the operators may contain state for performance reasons.) Irregular computation combined with irregular communication.

5. C OMPUTATIONAL R ESEARCH D IVISION 5 Software Characteristics Layered design to maximize reuse, hide details of parallelism - Rectangular arrays distributed over processers. - Operators to represent the coupling between different levels in the hierarchy. - Solvers, applications. - Fortran 77 on single patches for performance. Bulk-synchronous SPMD execution, alternates communication / computation. Locally static irregular data distributions, approximately load-balanced. C++ implementation: extensive use of templates; inheritance used mainly to define interfaces. Irregular computations implemented in C++. O(10 5 ) lines of code, supporting a range of applications from cosmology to cell biology. Prototype Titanium implementation.

6. C OMPUTATIONAL R ESEARCH D IVISION 6 What are our problems ? Mixed-language programming is a maintenance headache. Fortran is a better compromise between expressiveness and performance than C / C++ for multidimensional arrays, but still not entirely satisfactory (no dimension independent syntax). Bulk-synchronous communication on locally static data distributions limits scalability (Work queues? Overlapping communication and computation?). Load balancing should be based on runtime measurements, rather than approximate analytical models. Serial performance issues. Irregular operations written in C++ are not high-performance. O(1) flops / memory access are not a good match for deep memory hierarchies. Expressiveness and performance are a problem for I/O libraries. Tool-poor development environment.

7. C OMPUTATIONAL R ESEARCH D IVISION 7 Algorithm / Applications Future: More Complexity. Methods for complex geometry: more irregularity, more complex abstractions. Hybrid discrete / continuous models: particles, polymers, dynamic surfaces. Multiple physical processes: widely different workloads on the same grid distributions.