1. SIAM CSE’03 Combinatorial Scientific Computing: The Role of Discrete Algorithms in Computational Science & Engineering Bruce Hendrickson Sandia National Labs

2. Discrete Algorithms & Math Department MIT 4/03 »Compilers, system software, computer architecture, etc. CS and CS&E Computer Science Computational Science l What’s in the intersection?

3. Discrete Algorithms & Math Department MIT 4/03 » Combinatorial Scientific Computing CS and CS&E l What’s in the intersection? Computer Science Computational Science Algorithmics

4. Discrete Algorithms & Math Department MIT 4/03 Combinatorial Scientific Computing l Development, application and analysis of combinatorial algorithms to enable scientific and engineering computations Theory Practice This talk

5. Discrete Algorithms & Math Department MIT 4/03 World’s Apart l Computer Science = »Graph algorithms, set theory, complexity theory, etc. l Computational Science & Engineering = »Numerical analysis, PDEs, linear algebra, etc. l Differ in many ways »Vocabulary, concepts and abstractions »Culture – mathematics versus engineering »Definition of success »Aesthetics l Not an easy divide to span!

6. Discrete Algorithms & Math Department MIT 4/03 Sparse Direct Methods l Reorderings for sparse factorizations »Powerfully phrased as graph problems »Fill reducing orderings –Minimum degree (greedy) –Nested dissection (divide & conquer) »Bandwidth reducing orderings –graph traversals, graph eigenvectors »Heavy diagonal to reduce pivoting (matching) l Efficient exploitation of sparsity »Factorization, triangular solves, etc.

7. Discrete Algorithms & Math Department MIT 4/03 Graphs and sparse Gaussian elimination (1961-) 10 1 3 2 4 5 6 7 8 9 1 3 2 4 5 6 7 8 9 G(A) G + (A) [chordal] Cholesky factorization: for j = 1 to n add edges between j’s higher-numbered neighbors

8. Discrete Algorithms & Math Department MIT 4/03 Sparse Direct Methods l Reorderings for sparse factorizations »Powerfully phrased as graph problems »Fill reducing orderings –Minimum degree (greedy) –Nested dissection (divide & conquer) »Bandwidth reducing orderings –graph traversals, graph eigenvectors »Heavy diagonal to reduce pivoting (matching) l Efficient exploitation of sparsity »Factorization, triangular solves, etc.

9. Discrete Algorithms & Math Department MIT 4/03 Preconditioning l Incomplete Factorizations »Exploiting sparsity patterns, e.g. level-of-fill »Orderings l Partitioning for domain decomposition l Graph techniques in algebraic multigrid »Independent sets, matchings, etc. l Support Theory »Spanning trees & graph embedding techniques

10. Discrete Algorithms & Math Department MIT 4/03 Numerical Optimization l Sparse Jacobian Evaluation »Exploit sparsity to minimize function calls »Graph coloring on column intersection graph l Sparse basis construction »Matroids, graph colorings, spanning trees, etc. l Hybrid of combinatorics and numerics

11. Discrete Algorithms & Math Department MIT 4/03 Parallelizing Scientific Computations l Graph Algorithms »Partitioning »Coloring »Independent sets, etc. l Geometric algorithms »Space-filling curves & octrees for particles »Geometric partitioning l Reordering for memory locality

12. Discrete Algorithms & Math Department MIT 4/03 Parallelization Strategies l Observation: Parallelization is usually orthogonal to numerics l Issues are non-numerical »Load balancing »Communication minimization »Scheduling, etc. l Almost invariably combinatorial in spirit

13. Discrete Algorithms & Math Department MIT 4/03 Mesh Generation l Geometric algorithms & data structures »Delaunay/Voronoi decompositions »Convex hulls »Intersection checking, etc. l Topology of unstructured meshes »graph algorithms

14. Discrete Algorithms & Math Department MIT 4/03 More Mesh Generation l Rich amalgam of mathematical ideas l Differential geometry l Harmonic mappings & numerical PDEs l Optimization to improve mesh quality

15. Discrete Algorithms & Math Department MIT 4/03 Computational Biology l Genomics »Fragment assembly »Sequence analysis, etc. »Lots of string algorithms l Proteomics »Structural comparisons »NMR and Mass Spec analysis l Phylogenics l Literature mining l Microarray clustering & analysis l Etc, etc.

16. Discrete Algorithms & Math Department MIT 4/03 Statistical Physics l Ising spin models and percolation theory »Pfaffians, permanents & matching »Very rich graph theory »Several Nobel prizes awarded l Other percolation models »External fields, Connectivity, Rigidity, etc. »Network flow and other graph algorithms

17. Discrete Algorithms & Math Department MIT 4/03 Graphs in Chemisty l Categorizing molecules by graph properties »Various topological invariants, graph properties »Used to screen molecules for desired properties l Combinatorics of polymers »Geometric and graph properties l Statistically correct ensembles »Graph enumeration and sampling

18. Discrete Algorithms & Math Department MIT 4/03 CS&E Techniques in Computer Science l Continuous methods in discrete optimization »Using matrix eigenvectors to understand graphs »Approximation algorithms via linear or quadratic programming l Linear algebra in information analysis »Latent semantic indexing (SVD for info retrieval) »Google’s page ranking (eigenvector & Perron-Frobenius) »Kleinberg’s hubs and authorities (SVD) l Multilevel combinatorial algorithms »Dominant paradigm for practical graph partitioning »Being applied to range of combinatorial problems »Origins in algebraic multigrid

19. Discrete Algorithms & Math Department MIT 4/03 The Future … “It’s tough to make predictions, especially about the future” Yogi Berra l Prediction: All of the aforementioned and more.

20. Discrete Algorithms & Math Department MIT 4/03 Info Organization, Analysis & Mining l Graph algorithms and linear algebra »Importance ranking of documents/pages »Information retrieval l Publication mining is key tool in biology »Text analysis & inference l Simulation output already overwhelming »Learning theory »Advanced visualization

21. Discrete Algorithms & Math Department MIT 4/03 More Biology l Gene promotion and inhibition »Strings and learning theory l Multi-atom interactions »Protein complexes »Regulatory networks »Geometry, topology and graphs l Biological systems »Whole cell modeling »Ecological models »Topology and graph analysis

22. Discrete Algorithms & Math Department MIT 4/03 Fast Algorithms for Huge Problems l For computer science theorists, key distinction is between polynomial & exponential time l For scientific computing, key is often linear versus quadratic time

23. Discrete Algorithms & Math Department MIT 4/03 Examples l Approximate max-weight matching »[Monien, Preis, Diekmann], [Drake, Hougardy] »Useful for partitioning »Exact algorithm O(mn) time »2-approximation in O(m) time l Extreme Case: “Can we understand anything interesting about our data when we do not even have time to read all of it?” - Ronitt Rubinfeld

24. Discrete Algorithms & Math Department MIT 4/03 Sublinear Time Algorithms l Fast Monte Carlo algorithms for finding low-rank approximations to a matrix »[Frieze, Kannan, Vempala] »Find B 0 such that: ||A – B 0 || F  min B ||A - B|| F + ε ||A|| F »Run-time independent of size of matrix! l Approximating weight of MST in sublinear time »[Chazelle, Rubinfeld, Trevisani] »Key idea: estimate number of connected components in time independent of size of graph

25. Discrete Algorithms & Math Department MIT 4/03 Elsewhere at CSE-03… l MS23/47 Combinatorial Algorithms in Scientific Computing l CP8/32 Discrete Algorithms l IP4 Computational Proteomics (Mark Gerstein) l MS25 Computational Proteomics l CP20 Methods for Particle Simulations l CP26 Geometric Algorithms l MS81 Locality in Scientific Applications l Discrete algorithms play a role in numerous individual talks!

26. Discrete Algorithms & Math Department MIT 4/03 Hard Questions l How will combinatorial methods be used by people who don’t understand them in detail? l What are the implications … »for teaching? »for software development? »for journals? »for professional societies?

27. Discrete Algorithms & Math Department MIT 4/03 Morals l Things are clearer if you look at them from multiple perspectives l Combinatorial algorithms are pervasive in scientific computing and will become more so l Lots of exciting opportunities »High impact for discrete algorithms work »Enabling for scientific computing

28. Discrete Algorithms & Math Department MIT 4/03 Thanks l Yogi Berra, Erik Boman, Edmond Chow, Karen Devine, Jean-Loup Faulon, John Gilbert, Mike Heath, Pat Knupp, Esmond Ng, Ali Pınar, Steve Plimpton, Cindy Phillips, Alex Pothen, Robert Preis, Padma Raghavan, Jonathan Shewchuk, Shang- Hua Teng, Sivan Toledo, etc.

29. Discrete Algorithms & Math Department MIT 4/03 For More Information l www.cs.sandia.gov/~bahendr l lists.odu.edu/listinfo/csc l Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed-Martin Company, for the US DOE under contract DE-AC- 94AL85000