1. Combinatorial Scientific Computing:
The Role of Discrete Algorithms in
Computational Science & Engineering
Bruce Hendrickson
Sandia National Labs

2. Computational Science
CS and CS&E
Computer Science
Computational Science
What’s in the intersection?
Compilers, system software, computer architecture, etc.

3. Computational Science
CS and CS&E
Computer Science
Computational Science
Algorithmics
What’s in the intersection?
Combinatorial Scientific Computing

4. Combinatorial Scientific Computing
Development, application and analysis of combinatorial algorithms to enable scientific and engineering computations
Practice
Theory
This talk

5. World’s Apart Computer Science = Computational Science & Engineering =
Graph algorithms, set theory, complexity theory, etc.
Computational Science & Engineering =
Numerical analysis, PDEs, linear algebra, etc.
Differ in many ways
Vocabulary, concepts and abstractions
Culture – mathematics versus engineering
Definition of success
Aesthetics
Not an easy divide to span!
Formal proof versus practical improvement
19th Century giants: Gauss, Hilbert, etc.
False dichotomy. Fails to serve either community well.

6. Sparse Direct Methods Reorderings for sparse factorizations
Powerfully phrased as graph problems
Fill reducing orderings
Minimum degree (greedy)
Nested dissection (divide & conquer)
Bandwidth reducing orderings
graph traversals, graph eigenvectors
Heavy diagonal to reduce pivoting (matching)
Efficient exploitation of sparsity
Factorization, triangular solves, etc.
Genesis of CSC community

7. Graphs and sparse Gaussian elimination (1961-)
Fill: new nonzeros in factor 10 1 3 2 4 5 6 7 8 9
G(A)
G+(A) [chordal]
Cholesky factorization:
for j = 1 to n add edges between j’s higher-numbered neighbors

8. Matrix Reordering: Strongly Connected Components
Before After

9. Sparse Direct Methods Reorderings for sparse factorizations
Powerfully phrased as graph problems
Fill reducing orderings
Minimum degree (greedy)
Nested dissection (divide & conquer)
Bandwidth reducing orderings
graph traversals, graph eigenvectors
Heavy diagonal to reduce pivoting (matching)
Efficient exploitation of sparsity
Factorization, triangular solves, etc.
Genesis of CSC community

10. Preconditioning Incomplete Factorizations
Exploiting sparsity patterns, e.g. level-of-fill
Orderings
Partitioning for domain decomposition
Graph techniques in algebraic multigrid
Independent sets, matchings, etc.
Support Theory
Spanning trees & graph embedding techniques

11. Numerical Optimization
Sparse Jacobian Evaluation
Exploit sparsity to minimize function calls
Graph coloring on column intersection graph
Sparse basis construction
Matroids, graph colorings, spanning trees, etc.
Hybrid of combinatorics and numerics

12. Parallelizing Scientific Computations
Graph Algorithms
Partitioning
Coloring
Independent sets, etc.
Geometric algorithms
Space-filling curves & octrees for particles
Geometric partitioning
Reordering for memory locality
Parallelization issues are almost entirely combinatorial

13. Parallelization Strategies
Observation: Parallelization is usually orthogonal to numerics
Issues are non-numerical
Load balancing
Communication minimization
Scheduling, etc.
Almost invariably combinatorial in spirit

14. Mesh Generation Geometric algorithms & data structures
Delaunay/Voronoi decompositions
Convex hulls
Intersection checking, etc.
Topology of unstructured meshes
graph algorithms
Glorious amalgam of mathematical ideas applied to real, and important problem.

15. More Mesh Generation Rich amalgam of mathematical ideas
Differential geometry
Harmonic mappings & numerical PDEs
Optimization to improve mesh quality

16. Computational Biology
Genomics
Fragment assembly
Sequence analysis, etc.
Lots of string algorithms
Proteomics
Structural comparisons
NMR and Mass Spec analysis
Phylogenics
Literature mining
Microarray clustering & analysis
Etc, etc.

17. Statistical Physics Ising spin models and percolation theory
Pfaffians, permanents & matching
Very rich graph theory
Several Nobel prizes awarded
Other percolation models
External fields, Connectivity, Rigidity, etc.
Network flow and other graph algorithms
Cellular Automata

18. Graphs in Chemisty Categorizing molecules by graph properties
Various topological invariants, graph properties
Used to screen molecules for desired properties
Combinatorics of polymers
Geometric and graph properties
Statistically correct ensembles
Graph enumeration and sampling
Cayley coined “graph” in this context.

19. CS&E Techniques in Computer Science
Continuous methods in discrete optimization
Using matrix eigenvectors to understand graphs
Approximation algorithms via linear or quadratic programming
Linear algebra in information analysis
Latent semantic indexing (SVD for info retrieval)
Google’s page ranking (eigenvector & Perron-Frobenius)
Kleinberg’s hubs and authorities (SVD)
Multilevel combinatorial algorithms
Dominant paradigm for practical graph partitioning
Being applied to range of combinatorial problems
Origins in algebraic multigrid

20. The Future … “It’s tough to make predictions,
especially about the future”
Yogi Berra
Prediction: All of the aforementioned and more.

21. Info Organization, Analysis & Mining
Graph algorithms and linear algebra
Importance ranking of documents/pages
Information retrieval
Publication mining is key tool in biology
Text analysis & inference
Simulation output already overwhelming
Learning theory
Advanced visualization

22. More Biology Gene promotion and inhibition Multi-atom interactions
Strings and learning theory
Multi-atom interactions
Protein complexes
Regulatory networks
Geometry, topology and graphs
Biological systems
Whole cell modeling
Ecological models
Topology and graph analysis

23. Fast Algorithms for Huge Problems
For computer science theorists, key distinction is between polynomial & exponential time
For scientific computing, key is often linear versus quadratic time

24. Examples Approximate max-weight matching Extreme Case:
[Monien, Preis, Diekmann], [Drake, Hougardy]
Useful for partitioning
Exact algorithm O(mn) time
2-approximation in O(m) time
Extreme Case:
“Can we understand anything interesting about our data when we do not even have time to read all of it?” Ronitt Rubinfeld

25. Sublinear Time Algorithms
Fast Monte Carlo algorithms for finding low-rank approximations to a matrix
[Frieze, Kannan, Vempala]
Find B0 such that: ||A – B0||F  minB ||A - B||F + ε ||A||F
Run-time independent of size of matrix!
Approximating weight of MST in sublinear time
[Chazelle, Rubinfeld, Trevisani]
Key idea: estimate number of connected components in time independent of size of graph

26. Hard Questions
How will combinatorial methods be used by people who don’t understand them in detail?
What are the implications …
for teaching?
for software development?
for journals?
for professional societies?
CS students should learn about singular vectors.
CS&E students should learn about max-weight spanning trees.

27. Morals
Things are clearer if you look at them from multiple perspectives
Combinatorial algorithms are pervasive in scientific computing and will become more so
Lots of exciting opportunities
High impact for discrete algorithms work
Enabling for scientific computing
Combinatorialists can profit from
New problems (often special cases of existing ones)
Eager customers for advances
New algorithmic insights
Scientific computing folks can profit from
New tools & techniques
Novel points of view
For individual researchers
Lots of interesting stuff at boundaries between fields
Opportunities for enormous impact
Dialogue is Necessary!
What problem variants are useful?
Which application formulations are tractable?
Computer scientist as tool-maker
Sandia’s success in CS&E is partially due to early tolerance of discrete algorithms. Look at my department.

28. Thanks
Yogi Berra, Erik Boman, Edmond Chow, Karen Devine, Alan Edelman, Jean-Loup Faulon, John Gilbert, Mike Heath, Pat Knupp, Esmond Ng, Ali Pınar, Steve Plimpton, Cindy Phillips, Alex Pothen, Robert Preis, Padma Raghavan, Jonathan Shewchuk, Dan Spielman, Shang-Hua Teng, Sivan Toledo, etc.

29. For More Information www.cs.sandia.gov/~bahendr
lists.odu.edu/listinfo/csc
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed-Martin Company, for the US DOE under contract DE-AC-94AL85000