CS 290H Lecture 5 Elimination trees Read GLN section 6.6 (next time I’ll assign 6.5 and 6.7) Homework 1 due Thursday 14 Oct by 3pm turnin file1.

Slides:



Advertisements
Similar presentations
Lecture 15. Graph Algorithms
Advertisements

Partitional Algorithms to Detect Complex Clusters
Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence.
Lecture 3 Sparse Direct Method: Combinatorics Xiaoye Sherry Li Lawrence Berkeley National Laboratory, USA crd-legacy.lbl.gov/~xiaoye/G2S3/
Siddharth Choudhary.  Refines a visual reconstruction to produce jointly optimal 3D structure and viewing parameters  ‘bundle’ refers to the bundle.
CS 240A: Solving Ax = b in parallel Dense A: Gaussian elimination with partial pivoting (LU) Same flavor as matrix * matrix, but more complicated Sparse.
Chapter 8, Part I Graph Algorithms.
Graphs Graphs are the most general data structures we will study in this course. A graph is a more general version of connected nodes than the tree. Both.
Midwestern State University Department of Computer Science Dr. Ranette Halverson CMPS 2433 CHAPTER 4 - PART 2 GRAPHS 1.
Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.
Sparse Matrices in Matlab John R. Gilbert Xerox Palo Alto Research Center with Cleve Moler (MathWorks) and Rob Schreiber (HP Labs)
SOLVING SYSTEMS OF LINEAR EQUATIONS. Overview A matrix consists of a rectangular array of elements represented by a single symbol (example: [A]). An individual.
1 Modularity and Community Structure in Networks* Final project *Based on a paper by M.E.J Newman in PNAS 2006.
CSE 589 Applied Algorithms Spring 1999 Course Introduction Depth First Search.
1cs542g-term Sparse matrix data structure  Typically either Compressed Sparse Row (CSR) or Compressed Sparse Column (CSC) Informally “ia-ja” format.
CS 290H: Sparse Matrix Algorithms
Fast algorithm for detecting community structure in networks.
The Landscape of Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage (if sparse) More Robust.
Sparse Matrix Methods Day 1: Overview Day 2: Direct methods Nonsymmetric systems Graph theoretic tools Sparse LU with partial pivoting Supernodal factorization.
Sparse Matrix Methods Day 1: Overview Matlab and examples Data structures Ax=b Sparse matrices and graphs Fill-reducing matrix permutations Matching and.
CS240A: Conjugate Gradients and the Model Problem.
CSE 373, Copyright S. Tanimoto, 2002 Up-trees - 1 Up-Trees Review of the UNION-FIND ADT Straight implementation with Up-Trees Path compression Worst-case.
CSE 589 Applied Algorithms Course Introduction. CSE Lecture 1 - Spring Instructors Instructor –Richard Ladner –206.
The Evolution of a Sparse Partial Pivoting Algorithm John R. Gilbert with: Tim Davis, Jim Demmel, Stan Eisenstat, Laura Grigori, Stefan Larimore, Sherry.
15-853Page :Algorithms in the Real World Separators – Introduction – Applications.
DAST, Spring © L. Joskowicz 1 Data Structures – LECTURE 1 Introduction Motivation: algorithms and abstract data types Easy problems, hard problems.
Domain decomposition in parallel computing Ashok Srinivasan Florida State University COT 5410 – Spring 2004.
CS 290H Lecture 12 Column intersection graphs, Ordering for sparsity in LU with partial pivoting Read “Computing the block triangular form of a sparse.
Primal-Dual Meets Local Search: Approximating MST’s with Non-uniform Degree Bounds Author: Jochen Könemann R. Ravi From CMU CS 3150 Presentation by Dan.
Complexity of direct methods n 1/2 n 1/3 2D3D Space (fill): O(n log n)O(n 4/3 ) Time (flops): O(n 3/2 )O(n 2 ) Time and space to solve any problem on any.
MA/CSSE 473 Day 12 Insertion Sort quick review DFS, BFS Topological Sort.
The Landscape of Sparse Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage More Robust More.
Graph Algorithms. Definitions and Representation An undirected graph G is a pair (V,E), where V is a finite set of points called vertices and E is a finite.
Parallel Computing Sciences Department MOV’01 Multilevel Combinatorial Methods in Scientific Computing Bruce Hendrickson Sandia National Laboratories Parallel.
Algorithms  Al-Khwarizmi, arab mathematician, 8 th century  Wrote a book: al-kitab… from which the word Algebra comes  Oldest algorithm: Euclidian algorithm.
CS 219: Sparse Matrix Algorithms
An Efficient Algorithm for Enumerating Pseudo Cliques Dec/18/2007 ISAAC, Sendai Takeaki Uno National Institute of Informatics & The Graduate University.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Week 11 - Monday.  What did we talk about last time?  Binomial theorem and Pascal's triangle  Conditional probability  Bayes’ theorem.
Solution of Sparse Linear Systems
Lecture 4 Sparse Factorization: Data-flow Organization
Java Methods Big-O Analysis of Algorithms Object-Oriented Programming
Administrivia: October 5, 2009 Homework 1 due Wednesday Reading in Davis: Skim section 6.1 (the fill bounds will make more sense next week) Read section.
Graphs A graphs is an abstract representation of a set of objects, called vertices or nodes, where some pairs of the objects are connected by links, called.
Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley. Ver Chapter 13: Graphs Data Abstraction & Problem Solving with C++
Domain decomposition in parallel computing Ashok Srinivasan Florida State University.
Data Structures and Algorithms in Parallel Computing Lecture 7.
CS 290H Administrivia: May 14, 2008 Course project progress reports due next Wed 21 May. Reading in Saad (second edition): Sections
© 2006 Pearson Addison-Wesley. All rights reserved 14 A-1 Chapter 14 Graphs.
CS 290H 31 October and 2 November Support graph preconditioners Final projects: Read and present two related papers on a topic not covered in class Or,
CS 290H Lecture 15 GESP concluded Final presentations for survey projects next Tue and Thu 20-minute talk with at least 5 min for questions and discussion.
CS 290H Lecture 9 Left-looking LU with partial pivoting Read “A supernodal approach to sparse partial pivoting” (course reader #4), sections 1 through.
Symmetric-pattern multifrontal factorization T(A) G(A)
Conjugate gradient iteration One matrix-vector multiplication per iteration Two vector dot products per iteration Four n-vectors of working storage x 0.
Laplacian Matrices of Graphs: Algorithms and Applications ICML, June 21, 2016 Daniel A. Spielman.
CS 140: Sparse Matrix-Vector Multiplication and Graph Partitioning
Laplacian Matrices of Graphs: Algorithms and Applications ICML, June 21, 2016 Daniel A. Spielman.
The Landscape of Sparse Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage More Robust More.
Lecture 3 Sparse Direct Method: Combinatorics
CS 290N / 219: Sparse Matrix Algorithms
Solving Linear Systems Ax=b
CS 290H Administrivia: April 16, 2008
Parallel Algorithm Design using Spectral Graph Theory
The Landscape of Sparse Ax=b Solvers
Matrix Martingales in Randomized Numerical Linear Algebra
CS 290H Lecture 3 Fill: bounds and heuristics
Read GLN sections 6.1 through 6.4.
Nonsymmetric Gaussian elimination
Algorithm Course Algorithms Lecture 3 Sorting Algorithm-1
Presentation transcript:

CS 290H Lecture 5 Elimination trees Read GLN section 6.6 (next time I’ll assign 6.5 and 6.7) Homework 1 due Thursday 14 Oct by 3pm turnin file1 file2 … Makefile-or-README Temporary bug: “turnin” only works from csil.cs.ucsb.edu Final project: Either an algorithms experiment, or an application experiment, or a survey paper and talk Experiments can be solo or group; surveys are solo I’ll have some suggestions during the coming week Choose your project by the end of next week

Sparse accumulator (SPA) Abstract data type for a single sparse matrix column Operations: initialize spa O(n) time & O(n) space spa = spa + (scalar) * (CSC vector) O(nnz(spa)) time (CSC vector) = spa O(nnz(spa)) time (***) spa = 0 O(nnz(spa)) time … possibly other ops Implementation: dense n-element floating-point array “value” dense n-element boolean array “is-nonzero” (***) linked structure to sequence through nonzeros (***) (***) many possible variations in details

Nested dissection ordering A separator in a graph G is a set S of vertices whose removal leaves at least two connected components. A nested dissection ordering for an n-vertex graph G numbers its vertices from 1 to n as follows: Find a separator S, whose removal leaves connected components T 1, T 2, …, T k Number the vertices of S from n-|S|+1 to n. Recursively, number the vertices of each component: T 1 from 1 to |T 1 |, T 2 from |T 1 |+1 to |T 1 |+|T 2 |, etc. If a component is small enough, number it arbitrarily. It all boils down to finding good separators!

Separators in theory If G is a planar graph with n vertices, there exists a set of at most sqrt(6n) vertices whose removal leaves no connected component with more than 2n/3 vertices. (“Planar graphs have sqrt(n)-separators.”) “Well-shaped” finite element meshes in 3 dimensions have n 2/3 - separators. Also some other classes of graphs – trees, graphs of bounded genus, chordal graphs, bounded-excluded- minor graphs, … Mostly these theorems come with efficient algorithms, but they aren’t used much.

Separators in practice Graph partitioning heuristics have been an active research area for many years, often motivated by partitioning for parallel computation. See CS 240A. Some techniques: Spectral partitioning (uses eigenvectors of Laplacian matrix of graph) Geometric partitioning (for meshes with specified vertex coordinates) Iterative-swapping (Kernighan-Lin, Fiduccia-Matheysses) Breadth-first search (GLN 7.3.3, fast but dated) Many popular modern codes (e.g. Metis, Chaco) use multilevel iterative swapping Matlab graph partitioning toolbox: see course web page

Complexity of direct methods n 1/2 n 1/3 2D3D Space (fill): O(n log n)O(n 4/3 ) Time (flops): O(n 3/2 )O(n 2 ) Time and space to solve any problem on any well- shaped finite element mesh

Heuristic fill-reducing matrix permutations Nested dissection: Find a separator, number it last, proceed recursively Theory: approx optimal separators => approx optimal fill and flop count Practice: often wins for very large problems Minimum degree: Eliminate row/col with fewest nzs, add fill, repeat Hard to implement efficiently – current champion is “Approximate Minimum Degree” [Amestoy, Davis, Duff] Theory: can be suboptimal even on 2D model problem Practice: often wins for medium-sized problems Banded orderings (Reverse Cuthill-McKee, Sloan,...): Try to keep all nonzeros close to the diagonal Theory, practice: often wins for “long, thin” problems The best modern general-purpose orderings are ND/MD hybrids.

Fill-reducing permutations in Matlab Symmetric approximate minimum degree: p = symamd(A); symmetric permutation: chol(A(p,p)) often sparser than chol(A) Symmetric nested dissection: not built into Matlab several versions in meshpart toolbox (course web page references) Nonsymmetric approximate minimum degree: p = colamd(A); column permutation: lu(A(:,p)) often sparser than lu(A) also for QR factorization Reverse Cuthill-McKee p = symrcm(A); A(p,p) often has smaller bandwidth than A similar to Sparspak RCM

Column Cholesky Factorization for j = 1 : n for k = 1 : j -1 % cmod(j,k) for i = j : n A(i, j) = A(i, j) – A(i, k)*A(j, k); end; % cdiv(j) A(j, j) = sqrt(A(j, j)); for i = j+1 : n A(i, j) = A(i, j) / A(j, j); end; Column j of A becomes column j of L L L LTLT A j

Sparse Column Cholesky Factorization for j = 1 : n for k < j with A(j,k) nonzero % sparse cmod(j,k) A(j:n, j) = A(j:n, j) – A(j:n, k)*A(j, k); end; % sparse cdiv(j) A(j,j) = sqrt(A(j,j)); A(j+1:n, j) = A(j+1:n, j) / A(j, j); end; Column j of A becomes column j of L L L LTLT A j

Elimination Tree Cholesky factor G + (A)T(A) T(A) : parent(j) = min { i > j : (i, j) in G + (A) } parent(col j) = first nonzero row below diagonal in L T describes dependencies among columns of factor Can compute G + (A) easily from T Can compute T from G(A) in almost linear time

Facts about elimination trees If G(A) is connected, then T(A) is connected (it’s a tree, not a forest). If A(i, j) is nonzero and i > j, then i is an ancestor of j in T(A). If L(i, j) is nonzero, then i is an ancestor of j in T(A). T(A) is a depth-first spanning tree of G + (A). T(A) is the transitive reduction of the directed graph G(L T ).