CS 290N / 219: Sparse Matrix Algorithms

Slides:

Advertisements

Similar presentations

Matrices, Digraphs, Markov Chains & Their Use by Google Leslie Hogben Iowa State University and American Institute of Mathematics Leslie Hogben Iowa State.

Advertisements

CS 240A: Solving Ax = b in parallel Dense A: Gaussian elimination with partial pivoting (LU) Same flavor as matrix * matrix, but more complicated Sparse.

CS345 Data Mining Link Analysis Algorithms Page Rank Anand Rajaraman, Jeffrey D. Ullman.

Link Analysis: PageRank

CS 290H 7 November Introduction to multigrid methods

MATH 685/ CSI 700/ OR 682 Lecture Notes

Solving Linear Systems (Numerical Recipes, Chap 2)

Solution of linear system of equations

CSCI 317 Mike Heroux1 Sparse Matrix Computations CSCI 317 Mike Heroux.

Introduction to PageRank Algorithm and Programming Assignment 1 CSC4170 Web Intelligence and Social Computing Tutorial 4 Tutor: Tom Chao Zhou

Page Rank.  Intuition: solve the recursive equation: “a page is important if important pages link to it.”  Maximailly: importance = the principal eigenvector.

CS267 L12 Sources of Parallelism(3).1 Demmel Sp 1999 CS 267 Applications of Parallel Computers Lecture 12: Sources of Parallelism and Locality (Part 3)

Sparse Matrix Methods Day 1: Overview Day 2: Direct methods

The Landscape of Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage (if sparse) More Robust.

CS345 Data Mining Link Analysis Algorithms Page Rank Anand Rajaraman, Jeffrey D. Ullman.

Sparse Matrix Methods Day 1: Overview Matlab and examples Data structures Ax=b Sparse matrices and graphs Fill-reducing matrix permutations Matching and.

CS240A: Conjugate Gradients and the Model Problem.

CS240A: Computation on Graphs. Graphs and Sparse Matrices Sparse matrix is a representation.

Monica Garika Chandana Guduru. METHODS TO SOLVE LINEAR SYSTEMS Direct methods Gaussian elimination method LU method for factorization Simplex method of.

Arithmetic Operations on Matrices. 1. Definition of Matrix 2. Column, Row and Square Matrix 3. Addition and Subtraction of Matrices 4. Multiplying Row.

Elementary Linear Algebra Howard Anton Copyright © 2010 by John Wiley & Sons, Inc. All rights reserved. Chapter 1.

Matrices Write and Augmented Matrix of a system of Linear Equations Write the system from the augmented matrix Solve Systems of Linear Equations using.

Conjugate gradients, sparse matrix-vector multiplication, graphs, and meshes Thanks to Aydin Buluc, Umit Catalyurek, Alan Edelman, and Kathy Yelick for.

CS 290H Lecture 12 Column intersection graphs, Ordering for sparsity in LU with partial pivoting Read “Computing the block triangular form of a sparse.

Support-Graph Preconditioning John R. Gilbert MIT and U. C. Santa Barbara coauthors: Marshall Bern, Bruce Hendrickson, Nhat Nguyen, Sivan Toledo authors.

Google’s Billion Dollar Eigenvector Gerald Kruse, PhD. John ‘54 and Irene ‘58 Dale Professor of MA, CS and I T Interim Assistant Provost Juniata.

Advanced Computer Graphics Spring 2014 K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology.

Complexity of direct methods n 1/2 n 1/3 2D3D Space (fill): O(n log n)O(n 4/3 ) Time (flops): O(n 3/2 )O(n 2 ) Time and space to solve any problem on any.

The Landscape of Sparse Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage More Robust More.

Computation on meshes, sparse matrices, and graphs Some slides are from David Culler, Jim Demmel, Bob Lucas, Horst Simon, Kathy Yelick, et al., UCB CS267.

CS 290H Lecture 5 Elimination trees Read GLN section 6.6 (next time I’ll assign 6.5 and 6.7) Homework 1 due Thursday 14 Oct by 3pm turnin file1.

CS 219: Sparse Matrix Algorithms

PageRank. s1s1 p 12 p 21 s2s2 s3s3 p 31 s4s4 p 41 p 34 p 42 p 13 x 1 = p 21 p 34 p 41 + p 34 p 42 p 21 + p 21 p 31 p 41 + p 31 p 42 p 21 / Σ x 2 = p 31.

Administrivia: May 20, 2013 Course project progress reports due Wednesday. Reading in Multigrid Tutorial: Chapters 3-4: Multigrid cycles and implementation.

1 Incorporating Iterative Refinement with Sparse Cholesky April 2007 Doron Pearl.

Elliptic PDEs and the Finite Difference Method

Solution of Sparse Linear Systems

CS240A: Conjugate Gradients and the Model Problem.

CS240A: Computation on Graphs. Graphs and Sparse Matrices Sparse matrix is a representation.

Matrices and Systems of Equations

Administrivia: October 5, 2009 Homework 1 due Wednesday Reading in Davis: Skim section 6.1 (the fill bounds will make more sense next week) Read section.

CS 290H Administrivia: May 14, 2008 Course project progress reports due next Wed 21 May. Reading in Saad (second edition): Sections

CS 290H 31 October and 2 November Support graph preconditioners Final projects: Read and present two related papers on a topic not covered in class Or,

CS 290H Lecture 15 GESP concluded Final presentations for survey projects next Tue and Thu 20-minute talk with at least 5 min for questions and discussion.

Link Analysis Algorithms Page Rank Slides from Stanford CS345, slightly modified.

Ljiljana Rajačić. Page Rank Web as a directed graph  Nodes: Web pages  Edges: Hyperlinks 2 / 25 Ljiljana Rajačić.

CS 290H Lecture 9 Left-looking LU with partial pivoting Read “A supernodal approach to sparse partial pivoting” (course reader #4), sections 1 through.

A Parallel Hierarchical Solver for the Poisson Equation Seung Lee Deparment of Mechanical Engineering

Symmetric-pattern multifrontal factorization T(A) G(A)

Computation on Graphs. Graphs and Sparse Matrices Sparse matrix is a representation of.

Conjugate gradient iteration One matrix-vector multiplication per iteration Two vector dot products per iteration Four n-vectors of working storage x 0.

Web Mining Link Analysis Algorithms Page Rank. Ranking web pages  Web pages are not equally “important” v  Inlinks.

The Landscape of Sparse Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage More Robust More.

Numerical Algorithms Chapter 11.

Parallel Direct Methods for Sparse Linear Systems

Model Problem: Solving Poisson’s equation for temperature

Solving Linear Systems Ax=b

CS 290H Administrivia: April 16, 2008

Chapter 8: Lesson 8.1 Matrices & Systems of Equations

Computation on meshes, sparse matrices, and graphs

Lecture 22: Parallel Algorithms

Computational meshes, matrices, conjugate gradients, and mesh partitioning Some slides are from David Culler, Jim Demmel, Bob Lucas, Horst Simon, Kathy.

Iterative Aggregation Disaggregation

Piyush Kumar (Lecture 2: PageRank)

Numerical Algorithms • Parallelizing matrix multiplication

CS 290H Lecture 3 Fill: bounds and heuristics

Read GLN sections 6.1 through 6.4.

Administrivia: November 9, 2009

Ax = b Methods for Solution of the System of Equations:

Ax = b Methods for Solution of the System of Equations (ReCap):

Presentation transcript:

CS 290N / 219: Sparse Matrix Algorithms John R. Gilbert (gilbert@cs.ucsb.edu) www.cs.ucsb.edu/~gilbert/cs290

Systems of linear equations: Ax = b

Systems of linear equations: Ax = b Alice is four years older than Bob. In three years, Alice will be twice Bob’s age. How old are Alice and Bob now?

Link analysis of the web 1 5 2 3 4 6 7 1 2 3 4 7 6 5 Web page = vertex Link = directed edge Link matrix: Aij = 1 if page i links to page j

Web graph: PageRank (Google) [Brin, Page] An important page is one that many important pages point to. Markov process: follow a random link most of the time; otherwise, go to any page at random. Importance = stationary distribution of Markov process. Transition matrix is p*A + (1-p)*ones(size(A)), scaled so each column sums to 1. Importance of page i is the i-th entry in the principal eigenvector of the transition matrix. But, the matrix is 10,000,000,000 by 10,000,000,000.

Mark Adams: Bone Density Modeling

Poisson’s equation for temperature

The (2-dimensional) model problem Graph is a regular square grid with n = k^2 vertices. Corresponds to matrix for regular 2D finite difference mesh. Gives good intuition for behavior of sparse matrix algorithms on many 2-dimensional physical problems. There’s also a 3-dimensional model problem.

Solving Poisson’s equation for temperature k = n1/3 For each i from 1 to n, except on the boundaries: – x(i-k2) – x(i-k) – x(i-1) + 6*x(i) – x(i+1) – x(i+k) – x(i+k2) = 0 n equations in n unknowns: A*x = b Each row of A has at most 7 nonzeros.

Graphs and Sparse Matrices: Cholesky factorization Fill: new nonzeros in factor 10 1 3 2 4 5 6 7 8 9 10 1 3 2 4 5 6 7 8 9 Symmetric Gaussian elimination: for j = 1 to n add edges between j’s higher-numbered neighbors G(A) G+(A) [chordal]

The Landscape of Sparse Ax=b Solvers Direct A = LU Iterative y’ = Ay More Robust More General Non- symmetric Symmetric positive definite More Robust Less Storage D

Complexity of linear solvers Time to solve model problem (Poisson’s equation) on regular mesh n1/2 2D 3D Dense Cholesky: O(n3 ) Sparse Cholesky: O(n1.5 ) O(n2 ) CG, exact arithmetic: CG, no precond: O(n1.33 ) CG, modified IC: O(n1.25 ) O(n1.17 ) CG, support trees: O(n1.20 ) -> O(n1+ ) O(n1.75 ) -> O(n1.31 ) Multigrid: O(n)

Administrivia Course web site: www.cs.ucsb.edu/~gilbert/cs290 Join the email (Google) discussion group!! (see web site) First homework is on the web site, due next Monday About 5 weekly homeworks, then a final project (implementation experiment, application, or survey paper) Assigned readings: Davis book, Saad book (online), MGT. (Library reserve; see me for an extra copy of Davis.)