Pádraig Cunningham University College Dublin Matrix Tutorial Transition Matrices Graphs Random Walks

2 Objective To show how some advanced mathematics has practical application in data mining / information retrieval. To show how some practical problems in data mining / information retrieval can be solved using matrix decomposition. To give you a flavour of some aspects of the course.

3 Stochastic Matrix: Markov process From IFrom IIFrom III To I To II To III In 1998 (in some state) Land use is:  30% I (Res), 20% II (Com), 50% III (Ind) Over 5 year period, the probabilities for change of use are:

4 Stochastic Matrix: Markov process = Land Use after 5 years v 1 = Av 0 v 2 = A 2 v 0 similarly and so on…

5 Stochastic Matrix: Markov process When this converges:  v n = Av n  i.e. it converges to v n an eigenvector of A corresponding to an eigenvalue 1.  v n = [ ]

6 Brief Review of Eigenvectors The eigenvectors v and eigenvalues of a matrix A are the ones satisfying Av i = i v i i.e. v i is a vector that:  Pre-multiplying by matrix A is the same as  Multiplying by the corresponding eigenvalue i

7 The important property… Repeated application of the matrix to an arbitrary vector results in a vector proportional to the eigenvector with largest eigenvalue  What has this got to do with Random Walks?...

8 Transition Matrices & Random Walks Consider a random walk over a set of linked web pages. The situation is defined by a transition (links) matrix. The eigenvector corresponding to the largest eigenvalue of the transition matrix tells us the probabilities of the walk ending on the various pages.

9 Web Pages Example Eigenvector corresponding to largest Eigenvalue  0.38  0.20  0.49  0.26  0.71 EVD: A D C B E ABCDE A10001 B11000 C01101 D00110 E11111 From To

10 Review of Matrix Algebra Why matrix algebra now?  The Google PageRank algorithm uses Eigenvectors in ranking relevant pages. Resources   The Matrix Cookbook

11 Brief Review of Eigenvectors Eigenvectors are a special set of vectors associated with a linear system of equations (i.e., a matrix equation). Each eigenvector is paired with a corresponding so-called eigenvalue. The decomposition of a square matrix into eigenvalues and eigenvectors is known as eigen decomposition

12 Matrices in JAVA - e.g. JAMA Class EigenvalueDecomposition  Constructor EigenvalueDecomposition(Matrix Arg)  Methods Matrix GetV() Matrix GetD() Where A is the original matrix and:  AV=VD

13 Summary Data describing connections between objects can be described as a graph This graph can be represented as a matrix Interesting structure can be discovered in this data using Matrix Eigen-decomposition