Multivariate Statistical Analysis

Slides:



Advertisements
Similar presentations
Applied Informatics Štefan BEREŽNÝ
Advertisements

Matrix Algebra Matrix algebra is a means of expressing large numbers of calculations made upon ordered sets of numbers. Often referred to as Linear Algebra.
Matrix Algebra Matrix algebra is a means of expressing large numbers of calculations made upon ordered sets of numbers. Often referred to as Linear Algebra.
1.5 Elementary Matrices and a Method for Finding
Matrix Definition: An array of numbers in m rows and n colums is called an mxn matrix A square matrix of order n, is an (nxn) matrix.
Matrices and Systems of Equations
Chapter 2 Matrices Definition of a matrix.
1 Neural Nets Applications Vectors and Matrices. 2/27 Outline 1. Definition of Vectors 2. Operations on Vectors 3. Linear Dependence of Vectors 4. Definition.
Economics 2301 Matrices Lecture 13.
MOHAMMAD IMRAN DEPARTMENT OF APPLIED SCIENCES JAHANGIRABAD EDUCATIONAL GROUP OF INSTITUTES.
Boot Camp in Linear Algebra Joel Barajas Karla L Caballero University of California Silicon Valley Center October 8th, 2008.
Matrices and Determinants
3.8 Matrices.
A matrix having a single row is called a row matrix. e.g.,
Chapter 1: Matrices Definition 1: A matrix is a rectangular array of numbers arranged in horizontal rows and vertical columns. EXAMPLE:
1 Statistical Analysis Professor Lynne Stokes Department of Statistical Science Lecture 5QF Introduction to Vector and Matrix Operations Needed for the.
Compiled By Raj G. Tiwari
1 February 24 Matrices 3.2 Matrices; Row reduction Standard form of a set of linear equations: Chapter 3 Linear Algebra Matrix of coefficients: Augmented.
Presentation on Matrices and some special matrices In partial fulfillment of the subject Vector calculus and linear algebra ( ) Submitted by: Agarwal.
Matrix Algebra. Quick Review Quick Review Solutions.
Digital Image Processing, 3rd ed. © 1992–2008 R. C. Gonzalez & R. E. Woods Gonzalez & Woods Matrices and Vectors Objective.
A rectangular array of numbers (we will concentrate on real numbers). A nxm matrix has ‘n’ rows and ‘m’ columns What is a matrix? First column First row.
Statistics and Linear Algebra (the real thing). Vector A vector is a rectangular arrangement of number in several rows and one column. A vector is denoted.
8.1 Matrices & Systems of Equations
Matrices CHAPTER 8.1 ~ 8.8. Ch _2 Contents  8.1 Matrix Algebra 8.1 Matrix Algebra  8.2 Systems of Linear Algebra Equations 8.2 Systems of Linear.
Linear algebra: matrix Eigen-value Problems Eng. Hassan S. Migdadi Part 1.
Chapter 3 Determinants Linear Algebra. Ch03_2 3.1 Introduction to Determinants Definition The determinant of a 2  2 matrix A is denoted |A| and is given.
A Review of Some Fundamental Mathematical and Statistical Concepts UnB Mestrado em Ciências Contábeis Prof. Otávio Medeiros, MSc, PhD.
CALCULUS – III Matrix Operation by Dr. Eman Saad & Dr. Shorouk Ossama.
Review of Matrix Operations Vector: a sequence of elements (the order is important) e.g., x = (2, 1) denotes a vector length = sqrt(2*2+1*1) orientation.
Chapter 2 Determinants. With each square matrix it is possible to associate a real number called the determinant of the matrix. The value of this number.
Matrices and Determinants
MATRICES Operations with Matrices Properties of Matrix Operations
STROUD Worked examples and exercises are in the text Programme 5: Matrices MATRICES PROGRAMME 5.
STROUD Worked examples and exercises are in the text PROGRAMME 5 MATRICES.
Boot Camp in Linear Algebra TIM 209 Prof. Ram Akella.
Linear Algebra Engineering Mathematics-I. Linear Systems in Two Unknowns Engineering Mathematics-I.
1 Objective To provide background material in support of topics in Digital Image Processing that are based on matrices and/or vectors. Review Matrices.
Lecture XXVI.  The material for this lecture is found in James R. Schott Matrix Analysis for Statistics (New York: John Wiley & Sons, Inc. 1997).  A.
Matrices. Variety of engineering problems lead to the need to solve systems of linear equations matrixcolumn vectors.
MATRICES A rectangular arrangement of elements is called matrix. Types of matrices: Null matrix: A matrix whose all elements are zero is called a null.
Linear Algebra by Dr. Shorouk Ossama.
Matrices Introduction.
MTH108 Business Math I Lecture 20.
Mathematics-I J.Baskar Babujee Department of Mathematics
2. Matrix Methods
MAT 322: LINEAR ALGEBRA.
Matrices and Vector Concepts
3.8 Matrices L Al-zaid Math1101.
nhaa/imk/sem /eqt101/rk12/32
MATRICES.
Matrices.
Review of Matrix Operations
Eigenvalues and Eigenvectors
MATHEMATICS Matrix Algebra
L5 matrix.
Matrices and Vectors Review Objective
Systems of First Order Linear Equations
Sequences and Summations
2. Matrix Algebra 2.1 Matrix Operations.
CSE 541 – Numerical Methods
Chapter 3 Linear Algebra
Matrices Introduction.
CS100: Discrete structures
Matrices and Matrix Operations
Objective To provide background material in support of topics in Digital Image Processing that are based on matrices and/or vectors.
Systems of Linear Equations and Matrices
Matrices and Determinants
3.8 Matrices L Al-zaid Math1101.
Matrices and Determinants
Presentation transcript:

Multivariate Statistical Analysis Review of Matrix Algebra Jen-pei Liu, Ph.D., Professor Division of Biometry, Department of Agronomy National Taiwan University and Division of Biostatistics and Bioinformatics National Health Research Institutes 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Review of Matrix Dataset: Storm survival of sparrows 5 measurements of the first 4 birds Bird(Y) X1 X2 X3 X4 X5 1 156 245 31.6 18.5 20.5 2 154 240 30.4 17.9 19.6 3 153 240 31.0 18.4 20.6 4 153 236 30.9 17.7 20.2 X1: total length; x2: alar extent; x3: length of peak and head; x4: length of humerus; and x5; length of keel of sternum; X1 to X5 were in mm. 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra Scalar A simple real number Example, the total length of bird #1 is 156 mm Matrix An mxn matrix is an array of real numbers of m rows and n columns, considered as a single entity of the form 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra An mxn matrix contains m rows and n columns. aij is the entry at the ith row and jth column, i.e., (i,j)th position Matrices and vectors are represented by the bold face such as A, r , or c Example: Storm survival of sparrows Row represents the bird Column represents the 5 measurements Bird #3 and measurement #2 is 240 mm 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra Dataset: Storm survival of sparrows 5 measurements of the first 4 birds in a 4x5 matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Vectors A vector is a matrix with a single row or column The c-component column vector Similarly, the r-component row vector 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra A 4x5 matrix consists of 4 rows and 5 columns The first row contains the 5 measurements of the first bird It can represent as a row vector A 1x5 vector 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra The total length of the first 4 birds can be represented as a column vector A 4x1 column vector 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Definition of a matrix Among the variety of forms that can be found in the literature 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Definition of a matrix Dataset: Storm survival of sparrows 5 measurements of the first 4 birds in a 4x5 matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Square matrices When r=c, the number of rows equals the number of column, A is referred to as a square matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Triangular matrices A square matrix with all elements above (or below) the diagonal being zero is called a triangular matrix A is an upper triangular matrix B is a lower triangular matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Diagonal matrices A square matrix with off-diagonal entries equal to zero is called a diagonal matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Scalar matrices A scalar matrix is simply a diagonal matrix with all diagonal elements the same 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Identity matrices A diagonal matrix having all diagonal elements equal to unity is called an identity matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Zero matrices Zero matrix: a matrix with all entries equal to 0 2019/2/22 copyright by Jen-pei Liu, PhD

Equality of two matrices Two matrices are equal only if they are of the same size and all entries of all corresponding entries are equal aij = bij for all i and j, i = 1,…,m and j = 1,…,n 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Addition: The (i,j) entry of the sum of two matrices A and B is the sum of the (i,j) entry of A and the (i,j) entry of B (They must have the same size) 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Example: 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Substraction: The (i,j) entry of the difference of two matrices A and B is the difference between the (i,j) entry of A and the (i,j) entry of B (They must have the same size) 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Example: 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Theorem: A+A+A=3A A+(-A)=0 (zero matrix) A+B=B+A k(A+B)=kA+kB=(A+B)k (k is scalar) 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Multiplication: Multiplication of a scalar and a matrix 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Multiplication: Multiplication of two matrices A and B, denoted as AxB, is defined only if the number of the columns of A (mxc) is equal to the number of rows of B (cxn) The entry at the (i,k) position is equal to the sum of cross-products of the entries of the ith row of A and the entries of the jth column of B The resulting matrix, say C, has the size of mxn and cik = aijbjk 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Multiplication: Multiplication of two matrices A and B 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Example: 2019/2/22 copyright by Jen-pei Liu, PhD

Basic operation of matrices Theorem: A(B+C)=AB+AC (A+B)C=AC+BC A(BC)=(AB)C AB≠BA If AB=0 , It does not imply that A=0 or B=0 If AB=AC , It does not imply that B=C 2019/2/22 copyright by Jen-pei Liu, PhD

The transpose of a matrix Transpose: interchanging the rows and columns  a nxm matrix A transpose is denoted by A’ The transpose of a row vector is a column vector and the transpose of a column vector is a row vector 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Symmetric matrices A square matrix whose entries are symmetric about its diagonal, i.e., aij = aji. The transpose of a symmetric matrix is equal to itself 2019/2/22 copyright by Jen-pei Liu, PhD

The transpose of a matrix Example: 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Theorem: (A’)’=A (kA)’=kA’ (k is scalar) (A+B)’=A’+B’ (AB)’=B’A’ (ABC)’=C’B’A’ If A is a square matrix then A+A’=B is a symmetric matrix (A+A’)’=A’+(A’)’=A’+A If A is a symmetric matrix then A=A’ AA’ and A’A are the symmetric matrices 2019/2/22 copyright by Jen-pei Liu, PhD

The transpose of a matrix Example: (AB)’=B’A’ 2019/2/22 copyright by Jen-pei Liu, PhD

The transpose of a matrix Example: If A is a square matrix then A+A’=B is a symmetric matrix 2019/2/22 copyright by Jen-pei Liu, PhD

The transpose of a matrix Example: AA’ and A’A are symmetric matrices 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Idempotent matrix When A is such that A2=A , we say A is a idempotent matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD The trace of a matrix The sum of the diagonal elements of a square matrix is called the trace of the matrix, written tr(A); i.e., for A={aij} for i, j=1,…..,n Example 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Properties tr(kA)=ktr(A) (k is scalar) tr(A+B)=tr(A)+tr(B) tr(AB)=tr(BA) tr(x’x)=tr(xx’) (x is a vector) tr(A’A)=tr(AA’) tr(A’)=tr(A) If A is a symmetric matrix then x’Ax=tr(x’xA)=tr(Axx’) tr(ABC)=tr(BCA)=tr(CAB) If A is idempotent tr(A)=r(A) 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Orthogonal matrices When A is such that AA’=I=A’A , we say A is a orthogonal matrix For example: 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Orthogonal vectors The norm of a real vector X’ is defined as For example, the norm of x’=(1 2 2 4) is A vector is said to be either normal or a unit vector when its norm is unity; i.e. ,when x’x=1 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Normalized vectors Any non-null vector X can be changed into a unit vector by multiplying it by the scalar ;i.e., is the normalized form of X, because u’u=1 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Partitioned matrices Consider the matrix, suppose we draw dashed lines between certain rows and columns as in The matrices A11, A12, A21, A22 are said to be submatrices of A 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Quadratic Forms Quadratic Forms 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Quadratic Forms Quadratic Forms 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Bilinear Forms A slightly more general function is the second-degree function in two sets of variables x and y. For example: It is called a bilinear form, and the matrix A does not have to be square as does the matrix in a quadratic form 2019/2/22 copyright by Jen-pei Liu, PhD

Positive definite matrices When x’Ax >0 for all x other than x=0, then x’Ax is a positive definite quadratic form, and A=A’ a positive definite (p.d.) matrix For example 2019/2/22 copyright by Jen-pei Liu, PhD

Positive semidefinite matrices When x’Ax ≧0 for all x and x’Ax=0 for some x≠0, then x’Ax is a positive semidefinite quadratic form, and hence A=A’ is a positive semidefinite (p.s.d.) matrix 2019/2/22 copyright by Jen-pei Liu, PhD

Positive semidefinite matrices For example This is zero for x’=(2 1 3), and for any scalar multiple thereof, as well as for x=0 2019/2/22 copyright by Jen-pei Liu, PhD

Matrices with all elements equal Vectors whose every element is unity are called summing vectors For example: the row vector 1’=(1 1 1 1) is the summing vector and for x’=(3 6 8 -2) 2019/2/22 copyright by Jen-pei Liu, PhD

Matrices having all elements equal When necessary to avoid confusion the order of a summing vector can be denoted in the usual way: 1’4=(1 1 1 1) For example: 2019/2/22 copyright by Jen-pei Liu, PhD

Matrices having all elements equal The inner product of a summing vector with itself is scalar, the vector’s order: Outer products are matrices with all elements equal to unity: 2019/2/22 copyright by Jen-pei Liu, PhD

Matrices having all elements equal Products of J’s with each other and with 1’s are, respectively, J’s and 1’s (multiplied by scalars): Particularly useful are square J’s and their variants: 2019/2/22 copyright by Jen-pei Liu, PhD

Matrices having all elements equal For statistics, the centering matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Determinants The determinant of a square matrix of order n is referred to as an n-order determinant and the customary notation for the determinant of the matrix A is |A| or det(A) Second-order determinants: For example: 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Determinants Third-order determinants: 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Determinants For example: 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Determinants n-order determinants: Theorem: 1. |A|=|A’| 2. If two rows of A are the same, |A|=0 3. |A+B|≠|A|+|B| , |A-B|≠|A|-|B| 4. |AB|=|A||B|=|B||A| 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Determinants The determinants of orthogonal matrices are equal to +1 or -1 The determinants of idempotent matrices are equal to 1 or 0 2019/2/22 copyright by Jen-pei Liu, PhD

The inverse of a square Matrix If the inverse of a square matrix exists, the multiplication of the square matrix and its inverse will be equal to the identity matrix The inverse of a square matrix A is denoted as A-1 AA-1 = I A-1 A = I 2019/2/22 copyright by Jen-pei Liu, PhD

The inverse of a square Matrix Example: 2019/2/22 copyright by Jen-pei Liu, PhD

The inverse of a square Matrix 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Properties When A-1A=AA-1= I , A-1 unique for given A |A-1| =1/|A|=|A|-1 (A-1)-1=A (A’)-1= (A-1)’ If A is a symmetric matrix then (A-1)’= A-1 (AB)-1= B-1A-1 If A is a orthogonal matrix then A-1= A’ 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Properties 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Properties 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Properties 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD The rank of a Matrix Equations Ax=y can be solved as x=A-1y only the inverse A-1 exists, and A-1 does exist only if |A| is nonzero The rank of a matrix is the number of linearly independent rows (and columns) in the matrix The rank of A will be denoted equivalently by rA or r(A) Whether |A| is zero or not for determining the existence of A-1 can be replaced by ascertaining whether r(A)<n or r(A)=n 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Theorem: r(AB) ≦ min [r(A), r(B)] r(A+B) ≦ r(A)+r(B) r(Am×n) ≦ min (m, n) r(PAQ) ≦ r(A) if |P|≠0 , |Q|≠0 r(A) =r(A’) 2019/2/22 copyright by Jen-pei Liu, PhD

Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors (Characteristic roots and Characteristic Vectors) 2019/2/22 copyright by Jen-pei Liu, PhD

Eigenvalues and Eigenvectors 2019/2/22 copyright by Jen-pei Liu, PhD

Eigenvalues and Eigenvectors Example: 2019/2/22 copyright by Jen-pei Liu, PhD

Eigenvalues and Eigenvectors 2019/2/22 copyright by Jen-pei Liu, PhD

Eigenvalues and Eigenvectors A matrix of size nxn has n eigenvalues The eigenvectors are not unique and different algorithms may provide different eigenvectors The sum of the diagonal elements of a symmetric matrix is called the trace of that matrix The sum of the eigenvalues of a symmetric matrix is also equal to the trace of that matrix 2019/2/22 copyright by Jen-pei Liu, PhD

Eigenvalues and Eigenvectors The eigenvalues of a real symmetric matrix are real. For each eigenvalue of a real symmetric matrix, there exists a real eigenvectors Let A be a nxn symmetric matrix, there exists an orthogonal matrix P such that P’AP = D, where D is a diagonal matrix with eigenvalues of A on its diagonal 2019/2/22 copyright by Jen-pei Liu, PhD

Eigenvalues and Eigenvectors 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Aglebra Vectors of Means and Covariance Matrices 2019/2/22 copyright by Jen-pei Liu, PhD

Derivative of Quadratic Forms 2019/2/22 copyright by Jen-pei Liu, PhD

Derivative of Quadratic Forms 2019/2/22 copyright by Jen-pei Liu, PhD

Derivative of Quadratic Forms 2019/2/22 copyright by Jen-pei Liu, PhD

Derivative of Quadratic Forms 2019/2/22 copyright by Jen-pei Liu, PhD

Derivative of Quadratic Forms 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra Vectors of Means and Covariance Matrices 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra Vectors of Means and Covariance Matrices Consider p variables X1, X2, …, Xp in n subjects That is each subject has n measurements. These np measurements can be arranged in a data matrix of size of nxp Each row represents the measurements of p variables of a subject Each column represents n measurements of variable Xj 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra Dataset: Storm survival of sparrows 5 measurements of the first 4 birds in a 4x5 matrix 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra Data Matrix 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

Review of Matrix Algebra 2019/2/22 copyright by Jen-pei Liu, PhD

copyright by Jen-pei Liu, PhD Summary Definitions of vectors and matrices Symmetric matrices, diagonal matrices, identity matrix Addition, subtraction, multiplication, inverse Eigenvalues and eigenvectors Mean vectors and covariance matrix 2019/2/22 copyright by Jen-pei Liu, PhD