Statistical Data Analysis Daniel Svozil Laboratory of Informatics and Chemistry Faculty of Chemical Technology

What is it about? Though the name does not suggest, the lecture consists of two parts 1.numerical linear algebra vector and matrix multiplication, matrix inverse, vector spaces, least squares, eigenproblem, matrix decompositions, climax: Singular Value Decomposition 2.statistics probability (including Bayes), confidence intervals, linear regression, logistic regression, ANOVA, non-parametric statistics, climax: PCA

Exercises The subject will be more like 3/1. Numerical linear algebra –pen and paper Statistics –Excel

Organizační věci zápočet: písemka uprostřed semestru, cokoliv lepšího než F znamená zápočet zkouška: písemná zkouška ve zkouškovém období, známky zapisuji do SISu, ukázkové příklady na výše uvedené adrese - 10 otázek, u každé otázky více odpovědí, zaškrtnout správné (min 1 je správná)

The Geometry of Linear Equations Daniel Svozil based on excelent video lectures by Gilbert Strang, MIT

Vectors

Fundamental problem of linear algebra solve a system of linear equations start with a (normal and nice) case of equal number of equations and unknowns (n equations, n unknowns) two pictures will be presented: –row picture (one equation on a line) –column picture (that’s the important one, and probably new to you) matrix form (the algebra way of looking at the problem using matrix A)

2 equations, 2 unknowns 2x – y = 0 -x + 2y = 3

Row picture I take one row at a time 2x – y = 0 x y -x + 2y = 3 x = 1 y = 2 2x – y = 0 -x + 2y = 3

Column picture 2x – y = 0 -x + 2y = 3 this equation is asking us to combine 1 st and 2 nd vectors in right amounts to get RHS vector it’s asking us to find a right LINEAR COMBINATION

x y col 1 col 2 Let’s take some linear combination. And why not the correct one to produce RHS so let x = 1 and y = 2? 2 x col 2 b

The idea of linear combination is crucial. Let’s think about this question: What are all the combinations? i. e. what will be the results if I take any x and any y? Any idea? All possible linear combinations will give all possible RHS, so they will fill the whole plane.

3 equations, 3 unknowns 2x – y = 0 -x + 2y – z = -1 -3y + 4z = 4

Row picture Help me, please! Take the second equation. 2x – y = 0 -x + 2y – z = -1 -3y + 4z = 4 However, as you can see, row picture is getting a little hard to see.

Column picture 2x – y = 0 -x + 2y – z = -1 -3y + 4z = 4

2x – y = 0 -x + 2y – z = -1 -3y + 4z = 4

9-D space

Matrix by a vector multiplication

Matrix Multiplication Daniel Svozil based on excelent video lectures by Gilbert Strang, MIT

Rules for matrix multiplication Several ways how to do it, all give the same answer, and they’re all important.

1 st way row 2 column 3 c 23 c ij

2 nd way – column way A m x n B n x p C m x p col1 col2 Matrix multiplication works so that I can just think of having several columns, multiplying by A and getting the columns of the answer. So, what can we say about columns of C? Just a reminder, each column of C is matrix A times column of B. Columns of C are linear combinations of columns of what? of A

3 rd way – row way A m x n B n x p C m x p Rows of C are linear combinations of rows of what? of B

Can you somehow combine 2 nd and 3 rd ways to make some conclusion? C T = (AB) T = B T A T

4 th way So what is left? Column times row What is the result of column times row?a matrix a big, full-size matrix What’s the answer? Just follow rules of the 1 st way. What’ the first row of the resulting matrix? 2 12 And next? This is very special matrix. What are its columns? Multiples of column vector. And what are the rows? Multiples of row vector. What does it mean geometrically, that all rows are multiples of the [1 6]? They all lie on the same line. And the same is true for cols.

Let me write down a 4 th way: AB is a sum of (cols of A x rows of B) 4 th way What’s the answer for that matrix multiplication? outer product outer product expansion

Matrix Inverse Daniel Svozil based on excelent video lectures by Gilbert Strang, MIT

Analogy Consider the equation ax = b, where a, x, b are real numbers. 2x = 4 We want to find x. What is the solution? x = b/a = ba -1. What are the conditions? a ≠ 0

Imitate this procedure for Ax = b. We need to find a matrix A -1 such that AA -1 = I I is the identity matrix, a matrix equivalent of number 1. However, we will also require A -1 A = I. This requirement forces A and A -1 to have which shape? Square.

Square matrices Square matrix A may or may not have inverse, not all matrices have inverses! So the question about matrix A is if it is invertible or not? If it’s invertible, then there exists another matrix A -1 (A inverse). Synonyma for invertible is nonsingular.

If A -1 exists, then A -1 A = I = AA -1 How to identify, if A has inverse? Let’s talk about singular case (no inverse).

Singular matrix Example Why is this noninvertible? det(A) is zero. What is det? We don’t know yet! OK, another explanation. So, we’re looking for a matrix K, for which AK = I. 1 st column of I is a linear combination of columns of A. Can you write down the linear combination?

What is the solution of this system of equations? Write down the eqs. and try to solve. Geometrically, columns of A lie on the same line, so their linear combination MUST lie also on that line. But [1 2] T does not line on the [1 0] T line! This is logical result if you think geometrically. Try to do it !

Another reason Square matrix A does not have an inverse, if you can find an vector x such that Ax = 0. OK, tell me the x for our case. So, why A is not invertible if Ax = 0? Left multiply by A -1. A -1 Ax = Ix = 0 means, that x is also 0, but we found non-zero x.

Conclusion Singular matrices take some vector x and turn them into zero.

How to find an inverse? First column of I is a linear combination of columns of A with coefficients a and b. Second column of I is a linear combination of columns of A with coefficients c and d. So we need to solve two Ax = b systems. Gauss-Jordan elimination solve several equations at once (this is a difference compared to Gauss elimination)

Some rules