Download presentation
Presentation is loading. Please wait.
1
III. Multi-Dimensional Random Variables and Application in Vector Quantization
2
© Tallal Elshabrawy 2 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 u 1, u 2 are orthogonal to each other For distinct eigen vectors, values
![© Tallal Elshabrawy 2 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 u 1, u 2 are orthogonal to each other For distinct eigen vectors, values](http://images.slideplayer.com./27/8963680/slides/slide_2.jpg "© Tallal Elshabrawy 2 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 u 1, u 2 are orthogonal to each other For distinct eigen vectors, values")
3
© Tallal Elshabrawy 3 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Representation of X over u 1, u 2 Projection of X over u 1 unit vector u 1 Projection of X over u 2 unit vector u 2
![© Tallal Elshabrawy 3 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Representation of X over u 1, u 2 Projection of X over u 1 unit vector u 1 Projection of X over u 2 unit vector u 2](http://images.slideplayer.com./27/8963680/slides/slide_3.jpg "© Tallal Elshabrawy 3 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Representation of X over u 1, u 2 Projection of X over u 1 unit vector u 1 Projection of X over u 2 unit vector u 2")
4
© Tallal Elshabrawy 4 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X
![© Tallal Elshabrawy 4 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X](http://images.slideplayer.com./27/8963680/slides/slide_4.jpg "© Tallal Elshabrawy 4 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X")
5
© Tallal Elshabrawy 5 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X
![© Tallal Elshabrawy 5 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X](http://images.slideplayer.com./27/8963680/slides/slide_5.jpg "© Tallal Elshabrawy 5 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X")
6
© Tallal Elshabrawy 6 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X
![© Tallal Elshabrawy 6 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X](http://images.slideplayer.com./27/8963680/slides/slide_6.jpg "© Tallal Elshabrawy 6 Karhunen-Loeve Decomposition v1v1 u2u2 X=[X 1, X 2 ] X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let u 1, u 2 (unit vectors) be a new co-ordinate system such that u 1, u 2 are the eigen vectors of the covariance matrix R X u1u1 v2v2 Diagonalization of R X")
7
© Tallal Elshabrawy 7 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2
![© Tallal Elshabrawy 7 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2](http://images.slideplayer.com./27/8963680/slides/slide_7.jpg "© Tallal Elshabrawy 7 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2")
8
© Tallal Elshabrawy 8 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Define E[X 1 ] E[X 2 ] E[Y 1 ] E[Y 2 ]
![© Tallal Elshabrawy 8 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Define E[X 1 ] E[X 2 ] E[Y 1 ] E[Y 2 ]](http://images.slideplayer.com./27/8963680/slides/slide_8.jpg "© Tallal Elshabrawy 8 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Define E[X 1 ] E[X 2 ] E[Y 1 ] E[Y 2 ]")
9
© Tallal Elshabrawy 9 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R X
![© Tallal Elshabrawy 9 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R X](http://images.slideplayer.com./27/8963680/slides/slide_9.jpg "© Tallal Elshabrawy 9 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R X")
10
© Tallal Elshabrawy 10 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R Y
![© Tallal Elshabrawy 10 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R Y](http://images.slideplayer.com./27/8963680/slides/slide_10.jpg "© Tallal Elshabrawy 10 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R Y")
11
© Tallal Elshabrawy 11 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R Y
![© Tallal Elshabrawy 11 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R Y](http://images.slideplayer.com./27/8963680/slides/slide_11.jpg "© Tallal Elshabrawy 11 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Covariance Matrix R Y")
12
© Tallal Elshabrawy 12 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Y 1 and Y 2 are UNCORRELATED Covariance Matrix R Y
![© Tallal Elshabrawy 12 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Y 1 and Y 2 are UNCORRELATED Covariance Matrix R Y](http://images.slideplayer.com./27/8963680/slides/slide_12.jpg "© Tallal Elshabrawy 12 K-L Transformation of 2-D Random Variable v1v1 u2u2 X X = [X 1, X 2 ] is a sample vector from a two-dimensional random variable defined over v 1, v 2 (unit vectors) co-ordinate system Let Y = [Y 1, Y 2 ] be the transformation of the random variable X over the co- ordinate system u 1, u 2 u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 Y 1 and Y 2 are UNCORRELATED Covariance Matrix R Y")
13
© Tallal Elshabrawy 13 K-L Transformation of 2-D Random Variable Therefore using principle component analysis, it is possible to transform a random variable X with components X 1 and X 2 that are correlated into another random variable Y whose components Y 1 and Y 2 are uncorrelated v1v1 u2u2 X u1u1 v2v2 X1X1 X2X2 Y Y1Y1 Y2Y2 u 1, u 2 are eigen vectors of R X λ 1, λ 2 are eigen values of R X
14
© Tallal Elshabrawy 14 Two-Dimensional Gaussian Distribution In the previous slides, we have talked about the transformed random variable Y whose components are uncorrelated and have mean m Y and covariance matrix R Y. What would be the distribution of Y. Well this depends on what is the distribution of X Generally X and Y do not follow a Gaussian distribution. However, if X is a two-dimensional Gaussian distribution then Y as well would be a two-dimensional Gaussian distribution
15
© Tallal Elshabrawy 15 Two-Dimensional Gaussian Distribution Suppose we have a two-dimensional random variable Y whose components Y 1 and Y 2 are independent and Gaussian
16
© Tallal Elshabrawy 16 Two-Dimensional Gaussian Distribution Suppose we have a two-dimensional random variable Y whose components Y 1 and Y 2 are independent and Gaussian
17
© Tallal Elshabrawy 17 Two-Dimensional Gaussian Distribution Suppose we have a two-dimensional random variable Y whose components Y 1 and Y 2 are independent and Gaussian
18
© Tallal Elshabrawy 18 Two-Dimensional Gaussian Distribution Suppose we have a two-dimensional random variable Y whose components Y 1 and Y 2 are independent and Gaussian This formula is valid for any multi- dimensional Gaussian random variable whether its components are correlated or not
19
© Tallal Elshabrawy 19 Two-Dimensional Gaussian Distribution
20
© Tallal Elshabrawy 20 Two-Dimensional Gaussian Distribution
21
© Tallal Elshabrawy 21 Two-Dimensional Gaussian Distribution
22
© Tallal Elshabrawy 22 Two-Dimensional Gaussian Distribution
23
© Tallal Elshabrawy 23 Two-Dimensional Gaussian Distribution
24
© Tallal Elshabrawy 24 Two-Dimensional Gaussian Distribution
25
© Tallal Elshabrawy 25 Two-Dimensional Gaussian Distribution
26
© Tallal Elshabrawy 26 Two-Dimensional Gaussian Distribution
27
© Tallal Elshabrawy 27 Two-Dimensional Gaussian Distribution
28
© Tallal Elshabrawy 28 Two-Dimensional Gaussian Distribution
29
© Tallal Elshabrawy 29 Two-Dimensional Gaussian Distribution
30
© Tallal Elshabrawy 30 Two-Dimensional Gaussian Distribution More Energy Less Energy Y 1 and Y 2 Correlated
31
© Tallal Elshabrawy 31 Two-Dimensional Gaussian Distribution Equal Energy By rotating the axis, the resultant transformed random variables are uncorrelated
Similar presentations
