1 Sample Geometry and Random Sampling Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and Multimedia

2 Array of Data * a sample of size n from a p -variate population

3 Row-Vector View

4 Example 3.1

5 Column-Vector View

6 Example 3.2

7 Geometrical Interpretation of Sample Mean and Deviation

8 Decomposition of Column Vectors

9 Example 3.3

10 Lengths and Angles of Deviation Vectors

11 Example 3.4

12 Random Matrix

13 Random Sample Row vectors X 1 ’, X 2 ’, …, X n ’ represent independent observations from a common joint distribution with density function f(x)=f(x 1, x 2, …, x p ) Mathematically, the joint density function of X 1 ’, X 2 ’, …, X n ’ is

14 Random Sample Measurements of a single trial, such as X j ’ = [X j1,X j2,…,X jp ], will usually be correlated The measurements from different trials must be independent The independence of measurements from trial to trial may not hold when the variables are likely to drift over time

15 Geometric Interpretation of Randomness Column vector Y k ’ =[X 1k,X 2k,…,X nk ] regarded as a point in n dimensions The location is determined by the joint probability distribution f(y k ) = f(x 1k, x 2k,…,x nk ) For a random sample, f(y k )=f k (x 1k )f k (x 2k )…f k (x nk ) Each coordinate x jk contributes equally to the location through the same marginal distribution f k (x jk )

16 Result 3.1

17 Proof of Result 3.1

18 Proof of Result 3.1

19 Proof of Result 3.1

20 Some Other Estimators

21 Generalized Sample Variance

22 Geometric Interpretation for Bivariate Case 

23 Generalized Sample Variance for Multivariate Cases

24 Interpretation in p-space Scatter Plot Equation for points within a constant distance c from the sample mean

25 Example 3.8: Scatter Plots

26 Example 3.8: Sample Mean and Variance-Covariance Matrices

27 Example 3.8: Eigenvalues and Eigenvectors

28 Example 3.8: Mean-Centered Ellipse

29 Example 3.8: Semi-major and Semi-minor Axes

30 Example 3.8: Scatter Plots with Major Axes

31 Result 3.2 The generalized variance is zero when the columns of the following matrix are linear dependent

32 Proof of Result 3.2

33 Proof of Result 3.2

34 Example 3.9

35 Example 3.9

36 Examples Cause Zero Generalized Variance Example 1 –Data are test scores –Included variables that are sum of others –e.g., algebra score and geometry score were combined to total math score –e.g., class midterm and final exam scores summed to give total points Example 2 –Total weight of chemicals was included along with that of each component

37 Example 3.10

38 Result 3.3 If the sample size is less than or equal to the number of variables ( ) then | S | = 0 for all samples

39 Proof of Result 3.3

40 Proof of Result 3.3

41 Result 3.4 Let the p by 1 vectors x 1, x 2, …, x n, where x j ’ is the j th row of the data matrix X, be realizations of the independent random vectors X 1, X 2, …, X n. If the linear combination a ’ X j has positive variance for each non-zero constant vector a, then, provided that p 0 If, with probability 1, a ’ X j is a constant c for all j, then | S | = 0

42 Proof of Part 2 of Result 3.4

43 Generalized Sample Variance of Standardized Variables

44 Volume Generated by Deviation Vectors of Standardized Variables

45 Example 3.11

46 Total Sample Variance

47 Sample Mean as Matrix Operation

48 Covariance as Matrix Operation

49 Covariance as Matrix Operation

50 Covariance as Matrix Operation

51 Sample Standard Deviation Matrix

52 Result 3.5

53 Proof of Result 3.5

54 Proof of Result 3.5

55 Result 3.6