1. NOTES ON MULTIPLE REGRESSION USING MATRICES  Multiple Regression Tony E. Smith ESE 502: Spatial Data Analysis  Matrix Formulation of Regression  Applications to Regression Analysis

2. SIMPLE LINEAR MODEL  Data:  Parameters:  Model:

3. SIMPLE REGRESSION ESTIMATION  Data Points:  Predicted Value:  Estimate Conditional Mean:    Line of Best Fit where:

4. STANDARD LINEAR MODEL  Data:  Parameters:  Model:

5. STANDARD LINEAR MODEL (k = 2)  Data:  Parameters:  Model:

6. REGRESSION ESTIMATION (for k =2) Plane of Best Fit     Data Points:  Predicted Value: where:

7. MATRIX REPRESENTATION OF THE STANDARD LINEAR MODEL  Vectors and Matrices:  Matrix Reformulation of the Model:

8. LINEAR TRANSFORMATIONS IN ONE DIMENSION  Linear Function:  Graphic Depiction:   

9. LINEAR TRANSFORMATIONS IN TWO DIMENSIONS  Linear Transformation:

10.  Graphical Depiction of Linear Transformation:        

11. SOME MATRIX CONVENTIONS  Transposes of Vectors and Matrices:  Symmetric (Square) Matrices:  Important Example:

12.  Column Representation of Matrices:  Row Representation of Matrices:

13.  Matrix Multiplication:  Inner Product of Vectors:  Transposes:

14. MATRIX REPRESENTATIONS OF LINEAR TRANSFORMATIONS  For any Two-Dimensional Linear Transformation : with :

15.  Graphical Depiction of Matrix Representation:        

16.  Inversion of Square Matrices (as Linear Transformations):

17. DETERMINANTS OF SQUARE MATRICES      

18. NONSINGULAR SQUARE MATRICES      

19. LEAST-SQUARES ESTIMATION  General Sum-of-Squares:  General Regression Matrices:

20. DIFFERENTIATION OF FUNCTIONS  General Derivative:  Example:   

21. PARTIAL DERIVATIVES  

22. VECTOR DERIVATIVES  Derivative Notation for:  Gradient Vector:

23. TWO IMPORTANT EXAMPLES  Linear Functions:  Quadratic Functions:

24.  Quadratic Derivatives:  Symmetric Case:

25. MINIMIZATION OF FUNCTIONS  First-Order Condition:  Example:  

26.    TWO-DIMENSIONAL MINIMIZATION

27. LEAST SQUARES ESTIMATION  Solution for:

28. NON-MATRIX VERSION (k = 2)  Data:  Beta Estimates:

29. EXPECTED VALUES OF RANDOM MATRICES  Random Vectors and Matrices  Expected Values:

30. EXPECTATIONS OF LINEAR FUNCTIONS OF RANDOM VECTORS  Linear Combinations  Linear Transformations

31. EXPECTATIONS OF LINEAR FUNCTIONS OF RANDOM MATRICES  Left Multiplication  Right Multiplication (by symmetry of inner products):

32. COVARIANCE OF RANDOM VECTORS  Random Variables :  Random Vectors:

33. COVARIANCE OF LINEAR FUNCTIONS OF RANDOM VECTORS  Linear Combinations:  Linear Transformations: ( Right Mult ) ( Left Mult )

34. TRANSLATIONS OF RANDOM VECTORS  Translation:  Means:  Covariances:

35. RESIDUAL VECTOR IN THE STANDARD LINEAR MODEL  Linear Model Assumption:  Residual Means:  Residual Covariances:

36. MOMENTS OF BETA ESTIMATES  Linear Model:  Mean of Beta Estimates: (Unbiased Estimator)  Covariance of Beta Estimates:

37. ESTIMATION OF RESIDUAL VARIANCE  Residual Variance :  Residual Estimates :  Natural Estimate of Variance :  Bias-Correct Estimate of Variance : (Compensates for Least Squares)

38. ESTIMATION OF BETA COVARIANCE  Beta Covariance Matrix:  Beta Covariance Estimates: