NOTES ON MULTIPLE REGRESSION USING MATRICES  Multiple Regression Tony E. Smith ESE 502: Spatial Data Analysis  Matrix Formulation of Regression  Applications to Regression Analysis

SIMPLE LINEAR MODEL  Data:  Parameters:  Model:

SIMPLE REGRESSION ESTIMATION  Data Points:  Predicted Value:  Estimate Conditional Mean:    Line of Best Fit where:

STANDARD LINEAR MODEL  Data:  Parameters:  Model:

STANDARD LINEAR MODEL (k = 2)  Data:  Parameters:  Model:

REGRESSION ESTIMATION (for k =2) Plane of Best Fit     Data Points:  Predicted Value: where:

MATRIX REPRESENTATION OF THE STANDARD LINEAR MODEL  Vectors and Matrices:  Matrix Reformulation of the Model:

LINEAR TRANSFORMATIONS IN ONE DIMENSION  Linear Function:  Graphic Depiction:   

LINEAR TRANSFORMATIONS IN TWO DIMENSIONS  Linear Transformation:

 Graphical Depiction of Linear Transformation:        

SOME MATRIX CONVENTIONS  Transposes of Vectors and Matrices:  Symmetric (Square) Matrices:  Important Example:

 Column Representation of Matrices:  Row Representation of Matrices:

 Matrix Multiplication:  Inner Product of Vectors:  Transposes:

MATRIX REPRESENTATIONS OF LINEAR TRANSFORMATIONS  For any Two-Dimensional Linear Transformation : with :

 Graphical Depiction of Matrix Representation:        

 Inversion of Square Matrices (as Linear Transformations):

DETERMINANTS OF SQUARE MATRICES      

NONSINGULAR SQUARE MATRICES      

LEAST-SQUARES ESTIMATION  General Sum-of-Squares:  General Regression Matrices:

DIFFERENTIATION OF FUNCTIONS  General Derivative:  Example:   

PARTIAL DERIVATIVES  

VECTOR DERIVATIVES  Derivative Notation for:  Gradient Vector:

TWO IMPORTANT EXAMPLES  Linear Functions:  Quadratic Functions:

 Quadratic Derivatives:  Symmetric Case:

MINIMIZATION OF FUNCTIONS  First-Order Condition:  Example:  

   TWO-DIMENSIONAL MINIMIZATION

LEAST SQUARES ESTIMATION  Solution for:

NON-MATRIX VERSION (k = 2)  Data:  Beta Estimates:

EXPECTED VALUES OF RANDOM MATRICES  Random Vectors and Matrices  Expected Values:

EXPECTATIONS OF LINEAR FUNCTIONS OF RANDOM VECTORS  Linear Combinations  Linear Transformations

EXPECTATIONS OF LINEAR FUNCTIONS OF RANDOM MATRICES  Left Multiplication  Right Multiplication (by symmetry of inner products):

COVARIANCE OF RANDOM VECTORS  Random Variables :  Random Vectors:

COVARIANCE OF LINEAR FUNCTIONS OF RANDOM VECTORS  Linear Combinations:  Linear Transformations: ( Right Mult ) ( Left Mult )

TRANSLATIONS OF RANDOM VECTORS  Translation:  Means:  Covariances:

RESIDUAL VECTOR IN THE STANDARD LINEAR MODEL  Linear Model Assumption:  Residual Means:  Residual Covariances:

MOMENTS OF BETA ESTIMATES  Linear Model:  Mean of Beta Estimates: (Unbiased Estimator)  Covariance of Beta Estimates:

ESTIMATION OF RESIDUAL VARIANCE  Residual Variance :  Residual Estimates :  Natural Estimate of Variance :  Bias-Correct Estimate of Variance : (Compensates for Least Squares)

ESTIMATION OF BETA COVARIANCE  Beta Covariance Matrix:  Beta Covariance Estimates: