G-inverse and Solution of System of Equations and their Applications in Statistics Professor Dr. S. K. Bhattacharjee Department of Statistics University of Rajshahi, Bangladesh

System of Linear Equations

Linear Combination of Vectors

Vector Form

Solution of Linear System A solution of a linear system is an assignment of values to the variables x1, x2, ..., xn such that each of the equations is satisfied. The set of all possible solutions is called the solution set. A linear system may behave in any one of three possible ways: The system has infinitely many solutions. The system has a single unique solution. The system has no solution.

Geometric Interpretation For a system involving two variables (x and y), each linear equation determines a line on the xy-plane. Because a solution to a linear system must satisfy all of the equations, the solution set is the intersection of these lines, and is hence either a line, a single point, or the empty set. For three variables, each linear equation determines a plane in three-dimensional space, and the solution set is the intersection of these planes. Thus the solution set may be a plane, a line, a single point, or the empty set. For n variables, each linear equations determines a hyperplane in n-dimensional space. The solution set is the intersection of these hyperplanes, which may be a flat of any dimension

Solution of Two Vectors

Solution of Three Equations

Solution of Three Equations

Solution of Three Equations

Solution of Three Equations

Solution of Two Equations in Three Variables

Solution of Linear System In general, the behavior of a linear system is determined by the relationship between the number of equations and the number of unknowns: Usually, a system with fewer equations than unknowns has infinitely many solutions. Such a system is also known as an underdetermined system. Usually, a system with the same number of equations and unknowns has a single unique solution. Usually, a system with more equations than unknowns has no solution. Such a system is also known as an overdetermined system.

Examples The system has exactly 1 solution. Systems have 1 and only 1 solution when the two lines have different slope. Think about it, if the two lines have different slopes then eventually at some point they must meet. After all the lines are not parallel. system has no solutions Systems have no solution when the lines are parallel (ie have the same slope) and the lines have different y-intercepts. As an example look at the following two lines Line 1: y = 5x +13 Line 2: y = 5x + 12 The system has infinite solutions Systems have infinite solutions when the lines are parallel and the lines have the same y-intercept. If two lines have the same slope (ie are parallel) and the same y-intercept, they are actually the same exact line. In other words, systems have infinite solutions when the two lines are the same line! As an example consider the following two lines Line 1: y = x +3 Line 2: 2y = 2x +6 These two lines are exactly the same line. If you multiply line 1 by two you get line 2.

Solution of Linear System

Two Variables Three Eq.

Example The equations 3x + 2y = 6 and 3x + 2y = 12 are inconsistent.

Methods of Solution The Methods of finding the solution to systems of linear equations: graph : by looking at where lines intersect (meet) on a graph algebraic equation : by setting the equations of the system equal to each other then solving this equation. substitution : by solving for one of the variables and substituting its value in to the other equation. Elimination : Elimination involves algebraic manipulations of two or more equations. The end goal is to eliminate a variable by creating opposite coefficients (The examples below should clarify this straightforward approach).

Graphical Method

Algebraic Equation Method

Substitution Method The Substitution Method   The substitution method involves algebraic substitution of one equation into a variable of the other. A quick refresher on algebraic substitution: Refresher:Substitution Equation 1 : x = 5 Equation 2: y = x +2 How to Substitute 1) Use equation 1( x= 5) to substitute 5 for x in second equation y = (5) + 2 2) So love for Y y = 5+ 2 = 7

Example Substitution Example Two Line 1 : y=2x+1 Line 2 : 2y=3x-2 Step 1: Substitute one equation into the other 2(2x+1)=3x-2 Step 2: Now that you have a single variable equation, solve for that variable's equation 4x+2 = 3x-2 x+2= -2 x= –4 Step 3 : Once you have solved for the one variable insert that variable back into either equation to obtain the value of y at the solution. Insert x= –4 to find y value y = 2(–4)+1= –7 This example's solution is ( –4, –7).

Elimination Method Elimination method is an algebraic method for solving systems. To use elimination you perform an operation on 1 equation then add the two equations so that one of the variables cancels. Example of Elimination Line 1: y = x + 1 Line 2: y = –x

Elimination Method

Matrix Methods of Solution

Inverse Method The Matrix form of a linear system of equations is Ax= b If the coefficient matrix is non-singular then the solution is: x = A-1b

Elementary Row Transformation Solving a system of linear equations by reducing the augmented matrix of the system to row canonical form

Example

Example

Homogeneous System

Homogeneous System Homogeneous Systems are always Consistent Every homogeneous system has at least one solution, known as the zero solution (or trivial solution), which is obtained by assigning the value of zero to each of the variables. The solution set has the following additional properties: If u and v are two vectors representing solutions to a homogeneous system, then the vector sum u + v is also a solution to the system. If u is a vector representing a solution to a homogeneous system, and r is any scalar, then ru is also a solution to the system.

Homogeneous System Suppose that a homogeneous system of linear equations has m equations and n variables with n>m. Then the system has infinitely many solutions.

Example

Example

Example

Generalized Inverse

Generalized Inverse

Generalized Inverse

Method of Obtaining g-Inverse

Example

Method of Obtaining g- Inverse

Example

Properties of g Inverse of X’X The matrix X’X has an important role in statistics where it arises in least square equations X’Xb = X’y. When G is a generalized inverse of X’X: G’ is also a generalized inverse of X’X XGX’X=X, i.e., GX’ is a generalized inverse of X XGX’ is invariant to G XGX’ is symmetric whether G is or not. Let X+ be the Moore – Penrose inverse of X. Then XX+ = XGX’ but X+ may not be equal to GX’.

g-Inverse

g-Inverse

Properties Every singular symmetric matrix (of order two or more) has both symmetric and non-symmetric generalized inverses. Let A represent a matrix of full column rank and B a matrix of full row rank. Then, (1) a matrix G is a generalized inverse of A if and only if G is a left inverse of A. And, (2) a matrix G is a generalized inverse of B if and only if G is a right inverse of B.

Properties For any matrix A and any nonzero scalar k, (1/k)A− is a generalized inverse of kA. For any matrix A,−A− is a generalized inverse of −A. For any matrix A, (A−)’ is a generalized inverse of A’ . For any symmetric matrix A, (A−)’ is a generalized inverse of A’ .

Properties A linear system Ax = b is consistent if and only if AA- b = b or, equivalently, if and only if (I − AA-)B = 0

The Moore-Penrose Inverse Given any matrix A, there is a unique matrix M such that (i) AMA=A (ii) MAM=M (iii) AM is symmetric (iv) MA is symmetric Then matrix M is called the Moore-Penrose inverse of A. M = L’(K’AL’)-1 K’

The Moore-Penrose Inverse

Example

Example