Linear Algebra Textbook: J. Hefferon, “Linear Algebra” Reference: T.M.Apostol, “Linear Algebra” Website: http://ckw.phys.ncku.edu.tw Homework submission: class@ckw.phys.ncku.edu.tw

Preface Evolution of Linear Algebra: Linear simultaneous algebraic equations. Matrix formulation → matrix algebra. Vector space. Linear system of differential equations → Inner product space (Hilbert space). Applications of Linear Algebra: Minimax (optimization) problems. Linear operators (eigen-problems). Linear approximations to non-linear problems.

Computerization Procedural programming ( FORTRAN, C, … ) Routine libraries: EISPACK, LINPACK, IMSL, … Symbolic programming Mathematica, Maple, Reduce, … Dedicated packages: MATLAB, …

Being Computer Aided Leave drudgery to the C. Post-C learning process: Understand the subject: Reasoning. Applicability. Limitations / Error check. Comparison / relation with other subjects Apply by hand concept to simple cases. C. Make knowledge C. searchable.

Goals of this Course Concepts: Linear Systems of Equations Linear (Vector) Spaces Linear Transformations / Operators Matrix Algebra & Determinants Inner Product (Hilbert) Spaces Techniques: Mathematical Proofs Computational Skills

Hefferon Linear Systems Vector Spaces Maps Between Spaces Determinants Similarity

Chapter One: Linear Systems I. Solving Linear Systems II. Linear Geometry of n-Space III. Reduced Echelon Form Topic: Computer Algebra Systems Topic: Input-Output Analysis Topic: Accuracy of Computations Topic: Analyzing Networks

I. Solving Linear Systems I.1. Gauss’ Method I.2. Describing the Solution Set I.3. General = Particular + Homogeneous

Example of a Linear System: Balances Find the unknown masses h & c balanced as follows: → Example of a Linear System: Chemical Reaction Toluene + Nitric acid → Trinitrotoluene (TNT) + Water 甲苯 + 硝酸 → 三硝基甲苯 (黄色炸藥) + 水

Definition: An ordered set of n real numbers is called an n-tuple. Definition: The set of all n-tuples is called Rn. Definition 1.1a: A linear equation in variables x1, x2, …, xn has the form (1) where a1, a2, …, an R are the equation’s coefficients, and dR its constant. Definition 1.1b: An n-tuple ( s1, s2, …, sn ) R is a solution of, or satisfies, eq(1) if

Definition 1.1c: ( s1, s2, …, sn ) is a solution of the system of m linear equations (2) if it is a solution of all m equations in eq(2). Example 1.2: (1, 5 ) is a solution of since (5, 1) is not a solution because

I.1. Gauss’ Method Gauss’method  Systematic elimination of variables. Example 1.3:   Forward elimination ( upper triangularization ) : row 1  row 3 : 3 row 1 :  row 1 + row 2: Backward substitution (diagonalization) with ρi = row i : 1/3ρ3 ; ρ2 : 2ρ3 + ρ2 : 6ρ2 +ρ1 :

Theorem 1.4 (Gauss’ Method) : A linear system is invariant under the following operations : ρi  ρj , where i  j. ρi → c ρi , where c  0. ρi → ρi + c ρj , where i  j and c  0. Definition 1.5 (Elementary reduction / row / Gaussian operations): Swapping. Multiplying by a scalar. Rescaling / pivoting.

Example 1.6:

Leading variable = 1st variable with non-zero coefficient in a row. Definition 1.9: Leading variable = 1st variable with non-zero coefficient in a row. A system is in echelon form if each leading variable is to the right of the one above it. A linear system needs not have the same number of equations as unknowns.  Example 1.6:  

Example 1.12 ( System with no solution ) : A linear system can have exactly one, none, or infinitely many solutions. Example 1.12 ( System with no solution ) :   Inconsistent. Example 1.14 ( System with infinitely many solutions ) :  

Unique solution: (Triangular)  Unique solution: (Triangular)  No solution: (Echelon)  Many solution: (Echelon)

Exercise 1: Four positive integers are given. Select any three of the integers, find their arithmetic average, and add this result to the fourth integer. Thus the numbers 29, 23, 21, and 17 are obtained. One of the original integers is: (a) 19 (b) 21 (c) 23 (d) 29 (e) 17 Exercise 2: Laugh at this: AHAHA + TEHE = TEHAW. It resulted from substituting a code letter for each digit of a simple example in addition, and it is required to identify the letters and prove the solution unique.

I.2. Describing the Solution Set Example 2.1: System with many solutions. z = free variable Solution set = z = parameter Definition 2.2: The non-leading variables in an echelon form are free variables. Variables used to describe a solution set are parameters of the solution set.

Example 2.3: System with 2 free variables. z, w = free variables Solution set = z, w = parameters

Matrix Definition 2.6 : Matrix An mn matrix is a rectangular array of numbers with m rows and n columns. Each number in the matrix is an entry. Example: A 23 matrix i = row index j = column index

Example 2.7: Linear system  can be abbreviated as 

Definition 2.8: Vector An m-dimensional (column) vector is an m1 matrix. An n-dimensional row vector is an 1n matrix. The entries of a vector are its components. Definition 2.9: Solution A linear equation a1x1+ a2x2+…+ an xn = d of n unknowns x1 , x2 , …, xn is satisfied by a vector s  a1s1+ a2s2+…+ an sn = d A vector satisfies a linear system if it satisfies each equation in the system. Definition 2.10: Vector Addition (Sum)

Definition 2.10a: Matrix Addition (Sum) Definition 2.11: Scalar Multiplication Example 2.12:

Example 2.13:   Particular solution

Possible scenarios:             

Exercise: Solve the following using matrix notations & express the solution sets in vector notations. Exercise: Describe all functions f(x) = a x2 + b x + c such that f(1) = 2 & f(1) = 6. Describe all functions f(x) = a x2 + b x + c such that f(1) = 2 Exercise: Consider the linear system { a x + y = a2 , x + a y = 1 }. Find values of a so that it has 1) No solutions. 2) Infinitely many solutions.

I.3. General = Particular + Homogeneous Definition 3.2: Homogeneous Equations A linear equation is homogeneous if it can be put in the form a1x1 + a2x2 + … + anxn = 0 A linear system of equations is homogeneous if it can be put in the form Every linear system has an associated homogeneous system obtained by setting all constant terms to zero. Definition 3.4: Zero vector = 0 with 0i = 0  i. A homogeneous system is always consistent since 0 is always a solution. If s is a solution to a homogeneous system, so is c s  cR .

From Example 2.13: satisfy the assoicated homogeneous system: & satisfies the inhomogeneous system:

Example 3.5: Homogeneous system with unique solution 0.

Lemma 3.7: For any homogeneous linear system, there exists vectors β1 , …, βk s.t. where k is the number of free variables in an echelon form of the system. Proof : Let the system be of m equations in n unknowns with m  n. By means of the Gauss method, it can be reduced to an echelon form. Next, we make sure the leading variable of every row is on the diagonal position by moving any offending columns to the rightmost position. Let there be p equations of the form 0 = 0 at the bottom: Moving columns means re-ordering the unknowns.

By meaning of backward substitution, one gets With suitable re-labeling of the unknowns, we get where xj for j = mp+1, …, n are free variables. By definition:

The non-b part is a kk unit matrix. Proof is completed by setting βj to be the jth column of the nk matrix z The non-b part is a kk unit matrix. Particular solution:

Lemma 3.8: Proof: 1. Every solution s can be be written as s = p + h : Let s be a solution, then sp satisfies the associated homogeneous system since Setting h = s  p completes the proof. 2. Every s = p + h is a solution : Theorem 3.1: For any linear system, there exists vectors β1 , …, βk s.t. where p is a particular solution and k is the number of free variables.

Example 3.10: System with empty solution set Homogeneous solution set = → no particular solution → no general solution

Solution sets of linear systems are either Corollary 3.11: Solution sets of linear systems are either empty (system is inconsistent / example 3.10) have one element (solution is unique / example 3.5) have infinitely many elements (system is indeterminate / example 2.13) Dim( h) = 1 Dim( h) > 1 p exists Unique ( 1 general solution ) indeterminate (  general solutions ) p does not exist inconsistent ( 0 general solutions ) Definition 3.12 : Singular and Nonsingular Matrices A square matrix is nonsingular if it is the matrix of coefficients of a homogeneous system with a unique solution. It is singular otherwise, that is, if it is the matrix of coefficients of a homogeneous system with infinitely many solutions.

Definition: Rank of a Matrix The rank of a matrix is equal to the number of non-zero rows in its echelon form. Let A be an mn matrix, then Rank A  min( m, n ). Theorem A square matrix is nonsingular iff its rank is equal to its dimension.

Exercises