Solving Linear Equations UC Berkeley Fall 2004, E77 Copyright 2005, Andy Packard. This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

Linear equations Consider n equations in m unknowns. Think of the A ij as known coefficients, and the b i as known numbers. The goal is to solve for all of the unknowns x j

Example of Linear Equations Intersection of two lines Simple truss structures –Consist of beams –Frictionless “pin” joints Heat Transfer through conductive material Electrical current flow through resistive network Getting proper balance of nutrients from selection of foods

Simple truss analysis: Basic Concepts M4M4 M5M5 M1M1 M6M6 M2M2 M3M3 Truss members are beams held together with pin joints (no welding – drill a hole in each beam, push pin through). Beams only have a force acting at each end, no moments. These are called 2-force members. If the truss is in eqilibrium, total force on beam must be 0, and there cannot be a torque on the beam. Pins transfer force between beams. If the truss is in equilibrium, all forces acting on a pin must sum to zero.

Force balance on a pin F4F4 F3F3 F1F1 F2F2 Sum forces on pin in horizontal and vertical directions. For equilibrium, forces must sum to zero. Draw a free-body diagram of a given pin. The forces acting on it are the forces from the members (Newton ’ s 3 rd law) Measure θ from here (eg.)

Resultant forces on a 2-force member Force imbalance, beam would accelerate. These must be equal and opposite Force balanced, but beam would rotate. These must be along the beam 2-force member under load, in equilibrium

Simple truss structure analysis M4M4 M5M5 M1M1 M6M6 M2M2 M3M3 Let T i be the force in member M i.

Free-body diagrams on each pin T4T4 T5T5 T1T1 T6T6 T2T2 T3T3

Force balance on a pin T4T4 T5T5 T1T1 T2T2 Sum forces on pin in horizontal and vertical directions

Forces on a particular PIN T4T4 T5T5 T1T1 T6T6 T2T2 T3T3

T2T2 T3T3

T4T4 T5T5 T1T1 T6T6 T2T2 T3T3

T4T4 T6T6 T3T3

Assembling the equations If geometry is fixed, and external load is known, then this is 6 equations, 6 unknowns. We need some “ good notation ” for linear equations ….

Array-Vector multiplication If A is an n-by-m array, and x is an m-by-1 vector, then the “product Ax” is a n-by-1 vector, whose i’th component is = n-by-m m-by-1 n-by-1 = n-by-m m-by-1 n-by-1

Array-Vector multiplication If A is an n-by-m array, and x is an m-by-1 vector, then the “product Ax” is a n-by-1 vector, whose i’th component is

Ax as linear combination of columns of A add, to give

Linear equations Consider n equations in m unknowns. Collect –The A ij into an n-by-m array called A –The b i into a n-by-1 vector called b, and –The x j into an m-by-1 vector called x Then the equations above can be written concisely as matrix/vector multiply vector equality

Linear equations For the equation Ax=b, there are 3 distinct cases = = = Square, equal number of unknowns and equations Underdetermined: more unknowns than equations Overdetermined: fewer unknowns than equations

Types of solutions with “random” data “Generally” the following observations would hold = = = One solution (eg., 2 lines intersect at one point) Infinite solutions (eg., 2 planes intersect at many points) No solutions (eg., 3 lines don ’ t intersect at a point)

Linear equations But anything can happen. For example: = = = No solution (2 parallel lines) Many solutions (2 parallel lines) No solutions (2 parallel planes) Solutions (3 lines that do intersect at a point)

Assembling the equations If geometry is fixed, and external load is known, then this is 6 equations, 6 unknowns

Assembling the equations In matrix/vector form

Norms of vectors If v is an m-by-1 column (or row) vector, the “norm of v” is defined as Symbol to denote “norm of v” Square-root of sum-of-squares of components, generalizing Pythagorean ’ s theorem The norm of a vector is a measure of its length. Some facts: ||v||=0 if and only if every component of v is zero ||v + w|| ≤ ||v|| + ||w||

The “Least Squares” Problem If A is an n-by-m array, and b is an n-by-1 vector, let c * be the smallest possible (over all choices of m-by-1 vectors x) mismatch between Ax and b (ie., pick x to make Ax as much like b as possible). “is defined as” “the minimum, over all m-by-1 vectors x” “ the length (ie., norm) of the difference/mismatch between Ax and b. ”

Four cases for Least Squares Recall least squares formulation There are 4 scenarios c * = 0: the equation Ax=b has at least one solution –only one x vector achieves this minimum –many different x vectors achieves the minimum c * > 0: the equation Ax=b has no solutions –only one x vector achieves this minimum –many different x vectors achieves the minimum

The backslash operator If A is an n-by-m array, and b is an n-by-1 vector, then >> x = A\b solves the “least squares” problem. Namely –If there is an x which solves Ax=b, then this x is computed –If there is no x which solves Ax=b, then an x which minimizes the mismatch between Ax and b is computed. In the case where many x satisfy one of the criterion above, then a smallest (in terms of vector norm) such x is computed. So, mismatch is handled first. Among all equally suitable x vectors that minimize the mismatch, choose a smallest one.

Four cases: x=A\b as solution of No Mismatch: c * = 0, and only one x vector achieves this minimum Choose this x c * = 0, and many different x vectors achieves the minimum From all these minimizers, choose smallest x (ie., norm) Mismatch: c * > 0, and only one x vector achieves this minimum Choose this x c * > 0, and many different x vectors achieves the minimum From all minimizers, choose an x with the smallest norm