Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 1 : “shiv rpi” Linear Algebra A gentle introduction Linear Algebra has become as basic and as applicable as calculus, and fortunately it is easier. --Gilbert Strang, MIT

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 2 : “shiv rpi” What is a Vector ? q Think of a vector as a directed line segment in N-dimensions! (has “length” and “direction”) q Basic idea: convert geometry in higher dimensions into algebra! q Once you define a “nice” basis along each dimension: x-, y-, z-axis … q Vector becomes a 1 x N matrix! q v = [a b c] T q Geometry starts to become linear algebra on vectors like v! x y v

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 3 : “shiv rpi” Vector Addition: A+B A B A B C A+B = C (use the head-to-tail method to combine vectors) A+B

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 4 : “shiv rpi” Scalar Product: av v av Change only the length (“scaling”), but keep direction fixed. Sneak peek: matrix operation (Av) can change length, direction and also dimensionality!

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 5 : “shiv rpi” Vectors: Dot Product Think of the dot product as a matrix multiplication The magnitude is the dot product of a vector with itself The dot product is also related to the angle between the two vectors

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 6 : “shiv rpi” Inner (dot) Product: v.w or w T v v w  The inner product is a SCALAR! If vectors v, w are “columns”, then dot product is w T v

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 7 : “shiv rpi” Bases & Orthonormal Bases q Basis (or axes): frame of reference vs Basis: a space is totally defined by a set of vectors – any point is a linear combination of the basis Ortho-Normal: orthogonal + normal [Sneak peek: Orthogonal: dot product is zero Normal: magnitude is one ]

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 8 : “shiv rpi” What is a Matrix? q A matrix is a set of elements, organized into rows and columns rows columns

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 9 : “shiv rpi” Basic Matrix Operations q Addition, Subtraction, Multiplication: creating new matrices (or functions) Just add elements Just subtract elements Multiply each row by each column

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 10 : “shiv rpi” Matrix Times Matrix

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 11 : “shiv rpi” Multiplication q Is AB = BA? Maybe, but maybe not! q Matrix multiplication AB: apply transformation B first, and then again transform using A! q Heads up: multiplication is NOT commutative! q Note: If A and B both represent either pure “rotation” or “scaling” they can be interchanged (i.e. AB = BA)

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 12 : “shiv rpi” Matrix operating on vectors q Matrix is like a function that transforms the vectors on a plane q Matrix operating on a general point => transforms x- and y-components q System of linear equations: matrix is just the bunch of coeffs ! q x’ = ax + by q y’ = cx + dy              ' ' y x dc ba       y x

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 13 : “shiv rpi” Direction Vector Dot Matrix

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 14 : “shiv rpi” Matrices: Scaling, Rotation, Identity q Pure scaling, no rotation => “diagonal matrix” (note: x-, y-axes could be scaled differently!) q Pure rotation, no stretching => “orthogonal matrix” O q Identity (“do nothing”) matrix = unit scaling, no rotation! [cos , sin  ] T [1,0] T [0,1] T  [-sin , cos  ] T cos  -sin  sin  cos  [1,0] T [0,1] T r r 2 [r 1,0] T [0,r 2 ] T scaling rotation

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 15 : “shiv rpi” Scaling P P’ r 0 0 r a.k.a: dilation (r >1), contraction (r <1)

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 16 : “shiv rpi” Rotation P P’ cos  -sin  sin  cos 

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 17 : “shiv rpi” 2D Translation t P P’ P x y tx ty P’ t

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 18 : “shiv rpi” Inverse of a Matrix q Identity matrix: AI = A q Inverse exists only for square matrices that are non-singular q Maps N-d space to another N-d space bijectively q Some matrices have an inverse, such that: AA -1 = I q Inversion is tricky: (ABC) -1 = C -1 B -1 A -1 Derived from non- commutativity property

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 19 : “shiv rpi” Determinant of a Matrix q Used for inversion q If det(A) = 0, then A has no inverse ns/inverse/threeD/index.htm

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 20 : “shiv rpi” Projection: Using Inner Products (I) p = a (a T x) ||a|| = a T a = 1

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 21 : “shiv rpi” Homogeneous Coordinates q Represent coordinates as (x,y,h) q Actual coordinates drawn will be (x/h,y/h)

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 22 : “shiv rpi” Homogeneous Coordinates q The transformation matrices become 3x3 matrices, and we have a translation matrix! 1 0 t x 0 1 t y = x’ y’ 1 xy1xy1 New point Transformation Original point Exercise: Try composite translation.

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 23 : “shiv rpi” Homogeneous Transformations

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 24 : “shiv rpi”24 Order of Transformations q Note that matrix on the right is the first applied q Mathematically, the following are equivalent p’ = ABCp = A(B(Cp)) q Note many references use column matrices to represent points. In terms of column matrices p ’T = p T C T B T A T TRM

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 25 : “shiv rpi”25 Rotation About a Fixed Point other than the Origin Move fixed point to origin Rotate Move fixed point back M = T(p f ) R(  ) T(-p f )

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 26 : “shiv rpi” Vectors: Cross Product q The cross product of vectors A and B is a vector C which is perpendicular to A and B q The magnitude of C is proportional to the sin of the angle between A and B q The direction of C follows the right hand rule if we are working in a right-handed coordinate system B A A×BA×B

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 27 : “shiv rpi” MAGNITUDE OF THE CROSS PRODUCT

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 28 : “shiv rpi” DIRECTION OF THE CROSS PRODUCT q The right hand rule determines the direction of the cross product

Shivkumar Kalyanaraman Rensselaer Polytechnic Institute 29 : “shiv rpi” For more details q Prof. Gilbert Strang’s course videos: q /VideoLectures/index.htm /VideoLectures/index.htm q Esp. the lectures on eigenvalues/eigenvectors, singular value decomposition & applications of both. (second half of course) q Online Linear Algebra Tutorials: q