Rendering – Matrix Transformations and the Graphics Pipeline CSE 3541/5541 Matt Boggus

Overview Matrix format and operations Graphics pipeline overview Modeling transformations View and perspective transformations Polygon visibility and rasterization

What is a Matrix? A matrix is a set of elements, organized into rows and columns rows columns

Definitions n x m Array of Scalars (n Rows and m Columns) n: row dimension of a matrix, m: column dimension m = n  square matrix of dimension n Element Transpose: interchanging the rows and columns of a matrix Column Matrices and Row Matrices Column matrix (n x 1 matrix): Row matrix (1 x n matrix):

Matrix Operations Scalar-Matrix Multiplication Matrix-Matrix Addition Multiply every element by the scalar Matrix-Matrix Addition Add elements with same index Matrix-Matrix Multiplication A: n x l matrix, B: l x m  C: n x m matrix Easy to overlook that cij is a single element, so computing C requires iterating over rows and columns. cij = the sum of multiplying elements in row i of matrix a times elements in column j of matrix b

Matrix Operation Examples

Matrix Operations Properties of Scalar-Matrix Multiplication Properties of Matrix-Matrix Addition Commutative: Associative: Properties of Matrix-Matrix Multiplication Identity Matrix I (Square Matrix)

Matrix Multiplication Order Is AB = BA? Try it! Matrix multiplication is NOT commutative! The order of series of matrix multiplications is important! Generalize where possible, but don’t forget about special cases

Inverse of a Matrix Identity matrix: AI = A Some matrices have an inverse, such that: AA-1 = I

Inverse of a Matrix Do all matrices have a multiplicative inverse? Consider this example, try to solve for A-1: AA-100 = Given A, try to solve for the 9 variables in its inverse A-1 by setting the result of the multiplication AA-1 equal to I. If no solution exists, the matrix has no inverse. 0 * a + 0 * d + 0 * g = 0 ≠ 1 Note: 1 is the element at 00 in the identity matrix

Inverse of Matrix Concatenation Inversion of concatenations (ABC)-1 = ? A * B * C * X = I A * B * C * C-1 = A * B A * B * B-1 = A A * A-1 = I Order is important, so X = C-1B-1A-1 Given A, B, and C assuming each has an inverse, we can construct the inverse of a series of multiplications ABC by constructing a matrix such that each individual “cancels” with its inverse.

Row and Column Matrices + points By convention we will use column matrices for points Column Matrix Row matrix Concatenations Associative By Row Matrix

Computer Graphics The graphics pipeline is a series of conversions of points into different coordinate systems or spaces

Modeling or Geometric transformations

Common 2D transformations y y x x Translate Scale y x Rotate

Point representation We use a column vector (a 2x1 matrix) to represent a 2D point Points are defined with respect to origin (point) coordinate axes (basis vectors) Since points and vector share this notation, we can think of transformations applying to either type.

Arbitrary transformation of a 2D point

2D Translation Re-position a point along a straight line Given a point p = (x, y) and the translation vector t = (tx, ty) (x’,y’) The new point p’ = (x’, y’) x’ = x + tx y’ = y + ty ty (x,y) tx OR p’ = p + t where

2D Translation How to translate an object with multiple vertices? Translate individual vertices

2D Rotation Rotate with respect to origin (0,0) > 0 : Rotate counter clockwise q q < 0 : Rotate clockwise

2D Rotation (x,y) -> Rotate about the origin by q (x’, y’) r f How to compute (x’, y’) ? Represent the points using polar coordinate notation x = r cos (f) y = r sin (f) x’ = r cos (f + q) y’ = r sin (f + q)

2D Rotation x = r cos (f) y = r sin (f) (x,y) (x’,y’) q x = r cos (f) y = r sin (f) x’ = r cos (f + q) y = r sin (f + q) r f x’ = r cos (f + q) = r cos(f) cos(q) – r sin(f) sin(q) Use trigonometry angle sum identities = x cos(q) – y sin(q) y’ = r sin (f + q) = r sin(f) cos(q) +r cos(f) sin(q) = y cos(q) + x sin(q)

2D Rotation x’ = x cos(q) – y sin(q) y’ = y cos(q) + x sin(q) r (x,y) (x’,y’) q x’ = x cos(q) – y sin(q) y’ = y cos(q) + x sin(q) r f Matrix form: x’ cos(q) -sin(q) x y’ sin(q) cos(q) y =

2D Rotation How to rotate an object with multiple vertices? q Rotate individual Vertices

2D Scaling Scale: Alter the size of an object by a scaling factor (Sx, Sy), i.e. x’ = x * Sx y’ = y * Sy x’ Sx 0 x y’ 0 Sy y = (2,2) (4,4) (1,1) (2,2) Sx = 2, Sy = 2

2D Scaling Object size has changed, but so has its position! (4,4) (2,2) (4,4) (1,1) (2,2) Sx = 2, Sy = 2 Object size has changed, but so has its position!

2D Scaling special case – Reflection sx = -1 sy = 1 original sx = -1 sy = -1 sx = 1 sy = -1

Put it all together Translation: x’ x tx y’ y ty Rotation: x’ cos(q) -sin(q) x y’ sin(q) cos(q) y Scaling: x’ Sx 0 x y’ 0 Sy y = + = * = *

Translation as multiplication? Can we use only tx and ty to make a multiplicative translation matrix? (Boardwork derivation) Note: translation should only be a function of the translation vector (tx, ty) – that is why we don’t want to use the scalar values of x and y in the translation matrix. In practice, we are going to use the same translation matrix on multiple points, each with a different value for x and y, so those values are not constant over time as we translate more vertices. Step 1. Determine what the dimensions of [?] are Step 2. Create variables for each element of [?] Step 3. Multiply it out and try to solve for the variables ONLY IN TERMS OF tx and ty

Translation Multiplication Matrix x’ = x + tx y’ y ty Use 3 x 1 vector x’ 1 0 tx x y’ = 0 1 ty * y 1 0 0 1 1

3x3 2D Rotation Matrix x’ cos(q) -sin(q) x y’ sin(q) cos(q) * y r = (x,y) (x’,y’) q f r x’ cos(q) -sin(q) 0 x y’ sin(q) cos(q) 0 * y 1 0 0 1 1 =

3x3 2D Scaling Matrix x’ Sx 0 x y’ 0 Sy y = x’ Sx 0 0 x 1 0 0 1 1

3x3 2D Matrix representations Translation: Rotation: Scaling:

Visualization of composing transformations Translate then Scale Scale then Translate Note: in these figures, circles are not drawn to exact scale

Linear Transformations A linear transformation can be written as: x’ = ax + by + c OR y’ = dx + ey + f

Why use 3x3 matrices? So that we can perform all transformations using matrix/vector multiplications This allows us to pre-multiply all the matrices together The point (x,y) is represented using Homogeneous Coordinates (x,y,1)

Matrix concatenation Examine the computational cost of using four matrices ABCD to transform one or more points (i.e. p’ = ABCDp) We could: apply one at a time p' = D * p p'' = C * p' … 3x3 * 3x1 for each transformation for each point Or we could: concatenate (pre-multiply matrices) M=A*B*C*D p' = M * p 3x3 * 3x3 for each transformation 3x3 * 3x1 for each point

Local Rotation The standard rotation matrix is used to rotate about the origin (0,0) What if I want to rotate about an arbitrary center? cos(q) -sin(q) 0 sin(q) cos(q) 0 0 0 1 Smiley face from Han-Wei Shen’s rendering slides

Arbitrary Rotation Center To rotate about an arbitrary point P (px,py) by q: Translate the object so that P will coincide with the origin: T(-px, -py) Rotate the object: R(q) Translate the object back: T(px, py) (px,py)

Arbitrary Rotation Center Translate the object so that P will coincide with the origin: T(-px, -py) Rotate the object: R(q) Translate the object back: T(px,py) As a matrix multiplication p’ = T[px,py] * R[q] * T[-px, -py] * P x’ 1 0 px cos(q) -sin(q) 0 1 0 -px x y’ = 0 1 py sin(q) cos(q) 0 0 1 -py y 1 0 0 1 0 0 1 0 0 1 1

Local scaling The standard scaling matrix will only anchor at (0,0) Sx 0 0 0 Sy 0 0 0 1 What if I want to scale about an arbitrary pivot point?

Arbitrary Scaling Pivot To scale about an arbitrary pivot point P (px,py): Translate the object so that P will coincide with the origin: T(-px, -py) Scale the object: S(Sx, Sy) Translate the object back: T(px,py) (px,py)

Moving to 3D Translation and Scaling are very similar, just include z dimension Rotation is more complex

3D Translation

3D Scaling

3D Rotations – rotation about primary axes

Viewing and Projection Transformations

Viewing transformation parameters Camera (eye) position (ex,ey,ez) Center of interest (cx, cy, cz) Or equivalently, a “viewing vector” to specify the direction the camera faces Up vector (Up_x, Up_y, Up_z)

Eye coordinate system Camera (eye) position (ex,ey,ez) View vector : v Up vector : u Third vector (v x u) : w Viewing transform – construct a matrix such that: Camera is translated to the origin Camera is rotated so that v is aligned with (0,0,1) or (0,0,-1) u is aligned with (0,1,0)

Projection Transform a point from a high dimensional space to a low‐dimensional space. In 3D, the projection means mapping a 3D point onto a 2D projection plane (or called image plane). There are two basic projection types: Parallel (orthographic) Perspective

Orthographic projection

Orthographic projection

Properties of orthographic projection Not realistic looking Can preserve parallel lines Can preserve ratios Does not preserve angles between lines Mostly used in computer aided design and architectural drawing software

Foreshortening – Pietro Perugino fresco example

Orthographic projection no foreshortening

Perspective projection has foreshortening

Perspective projection viewing volume – frustum

Properties of perspective projection Realistic looking Lines are mapped to lines Parallel lines may not remain parallel Ratios are not preserved Distances cannot be directly measured, as in parallel projection

Visibility and Rasterization

In what order should the polygons be drawn? Visibility problem In what order should the polygons be drawn?

Painter’s algorithm Image source: https://en.wikipedia.org/wiki/Painter's_algorithm Draw polygons from back to front ; requires that the polygons be sorted

Z-buffer, with example Foreach polygon Foreach pixel in polygon Compare polygon depth at (x,y) with current depth at (x,y) If( polygon.z < midepth) Set mindepth Set pixel as polygon’s color Image source: http://cs.brown.edu/stc/summer/2ViewRender/2ViewRender_20.html

Additional slides Additional reference materials: Huamin Wang’s (http://www.cse.ohio-state.edu/~whmin/) real-time rendering notes Rick Parent’s (http://www.cse.ohio-state.edu/~parent/) ray-tracing notes Additional slides

Critical thinking – transformations and matrix multiplication Suppose we want to scale an object, then translate it. What should the matrix multiplication look like? A. p’ = Scale * Translate * p B. p’ = Translate * Scale * p C. p’ = p * Scale * Translate D. Any of these is correct

Shearing Y coordinates are unaffected, but x coordinates are translated linearly with y That is: y’ = y x’ = x + y * h x' 1 h 0 x y' = 0 1 0 * y 1 0 0 1 1

Shearing in y x' 1 0 0 x y' = g 1 0 * y 1 0 0 1 1 Interesting Facts: 1 0 0 1 1 Interesting Facts: Any 2D rotation can be built using three shear transformations. Shearing will not change the area of the object Any 2D shearing can be done by a rotation, followed by a scaling, and followed by a rotation

Another use of perspective Head Tracking for Desktop VR Displays using the WiiRemote