3D Geometry for Computer Graphics

2 The plan today Least squares approach  General / Polynomial fitting  Linear systems of equations  Local polynomial surface fitting

3 y = f (x) Motivation Given data points, fit a function that is “close” to the points y x P i = (x i, y i )

4 Motivation Local surface fitting to 3D points

5 Line fitting Orthogonal offsets minimization – we already learned (PCA) x y

6 Line fitting y- offsets minimization x y P i = (x i, y i )

7 Line fitting Find a line y = ax + b that minimizes E(a,b) is quadratic in the unknown parameters a, b Another option would be, for example: But – it is not differentiable, harder to minimize…

8 Line fitting – LS minimization To find optimal a, b we differentiate E(a, b) : E(a, b) = (–2x i )[y i – (ax i + b)] = 0 E(a, b) = (–2)[y i – (ax i + b)] = 0

9 Line fitting – LS minimization We get two linear equations for a, b : (–2x i )[y i – (ax i + b)] = 0 (–2)[y i – (ax i + b)] = 0

10 Line fitting – LS minimization We get two linear equations for a, b : [x i y i – ax i 2 – bx i ] = 0 [y i – ax i – b] = 0

11 Line fitting – LS minimization We get two linear equations for a, b : ( x i 2 ) a + ( x i ) b = x i y i ( x i ) a + ( 1) b = y i

12 Line fitting – LS minimization Solve for a, b using e.g. Gauss elimination Question: why the solution is the minimum for the error function? E(a, b) = [y i – (ax i + b)] 2

13 Fitting polynomials y x

14 Decide on the degree of the polynomial, k Want to fit f (x) = a k x k + a k-1 x k-1 + … + a 1 x+ a 0 Minimize: E(a 0, a 1, …, a k ) = [y i – (a k x i k +a k-1 x i k-1 + …+a 1 x i +a 0 )] 2 E(a 0,…,a k ) = (– 2x m )[y i – (a k x i k +a k-1 x i k-1 +…+ a 0 )] = 0 Fitting polynomials

15 Fitting polynomials We get a linear system of k+1 in k+1 variables

16 General parametric fitting We can use this approach to fit any function f(x)  Specified by parameters a, b, c, …  The expression f(x) linearly depends on the parameters a, b, c, …

17 General parametric fitting Want to fit function f abc… (x) to data points (x i, y i )  Define E(a,b,c,…) = [y i – f abc… (x i )] 2  Solve the linear system

18 General parametric fitting It can even be some crazy function like Or in general:

19 Solving linear systems in LS sense Let’s look at the problem a little differently:  We have data points (x i, y i )  We want the function f(x) to go through the points:  i =1, …, n: y i = f(x i )  Strict interpolation is in general not possible In polynomials: n+1 points define a unique interpolation polynomial of degree n. So, if we have 1000 points and want a cubic polynomial, we probably won’t find it…

20 Solving linear systems in LS sense We have an over-determined linear system n  k : f(x 1 ) = 1 f 1 (x 1 ) + 2 f 2 (x 1 ) + … + k f k (x 1 ) = y 1 f(x 2 ) = 1 f 1 (x 2 ) + 2 f 2 (x 2 ) + … + k f k (x 2 ) = y 2 … f(x n ) = 1 f 1 (x n ) + 2 f 2 (x n ) + … + k f k (x n ) = y n

21 Solving linear systems in LS sense In matrix form:

22 Solving linear systems in LS sense In matrix form: Av = y

23 Solving linear systems in LS sense More constrains than variables – no exact solutions generally exist We want to find something that is an “approximate solution”:

24 Finding the LS solution v  R k Av  R n As we vary v, Av varies over the linear subspace of R n spanned by the columns of A: Av = A2A2 A1A1 AkAk 1 2. k = 1 A1A1 A2A2 AkAk + 2 +…+ k

25 Finding the LS solution We want to find the closest Av to y : Subspace spanned by columns of A y RnRn Av closest to y

26 Finding the LS solution The vector Av closest to y satisfies: (Av – y)  {subspace of A ’s columns}  column A i, = 0  i, A i T (Av – y) = 0 A T (Av – y) = 0 (A T A)v = A T y These are called the normal equations

27 Finding the LS solution We got a square symmetric system (A T A)v = A T y (k  k) If A has full rank (the columns of A are linearly independent) then (A T A) is invertible.

28 Weighted least squares Sometimes the problem also has weights to the constraints:

29 Local surface fitting to 3D points Normals? Lighting? Upsampling?

30 Local surface fitting to 3D points Locally approximate a polynomial surface from points

31 Fitting local polynomial X Y Z Reference plane Fit a local polynomial around a point P P

32 Fitting local polynomial surface Compute a reference plane that fits the points close to P Use the local basis defined by the normal to the plane! z x y

33 Fitting local polynomial surface Fit polynomial z = p(x,y) = ax 2 + bxy + cy 2 + dx + ey + f z x y

34 Fitting local polynomial surface Fit polynomial z = p(x,y) = ax 2 + bxy + cy 2 + dx + ey + f z x y

35 Fitting local polynomial surface Fit polynomial z = p(x,y) = ax 2 + bxy + cy 2 + dx + ey + f z x y

36 Fitting local polynomial surface Again, solve the system in LS sense: ax bx 1 y 1 + cy dx 1 + ey 1 + f = z 1 ax bx 2 y 2 + cy dx 2 + ey 2 + f = z 1... ax n 2 + bx n y n + cy n 2 + dx n + ey n + f = z n Minimize  ||z i – p(x i, y i )|| 2

37 Fitting local polynomial surface Also possible (and better) to add weights:  w i ||z i – p(x i, y i )|| 2, w i > 0 The weights get smaller as the distance from the origin point grows.

See you next time