CIS 541 – Numerical Methods Mathematical Preliminaries

Derivatives Recall the limit definition of the first derivative. February 28, 2019 OSU/CIS 541

Partial Derivatives Same as derivatives, keep each other dimension constant. February 28, 2019 OSU/CIS 541

Tangents and Gradients Recall that the slope of a curve (defined as a 1D function) at any point x, is the first derivative of the function. That is, the linear approximation to the curve in the neighborhood of t is l(x) = b + f’(t)x f(x) t (1,f’(t)) y x February 28, 2019 OSU/CIS 541

Tangents and Gradients Since we also want this linear approximation to intersect the curve at the point t. l(t) = f(t) = b + f’(t)t Or, b = f(t) - f’(t)t We say that the line l(x) interpolates the curve f(x) at the point t. February 28, 2019 OSU/CIS 541

Functions as curves We can think of the curve shown in the previous slide as the set of all points (x,f(x)). Then, the tangent vector at any point along the curve is February 28, 2019 OSU/CIS 541

Side note on Curves There are other ways to represent curves, rather than explicitly. Functions are a subset of curves (x,y(x)). Parametric equations represent the curve by the distance walked along the curve (x(t),y(t)). Circle: (cos, sin) Implicit representations define a contour or level-set of the function: f(x,y) = c. February 28, 2019 OSU/CIS 541

Tangent Planes and Gradients In higher-dimensions, we have the same thing: A surface is a 2D function in 3D: Surface = (x, y, f(x,y) ) A volume or hyper-surface is a 3D function in 4D: Volume = (x, y, z, f(x,y,z) ) February 28, 2019 OSU/CIS 541

Tangent Planes and Gradients The linear approximation to the higher-dimensional function at a point (s,t), has the form: ax+by+cz+d=0, or z(x,y) = … What is this plane? February 28, 2019 OSU/CIS 541

Construction of Tangent Planes Images courtesy of TJ Murphy: http://www.math.ou.edu/~tjmurphy/Teaching/2443/TangentPlane/TangentPlane.html February 28, 2019 OSU/CIS 541

Construction of Tangent Planes February 28, 2019 OSU/CIS 541

Construction of Tangent Planes February 28, 2019 OSU/CIS 541

Tangent Planes and Gradients The formula for the plane is rather simple: z(s,t) = f(s,t) - interpolates z(s+dx,t) = f(s,t) + fx(s,t)dx = b + adx Linear in dx Of course, the plane does not stay close to the surface as you move away from the point (s,t). February 28, 2019 OSU/CIS 541

Tangent Planes and Gradients The normal to the plane is thus: The 2D vector: is called the gradient of the function. It represents the direction of maximal change. February 28, 2019 OSU/CIS 541

Gradients The gradient thus indicates the direction to walk to get down the hill the fastest. Also used in graphics to determine illumination. February 28, 2019 OSU/CIS 541

Review of Functions Extrema of a function occur where f’(x)=0. The second derivative determines whether the point is a minimum or maximum. The second derivative also gives us an indication of the curvature of the curve. That is, how fast it is oscillating or turning. February 28, 2019 OSU/CIS 541

The Class of Polynomials Specific functions of the form: February 28, 2019 OSU/CIS 541

The Class of Polynomials For many polynomials, the latter coefficients are zero. For example: p(x) = 3+x2+5x3 February 28, 2019 OSU/CIS 541

Taylor’s Series For a function, f(x), about a point c. I.E. A polynomial February 28, 2019 OSU/CIS 541

Taylor’s Theorem Taylor’s Theorem allows us to truncate this infinite series: February 28, 2019 OSU/CIS 541

Taylor’s Theorem Some things to note: (x-c)(n+1) quickly approaches zero if |x-c|<<1 (x-c)(n+1) increases quickly if |x-c|>>1 Higher-order derivatives may get smaller (for smooth functions). February 28, 2019 OSU/CIS 541

Higher Derivatives What is the 100th derivative of sin(x)? Compare 3100 to 100! What is the 100th derivative of sin(1000x)? February 28, 2019 OSU/CIS 541

Taylor’s Theorem Hence, for points near c we can just drop the error term and we have a good polynomial approximation to the function (again, for points near c). Consider the case where (x-c)=0.5 For n=4, this leads to an error term around 2.6*10-4 f() Do this for other values of n. Do this for the case (x-c) = 0.1 February 28, 2019 OSU/CIS 541

Some Common Derivatives February 28, 2019 OSU/CIS 541

Some Resulting Series About c=0 February 28, 2019 OSU/CIS 541

Some Resulting Series About c=0 February 28, 2019 OSU/CIS 541

Book’s Introduction Example Eight terms for first series not even yielding a single significant digit. Only four for second series with four significant digits. February 28, 2019 OSU/CIS 541

Mean-Value Theorem Special case of Taylor’s Theorem, where n=0, x=b. Assumes f(x) is continuous and its first derivative exists everywhere within (a,b). February 28, 2019 OSU/CIS 541

Mean-Value Theorem So what!?!?! What does this mean? Function can not jump away from current value faster than the derivative will allow. f(x) secant a b  February 28, 2019 OSU/CIS 541

Rolles Theorem If a and b are roots (f(a)=f(b)=0) of a continuous function f(x), which is not everywhere equal to zero, then f’(t)=0 for some point t in (a,b). I.e., What goes up, must come down. f(x) f’(t)=0 a t b February 28, 2019 OSU/CIS 541

Caveat For Taylor’s Series and Taylor’s Theorem to hold, the function and its derivatives must exist within the range you are trying to use it. That is, the function does not go to infinity, or have a discontinuity (implies f’(x) does not exist), … February 28, 2019 OSU/CIS 541

Implementing a Fast sin() const int Max_Iters = 100,000,000; float x = -0.1; float delta = 0.2 / Max_Iters; float Reimann_sum = 0.0; for (int i=0; i<Max_Iters; i++) { Reimann_Sum += sinf(x); x+=delta; } Printf(“Integral of sin(x) from –0.1->0.1 equals: %f\n”, Reimann_Sum*delta ); February 28, 2019 OSU/CIS 541

Implementing a Fast sin() const int Max_Iters = 100,000,000; float x = -0.1; float delta = 0.2 / Max_Iters; float Reimann_sum = 0.0; for (int i=0; i<Max_Iters; i++) { Reimann_Sum += my_sin(x); //my own sine func x+=delta; } Printf(“Integral of sin(x) from –0.1->0.1 equals: %f\n”, Reimann_Sum*delta ); February 28, 2019 OSU/CIS 541

Version 1.0 my_sin( const float x ) { float x2 = x*x; float x3 = x*x2; return (x – x3/6.0 + x2*x3/120.0 ); } February 28, 2019 OSU/CIS 541

Version 2.0 – Horner’s Rule Static const float fac3inv = 1.0 / 6.0f; Static const float fac5inv = 1.0 / 120.0f; my_sin( const float x ) { float x2 = x*x; return x*(1.0 – x2*(fac3inv - x2*fac5inv)); } February 28, 2019 OSU/CIS 541

Version 3.0 – Inline code const int Max_Iters = 100,000,000; float x = -0.1; float delta = 0.2 / Max_Iters; float Reimann_sum = 0.0; for (i=0; i<Max_Iters; i++) { x2 = x*x; Reimann_Sum += x*(1.0–x2*(fac3inv-x2*fac5inv); x+=delta; } Printf(“Integral of sin(x) from –0.1->0.1 equals: %f\n”, Reimann_Sum*delta ); February 28, 2019 OSU/CIS 541

Max( |sin(x)-my_sin(x)| ) Timings Pentium III, 600MHz machine Time in seconds Result Max( |sin(x)-my_sin(x)| ) Using sinf 27 -0.0041943 Version 1.0 20 1.93765*10-11 Version 2.0 13 8.0495*10-12 Version 3.0 2 February 28, 2019 OSU/CIS 541

Observations Is the result correct? Why did we gain some accuracy with version 2.0? Is (–0.1,0.1) a fair range to consider? Is the original sinf() function optimized? How did we achieve our speed-ups? We will re-examine this after Lab1. Ask these question for Lab1 !!! February 28, 2019 OSU/CIS 541

Homework Read Chapters 1 and 2 for next class. Start working on Lab 1 and Homework 1. February 28, 2019 OSU/CIS 541