1. Math for CSTutorial 5-61 Tutorial 5 Function Optimization. Line Search. Taylor Series for R n Steepest Descent

2. Math for CSTutorial 5-62 Line search runs as following. Let Be the scalar function of α representing the possible values of f(x) in the direction of p k. Let (a,b,c) be the three points of α, such, that the point of (constrained) minimum x’, is between a and c: a<x’<c. Then the following algorithm allows to apprach x’ arbitrarily close: If f(a) ≥ f(c), u = (a+b)/2; If f(u) < f(b) (a,b,c) = (a,u,b) Else (a,b,c) = (u,b,c) Line search a b c u If f(a) < f(c), u = (b+c)/2; If f(u) < f(b) (a,b,c) = (b,u,c) Else (a,b,c) = (a,b,u)

3. Math for CSTutorial 5-63 The Taylor series for f(x) is,where For the function of m variables, the expression is Taylor Series

4. Math for CSTutorial 5-64 Consider the elliptic function: f(x,y)=(x-1) 2 +(2y-2) 2 and find the first three terms of Taylor expansion. 2D Taylor Series: Example

5. Math for CSTutorial 5-65 Consider the elliptic function: f(x,y)=(x-1) 2 +(2y-2) 2 and find the first three terms of Taylor expansion. Find the first step of Steepest Descent. Steepest Descent 1 2 -f’(0)

6. Math for CSTutorial 5-66 Consider the elliptic function: f(x,y)=(x-1) 2 +(2y-2) 2 and find the first three terms of Taylor expansion. Find the first step of Steepest Descent. Now, find a step a, in the direction of gradient, minimizing the function: Steepest Descent

7. Math for CSTutorial 5-67 Consider the elliptic function: f(x,y)=(x-1) 2 +(2y-2) 2 and find the first three terms of Taylor expansion. Find the first step of Steepest Descent. Is it a minimum? Next step? Steepest Descent