1. L22 Numerical Methods part 2 Homework Review Alternate Equal Interval Golden Section Summary Test 4 1

2. 2

3. Problem 10.4 3 Yes, descent direction

4. Prob 10.10 4 No, not a descent direction

5. Prob 10.19 5 Slope

6. Prob 10.19 cont’d 6

7. 7

8. Prob 10.30 8

9. The Search Problem Sub Problem A Which direction to head next? Sub Problem B How far to go in that direction? 9

10. Search Direction… Min f(x): Let’s go downhill! 10 Descent condition

11. Step Size? How big should we make alpha? Can we step too “far?” i.e. can our step size be chosen so big that we step over the “minimum?” 11

12. 12 Figure 10.2 Conceptual diagram for iterative steps of an optimization method. We are here Which direction should we head?

13. Some Step Size Methods “Analytical” Search direction = (-) gradient, (i.e. line search) Form line search function f( α) Find f’( α)=0 Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section 13

14. 14 Figure 10.5 Nonunimodal function f(  ) for 0   Nonunimodal functions Unimodal if stay in locale?

15. Monotonic Increasing Functions 15

16. Monotonic Decreasing Functions 16 continous

17. 17 Figure 10.4 Unimodal function f(  ). Unimodal functions monotonic increasing then monotonic decreasing monotonic decreasing then monotonic increasing

18. Some Step Size Methods “Analytical” Search direction = (-) gradient, (i.e. line search) Form line search function f( α) Find f’( α)=0 Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section 18

19. 19 Figure 10.3 Graph of f(  ) versus . Analytical Step size Slope of line search= Slope of line at fmin

20. Analytical Step Size Example 20

21. Alternative Analytical Step Size 21 New gradient must be orthogonal to d for

22. Some Step Size Methods “Analytical” Search direction = (-) gradient, (i.e. line search) Form line search function f( α) Find f’( α)=0 Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section 22

23. 23 Figure 10.6 Equal-interval search process. (a) Phase I: initial bracketing of minimum. (b) Phase II: reducing the interval of uncertainty. “Interval Reducing” Region elimination “bounding phase” Interval reduction phase”

24. 2 delta! 24

25. Successive-Equal Interval Algorithm 25 “Interval” of uncertainty

26. More on bounding phase I Swan’s method Fibonacci sequence 26

27. Successive Alternate Equal Interval 27 Assume bounding phase has found Min can be on either side of But for sure its not in this region! Point values… not a line

28. Successive Alt Equal Int 28 Requires two function evaluations per iteration

29. 29 Figure 10.8 Initial bracketing of the minimum point in the golden section method. Fibonacci Bounding

30. 30 Figure 10.9 Graphic of a section partition. Golden section

31. Golden Section Example 31

32. Summary General Opt Algorithms have two sub problems: search direction, and step size Descent condition assures correct direction For line searches…in local neighborhood… we can assume unimodal! Step size methods: analytical, region elimin. Region Elimination (“interval reducing”) Equal interval Alternate equal interval Golden Section 32