1. MATLAB Optimization Greg Reese, Ph.D Research Computing Support Group Miami University

2. MATLAB Optimization © 2010-2013 Greg Reese. All rights reserved 2

3. Function Summary 3 FunctionParameterLinearityConstraintsLocation fgoalattain fminbnd fmincon fminimax fminsearch fminunc fzero lsqlin lsqnonlin lsqnonneg

4. 4 Optimization Optimization - finding value of a parameter that maximizes or minimizes a function with that parameter – Talking about mathematical optimization, not optimization of computer code! – "function" is mathematical function, not MATLAB language function

5. 5 Optimization Optimization - finding value of a parameter that maximizes or minimizes a function with that parameter – Can have multiple parameters – Can have multiple functions – Parameters can appear linearly or nonlinearly

6. 6 Linear programming – "programming" means determining feasible programs (plans, schedules, allocations) that are optimal with respect to a certain criterion – Most often used kind of optimization –Tremendous number of practical applications\ Won't discuss further

7. 7 fminbnd The basic optimizing 1D routine is fminbnd, which attempts to find a minimum of a function of one variable within a fixed interval. function must be continuous only real values (not complex) only scalar values (not vector) might only find local minimum

8. 8 fminbnd x = fminbnd( @fun, x1, x2 ) x1 < x2 is the minimization interval x is the x-value in [x1,x2] where the minimum occurs fun is the function whose minimum we’re looking for. fun must accept exactly one scalar argument and must return exactly one scalar value Because there are no restrictions on the value of fun, finding the minimum is called unconstrained optimization.

9. 9 fminbnd x = fminbnd( @fun, x1, x2 ) @fun is called a function handle. You must include the “@” when you call fminbnd.

10. 10 fminbnd Try It Find the minimum of f(x) = ( x – 3 ) 2 – 1 on the interval [0,5] Step 1 – make an m-file with the function

11. 11 fminbnd Try It Step 2 – call fminbnd with your function >> x = fminbnd( @parabola, 0, 5 ) x = 3 If you want to get the value of the function at its minimum, call the function with the minimum x value returned by fminbnd >> parabola(x) ans = -1

12. 12 fminbnd Try It Step 3 – verify result is a global minimum by plotting function over a range, say [-10 10] >> ezplot( @parabola, [-10 10] )

13. 13 fminunc The multidimensional equivalent of fminbnd is fminunc. It attempts to find a local minimum of a function of one or more variables. function must be continuous only real values (not complex) only scalar values (not vector) might only find local minimum

14. 14 fminunc x = fminunc( @fun, x0 ) x0 is your guess of where the minimum is fun is the function whose minimum we’re looking for. fun must accept exactly one scalar or vector argument and must return exactly one scalar value

15. 15 fminunc x = fminunc( @fun, x0 ) fun only has one argument. If the argument is a vector, each element represents the corresponding independent variable in the function Example f(x,y,z) = x 2 + y 2 + z 2 function w = f( x ) w = x(1)^2 + x(2)^2 + x(3)^2

16. 16 fminunc Optimizing routines such as fminunc are often sensitive to the initial value. For bad initial values they may converge to the wrong answer, or not converge at all. Tip – if fun is a function of one or two variables, plot it first to get an idea of where the minimum is

17. 17 fminunc Try It Find the minimum of f(x) = ( x – π ) 2 – 1 Step 1 – make an m-file with the function

18. 18 fminunc Try It Step 2 – plot the function to get an estimate of the location of its minimum Looks like min is at about 3

19. 19 fminunc Try It Step 3 – call fminunc with your function and an initial guess of 3 >> xmin = fminunc( @pi_parabola, 3 ) Warning: Gradient must be provided for trust-region method; using line-search method instead. > In fminunc at 247 Optimization terminated: relative infinity-norm of gradient less than options.TolFun. xmin = 3.1416 Ignore this stuff

20. 20 fminunc Try It Find the minimum of f(x,y) = ( x–22.22 ) 2 + ( y-44.44 ) 2 - 17 Step 1 – make an m-file with the function

21. 21 fminunc Try It Step 2 – plot the function to get an estimate of the location of its minimum ezsurf expects a function that takes two arguments. Since parabola only takes one, we need to make a different version that takes two. Let's call it paraboloid_xy

22. 22 fminunc Try It Step 2 – plot the function to get an estimate of the location of its minimum >> ezsurf( @parabola_xy, [ 0 50 0 50] ) After zooming and panning some, it looks like the min is at about x=20 and y=50

23. 23 fminunc Try It Step 3 – call fminunc with your function and an initial guess of [20 50] >> x = fminunc( @paraboloid, [20 50] ) x = 22.2200 44.4400 >> paraboloid( x ) ans = -17.0000

24. 24 fminunc Try It Find the minimum of a 5D hypersphere of centered at [ 1 2 3 4 5 ] f(x,y,z,a,b) = (x–1) 2 + (y–2) 2 + (z–3) 2 + (a–4) 2 + (b–5) 2 Step 1 – make an m-file with the function Use 3 dots to continue a line in a file.

25. 25 fminunc Try It Step 2 – can't plot 5D, so guess [ 0 0 0 0 0] Step 3 – call fminunc with your function and an initial guess of [0 0 0 0 0] >> xmin = fminunc( @hypersphere, [0 0 0 0 0] ) xmin = 1.0000 2.0000 3.0000 4.0000 5.0000 >> hypersphere( xmin ) ans = 3.9790e-012 ≈ 0

26. 26 MATLAB Optimization Questions?

27. 27 The End