1. ISM 206 Lecture 6 Nonlinear Unconstrained Optimization

2. Announcements This week: Nonlinear Optimization Unconstrained (today) Constrained (Thursday and Thursday!) Homework and solutions available Rescheduled midterm

3. Outline Today: Unconstrained –Helps us with constrained problems (recall primal/dual) One-dimensional rules Taylor’s series Searching for ‘zeros’ Multivariate problems –Necessary and Sufficient conditions –Line search methods

4. Reminder: Optimization Overview Variables: Objective: Subject to Constraints: Sometimes additional constraints: –Binary –Integer Today we move from linear to nonlinear constraints and objective functions

5. Single Variable unconstrained problems Remember basic calculus: Find the maximum / minimum of a polynomial function What are optimality conditions? Local vs. global optimum Smooth functions

6. Taylor’s Series Recall the Taylor expansion for a function This leads to necessary and sufficient conditions for optimality

7. Finding the turning points Bisection method Newton’s method

8. Questions and Break

9. Multivariate optimization Taylor’s series in higher dimensions Leads to more very similar optimality conditions!

10. Line search methods Steepest Descent Newton’s method Quasi-newton methods