1. Optimization Multi-Dimensional Unconstrained Optimization (Gradient Methods) Examples and Exercises

2. Example 1
Determine whether the stationary point of the following quadratic functions is a local maxima, local minima or saddle point?
A point x* is a stationary point iff
f '(x*) = 0 (if f is a function of one variable)
f (x*) = 0 (if f is a function of >1 variables)

3. Example 1 – Solution
We still have to test if the point is a local maxima, minima or saddle point (continue next page …)

4. Example 1 – Solution (Continue)
(ii) (… continue)

5. Example 1 – Solution (Continue)

6. Example 1 – Solution (Continue)
(continue next page …)

7. Example 1 – Solution (Continue)
(iv) (… continue from previous slide)
We can verify if a matrix is positive definite by checking if the determinants of all its upper left corner sub-matrices are positive.
Since H is neither positive definite nor negative definite (i.e., indefinite), the stationary point is a saddle point.

8. Exercise
For each of the following points, determine whether it is a local maxima, local minima, saddle point, or not a stationary point of
(0, 0)
(1, 0)
(-1, -1)
(1, 1)

9. Solution