Optimization Multi-Dimensional Unconstrained Optimization (Gradient Methods) Examples and Exercises

Example 1 Determine whether the stationary point of the following quadratic functions is a local maxima, local minima or saddle point? A point x* is a stationary point iff f '(x*) = 0 (if f is a function of one variable) f (x*) = 0 (if f is a function of >1 variables)

Example 1 – Solution We still have to test if the point is a local maxima, minima or saddle point (continue next page …)

Example 1 – Solution (Continue) (ii) (… continue)

Example 1 – Solution (Continue)

Example 1 – Solution (Continue) (continue next page …)

Example 1 – Solution (Continue) (iv) (… continue from previous slide) We can verify if a matrix is positive definite by checking if the determinants of all its upper left corner sub-matrices are positive. Since H is neither positive definite nor negative definite (i.e., indefinite), the stationary point is a saddle point.

Exercise For each of the following points, determine whether it is a local maxima, local minima, saddle point, or not a stationary point of (0, 0) (1, 0) (-1, -1) (1, 1)

Solution