L6 Optimal Design concepts pt B Homework Review Single variable minimization Multiple variable minimization Quadratic form Positive definite tests Summary Test 1 Wed formula sheet

Global/local optima Global Maximum? f(x*)≤ f(x) Anywhere in S Local Maximum? In small neighborhood N Closed & Bounded Weierstrass Theorem

Taylor Series Expansion Assume f(x) is: 1. Continuous function of a single variable x 2. Differentiable n times 3. x ∈ S, where S is non-empty, closed, and bounded 4. therefore x* is a possible optima

Taylor Series Approximations

Single variable minimization Given that x* is the minimum of f(x), then any movement away from x* is “uphill”, therefore to guarantee that a move goes uphill First-order necessary condition

Stationary point=max,min,neither Any points satisfying Are called “stationary” points. Those points are a: Min pt, or Max pt, or Neither (i.e. an inflection pt) We need another test!

Second-order sufficient condition Look at second order term Second-order sufficient condition for a minimum

Second-order condition? What if Then f(x*) is not a minimum of f(x*). It is a maximum of f(x*).

Single variable optimization First-order necessary condition Second-order sufficient condition for a minimum Second-order sufficient condition for a maximum

Higher—order tests? What if Then the second order test fails. We need higher order derivatives… Min Max

Second-order necessary conditions Note the “= 0” possibility A pt not satisfying this test is not a min! A pt not satisfying this test is not a max!

Multiple variable optimization If x* is the minimum of f(x), then any movement away from x* is “uphill”. How can we guarantee that for a move in any d, we make away from x*, we go “uphill”?

First-order necessary Condition For x* to be a local minimum: 1rst order term

Second-order sufficient condition For x* to be local minimum: That is H(x*) must be positive definite Remember that has “quadratic form”

Quadratic form of a matrix

Wuad From Ex

Positive definiteness? By inspection Leading principal minors Eigenvalues e.g. by inspection

Find leading principal minors to check PD of A(x)

Principal Minors Test for PD A matrix is positive definite if: 1.No two consecutive minors can be zero AND 2. All minors are positive, i.e. If two consecutive minors are zero The test cannot be used.

Principal Minors Test for ND A matrix is negative definite if: 1.No two consecutive minors can be zero AND 2. Mk<0 for k=odd 3. Mk>0 for k=even If two consecutive minors are zero The test cannot be used.

Eigenvalues Since x should not be zero… we should find values for lambda such that

Eigenvalue test Form Eigenvalue Test Positive Definite (PD) Positive Semi-def (PSD) Indefinite

Eigenvalue example Therefore A is NSD

Summary Single variable minimization Multiple variable minimization Quadratic form Positive definite tests