1. L6 Optimal Design concepts pt B
Homework
Review
Single variable minimization
Multiple variable minimization
Quadratic form
Positive definite tests
Summary
Test 1 Wed
formula sheet

2. Global/local optima Global Maximum? f(x*)≤ f(x) Anywhere in S
Local Maximum?
In small neighborhood N
Closed & Bounded
Weierstrass Theorem

3. Taylor Series Expansion
Assume f(x) is:
1. Continuous function of a single variable x
2. Differentiable n times
3. x ∈ S, where S is non-empty, closed, and bounded
4. therefore x* is a possible optima

4. Taylor Series Approximations

5. Single variable minimization
Given that x* is the minimum of f(x), then any movement away from x* is “uphill”, therefore to guarantee that a move goes uphill
First-order necessary condition

6. Stationary point=max,min,neither
Any points satisfying
Are called “stationary” points.
Those points are a:
Min pt, or
Max pt, or
Neither (i.e. an inflection pt)
We need another test!

7. Second-order sufficient condition
Look at second order term
Second-order sufficient condition for a minimum

8. Second-order condition?
What if
Then f(x*) is not a minimum of f(x*).
It is a maximum of f(x*).

9. Single variable optimization
First-order necessary condition
Second-order sufficient
condition for a minimum
Second-order sufficient
condition for a maximum

10. Higher—order tests? What if
Then the second order test fails. We need higher order derivatives…
Min
Max

11. Second-order necessary conditions
Note the “= 0” possibility
A pt not satisfying this
test is not a min!
A pt not satisfying this
test is not a max!

12. Multiple variable optimization
If x* is the minimum of f(x), then any movement away from x* is “uphill”.
How can we guarantee that for a move in any d, we make away from x*, we go “uphill”?

13. First-order necessary Condition
For x* to be a local minimum:
1rst order term

14. Second-order sufficient condition
For x* to be local minimum:
That is H(x*) must be positive definite
Remember that
has “quadratic form”

15. Quadratic form of a matrix

16. Wuad From Ex

17. Positive definiteness?
By inspection
Leading principal minors
Eigenvalues
e.g. by inspection

18. Find leading principal minors to check PD of A(x)

19. Principal Minors Test for PD
A matrix is positive definite if:
1.No two consecutive minors can be zero AND
2. All minors are positive, i.e.
If two consecutive minors are zero
The test cannot be used.

20. Principal Minors Test for ND
A matrix is negative definite if:
1.No two consecutive minors can be zero AND
2. Mk<0 for k=odd
3. Mk>0 for k=even
If two consecutive minors are zero
The test cannot be used.

21. Eigenvalues
Since x should not be zero… we should find values for lambda such that

22. Eigenvalue test Form Eigenvalue Test Positive Definite (PD)
Positive Semi-def (PSD)
Indefinite

23. Eigenvalue example
Therefore A is NSD

24. Summary Single variable minimization Multiple variable minimization
Quadratic form
Positive definite tests