L5 Optimal Design concepts pt A Homework Review “Minimum” definitions Weierstrass Theorem Gradient vector, Hessian matrix Taylor Series Expansions Summary Test 1 Wed formula sheet

Graphical Solution Sketch coordinate system Plot constraints Determine feasible region Plot f(x) contours Find opt solution x* & opt value f(x*)

5. Find Optimal solution & value Opt. solution point D x*= [4,12] Opt. Value P=4(400)+12(600) P=8800 f(x*)=8800 Figure 3.5 Graphical solution to the profit maximization problem: optimum point D = (4, 12); maximum profit, P = 8800.

Optimization Methods Figure 4.1 Classification of optimization methods.

Global/local optima Global Maximum? Local Maximum? Global Minimum? Local Minimum? Global = absolute Local = relative Figure 4.2 Representation of optimum points. (a) The unbounded domain and function (no global optimum). (b) The bounded domain and function (global minimum and maximum exist). Closed & Bounded

Global (Absolute) Min x* is a global minimum of f(x) IFF f(x*)≤ f(x) for all x in the feasible set Note that “Strong (strict)” versus “weak” minima f(x*)< f(x) f(x*)≤ f(x)

Local (Relative) Min x* is a global minimum of f(x) IFF f(x*)≤ f(x) For all x in a small neighborhood N of x* in the feasible set Where N is defined as:

Global/local optima Global Maximum? f(x*)≤ f(x) Anywhere in S Local Maximum? In small neighborhood N Figure 4.2 Representation of optimum points. (a) The unbounded domain and function (no global optimum). (b) The bounded domain and function (global minimum and maximum exist). Closed & Bounded

Unconstrained Minimum Figure 4.3 Representation of unconstrained minimum for Example 4.1.

Constrained Minimum Figure 4.4 Representation of constrained minimum for Example 4.2.

Existence of a Global Minimum Weierstrass Theorem: 1. If f(x) is continuous 2. On non-empty set S and 3. S is closed and bounded Then f(x) has a global minimum in S recall open interval (1,3) versus closed [1,3]

NOTE: If Weierstrass conditions are not satisfied A global minimum may still exist! The power of the theorem is that it guarantees a global min if conditions are satisfied. e.g. f(x)= x2 on open interval

Gradient is ⊥ to tangent plane Figure 4.5 Gradient vector for f(x1, x2, x3) at the point x*.

Gradient example 1.6 1.2 Figure 4.6 Gradient vector (that is not to scale) for the function f(x) of Example 4.5 at the point (1.8, 1.6).

Second Partial Deriivatives of a function f(x) Hessian Matrix What does the x* mean?

Hessian Example

Taylor Series Expansion Assume f(x) is: 1. Continuous function of a single variable x 2. Differentiable n times 3. x ∈ S, where S is non-empty, closed, and bounded 4. therefore x* is a possible optima

Taylor Series Approximations

Taylor Series – Mult. Variables Estimate f(x) using Taylor Series Let

Change in function Let’s define a change in the function with the symbol ∆ ∆-Delta ∇-Del Let’s define a first order change in the function

Taylor Exp of MV

Summary “Minimum” definitions Weierstrass Theorem Gradient vector, Hessian matrix Taylor Series Expansions