1. DECISION MODELING WITH MICROSOFT EXCEL Copyright 2001 Prentice Hall Publishers and Ardith E. Baker Nonlinear Chapter 7 Optimization Part 1

2. Introduction to Nonlinear Optimization Models Not all ___________relationships in business and economics problems are _________. In general, some of the prominent reasons for ________________are: 1. ________________Relationships 2. Nonadditive Relationships 3. Efficiencies or Inefficiencies of__________ Although common, nonlinear models are more difficult to ____________than linear models.

3. Unconstrained Optimization in Two or More Decision Variables Consider the case of two ___________variables, x 1 and x 2 and a given function f(x 1,x 2 ). For the case of 2 decision variables, partial ____________from calculus are used to describe local or global ____________of f. Let f xi denote the first partial derivative, f xi xi denote the second partial derivative, etc. Any point at which all first partial derivatives vanish is called a_____________________.

4. We have the following ___________________for optimality: At a local max or min both partial derivatives must equal ______(i.e., f x 1 = f x 2 = 0). That is, a local maximizer or a local minimizer is always a _____________point. However, not all stationary points provide maxima and___________. In this case, we can employ the second order ___________condition for optimality.

5. first-order ________conditionsoptimality conditions First and second-order tests (called first-order ________conditions and ___________optimality conditions, respectively) can be applied to locate unconstrained local optima for functions of more than one variable. NOTES: The first order conditions are____________; the second-order conditions are sufficient. The second-order conditions __________the first order ones (i.e., the second order conditions assume that x 1 *,x 2 * is a ________ point).

6. The first-derivative test (the necessary condition) says that the _____optima are contained among the stationary points of the__________. The second-derivative test (the _________ condition) allows us to distinguish between local ___________and minimizers, and points that are neither. For a ____________function of n variables, each local optimizer is a stationary point. In order to guarantee that a stationary point is, for example, a _______maximizer, second-order sufficiency conditions must be___________.

7. These two types of optimality conditions have limited practical ______________because: 1. Setting the first partial derivatives equal to zero gives a system of n equations in n ________. Unless the system is_____, it is not easy to find solutions. 2. The second-order sufficiency conditions are complicated and require the evaluation of _______________of certain entries in the matrix of second partial derivatives. criterion First-order necessary conditions in nonlinear optimizers serve as a _________criterion for the hill-climbing optimization methods that search for local__________.

8. function When maximizing a ___________function, any stationary point is a _________maximizer (for a convex function, any stationary point is a global minimizer). In the general case, an optimized solution could be a local maximizer or minimizer or neither, in the _________case, we are guaranteed that any solution is a global_____________.

9. Nonlinear Optimization: A Descriptive Geometric Introduction Software packages (such as Solver) are based on ________hill-climbing (or hill-descent) behavior. For a maximization problem: An _________point is chosen (i.e., a set of numerical values for the decision variables). An uphill direction is determined by approximating the _________of change in all directions (the first partial derivative) of the objective function at that initial point.

10. The method __________when the approximated rates of further OV change in all directions (the first ________derivatives) are close to zero (the first order conditions are__________). For _____________optimization, the method moves from the initial point, along a line in an uphill (____increasing) direction, to the highest point that can be attained on that line. Then, a new uphill direction is defined, and the ____________is continued.

11. unconstrained ____________ Although the focus has been on unconstrained ____________, we are interested in optimizing an objective function subject to______________. Just as in the case of LP modeling, constraints take the form of __________and/or inequalities. However, ____________of the constraints is not assumed in this case. Thus, the general NLP ___________model can be written as follows (f and g i ’s are just symbols for complicated nonlinear functions of the decision variables, x, to compute the OV and each constraint’s LHS).

12. Max f(x 1, x 2, …, x n )(objective) s.t. g 1 (x 1, …, x n ) = b 1 g 2 (x 1, …, x n ) = b 2 g m (x 1, …, x n ) = b m m equality constraints k inequality constraints h 1 (x 1, …, x n ) < r 1 h 2 (x 1, …, x n ) < r 2 h k (x 1, …, x n ) < r k

13. Nonlinear Optimization: A Descriptive Geometric Introduction Consider the following symbolic model: Graphical Analysis: Max x 1 - x 2 s.t. -x 1 2 + x 2 > 1 x 1 + x 2 < 3 -x 1 + x 2 < 2 x 1, x 2 > 0 If even one _____________constraint, objective function, or both exists, then it is a nonlinear model. This is called a ________________(NLP).

14. constraint set To use the _______approach, first we graph each constraint to find the _____________(constraint set) which is the set of points that simultaneously satisfy _____the constraints. This set represents the ___________decisions. We want to find the allowable decision that ___________the objective function. To do this, find the “most uphill” _________of the objective function that still touches the constraint set. The point at which it touches will be an ________ solution to the model.

15. x2x2 x1x1 2 2 There is only one __________constraint and the solution is not at a corner intersection.

16. Noncorner Optima: This graph shows a ________________ nonlinear inequality constrained ___________model in which all constraints are linear and the constraint set is a standard LP_____________. x2x2 x1x1 The objective function is __________.

17. Nonlinear Optimization: A Descriptive Geometric Introduction The following statements hold in either LP or NLP models: Comparisons between LP and NLP: 1. Increasing (decreasing) the ____of a ) constraint loosens the constraint. This cannot contract and may expand the _____________set. 2. Increasing (decreasing) the RHS of a ) constraint __________the constraint. This cannot expand and may ____________the constraint set.

18. 3. ________a constraint cannot hurt and may help the optimal ____________value. 4. ___________a constraint cannot help and may hurt the optimal objective value.

19. the rate of change in OV as the RHS of that constraint_________ In LP, the ____________on a specified constraint was defined as the rate of change in OV as the RHS of that constraint_________, with all other data unchanged. Lagrange Multiplier: In the NLP context, this rate of change is called the__________________. In an LP, the shadow price is ___________for a range of values for the RHS parameter of interest. In the NLP context, this property does not generally hold true. Consider the following simple NLP: Max x 2 s.t. x < b x > 0

20. In order to ___________x 2, make x as large as possible. Thus, the optimal solution is x * = b, and the optimal value of the _______function (OV) is (x * ) 2 = b 2. Thus, the OV is a function of b; OV(b) = b 2. The rate of change of this OV function as b increases is the _________of OV(b), namely 2b. The Lagrange multiplier is not ____________for a range of values of the RHS, b. It varies continuously with b, as might be expected.

21. In an LP, it is always true that there cannot be a ______solution that is not also________. Local versus Global Solutions: This is not usually true for ________NLP models. Such models may have local as well as global solutions. Consider the ___________Max model: x2x2 x1x1

22. local constrained ___________ In the previous graph, the point called “Local Max” is termed a local constrained ___________ because the value of the objective function at this point is no smaller than at its ____________ feasible points. global constrained maximizer The point called “__________” is termed a global constrained maximizer because the value of the ________function at this point is no smaller than at all other feasible points. In general for NLPs, additional conditions must be imposed upon the model, called ________and concavity conditions. These conditions must be satisfied to guarantee that a local constrained _____________is also global.

23. Nonlinear Optimization: A Descriptive Geometric Introduction Many non-linear problems in business and economics are of the following form: Equality-Constrained NLPs: The goal is to maximize or minimize an objective function in n ___________subject to a set of m equality constraints. Maximize or Minimize f(x 1, …, x n ) s.t. g(x 1, …, x n ) = b i i = 1, …, m (m<n)

24. A manufacturer can make a product on either of two machines. Example 1. Let x 1 denote the _______made on machine 1, x 2, the quantity made on machine 2. let ax 1 + bx 1 2 = cost of producing on machine 1 ax 2 + bx 2 2 = cost of producing on machine 2 Find the values of x 1 and x 2 that __________total cost subject to the requirement that total production quantity is to be some given number, say R. The _______________model is: Min ax 1 + bx 1 2 + ax 2 + bx 2 2 s.t. x 1 + x 2 = R

25. The classic _________________model. Let Example 2. Determine the ____________mix that maximizes that person’s utility subject to his/her budget constraint. p 1, p 2, and p 3 denote given ________prices of three goods s 1, s 2, and s 3 be given person-specific ____________ B, a specified constant, denotes a person’s available spending____________ x 1 s1 + x 2 s2 + x 3 s3 denote the person’s “____” derived from consuming x 1 units of good 1, x 2 units of good 2 and x 3 units of good 3.

26. The symbolic model is: Max x 1 s1 + x 2 s2 + x 3 s3 s.t. p 1 x 1 + p 2 x 2 + p 3 x 3 = B

27. Consider the model Example 3. Max x 1 - x 2 s.t. -x 1 2 + x 2 = 1 x2x2 x1x1 1.5 1.0 0.5 1.0 Solution Equality Constraint Optimal objective contour and constraint line are tangent at optimal solution.

28. Here is the Solved spreadsheet model and formulas:

29. Using Solver for NLP Models Solver can be used to enter and ________a model that could contain a nonlinear objective or nonlinear constraint functions or_______. For LP optimization, Solver uses the _________ method to move from corner to corner in the __________region. For NLP optimization, Solver uses a hill-climbing technique based on a “_______________” procedure, called GRG.

30. The following steps describe the GRG procedure: Changing Cells Using the _________values of the decision variables (specified in Solver’s Changing Cells field), the procedure finds a feasible solution. From that initial starting point, a ________is computed that most rapidly improves the OV. ___________ (i.e., changes in values of the decision variables), is made in that direction until a constraint boundary is encountered or until the ______no longer improves. A new direction is __________from that new point, and the process is repeated and continues until no further improvement in any ____________occurs.

31. A restaurant’s average daily expense for advertising is $100, all of which is to be allocated to newspaper ads and radio commercials. Let Example Nonlinear Models Marketing Expenditures: x 1 = avg. no. $/day spent on newspaper ads x 2 = avg. no. $/day spent on radio ads Total annual cost of running the advertising dept: Cost = C(x 1, x 2 ) = 20,000 – 440x 1 – 300x 2 + 20x 1 + 12x 2 + x 1 x 2

32. The goal is to find the restaurant’s ________ allocation that will minimize the total annual cost while adhering to the desired ad ___________of $100 per day. The symbolic model is as follows: Min 20,000 – 440x 1 – 300x 2 + 20x 1 + 12x 2 + x 1 x 2 s.t. x 1 + x 2 = 100 x 1, x 2 > 0

33. Here is the Solved Excel spreadsheet:

34. Here are the Solver parameters and the Sensitivity Report: This value indicates that the initial ___of increase in the annual cost of the adv. dept. would be about $1195 for each additional budget dollar spent daily on ____________.

35. Lagrange Multipliers Lagrange Multipliers in NLP have almost the same interpretation as the ____________in LP. Example Nonlinear Models Economic Interpretation of Lagrange Multipliers and Reduced Gradients: At_______, the value of the Lagrange multiplier is the ___________rate of change in the optimal value of the objective function as the ith RHS, b i, is increased, with all other data______________. In economic terminology, the ith Lagrange multiplier reflects the _______________of the ith resource.

36. The units of measure are: units of the objective function units of RHS of constraint i A __________Lagrange multiplier would indicate that increasing the RHS would initially increase the_____. A __________Lagrange multiplier would indicate that decreasing the RHS would initially ________ the OV. The Lagrange multiplier can be used to ________ what will happen to the _________value if the RHS is changed.

37. Similar to Reduced Cost, the _______________of a variable relates to the upper or lower bound constraints on __________variables. A ____________Reduced Gradient indicates that increasing the variable will initially decrease the OV. A ____________Reduced Gradient indicates that increasing the variable will initially increase the OV. If a decision variable is at its _______bound, the Reduced Gradient should be ___________for the solution to be optimal in a Max model. Otherwise, decreasing the variable would improve the __________function value.

38. If a decision variable is at its lower bound, the Reduced gradient should be __________in a Max model. Otherwise, _____________the variable would improve the objective function value. If a decision variable is between its upper and lower bounds, the Reduced gradient should be ______for the solution to be_________.

39. Consider the following NLP model of a previous LP model. Let Astro and Cosmo Revisited: Profit = (PA – 210)A + (PC – 230)C A = daily production of Astro model TV sets PA = selling price of Astros = 314 – 1.9A +.01A 2 C = daily production of Cosmo model TV sets PC = selling price of Cosmos = 243 -.14C If the unit variable cost of an Astro is $210 and the unit variable cost of a Cosmo is $230, then the total profit is

40. Max (PA – 210)A + (PC – 230)C Here is the symbolic model: PA =.01A 2 – 1.9A + 314 (selling price of Astros) s.t. PC = -.14C + 243 (selling price of Cosmos) A < 70 (capacity of Astro line) C < 50 (capacity of Cosmo line) A + 2C < 120 (department A labor hours) A + C < 90 (department B labor hours) A, PA, C, PC > 0 Nonlinear constraints

41. Here is the Excel spreadsheet model:

42. Here are the Solver parameters :

43. Here is the Sensitivity Report: This constraint is_______. The Lagrange multiplier indicates that _________the OV increases at the rate of about $0.86 per unit of additional labor hours in Dept. A.

44. This graph shows that for the Astro and Cosmo NLP model, the optimal solution does not occur at a _______of the feasible region, though it is on the boundary. 0 20 50 90 C A 7090120

45. Solver may not provide the optimal solution with NLP models. Here are some examples: Example Nonlinear Models Optimality in NLPs: Gulf Coast Oil blends gasoline from three components: Gulf Coast Oil Model: Domestic blend Foreign blend (a blending of two sources) Octane Additive (used only in premium gasoline)

46. The Foreign Blend is transported monthly to Gulf Coast Oil in the single 8,000,000 gallon storage compartment of a large tanker. Component Octane Cost Availability No. per Gallon (000s Gal/Mth) Domestic Blend 85$0.6510,000 Foreign Blend Source 1 93$0.80 * Source 2 97$0.90 * Premium Additive 900$30 50 * Because of the way Gulf Coast Oil obtains Source 1 and Source 2, no more than 8,000,000 gallons of Source 1 plus Source 2 may be obtained per month.

47. Because the oil purchased from the two sources loses its separate identities when “_______” into the storage compartment of the tanker, the model is called a_______________. The goal is to determine how many gallons of Regular, Midgrade, and Premium gasoline to ________each month, given that it must honor minimum supply contracts of 100 thousand gallons of each type of gasoline. In addition, each gasoline is subject to a ________octane requirement. The octane number is a weighted average of the octane numbers of its components where the weights are the ________of each component in the blend.

48. Minimum Octane No Wholesale Price Per Gallon Regular 87$1.18 Midgrade 89$1.25 Premium 94$1.40 Let regular R = thousand gal. of regular gasoline produced midgrade M = thousand gal. of midgrade gas produced premium P = thousand gallons of premium gas produced domestic D = thousand gallons of domestic blend produced additive A = thousand gallons of premium additive produced

49. RD = thousand gallons of domestic blend in regular gasoline RF = thousand gallons of foreign blend in regular gasoline MD = thousand gallons of domestic blend in midgrade gasoline PD = thousand gallons of domestic blend in premium gasoline PF = thousand gallons of foreign blend in premium gasoline MF = thousand gallons of foreign blend in midgrade gasoline S1= thous. gal. purchased, foreign source 1 S2= thous. gal. purchased, foreign source 2

50. OCT = octane number of pooled foreign blend = 93S1 + 97S2 S1 + S2 OCT(S1 + S2) = 93S1 + 97S2 In addition to the nonnegativity constraints, the symbolic model is: Max 1.18R + 1.25M + 1.40P -.65D -.8S1 -.9S2 – 30A s.t. R = RD + RF (composition of reg. gas) M = MD + MF (composition of midgrade gas) P = PD + PF + A (composition of premium gas) D = RD + MD + PD (total domestic blend used) S1 + S2 < 8,000 (tanker capacity for foreign sources)

51. 85RD + OCT*RF > 87R (min octane number for regular gasoline) The next 4 constraints are nonlinear: 85MD + OCT*MF > 89M (min octane number for midgrade gasoline) 85PD + OCT*PF + 900A > 94P (min octane number for premium gasoline) OCT(S1 + S2) = 93S1 + 97S2 (pooling constraint for foreign sources) D < 10,000 (supply limit for domestic blend) A < 50 (supply limit for premium blend) R, M, and P each > 100 (contract delivery min.)

52. Here is an Excel model with example decision values: Solver will use these values to find an initial starting point.

53. Here are the Solver parameters:

54. This is the first solution found by Solver:

55. Solver Results Solver _________ The previous Solver Results message means that Solver has __________its search because the rate of change in the OV was below the Solver _________value for 5 iterations (i.e., the rate of improvement in the OV was too low to continue the optimization method). For NLPs, Solver always starts from a given ______point. Now, run Solver again to force it to begin optimization again to see if it will _______ upon its solution.

56. This is the second solution found by Solver:

57. model Because of the form of the nonlinear constraints, this particular Gulf Coast Oil model is called a ___________model. The starting point for the NLP method is very important for a nonconcave model. You may need to try several different _______points. The best starting points are those near the ______ optimal solution to the model. For nonconcave models, there is no ___________ that the Solver solution is the global optimal one. localglobal In the Gulf Coast Oil example, after two attempts to _______the model, Solver has converged to a local optimum and not the global optimum.

58. The global optimum for this model is ________ to be the following: This solution was found by re-optimizing the model dozens of times, each time using a different starting point set of _______cell values.

59. Here is the Sensitivity Report for the optimal solution:

60. Solver Options For __________or highly nonlinear models, some of the Solver Options can be used to try and improve Solver’s GRG’s ____________tactics. Solver Options Settings This value is used to ________ Solver’s search when the OV is improving very slowly. If the improvement is < the default value of.00001 for 5________, Solver stops. Setting this value smaller forces Solver to continue the optimization method even if the rate of change in ____is small.

61. Quadratic Setting the ________option to Quadratic forces Solver to approximate its estimates of the variable equations in its one-dimensional searches by a __________function instead of a linear (tangent) one. Selecting _______forces Solver to produce a more accurate approximation by estimating each directional ___________using two adjacent points to each iterative solution point instead of just one.

62. Search These Search options determine how Solver chooses the search ________along which an improvement in the OV will be sought. This setting determines how closely the _____ calculations must match the RHS in order for a given constraint to be ____________. If a constraint’s LHS differs from its RHS by an amount less than this setting, then the two are considered equal, and thus, the constraint is_______.

63. The settings shown below are suggested for highly __________or nonconcave NLPs. If the NLP model involves some integer decision variables, then setting the ____________to 0% will force Solver to continue its search.

64. End of Part 1 Please continue to Part 2