1. DECISION MODELING WITH MICROSOFT EXCEL Copyright 2001 Prentice Hall Publishers and Ardith E. Baker Nonlinear Chapter 7 Optimization Part 2

2. There is no single preferred ___________method for optimizing NLPs. Solvability of NLP Models Three classes of procedures currently seem to be most useful: GRG (Generalized __________Gradient)SLP (_____________Linear Programming) SQP (Successive _________Programming) Nonlinear models are divided into two classes: 1. Those that can be_____________ 2. Those that one can try to optimize These models must typically conform to certain qualifications of structure and size.

3. Hierarchy of increasing computational difficulty: Concave or convex quadratic programs General concave or convex programs More general nonlinear programs NONLINEAR PROGRAMS II I III IV LINEAR PROGRAMS

4. __________set of points __________set of points is any set of points that has the following property: Solvability of NLP Models Nonlinear Programs that can be Solved: Concave and Convex Programs Consider all possible _______of points in the set, and consider the line segment _____________any such pair. All such line segments must lie entirely within the set. Any ______________set for a linear program is a convex set. The _______programs that we can be reasonably sure of solving, must also have convex constraint sets.

5. ConvexNonconvex

6. Concave and Convex Functions: _________function A _________function is shaped like an upside- down bowl. concave function A concave function has the property that the line ___________connecting any two points on the graph of the function never enters the space ________the graph. _________function A _________function is shaped like a bowl. convex function A convex function has the property that the line segment connecting any two _______on the graph of the function never enters the space _________the graph.

7. Suppose that we have a nonlinear program with only ____________constraints: If the LHS constraint function associated with each __________is concave, the constraint set will be a convex set.

8. -212 x such that g(x) < 2 is convex 2 3 g(x) = x 2 + 1 Convex set g(x) is convex and the set defined by g(x) < 2 is convex. The set defined by g(x) > 2 is not convex.

9. convex Note that the term __________applies only to functions, whereas convex can apply either to a __________or to a set of points. concave NLP A concave NLP is a ______model with a concave objective function and a convex _____________set. convex NLP A convex NLP is a _____model with a convex objective function and a ________constraint set. Because of____________, an LP satisfies both of the conditions above. An important characteristic of concave (or convex) ______is that for such models any local solution is necessarily a ___________solution.

10. Nonlinear Programs that we Try to Solve: General NLPs are often called _______________ (the convexity and concavity properties are absent). When solving, any ____________will generally terminate at a point at which the necessary (i.e., __________) optimality conditions are satisfied. For a concave or convex program, such a point is guaranteed to be a _________optimizer, but for general NLPs, this need not be true.

11. Feasible Region x1x1 x2x2 x3x3 x*x* x C P F(x) The Gulf Coast Model (a __________constrained Max model) is illustrated above. The objective function is neither concave nor convex. The solution is x * but the optimizer may _________at any of the points x 1, x 2, x 3, or x *.

12. How can you tell whether a ________program in many variables is concave, convex, or neither? Consider the following answers: 1. Sometimes there are mathematical _____ that can be applied to the problem _________to determine whether they are concave, convex, or___________. 2. Sometimes __________intuition is used to assert that some phenomenon reflects diminishing _______returns or increasing marginal costs, and hence the associated function is concave or___________.

13. 3. In many real problems nothing is done to address the question. The model is just ___________and then inquiries as to the practical usefulness of the terminal point are made. Solver is frequently _________ from many different initial points, to explore the possibility of producing a better solution with higher_______. Finally, avoid using certain non-smooth Excel __________[e.g., IF(), ABS(), CHOOSE(), MIN(), MAX(), VLOOKUP()] that typically produce extreme ___________behavior. These functions destroy linearity and can cause “kinks” or dis- continuities in the objective function or _______ values for some values of the decision variables.

14. Highly nonlinear models occur frequently in _______situations. In addition, many real-world models make heavy use of Excel’s __________ functions. Introduction to Evolutionary Solver The presence of 1. highly ________Excel models often having 2. nonsmooth ____________and 3. many local_____________ usually means that the convexity and concavity properties necessary for Solver to reliably optimize and NLP to find a global optimum are _______________.

15. These three irregularities are shown in this Excel model. The _______is the product of two nonsmooth nonlinear functions in two __________ variables X2 and X2. Here, we have used Sover’s GRG ___________at an arbitrary pair of values for X1 and X2.

16. Here are the Solver parameters:

17. This is a chart of the Y Total objective function for X1 and X2. The chart verifies the ____________ of Y Total. There are numerous _______optima. The unique global optimum occurs at X1=1.42 and X2=1.41.

18. Here, X1 and X2 were first _____________to 3.3. Solver’s GRG nonlinear optimizer converged to a ______optimum far below the global one.

19. In this run, X1 and X2 were each initialized to 1, an ______point very close to the global_______. However, Solver failed to find it. Recent research in optimization theory has focused on creating ________procedures that are more immune to the previously described problems.

20. Evolutionary Solver optimizer provides a __________way to select good initial starting values for the decision variables. Solver’s Evolutionary search procedure for ____________is different from the LP, ILP and NLP methods in that 1. It relies in part on __________determined starting points (a _________________ optimizer). 2. It keeps a large set of results, called a ________of candidate solutions, not all of which are good solutions. The population is used to help create new _______solution points.

21. 3. It makes ________changes in one or more members of the population at times to create new “________” candidate starting points that may be far removed from other members of the population. 4. Like sexual reproduction, elements of __________solutions in the population are combined with each other by a DNA-like strand _______operation in order to create a new solution candidate with some of the features of each parent solution. 5. Any constraint ________produced by a new solution are reflected by subtracting or adding a penalty to the solution OV. This modified OV becomes that solutions “____________” measure.

22. 6. Much like natural selection, offspring starting points that do not produce ________OV and do not help produce other candidate starting points with improved OV are ultimately ______from the population. A limited version of Solver Evolutionary procedure is contained in the Premium Edition solver for Education.

23. The default setting requires that all the ________ variables have both upper and lower bound ________which must be added as new constraints to the Solver Parameters dialog if not already present. The progress of the Evolutionary Solver is marked by a ______________at the bottom of the Excel window.

24. Instead of an_________, Evolutionary Solver can be thought of as a search engine that intelligently searches the _____________of candidate model solutions looking for better solutions. Unfortunately, you will never know how close the “_______” solution will be to a global optimum solution. When Evolutionary Solver terminates, a Solver Results dialog appears offering a ________ report:

25. Here is the resulting solution for the Gulf Coast model.

26. Here is the Population Report for the model. This report summarizes the characteristics of the population of solutions it created. This portion of the report tabulates the best and the average values of all decision variables tried and several measures of their variability.

27. Optimizer Attribute LP ILP QP NLP Evolutionary Range of Models Supported Range of Models Supported NarrowWider Wider Wide Widest Speed of Optimization Speed of Optimization Very Fast Slow Fast Slow Extremely Slow Finds Global Optimum Finds Global Optimum Yes Yes Yes * ** Sensitivity Report Sensitivity Report Yes No Yes Yes No Scales to Large Models Scales to Large Models Yes *** Yes * No Tolerates nonsmoothExcel Tolerates nonsmooth Excel No No No No Yes functions *Only for well behaved models ** Unlikely, except for well behaved models ***Only for models with special structure

28. A ____________programming model has the important _________(convexity) property that avoids the optimization difficulties inherent with more generalized_________. Introduction to Quadratic Programming linear quadratic Whereas ___________programming strives to maximize or minimize the value of a linear objective function subject to a set of linear constraints, the __________programming model strives to maximize or minimize the value of a quadratic objective function subject to a set of linear constraints.

29. Quadratic Functions: Quadratic Functions: Here are some examples of quadratic functions: 9x 1 2 + 4x 1 + 7 3x 1 2 – 4x 1 x 2 + 15x 2 2 + 20x 1 – 13x 2 – 14 In general, a quadratic function in N variables is:  A i x i 2 +  i=1 N N-1 j = i+1 N B ij x i x j +Cixi +DCixi +D N i=1 Note that when A i and B ij are 0, then the function is linear.

30. Geometric Representation: Geometric Representation: Consider the following symbolic QP model: Min (x 1 - 6) 2 + (x 2 - 8) 2 s.t. x 1 < 7 x 2 < 5 x 1 + 2x 2 < 12 x 1 + x 2 < 9 x 1, x 2 > 0 The objective function can be rewritten as: x 1 2 - 12x 1 + 36 + x 2 2 - 16x 2 + 64 (x 1 - 6) 2 + (x 2 - 8) 2 = k Also, (x 1 - 6) 2 + (x 2 - 8) 2 = k is the equation of a circle with radius k and center at the point (6,8).

31. The contours of the objective function are concentric circles around the point (6,8).

32. Comparison with LP: Comparison with LP: Like NLP models in general, there need not be an optimal _________solution. As a direct result, there may be more positive variables in the ________solution than there are __________constraints. There are 2 approaches to optimizing QP models: 1. Use a general ___________programming optimizer, such as Solver. 2. Use a specially written _____________ programming optimizer. Solver Solution of QP Problems:

33. Here is an example of an QP model optimized using Solver:

34. Here is the resulting Sensitivity Analysis: These values represent the ______________in the OV as the ith RHS is__________, with all other data unchanged. In the _______of direct upper or lower bound constraints, these values apply to a non- negative variable whose optimal value is____. The reduced gradient is the rate at which the objective value is “______” as that variable is forced to assume positive values in an optimal solution.

35. Portfolio selection is a fundamental model in modern___________. Portfolio Selection The Portfolio Model: An investor has P dollars to invest in a set of n stocks and would like to know how much to invest in each stock. The chosen collection is called the investor’s _____________. There are ______________goals in this model: a large expected return and a small risk

36. An example of a ___________is: return Suppose an investment of D i dollars is put into asset i. Over some specified time period, D i dollars grows to 1.3D i. Then the return over that period is (1.3D i - D i )/D i = 0.3. _____is measured by the variance of the return on the portfolio. Since the portfolio manager seeks low risk and high expected return, one way to __________the model is to minimize the variance of the return (i.e., risk) subject to a given _________bound on expected return. This model turns out to be a quadratic _______________model.

37. Formulating the Portfolio Model: Let x i be the proportion of the portfolio invested in stock i P be the amount in dollars to invest  i 2 = variance of yearly returns from stock i, i = 1, 2  12 = covariance of yearly returns from stocks 1 and 2 R i = expected yearly return from stock i, i = 1, 2 G = lower bound on expected yearly return from total investment S i = upper bound on investment stock i, i = 1, 2

38. Model Facts: of the yearly returns from stock i 1. The __________of the yearly returns from stock i is a number describing the “_________” of these returns from year to year. of the yearly returns from stock i 2. The __________of the yearly returns from stock i is a number that describes the extent to which the returns of the two stocks move up or down together. of the portfolio 3. The _________________of the portfolio is defined as the number x 1 R 1 + x 2 R 2. of the return of the portfolio 4. The ________of the return of the portfolio is defined as the number 2  12 x 1 +  2 2 x 2 2. of the portfolio 5. The __________________of the portfolio is the square root of the variance.

39. For a two-stock example, the symbolic model is: Min  1 2 x 1 2 +2  12 x 1 x 2 +  2 2 x 2 2 (Variance of return) s.t. x 1 + x 2 = 1 (All funds must be invested) x 1 R 1 + x 2 R 2 > G (Lower bound on the expected return of the portfolio) x 1 < S 1 (Upper bound on investments in stock 1) x 2 < S 2 (Upper bound on investments in stock 2) x 1, x 2 > 0 (nonnegativity implies that “short selling” of a stock is not allowed) Let  1 2 = 0.09 R 1 = 0.06 S 1 = 0.75 G = 0.03  2 2 = 0.06 R 2 = 0.02 S 2 = 0.90  12 = 0.02

40. This graph shows the feasible set for the model: 26.55 o k= 4.54 Contour k= 2 Contour.5.751.0 6x 1 + 2x 2 > 3 (x 1 =.25, x 2 =.75) Optimal Solution (x 1 * =.36, x 2 * =.64) (x 1 =.75, x 2 =.25) x 1 + x 2 = 1.9 1.0 x2x2 x1x1

41. Here is the spreadsheet model for the problem:

42. In this three-asset problem, data will be used to estimate the _________in this model. Solver will be used to optimize the model. A Portfolio Example with Data Let X = fraction of asset x in the portfolio Y = fraction of asset y in the portfolio Z = fraction of asset z in the portfolio In the real world, the____________________, variances, and covariances must be estimated with ____________data. Formulating the Model:

43. In general, if n periods of data are available, there will be, for each _____i, an actual historical return R i associated with period t where t ______ from 1 to n. The expected _______return from asset i (i.e. the average of the asset’s historical__________) is estimated with:  i=1 n R i = 1n1n RitRit _________of the variance of return for asset i =  t=1 n (R i t – R i ) 1n1n The expected periodic historic returns R i t, are used to estimate ___________and covariances.

44. Estimate of the covariance of returns for assets i and j =  t=1 n (R i t – R i ) 1n1n (R j t – R j ) G = lower bound on expected return of portfolio S i = upper bound on the fraction of asset i that can be in the portfolio

45. The quadratic programming formulation of the three-asset model is as follows: Min  x 2 X 2 +  y 2 Y 2 +  z 2 Z 2 + 2  xy XY + 2  xz XZ + 2  yz YZ s.t. R x X + R y Y + R z Z > G X + Y + Z = 1 X < S x Y < S y Z < S z X, Y, Z > 0

46. Consider the following historical stock returns: The return for year n is defined by: (closing price, n) – (closing price, n-1) + (dividends, n) (closing price, n-1) Solver Solution:

47. The spreadsheet model is given below:

48. Here are the Solver parameters and Sensitivity Report:

49. The variance of _________yearly return is about 0.0205 and the standard deviation is 0.0205 = 14.33%. Assuming that the portfolio returns are ________ distributed with a mean of 15% and standard deviation of 14.33%, then (at a 95% _________ level) the expected return would roughly be between –13.7% and +43.7% (i.e., 15% + 2*14.33%).

50. The returns over the ensuing three years for the three stocks and the actual portfolio returns are: AT&T 10.3% 3.9% 3.0% GM 51.2% -5.0% -20.0% USS 64.7% 32.2% -26.6% Portfolio Returns 31.1% 3.9% -8.6% Stock Year 1 Year 2 Year 3 The __________multiplier indicates that a 1% increase in expected return would lead to an increase of 0.00324 in_________. Hence, the new portfolio variance would be about 0.0238.

51. This graph (a ____________quadratic convex function) shows that tightening the expected return ___________(i.e., increasing the expected return, b) hurts the OV more and more.

52. End of Part 2 Please continue to Part 3