Non Linear Programming 1 Nonlinear Programming (NLP) Modeling Examples MIT and James Orlin © 2003

Linear Programming Model ASSUMPTIONS: Proportionality Assumption Objective function Constraints Additivity Assumption MIT and James Orlin © 2003

What is a non-linear program? maximize 3 sin x + xy + y3 - 3z + log z Subject to x2 + y2 = 1 x + 4z  2 z  0 A non-linear program is permitted to have non-linear constraints or objectives. A linear program is a special case of non-linear programming! MIT and James Orlin © 2003

Nonlinear Programs (NLP) Nonlinear objective function f(x) and/or Nonlinear constraints gi(x). Today: we will present several types of non-linear programs. MIT and James Orlin © 2003

Unconstrained Facility Location This is the warehouse location problem with a single warehouse that can be located anywhere in the plane. Distances are “Euclidean.” 2 4 6 8 10 12 14 16 y C (2) (7) B A (19) P ? D (5) x Loc. Dem. A: (8,2) 19 B: (3,10) 7 C: (8,15) 2 D: (14,13) 5 P: ? MIT and James Orlin © 2003

An NLP Costs proportional to distance; known daily demands d(P,A) = … d(P,D) = minimize 19 d(P,A) + … + 5 d(P,D) subject to: P is unconstrained MIT and James Orlin © 2003

Here are the objective values for 55 different locations. MIT and James Orlin © 2003

Facility Location. What happens if P must be within a specified region? 2 4 6 8 10 12 14 16 y C (2) (7) B A (19) P ? D (5) x MIT and James Orlin © 2003

The model Minimize + …+ Subject to x  7 5  y  11 x + y  24 MIT and James Orlin © 2003

0-1 integer programs as NLPs minimize Sj cj xj subject to Sj aij xj = bi for all i xj is 0 or 1 for all j is “nearly” equivalent to minimize Sj cj xj + 106 Sj xj (1- xj). subject to Sj aij xj = bi for all i 0  xj  1 for all j MIT and James Orlin © 2003

Some comments on non-linear models The fact that non-linear models can model so much is perhaps a bad sign How can we solve non-linear programs if we have trouble with integer programs? Recall, in solving integer programs we use techniques that rely on the integrality. Fact: some non-linear models can be solved, and some are WAY too difficult to solve. More on this later. MIT and James Orlin © 2003

Variant of exercise from Bertsimas and Freund Buy a machine and keep it for t years, and then sell it. (0  t  10) all values are measured in $ million Cost of machine = 1.5 Revenue = 4(1 - .75t) Salvage value = 1/(1 + t) MIT and James Orlin © 2003

MIT and James Orlin © 2003

How long should we keep the machine? Work with your partner on how long we should keep the machine, and why? MIT and James Orlin © 2003

Non-linearities Because of Time Discount rates decreasing value of equipment over time wear and tear, improvements in technology Tax implications (Depreciation) Salvage value Secondary focus of the previous model(s): Finding the right model can be subtle MIT and James Orlin © 2003

Non-linearities in Pricing The price of an item may depend on the number sold quantity discounts for a small seller price elasticity for monopolist Complex interactions because of substitutions: Lowering the price of GM automobiles will decrease the demand for the competitors MIT and James Orlin © 2003

Non-linearities because of congestion The time it takes to go from MIT to Harvard by car depends non-linearly on the congestion. As congestion increases just to its limit, the traffic sometimes comes to a near halt. MIT and James Orlin © 2003

Non-linearities because of “penalties” Consider any linear equality constraint: e.g., 3x1 + 5x2 + 4x3 = 17 Suppose it is a “soft” constraint and we permit solutions violating it. We can then write: 3x1 + 5x2 + 4x3 - y = 17 And we may include a term of –10y2 in the objective function. This adds flexibility to the solution by discourages violation of our “goals” MIT and James Orlin © 2003

Portfolio Optimization In the following slides, we will show how to model portfolio optimization as NLPs The key concept is that risk can be modeled using non-linear equations Since this is one of the most famous applications of non-linear programming, we cover it in much more detail MIT and James Orlin © 2003

Risk vs. Return In finance, one trades of risk and return. For a given rate of return, one wants to minimize risk. For a given rate of risk, one wants to maximize return. Return is modeled as expected value. Risk is modeled as variance (or standard deviation.) MIT and James Orlin © 2003

Expectations Add Suppose that X and Y are random variables E(X + Y) = E(X) + E(Y) Interpretation: Suppose that the expected return in one year for Stock 1 is 9%. Suppose that the expected return in one year for Stock 2 is 10% If you put $100 in Stock 1, and $200 in Stock 2, your expected return is $9 + $20 = $29. MIT and James Orlin © 2003

Variances do not add (at least not simply) Suppose that X and Y are random variables Var(aX + bY) = a2 Var(X) + b2 Var(Y) + 2ab Cov(X, Y) Example. The risk of investing in “umbrellas” and “sunglasses” is less than the risk of either investment by itself. In general: Var(X1 + X2 + …+ Xn) = MIT and James Orlin © 2003

Reducing risk Diversification is a method of reducing risk, even when investments are positively correlated (which they often are). If only two investments are made, then the risk reduction depends on the covariance. MIT and James Orlin © 2003

Portfolio Selection (cont’d) Two Methods are commonly used: Min Risk s.t. Expected Return ³ Bound Max Expected Return - q (Risk) where q reflects the tradeoff between return and risk. MIT and James Orlin © 2003

Portfolio Selection Example There are 3 candidate assets for out portfolio, X, Y and Z. The expected returns are 30%, 20% and 8% respectively (if possible we would like at least a 12% return). Suppose the covariance matrix is: What are the variables? Let X,Y,Z be percentage of portfolio of each asset. MIT and James Orlin © 2003

Portfolio Selection Example Min st Max MIT and James Orlin © 2003

More on Portfolio Selection There can be institutional constraints as well, especially for mutual funds. No more than 15% in the energy sector Between 20% to 25% high growth At most 3% in any one firm etc. We end up with a large non-linear program. The unconstrained version becomes the “CapM model” in finance. Portfolio Example MIT and James Orlin © 2003

Regression Find the best linear fit for estimating the midterm grade from the homework grades MIT and James Orlin © 2003

Writing regression as an NLP Minimize Sj (rj)2 subject to r1 = (91x + y) – 89 r2 = (80x + y) – 97.5 r3 = (61x + y) – 58.5 … r9 = (50x + y) – 67 Minimize Sj (rj)2 subject to rj = Hj x + y – Mj for each j In an optimization framework, one can constrain coefficients. MIT and James Orlin © 2003

r2 =.082 MIT and James Orlin © 2003

r2 =.29 MIT and James Orlin © 2003

An application of regression to finance A famous application in Finance of determining the best linear fit is determining the b of a stock. CAPM assumes that the return of a stock s in a given time period is rs = a + b rm + e, rs = return on stock s in the time period rm = return on market in the time period b = a 1% increase in stock market will lead to a b% increase in the return on s (on average) MIT and James Orlin © 2003

Regression, and estimating b Return on Stock A vs. Market Return 80.00% 60.00% 40.00% 20.00% Stock 0.00% -40.00% -20.00% 0.00% 20.00% 40.00% 60.00% 80.00% -20.00% What is the best linear fit for this data? What does one mean by best? -40.00% -60.00% Market MIT and James Orlin © 2003

Regression, and estimating b Return on Stock A vs. Market Return 80.00% 60.00% 40.00% 20.00% Stock 0.00% -40.00% -20.00% 0.00% 20.00% 40.00% 60.00% 80.00% -20.00% Market The value b is the slope of the regression line. Here it is around .6 (lower expected gain than the market, and lower risk.) -40.00% -60.00% MIT and James Orlin © 2003

Solving NLP’s by Excel Solver MIT and James Orlin © 2003

Summary Applications of NLP to location problems, portfolio management, regression Non-linear programming is very general and very hard to solve Special case of convex minimization NLP is easier, because a local minimum is a global minimum MIT and James Orlin © 2003