1. Non Linear Programming 1
Nonlinear Programming (NLP)
Modeling Examples
MIT and James Orlin © 2003

2. Linear Programming Model
ASSUMPTIONS:
Proportionality Assumption
Objective function
Constraints
Additivity Assumption
MIT and James Orlin © 2003

3. What is a non-linear program?
maximize sin x + xy + y3 - 3z + log z Subject to x2 + y2 = x + 4z  z  0
A non-linear program is permitted to have non-linear constraints or objectives.
A linear program is a special case of non-linear programming!
MIT and James Orlin © 2003

4. Nonlinear Programs (NLP)
Nonlinear objective function f(x) and/or Nonlinear constraints gi(x).
Today: we will present several types of non-linear programs.
MIT and James Orlin © 2003

5. Unconstrained Facility Location
This is the warehouse location problem with a single warehouse that can be located anywhere in the plane. Distances are “Euclidean.” 2 4 6 8 10 12 14 16 y C
(2)
(7) B A
(19)
P ?
D (5) x
Loc. Dem.
A: (8,2)
B: (3,10)
C: (8,15)
D: (14,13) 5
P: ?
MIT and James Orlin © 2003

6. An NLP
Costs proportional to distance; known daily demands
d(P,A) = …
d(P,D) =
minimize 19 d(P,A) + … + 5 d(P,D) subject to: P is unconstrained
MIT and James Orlin © 2003

7. Here are the objective values for 55 different locations.
MIT and James Orlin © 2003

8. Facility Location. What happens if P must be within a specified region?2 4 6 8 10 12 14 16 y C
(2)
(7) B A
(19)
P ?
D (5) x
MIT and James Orlin © 2003

9. The model Minimize + …+ Subject to x  7 5  y  11 x + y  24
MIT and James Orlin © 2003

10. 0-1 integer programs as NLPs
minimize Sj cj xj
subject to Sj aij xj = bi for all i
xj is 0 or 1 for all j
is “nearly” equivalent to
minimize Sj cj xj Sj xj (1- xj).
subject to Sj aij xj = bi for all i
0  xj  1 for all j
MIT and James Orlin © 2003

11. Some comments on non-linear models
The fact that non-linear models can model so much is perhaps a bad sign
How can we solve non-linear programs if we have trouble with integer programs?
Recall, in solving integer programs we use techniques that rely on the integrality.
Fact: some non-linear models can be solved, and some are WAY too difficult to solve. More on this later.
MIT and James Orlin © 2003

12. Variant of exercise from Bertsimas and Freund
Buy a machine and keep it for t years, and then sell it. (0  t  10)
all values are measured in $ million
Cost of machine = 1.5
Revenue = 4( t)
Salvage value = 1/(1 + t)
MIT and James Orlin © 2003

13. MIT and James Orlin © 2003

14. How long should we keep the machine?
Work with your partner on how long we should keep the machine, and why?
MIT and James Orlin © 2003

15. Non-linearities Because of Time
Discount rates
decreasing value of equipment over time
wear and tear, improvements in technology
Tax implications (Depreciation)
Salvage value
Secondary focus of the previous model(s): Finding the right model can be subtle
MIT and James Orlin © 2003

16. Non-linearities in Pricing
The price of an item may depend on the number sold
quantity discounts for a small seller
price elasticity for monopolist
Complex interactions because of substitutions:
Lowering the price of GM automobiles will decrease the demand for the competitors
MIT and James Orlin © 2003

17. Non-linearities because of congestion
The time it takes to go from MIT to Harvard by car depends non-linearly on the congestion.
As congestion increases just to its limit, the traffic sometimes comes to a near halt.
MIT and James Orlin © 2003

18. Non-linearities because of “penalties”
Consider any linear equality constraint:
e.g., 3x1 + 5x2 + 4x3 = 17
Suppose it is a “soft” constraint and we permit solutions violating it. We can then write:
3x1 + 5x2 + 4x3 - y = 17
And we may include a term of –10y2 in the objective function.
This adds flexibility to the solution by discourages violation of our “goals”
MIT and James Orlin © 2003

19. Portfolio Optimization
In the following slides, we will show how to model portfolio optimization as NLPs
The key concept is that risk can be modeled using non-linear equations
Since this is one of the most famous applications of non-linear programming, we cover it in much more detail
MIT and James Orlin © 2003

20. Risk vs. Return
In finance, one trades of risk and return. For a given rate of return, one wants to minimize risk.
For a given rate of risk, one wants to maximize return.
Return is modeled as expected value. Risk is modeled as variance (or standard deviation.)
MIT and James Orlin © 2003

21. Expectations Add Suppose that X and Y are random variables
E(X + Y) = E(X) + E(Y)
Interpretation:
Suppose that the expected return in one year for Stock 1 is 9%.
Suppose that the expected return in one year for Stock 2 is 10%
If you put $100 in Stock 1, and $200 in Stock 2, your expected return is $9 + $20 = $29.
MIT and James Orlin © 2003

22. Variances do not add (at least not simply)
Suppose that X and Y are random variables
Var(aX + bY) = a2 Var(X) + b2 Var(Y) + 2ab Cov(X, Y)
Example. The risk of investing in “umbrellas” and “sunglasses” is less than the risk of either investment by itself.
In general:
Var(X1 + X2 + …+ Xn) =
MIT and James Orlin © 2003

23. Reducing risk
Diversification is a method of reducing risk, even when investments are positively correlated (which they often are).
If only two investments are made, then the risk reduction depends on the covariance.
MIT and James Orlin © 2003

24. Portfolio Selection (cont’d)
Two Methods are commonly used:
Min Risk
s.t. Expected Return ³ Bound
Max Expected Return - q (Risk)
where q reflects the tradeoff between return and risk.
MIT and James Orlin © 2003

25. Portfolio Selection Example
There are 3 candidate assets for out portfolio, X, Y and Z. The expected returns are 30%, 20% and 8% respectively (if possible we would like at least a 12% return). Suppose the covariance matrix is:
What are the variables?
Let X,Y,Z be percentage of portfolio of each asset.
MIT and James Orlin © 2003

26. Portfolio Selection Example
Min st
Max
MIT and James Orlin © 2003

27. More on Portfolio Selection
There can be institutional constraints as well, especially for mutual funds.
No more than 15% in the energy sector
Between 20% to 25% high growth
At most 3% in any one firm
etc.
We end up with a large non-linear program.
The unconstrained version becomes the “CapM model” in finance.
Portfolio Example
MIT and James Orlin © 2003

28. Regression
Find the best linear fit for estimating the midterm grade from the homework grades
MIT and James Orlin © 2003

29. Writing regression as an NLP
Minimize Sj (rj)2
subject to
r1 = (91x + y) – 89
r2 = (80x + y) – 97.5
r3 = (61x + y) – 58.5 …
r9 = (50x + y) – 67
Minimize Sj (rj)2
subject to
rj = Hj x + y – Mj for each j
In an optimization framework, one can constrain coefficients.
MIT and James Orlin © 2003

30. r2 =.082
MIT and James Orlin © 2003

31. r2 =.29
MIT and James Orlin © 2003

32. An application of regression to finance
A famous application in Finance of determining the best linear fit is determining the b of a stock.
CAPM assumes that the return of a stock s in a given time period is
rs = a + b rm + e,
rs = return on stock s in the time period
rm = return on market in the time period
b = a 1% increase in stock market will lead to a b% increase in the return on s (on average)
MIT and James Orlin © 2003

33. Regression, and estimating b
Return on Stock A vs. Market Return
80.00%
60.00%
40.00%
20.00%
Stock
0.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
80.00%
-20.00%
What is the best linear fit for this data? What does one mean by best?
-40.00%
-60.00%
Market
MIT and James Orlin © 2003

34. Regression, and estimating b
Return on Stock A vs. Market Return
80.00%
60.00%
40.00%
20.00%
Stock
0.00%
-40.00%
-20.00%
0.00%
20.00%
40.00%
60.00%
80.00%
-20.00%
Market
The value b is the slope of the regression line. Here it is around .6 (lower expected gain than the market, and lower risk.)
-40.00%
-60.00%
MIT and James Orlin © 2003

35. Solving NLP’s by Excel Solver
MIT and James Orlin © 2003

36. Summary
Applications of NLP to location problems, portfolio management, regression
Non-linear programming is very general and very hard to solve
Special case of convex minimization NLP is easier, because a local minimum is a global minimum
MIT and James Orlin © 2003