1 OR II GSLM 52800

2 Outline  separable programming  quadratic programming

3 Separable Programs  a separable NLP if  f and all g j are separable functions  0  x i   i, a finite number

4 Idea of Separable Program  min f(x), s.t.s.t.  g j (x)  0 for j = 1, …, m.  hard NLP but simple LP problems  approximating a separable NL program by a LP  a non-linear function by a piecewise linear one

5 A Fact About Convex Functions  f: a convex function  for any  > 0, possible to find a sequence of piecewise linear convex functions f n such that |f  f n |  

6 Example 6.1  a separable program

7 Example 6.1  approximating by a piecewise linear function  two representations, -form and  -form pointsOABC x1x y C B A O

8  Form  a piecewise linear function with (segment) break points  any point = the convex combination of the two break points of the linear segment  i (  0) = the weight of break point i 1 C B A O

9 Example 6.1  the program becomes  the last but one type of constraints is non-linear

10 Fact  nonlinear constraint: at most two adjacent i taking non-zero values  possible to have only one i = 1  for convex f and g j : no need to have the non- linear constraint  non-optimal to have more than two non-zero i, or two i not adjacent

11 Fact  non-optimal to have more than two non-zero i, or two i not adjacent  e.g., f being an objective function  any convex combination between two non-adjacent break points being above the piecewise non-linear function  similarly, the point for three or more non-zero I ’s lying above the piecewise non-linear function  think about A = 0.3, B = 0.4, and C = C B A O

12 Fact  non-optimal to have more than two non-zero i, or two i not adjacent  e.g., g j being a constraint  g j (0.3A+0.7B)  g j (0.3A+0.7C)  b j  the feasible set of {0.3A+0.7B} is larger than that by {0.3A+0.7C}  the solution from {0.3A+0.7C} cannot be minimum  similar argument for three or more non-zero i ’s lying above the piecewise non-linear function C B A O

13 Example 6.1  the program becomes a linear program

14 Example 6.2: Non-Convex Problem  min f(x),  s.t.1  x  3.  approximating f(x) by a piecewise linear function  y = A + 6 B  x = A + 3 B

15 Example 6.2: Non-Convex Problem  adding slack variable s, surplus variable u, and artificial variable a 1 and a 2 :

16 Example 6.2: Non-Convex Problem

17 Example 6.2: Non-Convex Problem

18 Example 6.2: Non-Convex Problem  most negative 0  B in basis  only A qualified to enter, not O

19 Example 6.2: Non-Convex Problem

20 Example 6.2: Non-Convex Problem

21 Example 6.2: Non-Convex Problem

22  Form  again, the last constraint is unnecessary for a convex program 1 C B A O

23 Quadratic Programming

24 Quadratic Objective Function & Linear Constraints  Langrangian function

25 KKT Conditions  positive definite Q  a convex program  a unique global minimum  the KKT sufficient  otherwise, KKT necessary

26 KKT Conditions  c T + x T Q +  T A  0  Qx + A   y =  c  Ax  b  0  Ax + v = b  x T (c + Qx + A  ) = 0  x T y = 0   T (Ax  b) = 0   T v = 0  x  0,   0, y  0, v  0  solving the set of equations  phase-1 of a linear program

27 Example 7.1 (Example of JB)

28 Example 7.1 (Example of JB)  KKT conditions  2x 1 +  1 +  2  y 1 = 8,  8x 2 +  1  y 2 = 16,  x 1 + x 2 + v 1 = 5,  x 1 + v 2 = 3.  x 1 y 1 = x 2 y 2 =  1 v 1 =  2 v 2 = 0  x 1, y 1, x 2, y 2,  1, v 1,  2, v 2  0

29 Example 7.1 (Example of JB)

30 Example 7.1 (Example of JB) Example 7.1 (Example of JB)