Linear Programming and Simplex Algorithm Reference: Numerical Recipe Sec. 10.8

2 Content Linear programming problems Simplex algorithm QSOpt solver LP Applications –Collision detection –Melodic similarity –Underdetermined linear systems

3 The Diet Problem Dietician preparing a diet consisting of two foods, A and B. CostProteinFatCarbohydrate A B Minimum requirement: protein 60g; fat 24g; carbohydrate 30g Looking for minimum cost diet

4 Linear Programming subject to Cp. NLP

5 The Problem Maximize Subject to N: dimension of x M: number of constraints (M = m 1 +m 2 +m 3 ) N: dimension of x M: number of constraints (M = m 1 +m 2 +m 3 )

6 Theory of Linear Optimization Terminology: Feasible vector Feasible basic vector Optimal feasible vector

7 Theory (cont) Feasible region is convex and bounded by hyperplanes Feasible vectors that satisfy N of the original constraints as equalities are termed feasible basic vectors. Optimal occur at boundary (gradient vector of objective function is always nonzero) Combinatorial problem: determining which N constraints (out of the N+M constraints) would be satisfied by the optimal feasible vector

8 Simplex Algorithm (Dantzig 1948) A series of combinations is tried to increase the objective function Number of iterations less than O(max(M,N)) Worst case complexity is exponential (in number of variables); yet has polynomial smoothed complexity (and works well in practice)

9 The Canonical Problem Maximize Subject to N: dimension of x M: number of constraints (M = m 1 +m 2 +m 3 ) N: dimension of x M: number of constraints (M = m 1 +m 2 +m 3 ) I. II. III.

11 QSopt [ref]ref Contains a GUI and a callable library

12 LP Format Example Problem smallExample Minimize obj: 0.60x x2 Subject to c1: 12x1 + 6x2 >= 24 30x1 + 15x2 >= 30 Bounds 0 <= x1 0 <= x2 End Somehow x1 >= 0 x2 >= 0 Doesn’t work!? BNF BNF for QSOpt scripts

QSOpt Examples 13

14 Duality Primal problemDual problem

15

16 Duality If a linear program has a finite optimal solution then so does the dual. Furthermore, the optimal values of the two programs are the same. If either problem has an unbounded optimal solution, then the other problem has no feasible solutions.

17 Diet Problem

Applications

19 The Problem Under-determined system (infinite number of solutions) Find the solution with minimum magnitude (i.e., closest to origin)

20 Example Approach 1: find complete solution (-2,0,0) subtract its projection along (1,-1,1) (1,-1,1) (-2,0,0)

21 Linear Programming Solution Most simplex algorithm assume nonnegative variables Set

22 Example

23 Solved by QSopt The real minimum (-4/3, -2/3, 2/3) The LP constraint is not exactly ||x|| 2 (but ||x||  ) This problem can also be solved by SVD (singular value decomposition)

24 LP in CD (narrow phase, ref)ref A pair of convex objects: each facet represented by the plane inequality a i x+b i y+c i z  d i If two sets of inequality (from two convex objects) define a feasible region, then a collision is detected The type of optimization (min|max) and the objective function used is irrelevant.

25 LP for CD Convert to dual problem for better efficiency(?!)

Transportation Distance as a Measure to Melodic Similarity Ref: Typke et al. 2003Typke et al. 2003

27 Example Partially matched

28 Earth Mover Distance

29 EMD as Linear Program Solve by simplex algorithm

30 EMD for Melodic Similarity Ground distance: Euclidean distance Scale time coordinate (so that they are comparable) Transposition: transpose one of the melodies so that the weighted average pitch is equal (not optimum but acceptable)

31 HW: Verify planar CD using LP Warning: Non-negative assumption (for some solvers)!! Warning: Non-negative assumption (for some solvers)!!

32 Verify planar CD using LP

33 LP Solvers Lp_solve: MILP QSopt GLPK

35 QSopt [ref]ref Contains a GUI and a callable library

36 LP Format Example Problem smallExample Minimize obj: 0.60x x2 Subject to c1: 12x1 + 6x2 >= 24 30x1 + 15x2 >= 30 Bounds 0 <= x1 0 <= x2 End [weird] Somehow x1 >= 0 x2 >= 0 Doesn’t work!?

37 LP format BNF definition 1

38 LP format BNF definition 2

QSOpt API

40 Scenario: Read and Solve

41 Accessing a Solution value: objective function x: solution vector (columns) pi: dual variables slack: slack variables rc: reduced cost In our applications, we don’t really care about pi, slack, and rc. Set them to NULL

42 # of nonzeros in jth column Location of start entry cmatind QS_MAX or QS_MIN

43 Modify an LP Problem QSnew_row QSadd_rows QSnew_col QSadd_cols QSdelete_row QSdelete_col QSchange_coef QSchange_objcoef QSchange_rhscoef QSchange_sense See reference manuals for more details (Ch.5) Use QSwrite_prob to verify the change