1. Computational Geometry Piyush Kumar (Lecture 5: Linear Programming) Welcome to CIS5930

2. Linear Programming Significance  A lot of problems can be converted to LP formulation oPerceptrons (learning), Shortest path, max flow, MST, matching, … Accounts for major proportion of all scientific computations Helps in finding quick and dirty solutions to NP-hard optimization problems  Both optimal (B&B) and approximate (rounding)

3. Graphing 2-Dimensional LPs Example 1: x 3 012 y 0 1 2 4 3 Feasible Region x  0y  0 x + 2 y  2 y  4 x  3 Subject to: Maximize x + y Optimal Solution These LP animations were created by Keely Crowston.

4. Graphing 2-Dimensional LPs Example 2: Feasible Region x  0y  0 -2 x + 2 y  4 x  3 Subject to: Minimize ** x - y Multiple Optimal Solutions! 4 1 x 3 12 y 0 2 0 3 1/3 x + y  4

5. Graphing 2-Dimensional LPs Example 3: Feasible Region x  0y  0 x + y  20 x  5 -2 x + 5 y  150 Subject to: Minimize x + 1/3 y Optimal Solution x 30 1020 y 0 10 20 40 0 30 40

6. y x 0 1 2 3 4 01 2 3 x 30 1020 y 0 10 20 40 0 30 40 Do We Notice Anything From These 3 Examples? x y 0 1 2 3 4 012 3 Extreme point

7. A Fundamental Point If an optimal solution exists, there is always a corner point optimal solution! y x 0 1 2 3 4 01 2 3 x 30 1020 y 0 10 20 40 0 30 40 x y 0 1 2 3 4 012 3

8. Graphing 2-Dimensional LPs Example 1: x 3 012 y 0 1 2 4 3 Feasible Region x  0y  0 x + 2 y  2 y  4 x  3 Subject to: Maximize x + y Optimal Solution Initial Corner pt. Second Corner pt.

9. And We Can Extend this to Higher Dimensions

10. Then How Might We Solve an LP? o The constraints of an LP give rise to a geometrical shape - we call it a polyhedron. o If we can determine all the corner points of the polyhedron, then we can calculate the objective value at these points and take the best one as our optimal solution. o The Simplex Method intelligently moves from corner to corner until it can prove that it has found the optimal solution.

11. But an Integer Program is Different x y 0 1 2 3 4 012 3  Feasible region is a set of discrete points.  Can’t be assured a corner point solution.  There are no “efficient” ways to solve an IP.  Solving it as an LP provides a relaxation and a bound on the solution.

12. Linear Programs in higher dimensions maximize z = -4x 1 + x 2 - x 3 subject to -7x 1 + 5x 2 + x 3 <= 8 -2x 1 + 4x 2 + 2x 3 <= 10 x 1, x 2, x 3  0

13. In Matrix terms

14. LP Geometry Forms a d dimensional polyhedron Is convex : If z 1 and z 2 are two feasible solutions then λz 1 + (1- λ)z 2 is also feasible. Extreme points can not be written as a convex combination of two feasible points.

15. LP Geometry Extreme point theorem: If there exists an optimal solution to an LP Problem, then there exists one extreme point where the optimum is achieved. Local optimum = Global Optimum

16. LP: Algorithms Simplex. (Dantzig 1947)  Developed shortly after WWII in response to logistical problems: used for 1948 Berlin airlift.  Practical solution method that moves from one extreme point to a neighboring extreme point.  Finite (exponential) complexity, but no polynomial implementation known. Courtesy Kevin Wayne

17. LP: Polynomial Algorithms Ellipsoid. (Khachian 1979, 1980)  Solvable in polynomial time: O(n 4 L) bit operations. on = # variables oL = # bits in input  Theoretical tour de force.  Not remotely practical. Karmarkar's algorithm. (Karmarkar 1984)  O(n 3.5 L).  Polynomial and reasonably efficient implementations possible. Interior point algorithms.  O(n 3 L).  Competitive with simplex! oDominates on simplex for large problems.  Extends to even more general problems.

18. LP: The 2D case Wlog, we can assume that c=(0,-1). So now we want to find the Extreme point with the smallest y coordinate. Lets also assume, no degeneracies, the solution is given by two Halfplanes intersecting at a point.

19. Incremental Algorithm? How would it work to solve a 2D LP Problem? How much time would it take in the worst case? Can we do better?