Computational Geometry Piyush Kumar (Lecture 5: Linear Programming) Welcome to CIS5930

Linear Programming Significance  A lot of problems can be converted to LP formulation oPerceptrons (learning), Shortest path, max flow, MST, matching, … Accounts for major proportion of all scientific computations Helps in finding quick and dirty solutions to NP-hard optimization problems  Both optimal (B&B) and approximate (rounding)

Graphing 2-Dimensional LPs Example 1: x y Feasible Region x  0y  0 x + 2 y  2 y  4 x  3 Subject to: Maximize x + y Optimal Solution These LP animations were created by Keely Crowston.

Graphing 2-Dimensional LPs Example 2: Feasible Region x  0y  0 -2 x + 2 y  4 x  3 Subject to: Minimize ** x - y Multiple Optimal Solutions! 4 1 x 3 12 y /3 x + y  4

Graphing 2-Dimensional LPs Example 3: Feasible Region x  0y  0 x + y  20 x  5 -2 x + 5 y  150 Subject to: Minimize x + 1/3 y Optimal Solution x y

y x x y Do We Notice Anything From These 3 Examples? x y Extreme point

A Fundamental Point If an optimal solution exists, there is always a corner point optimal solution! y x x y x y

Graphing 2-Dimensional LPs Example 1: x y Feasible Region x  0y  0 x + 2 y  2 y  4 x  3 Subject to: Maximize x + y Optimal Solution Initial Corner pt. Second Corner pt.

And We Can Extend this to Higher Dimensions

Then How Might We Solve an LP? o The constraints of an LP give rise to a geometrical shape - we call it a polyhedron. o If we can determine all the corner points of the polyhedron, then we can calculate the objective value at these points and take the best one as our optimal solution. o The Simplex Method intelligently moves from corner to corner until it can prove that it has found the optimal solution.

But an Integer Program is Different x y  Feasible region is a set of discrete points.  Can’t be assured a corner point solution.  There are no “efficient” ways to solve an IP.  Solving it as an LP provides a relaxation and a bound on the solution.

Linear Programs in higher dimensions maximize z = -4x 1 + x 2 - x 3 subject to -7x 1 + 5x 2 + x 3 <= 8 -2x 1 + 4x 2 + 2x 3 <= 10 x 1, x 2, x 3  0

In Matrix terms

LP Geometry Forms a d dimensional polyhedron Is convex : If z 1 and z 2 are two feasible solutions then λz 1 + (1- λ)z 2 is also feasible. Extreme points can not be written as a convex combination of two feasible points.

LP Geometry Extreme point theorem: If there exists an optimal solution to an LP Problem, then there exists one extreme point where the optimum is achieved. Local optimum = Global Optimum

LP: Algorithms Simplex. (Dantzig 1947)  Developed shortly after WWII in response to logistical problems: used for 1948 Berlin airlift.  Practical solution method that moves from one extreme point to a neighboring extreme point.  Finite (exponential) complexity, but no polynomial implementation known. Courtesy Kevin Wayne

LP: Polynomial Algorithms Ellipsoid. (Khachian 1979, 1980)  Solvable in polynomial time: O(n 4 L) bit operations. on = # variables oL = # bits in input  Theoretical tour de force.  Not remotely practical. Karmarkar's algorithm. (Karmarkar 1984)  O(n 3.5 L).  Polynomial and reasonably efficient implementations possible. Interior point algorithms.  O(n 3 L).  Competitive with simplex! oDominates on simplex for large problems.  Extends to even more general problems.

LP: The 2D case Wlog, we can assume that c=(0,-1). So now we want to find the Extreme point with the smallest y coordinate. Lets also assume, no degeneracies, the solution is given by two Halfplanes intersecting at a point.

Incremental Algorithm? How would it work to solve a 2D LP Problem? How much time would it take in the worst case? Can we do better?