1. Introduction to Algorithms
Linear Programming
My T. UF

2. New crop problem A farmer:
Has 10 acres to plant in wheat and rye
Has Only $1200 to spend
Has to plant at least 7 acres
Has to get the planting done in 12 hours
Each acre of wheat costs $200 and takes an hour to plant and gives $500 profit
Each acre of rye costs $100 and takes 2 hours to plant and gives $300 profit
Goal: Compute the numbers of acres of each should be planted to maximize profits
x: # of wheat acres
y: # of rye acres
My T. Thai

3. Linear program
A linear programming problem is the problem of maximizing or minimizing a linear function subject to linear constraints (equalities and inequalities)
E.g.
Objective function
Constraints
My T. Thai

4. Formulate shortest path as an LP
Triangle inequalities:
At the source vertex: ds = 0
dt is less than weights of all paths from s to t => dt is the maximum value that less than weights of all paths from s to t
We have the following:
My T. Thai

5. Max Flow Capacity constraints: Conservation constraints:
We have the following:
My T. Thai

6. Minimum-cost flow Pay cost to transmit flow fuv through edge (u, v)
Wish to send d units with the minimized cost
My T. Thai

7. Multicommodity flow
Has k commodities; ith commodity is defined by a triple
Total flow of all commodities through an edge does not exceed its capacity
Find a feasible flow
My T. Thai

8. Some Notations
Minimization (maximization) linear program: the value of objective function is minimized (maximized)
Feasible (infeasible) solution: a setting of variables satisfies all the constraints (conflicts at least one constraint)
Feasible region: set of feasible solutions
Objective value: value of the objective function at a particular point
Optimal solution: objective value is maximum over all feasible solutions
Optimal objective value: objective value of optimal solution
The linear program is infeasible if it has no feasible solutions; otherwise it is feasible
The linear program is unbounded if the optimal objective value is infinite
My T. Thai

9. Standard form
Concise representation:
My T. Thai

10. Convert into standard form
My T. Thai

11. Example My T. Thai mythai@cise.ufl.edu (negate objective function)
(replace equality)
(replace x2)
(negate constrain and change variable name)
My T. Thai

12. Geometry of Linear Programming
Theorem 1
Feasible region of an LP is convex
Proof:
Feasible region is the intersection of half spaces defined by the constraints. Each half space is convex.
My T. Thai

13. Example
My T. Thai

14. Feasibility and Infeasibility
Simple solution: try all vertices of polyhedron  running time:
 How to refine this approach?
My T. Thai

15. Simplex method
Start off from a vertex, which is called a basic feasible solution
Iteratively move along an edge of the polyhedron to another vertex toward the direction of optimization
For each move, need to make sure that the objective function is not decreased
Observation: when moving from a vertex to another vertex, an inequality achieves equality
My T. Thai

16. Questions Arise
My T. Thai

17. Slack form All inequality constraints are non-negativity constraints
Convert by introducing slack variables
nonbasic variables
Basic variables
My T. Thai

18. Concise representation of slack form
z: the value of the objective function
N: the set of indices of the nonbasic variables (|N| = n)
B: the set of indices of the basic variables (|B| = m)
v: an optional constant term in the objective function
A tuple (N, B, A, b, c, v) represents the slack form
maximize
My T. Thai

19. Simplex algorithm Basic solution: all nonbasic variables equals 0
Each iteration converts one slack form into an equivalent slack form s.t. the objective value is not decreased
Choose a nonbasic variable xe (entering variable) such that its increase makes the objective value increase
Keeping all constraint satisfied, raise the variable raise it until some basic variable xl (leaving variable) becomes 0
Exchange the roles of that basic variable and the chosen nonbasic variable
My T. Thai

20. Example
My T. Thai

21. Substitute x_2 = x_3 = 0 to the slack variables, we have:
My T. Thai

22. My T. Thai

23. Cycling
SIMPLEX may run forever if the slack forms at two different iterations of SIMPLEX are identical (cycling phenomenon)
We need specific rule of picking the entering and leaving variables
There are quite a few methods to prevent cycling. The one we just used is called Bland’s pivoting rule
My T. Thai

24. Other Pivoting Rules
My T. Thai

25. Time Complexity
My T. Thai

26. Duality Given a primal problem: The dual is:
P: min cTx subject to Ax ≥ b, x ≥ 0
The dual is:
D: max bTy subject to ATy ≤ c, y ≥ 0
My T. Thai

27. An Example
My T. Thai

28. Weak Duality Theorem Weak duality Theorem:
Let x and y be the feasible solutions for P and D respectively, then:
Proof: Follows immediately from the constraints
My T. Thai

29. Weak Duality Theorem This theorem is very useful
Suppose there is a feasible solution y to D. Then any feasible solution of P has value lower bounded by bTy. This means that if P has a feasible solution, then it has an optimal solution
Reversing argument is also true
Therefore, if both P and D have feasible solutions, then both must have an optimal solution.
My T. Thai

30. Hidden Message ≥
Strong Duality Theorem: If the primal P has an optimal solution x* then the dual D has an optimal solution y* such that:
cTx* = bTy*
My T. Thai

31. Complementary Slackness
Theorem:
Let x and y be primal and dual feasible solutions
respectively. Then x and y are both optimal iff two
of the following conditions are satisfied:
(ATy – c)j xj = 0 for all j = 1…n
(Ax – b)i yi = 0 for all i = 1…m
My T. Thai

32. Proof of Complementary Slackness
As in the proof of the weak duality theorem, we
have: cTx ≥(ATy)Tx = yTAx ≥ yTb (1)
From the strong duality theorem, we have:
(2)
(3)
My T. Thai

33. Proof (cont) Note that and We have: x and y optimal  (2) and (3) hold
 both sums (4) and (5) are zero
 all terms in both sums are zero (?)
 Complementary slackness holds
(4)
(5)
My T. Thai

34. Why do we care?
It’s an easy way to check whether a pair of primal/dual feasible solutions are optimal
Given one optimal solution, complementary slackness makes it easy to find the optimal solution of the dual problem
May provide a simpler way to solve the primal
My T. Thai

35. Some examples
Solve this system:
My T. Thai

36. Min-Max Relations What is a role of LP-duality Max-flow and Min-Cut
My T. Thai

37. Max Flow in a Network
Definition: Given a directed graph G=(V,E) with two distinguished nodes, source s and sink t, a positive capacity function c: E → R+, find the maximum amount of flow that can be sent from s to t, subject to:
Capacity constraint: for each arc (i,j), the flow sent through (i,j), fij bounded by its capacity cij
Flow conservation: at each node i, other than s and t, the total flow into i should equal to the total flow out of i
My T. Thai

38. An Example 3 3 4 4 4 3 2 4 3 4 3 2 4 2 1 t s 1 3 1 1 3 2 2 3 2 1 2 5
My T. Thai

39. Formulate Max Flow as an LP
Capacity constraints: 0 ≤ fij ≤ cij for all (i,j)
Conservation constraints:
We have the following:
My T. Thai

40. LP Formulation (cont) 3 3 4 4 4 3 2 4 3 4 3 2 4 3 2 1 t 1 s 1 1 2 3 23 2 4 3 4 3 2 4 3 2 1 t 1 s 1 1 2 3 2 3 1 2 2 5 ∞
My T. Thai

41. LP Formulation (cont)
My T. Thai

42. Min Cut Capacity of any s-t cut is an upper bound on any feasible flow
If the capacity of an s-t cut is equal to the value of a maximum flow, then that cut is a minimum cut
My T. Thai

43. Max Flow and Min Cut
My T. Thai

44. Solutions of IP Consider:
Let (d*,p*) be the optimal solution to this IP. Then:
ps* = 1 and pt* = 0. So define X = {pi | pi = 1} and $\bar X$ = {pi | pi = 0}. Then we can find the s-t cut
dij* =1. So for i in X and j in $\bar X$, define dij = 1, otherwise dij = 0.
Then the object function is equal to the minimum s-t cut
My T. Thai