UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2002 Lecture 8 Tuesday, 11/19/02 Linear Programming

Overview ä Motivation & Basics ä Standard & Slack Forms ä Formulating Problems as Linear Programs ä Simplex Algorithm ä Duality ä Initial Basic Feasible Solution ä Literature Case Study

Motivation & Basics

Motivation: A Political Problem source: textbook Cormen et al. Goal: Win election by winning majority of votes in each region. Subgoal: Win majority of votes in each region while minimizing advertising cost. 100,000 voters 200,000 voters 50,000 voters Thousands of voters who could be won with $1,000 of ads

Motivation: A Political Problem (continued) Thousands of voters representing majority. urban suburban rural source: textbook Cormen et al.

General Linear Programs real numbers variables Linear function Linear inequalities Linear constraints source: textbook Cormen et al.

Overview of Linear Programming Convex feasible region Objective function Objective value source: textbook Cormen et al.

Standard & Slack Forms

Standard Form objective function constraints source: textbook Cormen et al.

Standard Form (compact) mxn matrix m-dimensional vector n-dimensional vectors source: textbook Cormen et al. Can specify linear program in standard form by (A,b,c).

Converting to Standard Form source: textbook Cormen et al.

Converting to Standard Form (continued) source: textbook Cormen et al. Negate coefficients Transforming minimization to maximization

Converting to Standard Form (continued) source: textbook Cormen et al. If x j has no non-negativity constraint, replace each occurrence of x j with x j ’ – x j ”. Giving each variable a non-negativity constraint New non-negativity constraints

Converting to Standard Form (continued) source: textbook Cormen et al. Transforming equality constraints to inequality constraints

Converting to Standard Form (continued) source: textbook Cormen et al. Changing sense of an inequality constraint Rationale:

Converting Linear Programs into Slack Form source: textbook Cormen et al. for algorithmic ease, transform all constraints except non-negativity ones into equalities for inequality constraint: define slack slack variable instead of s basic variables non-basic variables

Converting Linear Programs into Slack Form (continued) source: textbook Cormen et al. objective function

Converting Linear Programs into Slack Form (continued) source: textbook Cormen et al. Compact Form: (N, B, A, b, c, v) set of indices of non-basic variables set of indices of basic variables Slack Form Example Compact Form negative of slack form coefficients

Formulating Problems as Linear Programs

Shortest Paths source: textbook Cormen et al. Single-pair shortest path: minimize “distance” from source s to sink t. Can we replace maximize with minimize here? Why or why not?

Maximum Flow source: textbook Cormen et al.

Minimum Flow source: textbook Cormen et al.

Multicommodity Flow source: textbook Cormen et al. should be s i

Simplex Method

Solving a Linear Program source: textbook Cormen et al. ä Simplex algorithm ä Geometric interpretation ä Visit vertices on the boundary of the simplex representing the convex feasible region ä Transforms set of inequalities using process similar to Gaussian elimination ä Run-time ä not polynomial in worst-case ä often very fast in practice ä Ellipsoid method ä Run-time ä polynomial ä slow in practice ä Interior-Point methods ä Run-time ä polynomial ä for large inputs, performance can be competitive with simplex method ä Moves through interior of feasible region

Simplex Algorithm: Example Basic Solution source: textbook Cormen et al. Standard Form Slack Form Basic Solution: set each nonbasic variable to 0. Basic Solution:

Simplex Algorithm: Example Reformulating the LP Model source: textbook Cormen et al. Main Idea: In each iteration, reformulate the LP model so basic solution has larger objective value Select a nonbasic variable whose objective coefficient is positive: x 1 Increase its value as much as possible. Identify tightest constraint on increase. For basic variable x 6 of that constraint, swap role with x 1. Rewrite other equations with x 6 on RHS. PIVOT leaving variable entering variable new objective value

Simplex Algorithm: Example Reformulating the LP Model source: textbook Cormen et al. Next Iteration: select x 3 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

Simplex Algorithm: Example Reformulating the LP Model source: textbook Cormen et al. Next Iteration: select x 2 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

Simplex Algorithm: Issues to be Addressed… source: textbook Cormen et al.

Simplex Algorithm: Pivoting source: textbook Cormen et al. leaving variable entering variable Rewrite the equation that has x l on LHS to have x e on LHS Update remaining equations by substituting RHS of new equation for each occurrence of x e. Do the same for objective function. Update sets of nonbasic, basic variables.

Simplex Algorithm: Pivoting (continued) source: textbook Cormen et al.

Simplex Algorithm: Pseudocode source: textbook Cormen et al. to be defined later (detects infeasibility) Feasibility initial basic solution optimal solution

Simplex Algorithm: Correctness Proof Strategy source: textbook Cormen et al. ä If SIMPLEX has initial feasible solution and eventually terminates, then it either ä returns a feasible solution or ä determines that the LP is unbounded. ä SIMPLEX terminates. ä SIMPLEX’s solution is optimal.

Simplex Algorithm: Correctness Proof Approach, Part I source: textbook Cormen et al. ä If SIMPLEX has initial feasible solution and eventually terminates, then it either ä returns a feasible solution or ä determines that the LP is unbounded.

Simplex Algorithm: Correctness Proof Approach, Part II source: textbook Cormen et al.

Duality Simplex Algorithm: Correctness Proof Approach, Part III

Linear Programming Duality source: textbook Cormen et al. max becomes min RHS coefficients swap places with objective function coefficients sense changes x variables go away y variables appear

Duality Example source: textbook Cormen et al.

Weak Linear Programming Duality source: textbook Cormen et al. Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP.

Weak Linear Programming Duality (continued) source: textbook Cormen et al.

Finding a Dual Solution source: textbook Cormen et al. Finding a dual solution whose value is equal to that of an optimal primal solution…

Optimality source: textbook Cormen et al.

Initial Basic Feasible Solution

Finding an Initial Solution source: textbook Cormen et al. An LP model whose initial basic solution is not feasible

Finding an Initial Solution (continued) source: textbook Cormen et al. Auxiliary LP model L aux :

Finding an Initial Solution (continued) source: textbook Cormen et al.

Finding an Initial Solution (continued) source: textbook Cormen et al. Original LP model L aux L aux in slack form

Finding an Initial Solution (continued) source: textbook Cormen et al. PIVOT

Finding an Initial Solution (continued) source: textbook Cormen et al.

Fundamental Theorem of Linear Programming source: textbook Cormen et al.

Literature Case Study