Introduction to Linear Programming Romil Jain

The Nutrition Problem Each fruit contains different nutrients Each fruit has different cost An apple a day keeps the doctor away – but apples are costly! A customer’s goal is to fulfill daily nutrition requirements at lowest cost.

The Nutrition Problem (cont’d) Lets take a simpler case of just apples and bananas. Must take at least 100 units of Calories & 90 units of Vitamins for good nutrition. A customer’s goal is to buy fruits in such a quantity that it minimizes cost but fulfills nutrition. CaloriesVitaminsCost($) Cost Function Cost = 5x 1 + 7x 2 Constraint Functions C1: 2x 1 + 4x 2  100 C2: 3x 1 + 3x 2  90 x 1, x 2  0

The Nutrition Problem (cont’d) Matrix Representation Constraints: 2x 1 + 4x 2  100 3x 1 + 3x 2  90 Non-negativity: x 1, x 2  0 Cost function = 5x 1 + 7x 2 Real life problems may have many variables and constraints ! CiCi XiXi NiNi M i,j  XiXi

Approach to Solution C1 C2 (0,0) x2x2 x1x1 Cost Function Cost = 5x 1 + 7x 2 Constraint Functions C1: 2x 1 + 4x 2  100 C2: 3x 1 + 3x 2  90 x 1, x 2  0

Cost Function Cost = 5x 1 + 7x 2 Constraint Functions C1: 2x 1 + 4x 2  100 C2: 3x 1 + 3x 2  90 x 1, x 2  0 Simplex Algorithm C1 C2 (0,0) x2x2 x1x1 Hill Climbing - Evaluate the vertices! Same approach for n variables with m constrains (in n-dimensional space). Maximum of m C n vertices

Simplex Algorithm (cont’d) A fast algorithm to solve Linear Programs Invented by George Dantzig in 1947 Computes solution to a Linear Program by evaluating vertices where constraints intersect each other.

Simplex Algorithm (cont’d) Minimize Cost : 21x 1 - 6x 2 – 100x 3 Constraint Functions: 5x 1 + 2x 2 +31x 3  21 1x 1 - 4x 2 +3x 3  56 6x x x 3  200. x 1, x 2, x 3  0 Evaluate vertices which satisfy constraints! Carbohydrates, vitamins (A,B,C,D,E), Proteins, minerals, fiber

Linear Programs Linear Program: An optimization problem whose constraints and cost function are linear functions Goal: Find a solution which optimizes the cost. E.g. Maximize Cost Function : 21x 1 - 6x 2 – 100x x 4 Constraint Functions: 5x 1 + 2x 2 +31x x 4  21 1x 1 - 4x 2 +3x x 1  56 6x x x x 4  200 ….. Applied in various industrial fields: Manufacturing, Supply- Chain, Logistics, Marketing… To save money and increase profits !

Linear Programs (cont’d) Do all Linear Programs have optimal solutions ? No ! Three types of Linear Programs: 1. Has an optimal solution with a finite cost value: e.g. nutrition problem 2. Unbounded: e.g maximize x, x  x  3. Infeasible: e.g maximize x, x  x  x 

Duality primal linear program minimize C.x M.x  N CiCi XiXi NiNi M i,j  XiXi dual linear program maximize N T.y M T.y  C T NiNi YiYi CiCi M j,i  YiYi It can be shown that, for every primal linear program, there is a dual linear program. This can be proved mathematically. But how do we understand the dual linear program semantically?

Semantics of Duality A customer’s goal is to buy fruits in such a quantity that it minimizes cost but fulfills nutrition. CiCi XiXi NiNi M i,j  XiXi Primal LP: minimize C.x Q.x  N Coefficients in each column represent the amount of nutrients in a particular food Cost of each fruit Daily nutrition Quantity of each fruit

Semantics of Duality Dual LP: maximize N T.y Q T.y  C T Coefficients in each row represent the amount of nutrients in a particular fruit NiNi YiYi CiCi M j,i  YiYi But what are Y i s in the dual ? Price of each nutrient! Daily nutrition Cost of each fruit Imagine a salesman trying to sell supplements for each fruit.

Semantics of Duality Primal Problem: A customer’s goal is to buy fruits in such a quantity that it minimizes cost but fulfills nutrition. Dual Problem: A salesman goal is to set price on each nutrient, so that it maximizes profit but his supplements are cheaper than fruits. (Otherwise who will by them?!) Primal (Customer) Dual (Salesman)

References Introduction To Algorithms, Cormen, Leiserson, Rivest, Stein How To Think About Algorithms, Edmonds Course Presentations, Edmonds An Introduction To Linear Programming and the Simplex Algorithm, Reveliotis ( P.html) P.html

Thank You! Questions ?

Simplex C1 C2 (0,0) y x Cost Function P = 5x + 7y Constraint Functions C1: 2x + 4y  100 C2: 3x + 3y  90 Non-Negativity: x,y  0 Recall we need to evaluate our cost function at the vertices where the constraint functions intersect each other

Simplex (cont’d) Our Equations P = 5x + 7y C1: 2x + 4y  100 C2: 3x + 3y  90 x,y  0 Slack Form Can be re-written as: P = 5x + 7y s 1 = x - 4y s 2 = x - 3y x, y  0 s 1,, s 2  We introduce 2 new variables called slack variables We don’t want to deal with complex inequalities

Simplex (cont’d) Cost Function P = 5x + 7y s 1 = x - 4y s 2 = x - 3y s 1,, s 2, x, y  0 STEP 1: We want an initial point Lets put x=0, y=0 Feasible solution x=0, y=0 P = 0

Simplex (cont’d) Cost Function P = 5x + 7y s 1 = x - 4y s 2 = x - 3y s 1,, s 2, x, y  0 STEP 2: We want next point Lets try to increase x. x can be increased maximum to 30 ( s 2 becomes zero) Rewrite equations (Pivoting) Now put y, s 2 = 0 Feasible solution x = 30 – y – s 2 /3 s 1 = /3s 2 – 2y P = 150 – 5/3s 2 + 2y x=30, y=0 P = 150

Simplex (cont’d) Cost Function P = 150 – 5/3s 2 + 2y x = 30 – y – s 2 /3 s 1 = /3s 2 – 2y s 1,, s 2, x, y  0 STEP 3: We want next point Lets try to increase y. y can be increased maximum to 20 ( s 1 becomes zero) Rewrite equations (Pivoting) Now put s 1, s 2 = 0 y = /3s 2 – 1/2s 1 x = 10 – 1/2s 1 - 2/3s 2 P = s 1 – s 2 x=10, y=20 P = 190 (We don’t increase s 2 because it will decrease P) Note that we cannot increase s1 & s2 without decreasing P. So we stop ! Feasible solution Is this solution optimal? Or have we run into a local minimum? ?