EMGT 6412/MATH 6665 Mathematical Programming Spring 2016 Mathematical Programming: Introduction to Linear Programming Dincer Konur Engineering Management and Systems Engineering

Outline Chapter 1 Introduction to Mathematical Programming Linear Programming Formulation Steps of Formulation Examples Assumptions Geometric Interpretation Geometry of LPs Properties of LPs Linear Programming Format Matrix Notation Standard Format Chapter 1

Outline Introduction to Mathematical Programming Linear Programming Formulation Steps of Formulation Examples Assumptions Geometric Interpretation Geometry of LPs Properties of LPs Linear Programming Format Matrix Notation Standard Format

Mathematical Programming Mathematical programming is selecting the best option(s) from a set of alternatives mathematically Minimizing/maximizing a function where your alternatives are defined by functions Many industries use mathematical programming Supply chain, logistics, and transportation Health industry, energy industry, finance, airlines Manufacturing industry, agriculture industry Education, Military Operations Research raised with WWII

Linear Programming Linear Programming problem (LP) is a mathematical programming problem where all of your functions are linear A function is linear when The variables have power of 1 and the variables are not multiplied with each other 𝑓 𝑥 =3𝑥+5  linear 𝑓 𝑥 =3 𝑥 2 +5  not linear 𝑓 𝑥 1 , 𝑥 2 =3 𝑥 1 +5 𝑥 2 +7  linear 𝑓 𝑥 1 , 𝑥 2 =3 𝑥 1 𝑥 2 +7  not linear 𝑓 𝑥 𝑥

Outline Introduction to Mathematical Programming Linear Programming Formulation Steps of Formulation Examples Assumptions Geometric Interpretation Geometry of LPs Properties of LPs Linear Programming Format Matrix Notation Standard Format

Linear Programming Formulation There are 4 main steps for formulating a mathematical programming model: Step 0: Understand the problem and problem data Objective function (cost) coefficients, technological coefficients Step 1: Identify the decision variables Things we control, and want to solve for Step 2: Construct the objective function Measure of performance (criterion function) Step 3: Construct the constraints Restrictions

Linear Programming Formulation x1,…, xn are the decision variables c1, c2,…, cn are the cost coefficients c1x1 + c2x2 + … + cnxn is the objective function ith constraint: Technological coefficient:

Linear Programming Formulation Deserted Island: Linear programming will save you! Suppose that you will be left in a deserted island You want to live as many days as you can Fortunately (!), you can take a bag with you to the island The bag can carry at most 50 lbs and there are limited set of items you can carry Bread: You can survive for 2 days with 1 lb of bread Steak: You can survive for 5 days with 1 lb of steak (Memphis style grilled!) Chicken: You can survive for 3 days with 1 lb of chicken (southern style deep fried!) Chocolate: You can survive for 6 days with 1 lb of chocolate How much of each item to take with you?

Linear Programming Formulation STEP 0: Know the problem and gather your data Your problem is to increase your chance of rescue by maximizing the number of days you survive You can have 1 bag which can carry 50 lbs at most You can only put bread, steak, chicken and chocolate into you bag Each item enables you survive for a specific number of days for each pound you take Bread: 2 days/lb Steak: 5 days/lb Chicken: 3days/lb Chocolate: 6days/lb STEP 1: Identify your decision variables, which are the things you control The amount of each item you will take with you 𝑥 1 :𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑏𝑟𝑒𝑎𝑑 (𝑙𝑏𝑠) 𝑥 2 :𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑠𝑡𝑒𝑎𝑘 (𝑙𝑏𝑠) 𝑥 3 :𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑐ℎ𝑖𝑐𝑘𝑒𝑛 (𝑙𝑏𝑠) 𝑥 4 :𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑐ℎ𝑜𝑐𝑜𝑙𝑎𝑡𝑒 𝑙𝑏𝑠 Warning: Always be careful with the metrics (try to use the same metrics)

Linear Programming Formulation STEP 2: Define your objective function and objective Your objective function is the measure of performance as a result of your decisions Your objective is what you want to do with your objective function Recall that you want to maximize the number of days you will survive We know our objective (maximize) and our performance of measure (number of days). Express the performance of measure as a function of your decision variables to get the objective function If you take 𝑥 1 lbs of bread, you will survive for 2𝑥 1 days If you take 𝑥 2 lbs of steak, you will survive for 5𝑥 1 days If you take 𝑥 3 lbs of chicken, you will survive for 3𝑥 1 days If you take 𝑥 4 lbs of chocolate, you will survive for 6𝑥 1 days Then objective function in terms of decision variables: 2𝑥 1 + 5𝑥 2 + 3𝑥 3 + 6𝑥 4 Objective Performance measure, Not a function though

Linear Programming Formulation STEP 3: Define your restrictions (constraints) There may be some restrictions which limit what you can do (hence, they define set of your alternatives) You can carry 1 bag and it can carry 50 lbs at most 𝑥 1 + 𝑥 2 + 𝑥 3 + 𝑥 4 ≤50 You cannot get negative amounts!!! 𝑥 1 ≥0, 𝑥 2 ≥0, 𝑥 3 ≥0, 𝑥 4 ≥0 Limit on how much you can carry Total amount you decide to carry

Linear Programming Formulation Combine your objective, objective function, and constraints 2𝑥 1 + 5𝑥 2 + 3𝑥 3 + 6𝑥 4 𝑥 1 + 𝑥 2 + 𝑥 3 + 𝑥 4 ≤50 𝑥 1 ≥0, 𝑥 2 ≥0, 𝑥 3 ≥0, 𝑥 4 ≥0 This is your LP model Note that everything is linear! 𝑀𝑎𝑥𝑖𝑚𝑖𝑧𝑒 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜

Linear Programming Formulation Feed Mix Problem: An agricultural mill manufactures feed for chickens. This is done by mixing several ingredients, such as corn, limestone, or alfalfa. The mixing is to be done in such a way that the feed meets certain levels for different types of nutrients, such as protein, calcium, carbohydrates, and vitamins. Suppose that there are n ingredients j = 1,…, n Suppose that there are m nutrients i = 1,…, m Let the unit cost of ingredient j be cj Suppose that aij is the amount of nutrient i present in a unit of ingredient j Acceptable lower and upper limits of nutrient i in a unit of the chicken feed are li’ and ui’, respectively Suppose that the mill cannot acquire more than uj units of ingredient j What should be the mixing the ingredients such that the cost is minimized and the restrictions are met?

Linear Programming Formulation Let the amount of ingredient j to be used be xj

Linear Programming Formulation Cutting Stock Problem: A manufacturer of metal sheets produces rolls of standard fixed width w and of standard length l. A large order is placed by a customer who needs sheets of width w and varying lengths. The manufacturer would like to cut the standard rolls in such a way as to satisfy the order and minimize the waste. Suppose that bi sheets with length li and width w needed, i = 1,…, m There are many ways of cutting the standard sheet in different lengths The cutting patterns: j=1,2,…,n Minimize the number of standard rolls used by determining how many of each pattern cut so that you satisfy the customer needs? aij is the number of sheets of length li in the jth pattern

Linear Programming Formulation Let xj be the number of standard rolls cut according to the jth pattern This is not a LP unless we remove the integer constraints!! LP models assume that decision variables are continuous

Linear Programming Formulation Transportation Problem: The Brazilian coffee company processes coffee beans into coffee at m plants. The coffee is then shipped every week to n warehouses in major cities for retail, distribution, and exporting. Suppose that the unit shipping cost from plant i to warehouse j is cij. Furthermore, suppose that the production capacity at plant i is ai and that the demand at warehouse j is bj. Practice: Formulate Tanker Scheduling problem

Linear Programming Formulation More numerical examples in Supplement 1 Assumptions of Linear Programming: Proportionality: Given variable xj, its contribution to cost is cixj and its contribution to the ith constraint is aijxj Additivity: This assumption guarantees that the total cost is the sum of the individual costs, and that the total contribution to the ith restriction is the sum of the individual contributions of the individual activities. Divisibility:This assumption ensures that the decision variables can be divided into any fractional levels so that non-integral values for the decision variables are permitted. Deterministic. The coefficients ci, aij, and bi are all known deterministically.

Outline Introduction to Mathematical Programming Linear Programming Formulation Steps of Formulation Examples Assumptions Geometric Interpretation Geometry of LPs Properties of LPs Linear Programming Format Matrix Notation Standard Format

Geometric Interpretation A simple graphical procedure to solve simple LPs (with two decision variables)

Geometric Interpretation Illustration: Examples in Supplement 1

Geometric Interpretation There are 4 cases possible for a LP: Unique Optimal Solution. If the optimal solution is unique, then it occurs at an extreme point.

Geometric Interpretation There are 4 cases possible for a LP: Alternative Optimal Solutions. If there are more than one feasible solutions that are optimal.

Geometric Interpretation There are 4 cases possible for a LP: Unbounded Optimal Objective Value. If there is always a better solution, we have unboundedness. If the feasible region is bounded, you cannot have unbounded optimal objective value

Geometric Interpretation There are 4 cases possible for a LP: Empty Feasible Region. The case when there is no feasible solution.

Outline Introduction to Mathematical Programming Linear Programming Formulation Steps of Formulation Examples Assumptions Geometric Interpretation Geometry of LPs Properties of LPs Linear Programming Format Matrix Notation Standard Format

Linear Programming Format Matrix Notation: A LP can be stated in a more convenient form using matrix notation

Linear Programming Format Standard format: Objective is minimization All variables are non-negative All constraints (except non-negativity) are equality constraints Any linear programming model can be converted to the standard format!!! Convert the variables if needed, convert the constraints if needed and convert the objective function if needed

Linear Programming Format Converting to standard format: Variables: X1, X2,…, Xn Non-negative: 𝑋𝑗≥0 Do nothing  Non-positive: 𝑋𝑗≤0 Then define 𝑌𝑗=−𝑋𝑗 and replace 𝑋𝑗 with −𝑌𝑗 in the formulation Unrestricted: 𝑋𝑗 can be both negative or positive Any number can be written as difference of two positive numbers Then define 𝑋𝑗=𝑋 𝑗 ′ −𝑋𝑗′′ where 𝑋𝑗′≥0 and 𝑋𝑗′′≥0 Replace 𝑋𝑗 with 𝑋 𝑗 ′ −𝑋𝑗′′ (one variable is replaced with two) If 𝑋1, 𝑋2, …, 𝑋𝑘 for some k, then define 𝑋𝑗=𝑋 𝑗 ′ −𝑋′′ for all j=1,2,…,k where 𝑋𝑗′≥0 and 𝑋′′≥0 Bounded….

Linear Programming Format Converting to standard format: Variables: X1, X2,…, Xn Bounded: 𝑋𝑗≥𝐿𝑗 or 𝑋𝑗≤𝑈𝑗 If 𝑋𝑗≥𝐿𝑗 define 𝑋 𝑗 ′ =𝑋𝑗−𝐿𝑗 note that 𝑋 𝑗 ′ ≥0 Replace 𝑋𝑗 with 𝑋 𝑗 ′ +𝐿𝑗 If 𝑋𝑗≤𝑈𝑗 define 𝑋 𝑗 ′ =𝑈𝑗−𝑋𝑗 Replace 𝑋𝑗 with U 𝑗 ′ −𝑋𝑗′ You can also treat bounded variables as regular <= or >= constraints and convert them to ‘=‘ constraints Constraints: Less than or equal to constraints Greater than or equal to constraints Must be converted to ‘=‘ constraints

Linear Programming Format Converting to standard format: Constraints: Greater than or equal to constraint: This can be made an ‘=‘ constraint by subtracting a positive slack variable, (sometimes referred to si) from left-hand-side We will have and Less than or equal to constraint: This can be made an ‘=‘ constraint by adding a positive slack variable, (sometimes referred to si) to the left-hand-side We will have and Note: An equality constraint can be transformed into two inequality constraints:

Linear Programming Format Converting to standard format: Objective: Example: Objective must be ‘minimize’ Maximize - + + Must be ‘=‘ Must be ‘=‘ This is ‘=‘ Convert to >=0 Convert to >=0

Linear Programming Format Converting to standard format: Objective converted! Objective is ‘minimize’ now Must be ‘=‘ Must be ‘=‘ This is ‘=‘ Convert to >=0 Convert to >=0

Linear Programming Format Converting to standard format: First constraint: ≤ constraint Define a slack variable 𝑥 4 ≥0 and add it to the left side − 𝑥 1 + 3𝑥 2 + 𝑥 3 +𝑥 4 =13 and 𝑥 4 ≥0 This is ‘=‘ − 𝒙 𝟏 + 𝟑𝒙 𝟐 + 𝒙 𝟑 +𝒙 𝟒 =𝟏𝟑 Must be ‘=‘ Objective is ‘minimize’ This is ‘=‘ Convert to >=0 𝑥 4 ≥0 Convert to >=0 This is ≥0

Linear Programming Format Converting to standard format: Second constraint: ≥ constraint Define a slack variable 𝑥 5 ≥0 and subtract it from the left side 𝑥 1 + 2𝑥 2 + 3𝑥 3 −𝑥 5 =12 and 𝑥 5 ≥0 This is ‘=‘ − 𝑥 1 + 3𝑥 2 + 𝑥 3 +𝑥 4 =13 This is ‘=‘ 𝒙 𝟏 + 𝟐𝒙 𝟐 + 𝟑𝒙 𝟑 −𝒙 𝟓 =𝟏𝟐 Objective is ‘minimize’ This is ‘=‘ Convert to >=0 𝑥 4 ≥0 Convert to >=0 These are ≥0 𝑥 5 ≥0

Linear Programming Format Converting to standard format: Objective and constraints are converted, we have: 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑥 1 − 2𝑥 2 −3 𝑥 3 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 − 𝑥 1 + 3𝑥 2 + 𝑥 3 +𝑥 4 =13 𝑥 1 + 2𝑥 2 + 3𝑥 3 −𝑥 5 =12 2𝑥 1 − 𝑥 2 + 𝑥 3 =4 𝑥 1 , 𝑥 2 𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑 Convert to >=0 𝑥 3 ≤−3 𝑥 4 ≥0 𝑥 5 ≥0

Linear Programming Format Converting to standard format: Variables 𝑥 1 , 𝑥 2 𝑎𝑟𝑒 𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑 Let 𝑥 1 = 𝑥 1 ′− 𝑥 1 ′′ where 𝑥 1 ′ ≥0, 𝑥 1 ′′ ≥0, replace 𝑥 1 with 𝑥 1 ′− 𝑥 1 ′′ Let 𝑥 2 = 𝑥 2 ′− 𝑥 2 ′′ where 𝑥 2 ′ ≥0, 𝑥 2 ′′ ≥0, replace 𝑥 2 with 𝑥 2 ′− 𝑥 2 ′′ 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 ( 𝑥 1 ′ − 𝑥 1 ′′ )− 2(𝑥 2 ′− 𝑥 2 ′′)−3 𝑥 3 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 −( 𝑥 1 ′ − 𝑥 1 ′′ )+ 3(𝑥 2 ′− 𝑥 2 ′′)+ 𝑥 3 +𝑥 4 =13 ( 𝑥 1 ′ − 𝑥 1 ′′ )+ 2(𝑥 2 ′− 𝑥 2 ′′)+ 3𝑥 3 −𝑥 5 =12 2( 𝑥 1 ′ − 𝑥 1 ′′ )− (𝑥 2 ′− 𝑥 2 ′′) + 𝑥 3 =4 𝑥 1 ′≥0, 𝑥 1 ′′≥0, 𝑥 2 ′≥0, 𝑥 2 ′′≥0,𝑥 4 ≥0, 𝑥 5 ≥0 𝑥 3 ≤−3 Convert to >=0

Linear Programming Format Converting to standard format: Variable 𝑥 3 ≤−3 Let 𝑥 3 ′ =−3− 𝑥 3 note that 𝑥 3 ′ ≥0, 𝑥 1 ′′ ≥0, Then we have 𝑥 3 =−3− 𝑥 3 ′ so replace 𝑥 3 with −3− 𝑥 3 ′ 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 ( 𝑥 1 ′ − 𝑥 1 ′′ )− 2(𝑥 2 ′− 𝑥 2 ′′)−3(−3− 𝑥 3 ′ ) 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 −( 𝑥 1 ′ − 𝑥 1 ′′ )+ 3(𝑥 2 ′− 𝑥 2 ′′)+(−3− 𝑥 3 ′ ) +𝑥 4 =13 ( 𝑥 1 ′ − 𝑥 1 ′′ )+ 2(𝑥 2 ′− 𝑥 2 ′′)+3(−3− 𝑥 3 ′ ) −𝑥 5 =12 2( 𝑥 1 ′ − 𝑥 1 ′′ )− (𝑥 2 ′− 𝑥 2 ′′) +(−3− 𝑥 3 ′ ) =4 𝑥 1 ′≥0, 𝑥 1 ′′≥0, 𝑥 2 ′ ≥0, 𝑥 2 ′′ ≥0, 𝑥 3 ′ ≥0,𝑥 4 ≥0, 𝑥 5 ≥0

Linear Programming Format Converted to standard format: 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑥 1 ′ − 𝑥 1 ′′ − 2𝑥 2 ′ +2 𝑥 2 ′′ +3 𝑥 3 ′ 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 − 𝑥 1 ′ + 𝑥 1 ′′ + 3𝑥 2 ′−3 𝑥 2 ′′− 𝑥 3 ′ +𝑥 4 =16 𝑥 1 ′ − 𝑥 1 ′′ + 2𝑥 2 ′−2 𝑥 2 ′′− 𝑥 3 ′ −𝑥 5 =21 2 𝑥 1 ′ −2 𝑥 1 ′′ − 𝑥 2 ′+ 𝑥 2 ′′− 𝑥 3 ′ =7 𝑥 1 ′≥0, 𝑥 1 ′′≥0, 𝑥 2 ′ ≥0, 𝑥 2 ′′ ≥0, 𝑥 3 ′ ≥0,𝑥 4 ≥0, 𝑥 5 ≥0

Linear Programming Format Let’s put it in matrix notation Let 𝐱= 𝑥 1 ′ 𝑥 1 ′′ 𝑥 2 ′ 𝑥 1 ′′ 𝑥 3 ′ 𝑥 4 𝑥 5 , 𝐀= −1 1 3 −3 −1 1 0 1 −1 2 −2 −1 0 1 2 −2 −1 1 −1 0 0 , and 𝐛= 16 21 7 Let 𝒄= 1 −1 −2 2 3 0 0

Linear Programming Format Standard form can have maximization objective as well but we will discuss Simplex mostly for minimization problems Another format that can be helpful is canonical form Examples in Supplement 1

Next time… Linear algebra review Basic linear algebra Affine and Linear combinations Linear independence, spanning Convex sets and convex functions Polyhedral sets, extreme points, representation