1. Introduction to Linear and Integer Programming
Saba Neyshabouri

2. Operations Research OR was developed and used during world war II
While the technology was advancing fast, the problems that analysts were facing were getting bigger and more complex.
Decision making for complex systems are very complicated and is out of human’s mind capability to solve problems with many variables.
Operations Research and optimization methods try to find “The Best” solution for a problem.
The best example of first application of OR was how to deploy radars to get the maximum coverage.

3. Operation Research
The founder of the field is George B. Dantzig who invented Simplex method for solving Linear Programming (LP) problems.
With Simplex it was shown that the optimal solution of LP’s can be found.
His book, linear programming and extensions , is where he has gathered all of his ideas and notable research.

4. LP Structure
There are 3 main parts that forms an optimization problem:
Decision Variables: Variables that represent the decision that can be made.
Objective function: Each optimization problem is trying to optimize (maximize/minimize) some goal such as costs, profits, revenue.
Constraints: Set of real restricting parameters that are imposed in real life or by the structure of the problem. Example for constraints can be:
Limited budget for a project
Limited manpower or resources
Being limited to choose only one option out of many options (Assignment)
In real world problems, we are always constrained in our decision making. We are bounded to choose from our options and not exceed limits.

5. General Form of LP The general form for LP is:
(1) is the objective function
(2),(3) are the set of constraints
X: vector of decision variables (n*1)
C: vector of objective function coefficients (n*1)
A: Technology matrix (m*n)
b: vector of resource availability (m*1)
This is the matrix form of the LP. There are algebraic representations which are exactly the equivalent of this representation.

6. LP example
Production planning: a producer of furniture has to make the decision about the production planning for 2 of its products: work desks and lunch tables. There are 2 resources available for production which these products are using: lumber and carpentry. 20 units of each resource is available. Here is the table containing the data for the problem:
Desk
Table
Selling Price
$15
$20
Desk
Table
Availability
Lumber 1 2 20
Carpentry

7. LP example
First table shows the selling prices of each product, and second table show the amount of each resource that is used to produce one unit of product as well as resource availability. The question is how many of each product should be produced in order to maximize the revenue?
To model this problem as a linear programming formulation we should define our decision variables:
Here explain what the goal is and how do we go about identifying the decision variables.

8. Example Formulation
Defining the decision variables of the problem, the formulation will be:
(1) is the total revenue generated by producing x1 desks and x2 tables.
(2) is the constraint for the total number of lumbers that exists.
(3) is the constraint for the total carpentry hours available.
(4) is the non-negativity constraints, saying that production can not be negative.

9. Feasible Region
Feasible region is defined by the set of constraints of the problem, which is all the possible points that satisfy the all the constraints.
In production planning problem, the feasible set defined by the constraints looks like this plot:
Red line is for the first constraint
And the green line is the second constraint

10. Feasible Region The red line represents the line for (2)
The green line represents the line for (3)
The region with blue line represents entire feasible region of the problem.

11. Geometric Solution
To find the solution of LP problems, the line for objective function should be plotted and the point in feasible region which maximize (minimize) the function is the solution.
Dashed parallel lines are representing the objective function line for 2 different values.
Note that to draw the objective function line, we have to fix the value of Z (revenue) so that we can draw the line, and the goal is to find the biggest Z such that the line still intersects the feasible region.

12. Simplex and LP
Solution of LP problems are always at the extreme points of the feasible region:
Thanks to the properties of linear constraints and convex sets, the LP solution is always on the edge of the feasible region and if we know the edges of the feasible region we can try them all and fine the best solution.
Pay attention to the extreme points {(0,0),(10,0),(0,10),(6.66,6.66)}

13. Simplex and LP
Knowing the structure of LP problems and where the solutions are, there are finite number of extreme points that need to be checked in order to find the optimal solution.
However the number of extreme points are finite, but for a problem with n variables and m constraints the upper bound for the number of extreme points is (n>m) :
Simplex is an algorithm that systematically explores the extreme points of the feasible region and moves from an extreme point to its improving neighbor.
For C(50, 10) the number is 10,272,278,170 which is huge. And the problems of real size are much larger (problems with million variables and constraints are common)
Knowing how big the number of edges can be, checking all of them is not the smartest way and it is computationally vey expensive (or impossible)
Simplex algorithm, takes advantage of the properties of convex optimization problems and moves on the edges of the feasible region trying to improve the objective function by moving to a better neighbor solution.

14. LP Solution (Production Planning)
For the production planning example the solutions are:
Since the number of desks and tables can not be fractional, we might be able to round the solution down to get our integer solution, with a revenue of 210.
Does rounding the solution yield the optimal solution to integer problem? X1 X2
Revenue 10
150
200
6.66
233.33

15. Optimization Software
There are various optimization tools developed based on Simplex and other optimization algorithms that are developed for LP problems.
The MPL code for our example is :
TITLE
Production_Planning
INDEX
Product=(Desk,LunchTable);
Resource=(Lumber, Carpentry);
DATA
Price[Product] = (15 , 20);
Availability[Resource] = (20 , 20);
Usage [Product, Resource]= (1 , 2,
2 , 1);
VARIABLES
X[Product];
MODEL
Max Revenue= SUM (Product: Price * X);
SUBJECT TO
ResourceLevel[Resource]: SUM (Product: Usage * X) <= Availability;
END
Software such as MPL, AMPL, Lingo, Gams are used to generate mathematical models efficiently using algebraic languages and taking advantage of data structure in order to be able to handle problems with high number of variables and constraints.
Usually these languages are very close to the condensed algebraic formulation of the problem which makes the use of these tools very easy.

16. Integer Optimization
Many of decisions and variables in real cases are inherently integers.
In our example the number of products to be manufactured can not be fractional and it has to be integer.
Many of the problems that are faced in real cases, have choosing an option in their structure, for example:
Which stock to invest in
Which route to choose to move in
Which arc of the graph to choose
Integer optimization is being used extensively . Many of the real life problems can only be modeled as integer programming problems due to the structure of the decisions that have to be made.
Other famous examples are :
Traveling salesman problem,
Scheduling problems
Facility location problem

17. Integer Variables
The only difference between LP problems and IP problems in structure is the definition of the decision variables.
How ever this change will fundamentally change the characteristics of the problem in hand.
LP problems have a convex set as their feasible region while IP problems have set of integer vectors as their feasible region (non-convex).
Dealing with non-convex optimization problems are much harder than convex optimization problems.
In order to have a picture in mind, the feasible set of the production planning problem for integer variables will only be the integer points inside the LP feasible region which is a disjoint set of points.

18. How to Solve IP Problems
LP relaxation: LP relaxation of and IP problem is when we allow the variables to take on fractional values (Like our example).
The idea to solve IP problems is to solve series of specific LP problems which will guide us towards an integer feasible solution.
This method is called Branch-and-Bound.
In this method, we branch on fractional values and make them to be the lower or upper integer value.
It is important to note that rounding the LP solution will not necessarily give the optimal or even feasible solution.
Branch and bound is method that implicitly enumerates all the feasible point of the feasible region.

19. Example (Revisited)
Remember we got 210 as our objective function by rounding down the LP solutions.
If we solve the same problem as an IP the optimal solution will be:
It is important to note that in our problem, rounding up the solution would give us infeasible solution to the problem.

20. Important Characteristics of IP Problems
IP problems generally are much more complicated than LP problems.
Many of IP classes are known as NP-hard problems, which takes very long processing time for computers to find the optimal solution (Years, for problems of real case size).
There has been extensive research in this area in order to find ways to be able to tackle these problems.
Cutting Planes
Branch-and-Cut
Column Generation techniques
Branch-and-Price
Meta Heuristics