IE 312 Review 1

The Process 2 Problem Model Conclusions Problem Formulation Analysis

Problem Formulation 3  What is the objective?  Maximize profit,  Minimize inventory,...  What are the decision variables?  Capacity, routing, production and stock levels  What are the constraints?  Capacity is limited by capital  Production is limited by capacity

Analysis 4  Optimization Algorithm  Computer Implementation  Excel (or other spreadsheet)  Optimization software (e.g., LINDO)  Modeling software (e.g., LINGO) Increasing Complexity

Optimization Algorithms 5  Find an initial solution  Loop:  Look at “ neighbors ” of current solution  Select one of those neighbors  Decide if to move to selected solution  Check stopping criterion

Tractability and Validity 6  Is a model tractable?  Can it be solved, or is it too complex and/or large?  Is a model valid?  Do we reach the same conclusions experimenting with the model as we would experimenting with the real system?

Mathematical Programming 7  General Model  Minimize or maximize some objective function  Subject to some constraints

Linear versus Nonlinear 8  A function is linear if it is a weighted sum of the decision variables, otherwise nonlinear  A linear program (LP) has a linear objective function f and constraint functions g 1,…,g m  A nonlinear program has at least one nonlinear function

Integer Programs 9  A variable is discrete if it can only take a limited or countable number of values  A variable is continuous if it can take values in a specific interval  Mathematical programs can be  continuous  discrete (integer/combinatorial)  mixed

Classification Summary 10 Linear Program (LP)ILP Nonlinear Program (NLP)INLP Integer Program (IP) Increased difficulty

Solution Techniques 11  Models  Thousands (or millions) of variables  Thousands (or millions) of constraints  Complex models tend to be valid  Is the model tractable?  Solution techniques

Improving Search 12  Begin at feasible solution  Advance along a search path  Ever-improving objective function value  Neighborhood: points within small positive distance of current solution  Stopping criterion: no improvement

Local Optima 13  Improving search finds a local optimum  May not be a global optimum (only a heuristic solution)  Tractability: for some models there is only one local optimum (which hence is global)

Tractability 14  The most tractable models for improving search are models with  unimodal objective function (linear special case)  convex feasible region (linear special case)  Here every local optimum is global

Solving LP Models 15  Improving Search  Unimodal  Convex feasible region  Should be successful!  Special Form of Improving Search  Simplex method

Optimal Solutions 16  Every optimal solution is a boundary point  We can find an improving direction whenever we are at an interior point  If optimum unique the it must be an extreme point of the feasible region  If optimal solution exist, an optimal extreme point exists

LP Standard Form 17  Easier if we agree on exactly what a LP should look like  Standard form  only equality main constraints  only nonnegative variables  variables appear at most once in left-hand-side and objective function  all constants appear on right hand side

Extreme Points 18  Know that an extreme point optimum exists  Will search trough extreme points  An extreme point is define by a set of constraints that are active simultaneously

Improving Search 19  Move from one extreme point to a neighboring extreme point  This defines the directions  Simplex is a special case of improving search that only uses these directions  Extreme points are adjacent if they are defined by sets of active constraints that differ by only one element  An edge is a line segment determined by a set of active constraints

Basic Solutions 20  An easy way of keeping track of the Simplex directions is by using some linear algebra  Extreme points are defined by set of active nonnegativity constraints  A basic solution is a solution that is obtained by fixing enough variable to be equal to zero, so that the equality constraints have a unique solution

Simplex Algorithm 21  Starting point  A basic feasible solution (extreme point)  Direction  Follow an edge to adjacent extreme point:  Increase one nonbasic variable  Compute changes needed to preserve equality constraints  One direction for each nonbasic variable

Sensitivity Analysis 22  Basic Question: How does our solution change as the input parameters change?  The objective function?  More/less profit or cost  The optimal values of decision variables?  Make different decisions!  Why?  Only have estimates of input parameters  May want to change input parameters

What We Know 23  Qualitative Answers for All Problems  Quantitative Answers for Linear Programs (LP)  Dual program  Same input parameters  Decision variables give sensitivities  Dual prices  Easy to set up  Theory is somewhat complicated

Discrete Optimization 24  Wide range of problems  LPs with additional integer constraints  Knapsack & capital budgeting  Set packing, covering, and partitioning  Traveling salesman and routing  How do we solve these problems?  Much more difficult that the corresponding continuous problems

Exponential Growth 25 2k2k k2k2

Relaxations 26  Discrete problems are hard  Relax them to an easier problem  Our original Swedish Steel formulation was a relaxation of the real problem  Can always relax a zero-one problem by allowing the variable to take any value on the interval [0,1]

Continuous Relaxations 27  LP relaxation or continuous relaxation is when a model with discrete variables is assume to have only continuous variables If a constraint relaxation is infeasible, the original model is infeasible as well

Relationship with Relaxed Solution 28 The optimal value of the relaxed problem is an upper/lower bound for the original maximization/minimization model Feasible solutions in relaxed model Feasible solutions in original model Optimum

Final Relationship 29 If an optimal solution to the relaxed model is feasible for the original model, it is also optimal for the original model. Feasible solutions in relaxed model Feasible solutions in original model Optimum for the relaxed model

Further Bound 30  Have that the optimal value of the relaxed problem is an upper/lower bound for the original maximization/minimization model The objective value of any integer feasible solution to a maximization/minimization problem is a lower/upper bound on the integer optimal value

Rounding Example 31 Solve LP relaxation for ( x 1, x 2 )=(376/193,950/193) Integer Programming (IP) solution is ( x 1, x 2 )=(5,0)

More Efficient Enumeration 32  Looking at every solution takes prohibitively long  Can we somehow account for every solution without actually looking at every solution?  Would like to be able to eliminate a bunch of solutions without evaluating them!  Branch and bound

Using Relaxations 33  If a relaxation of a candidate problem is infeasible that branch can be fathomed  If optimal solution of a relaxed candidate problem is worse than incumbent then fathom branch  If optimal solution of a relaxed candidate problem is feasible for full candidate then fathom branch and update candidate if necessary

Knapsack Model 34  Intuitive idea: what is the most valuable collection of items that can be fit into a backpack?

Capital Budgeting 35  Multidimensional knapsack problems are often called capital budgeting problems  Idea: select collection of projects, investments, etc, so that the value is maximized (subject to some resource constraints)

Assignment Problems 36  Assignment problems deal with optimal pairing or matching of objects in two distinct sets  Decision variable  Let A be the set of allowed assignments and c ij be the cost of assigning i to j.

Traveling Salesman Problem (TSP) 37 Ames Fort Dodge Boone Carroll Marshalltown West Des Moines Waterloo What is the shortest route, starting in Ames, that visits each city exactly ones?

Solving TSP 38  We can use branch-and-bound to get an exact solution to the TSP problem  As always, the key to implementing branch-and-bound is to relax the problem so that we can easily solve the relaxed problem, but we still get good bounds  How can we relax the TSP?

Single Machine Sequencing 39  A set of jobs  Find the best order in which to sequence the jobs  Solution  Tabu search  Branch-and-bound

Neighborhood/Local Search 40  Find an initial solution  Loop:  Look at “ neighbors ” of current solution  Select one of those neighbors  Decide if to move to selected solution  Check stopping criterion

Random Search Methods 41  Tabu Search  Maintain a tabu list of solution changes  A move made entered at top of tabu list  Fixed length (5-9)  Neighbors restricted to solutions not requiring a tabu move