Chap 10. Integer Prog. Formulations

Slides:



Advertisements
Similar presentations
1 Material to Cover  relationship between different types of models  incorrect to round real to integer variables  logical relationship: site selection.
Advertisements

BU Decision Models Integer_LP1 Integer Optimization Summer 2013.
Introduction to Algorithms
Lecture 10: Integer Programming & Branch-and-Bound
Chapter 7 Maximum Flows: Polynomial Algorithms
Integer Programming 3 Brief Review of Branch and Bound
Branch and Bound Searching Strategies
1 Optimization problems such as MAXSAT, MIN NODE COVER, MAX INDEPENDENT SET, MAX CLIQUE, MIN SET COVER, TSP, KNAPSACK, BINPACKING do not have a polynomial.
Mathematical Modeling and Optimization: Summary of “Big Ideas”
Approximation Algorithms
1 Branch and Bound Searching Strategies 2 Branch-and-bound strategy 2 mechanisms: A mechanism to generate branches A mechanism to generate a bound so.
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract.
CPSC 411, Fall 2008: Set 4 1 CPSC 411 Design and Analysis of Algorithms Set 4: Greedy Algorithms Prof. Jennifer Welch Fall 2008.
LP formulation of Economic Dispatch
Daniel Kroening and Ofer Strichman Decision Procedures An Algorithmic Point of View Deciding ILPs with Branch & Bound ILP References: ‘Integer Programming’
Decision Procedures An Algorithmic Point of View
The Theory of NP-Completeness 1. Nondeterministic algorithms A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking If the checking.
1.3 Modeling with exponentially many constr.  Some strong formulations (or even formulation itself) may involve exponentially many constraints (cutting.
Approximating Minimum Bounded Degree Spanning Tree (MBDST) Mohit Singh and Lap Chi Lau “Approximating Minimum Bounded DegreeApproximating Minimum Bounded.
1 Introduction to Approximation Algorithms. 2 NP-completeness Do your best then.
Operations Research Assistant Professor Dr. Sana’a Wafa Al-Sayegh 2 nd Semester ITGD4207 University of Palestine.
Linear Programming Data Structures and Algorithms A.G. Malamos References: Algorithms, 2006, S. Dasgupta, C. H. Papadimitriou, and U. V. Vazirani Introduction.
Notes 6IE3121 Knapsack Model Intuitive idea: what is the most valuable collection of items that can be fit into a backpack?
15.082J and 6.855J and ESD.78J Lagrangian Relaxation 2 Applications Algorithms Theory.
Notes 5IE 3121 Knapsack Model Intuitive idea: what is the most valuable collection of items that can be fit into a backpack?
Lecture 6 – Integer Programming Models Topics General model Logic constraint Defining decision variables Continuous vs. integral solution Applications:
Chapter 1. Formulations 1. Integer Programming  Mixed Integer Optimization Problem (or (Linear) Mixed Integer Program, MIP) min c’x + d’y Ax +
and 6.855J Lagrangian Relaxation I never missed the opportunity to remove obstacles in the way of unity. —Mohandas Gandhi.
CSE 589 Part VI. Reading Skiena, Sections 5.5 and 6.8 CLR, chapter 37.
1 Branch and Bound Searching Strategies Updated: 12/27/2010.
Integer Linear Programming Terms Pure integer programming mixed integer programming 0-1 integer programming LP relaxation of the IP Upper bound O.F. Lower.
Branch-and-Cut Valid inequality: an inequality satisfied by all feasible solutions Cut: a valid inequality that is not part of the current formulation.
Integer Programming (정수계획법)
Implicit Hitting Set Problems Richard M. Karp Erick Moreno Centeno DIMACS 20 th Anniversary.
OR Chapter 8. General LP Problems Converting other forms to general LP problem : min c’x  - max (-c)’x   = by adding a nonnegative slack variable.
1.2 Guidelines for strong formulations  Running time for LP usually depends on m and n ( number of iterations O(m), O(log n)). Not critically depend on.
Chapter 2. Optimal Trees and Paths Combinatorial Optimization
8/14/04 J. Bard and J. W. Barnes Operations Research Models and Methods Copyright All rights reserved Lecture 6 – Integer Programming Models Topics.
Lecture 6 – Integer Programming Models Topics General model Logic constraint Defining decision variables Continuous vs. integral solution Applications:
IE 312 Review 1. The Process 2 Problem Model Conclusions Problem Formulation Analysis.
Branch and Bound Searching Strategies
1.3 Modeling with exponentially many constr. Integer Programming
TU/e Algorithms (2IL15) – Lecture 12 1 Linear Programming.
Approximation Algorithms Duality My T. UF.
::Network Optimization:: Minimum Spanning Trees and Clustering Taufik Djatna, Dr.Eng. 1.
EMIS 8373: Integer Programming Combinatorial Optimization Problems updated 27 January 2005.
Discrete Optimization MA2827 Fondements de l’optimisation discrète Material from P. Van Hentenryck’s course.
The Theory of NP-Completeness
The minimum cost flow problem
6.5 Stochastic Prog. and Benders’ decomposition
Chapter 5. Optimal Matchings
1.3 Modeling with exponentially many constr.
3.3 Applications of Maximum Flow and Minimum Cut
Chapter 6. Large Scale Optimization
Integer Programming (정수계획법)
Chapter 6. Large Scale Optimization
András Sebő and Anke van Zuylen
Chapter 1. Formulations (BW)
1.3 Modeling with exponentially many constr.
Integer Programming (정수계획법)
Flow Feasibility Problems
11.5 Implicit Partitioning/Packing Problems
The Theory of NP-Completeness
11.5 Implicit Partitioning/Packing Problems
6.5 Stochastic Prog. and Benders’ decomposition
Chapter 1. Formulations.
Branch-and-Bound Algorithm for Integer Program
1.2 Guidelines for strong formulations
Chapter 6. Large Scale Optimization
1.2 Guidelines for strong formulations
Presentation transcript:

Chap 10. Integer Prog. Formulations Mixed Integer Programming Problem minimize c’x + d’y subject to Ax + By = b x, y  0 x integer Integer programming problem: no continuous variables y Zero-one (or binary) integer programming problem: no continuous variables y and x restricted to be 0 or 1 Note that the feasible solution set is no longer convex compared to the linear programming problem. Powerful modeling tool, but usually difficult to solve. Linear Programming 2009

10.1 Modeling Techniques Binary choice Example: zero-one knapsack problem Given n items, j th item has weight wj and value cj. Given capacity of the sack is K. Select the items to carry in the sack which maximizes the total value. maximize j = 1, …, n cjxj subject to j = 1, … , n wjxj  K xj  {0, 1}, j = 1, … , n (only one constraint. typical case in resource allocation problems. Has many applications in theory and algorithms.) Variations: integer knapsack problem, min form, precedence-constrained knapsack. Linear Programming 2009

Example: Facility location problem (uncapacitated) Forcing constraints decision A can be made ( x = 1 ) only if decision B also has been made ( y = 1 ) ( If y = 0 (decision B not made), then x = 0 (decision A cannot be made)  x  y , x, y  {0, 1} Example: Facility location problem (uncapacitated) n potential facility locations, m clients who needs to be serviced from these locations. fixed cost cj of opening a facility at location j. cost dij of serving client i from facility j. Select a set of facility locations and assign each client to one facility , while minimizing the total cost. Linear Programming 2009

minimize j = 1, …, n cjyj + i = 1, …, m j = 1, …, n dijxij Formulation minimize j = 1, …, n cjyj + i = 1, …, m j = 1, …, n dijxij subject to j = 1, … , n xij = 1 , for all i xij  yj , for all i, j xij , yj  {0, 1}, for all i, j ( or i xij  myj for all j ) yj = 1 if facility j is open, 0 otherwise. xij = 1 if client i is served by facility j, 0 otherwise. (or we may set 0  xij  1, and interpret it as the fraction of customer i’s demand serviced by facility j) Note that even without the binary constraints on xij , each client will be served entirely from a location which can serve it most cheaply. Then the meaning of xij is changed as the fraction of client i’s demand satisfied from facility j. Linear Programming 2009

Relations between variables Generalized upper bound constraints (GUB constraints) j = 1, … , n xj  1 xj  {0, 1} for all j Linear Programming 2009

Disjunctive constraints at least one of two constraints (a’x  b, c’x  d) needs to be satisfied. Let M be a sufficiently large number a’x  b - My, c’x  d - M(1 – y) y  {0, 1} In the text, following form is used assuming a, c  0 and x  0. a’x  yb, c’x  (1 – y)d Note that the feasible solution set is not convex for disjunctive constraints. Extension for more than 2 constraints. Linear Programming 2009

Restricted range of values x  { a1, a2, … , am } x = j = 1, …, m ajyj j = 1, …, m yj = 1, yj  {0, 1} Linear Programming 2009

Arbitrary piecewise linear cost functions f(x) x = i = 1, … , k i ai f(x) = i = 1, … , k i f(ai) x a1 a2 a3 a4 1 y1 2 y2 3 y3 4 Linear Programming 2009

minimize i = 1, … , k i f(ai) subject to i = 1, … , k i = 1 , 1  y1 , i  yi – 1 + yi , i = 2, … , k – 1 k  yk - 1 , i = 1, … , k-1 yi = 1 , i  0 , yi  {0, 1} Linear Programming 2009

f(x) c4 c2 c3 c1 K x L2 L3 L1 (x3) (x1) (x2) Alternative formulation

Want to express x as x = x1 + x2 + … + xk Formulation: Want to express x as x = x1 + x2 + … + xk Ljwj  xj  Ljwj-1 j = 1, 2, … , k w0 = 1 wj  {0, 1} j = 1, 2, … , k xj  0 j = 1, 2, … , k note that we have wj  wj-1 for j = 1, 2, … , k. If wj = 0 for some j  wk = 0 for k  j+1 If wj = wj-1 = 1  xj = Lj If wj = wj-1 = 0  xj = 0 If wj = 0, wj-1 = 1  0  xj  Lj Write the piecewise linear function as K + c1x1 + c2x2 + … + ckxk . OR-1 2009

Set covering, set packing, set partitioning problems M = { 1, … , m}, N = { 1, … , n} M1 , M2 , … , Mn  M weight cj for each Mj F  N is a cover of M if j  F Mj = M F  N is a packing of M if Mj  Mk =  for all j, k  F, j  k. F  N is a partition of M if it is both a cover and a packing of M. Find a cover F of minimum weight. Find a packing F of maximum weight. Find a partitioning of (min, max) weight. Linear Programming 2009

define xj = 1 if j  F. ( i.e. Mj is selected) 0, otherwise A : m  n, (0, 1) matrix Let aij = 1 , if i  Mj 0 , otherwise define xj = 1 if j  F. ( i.e. Mj is selected) 0, otherwise constraints Ax  e , Ax  e , Ax = e, where e is an m-dimensional vector with all components equal to 1. Has many practical applications in scheduling, graphs, ... Linear Programming 2009

Boolean Quadratic Function min f(x) = i = 1n dixi + i, j, i  j cijxixj , xi  {0, 1} for all i Let yij = xixj  min i = 1n dixi + i, j, i  j cijyij xi + xj – yij  1 -xi + yij  0 -xj + yij  0 for all i, j, i  j xi, yij  {0, 1} for all i, j constraints ensure that yij = 1  xi = 1, xj = 1 Linear Programming 2009

max (i, j)  E cij ( xi(1-xj) + (1-xi)xj ) xi  {0, 1} for all i Ex: quadratic knapsack problem, max cut of a graph, lift-and-project algorithm for 0-1 IP. Def: Given a graph G = (V, E), and subset S  V of vertices, the set of edges with exactly one endpoint in S is called a cut (relative to S). Given G = (V, E), and edge weights cij, e = (i, j)  E, find a maximum weight cut of G. max (i, j)  E cij ( xi(1-xj) + (1-xi)xj ) xi  {0, 1} for all i (may add constraint x1 = 1 ) Note: max cut problem is difficult to solve (NP-hard), but min cut problem is easy (max-flow min-cut theorem). Linear Programming 2009

10.2 Guidelines for strong formulations Time to solve LP generally depends on the number of constraints (m) and the number of variables (n). Empirically, number of iterations in simplex method: O(m), O(log n) Each iteration: O(m2) However, for integer programming problems, running time is usually exponential (NP-hard problems) of the problem size and very erratic. Strong formulation is important. A formulation which describes the feasible integer points closely is desirable. (Why?) Linear Programming 2009

Branch-and-Bound Algorithm for IP (IP) minimize c’x subject to Ax  b x  0 and integer Let zIP be the optimal value of IP and zLP be the optimal value of the LP relaxation of IP, i.e. problem with integer requirement dropped. Then zLP  zIP holds, i.e. zLP provides a lower bound on zIP . Usually upper bound can be obtained from a feasible solution. If lower bound = upper bound, then optimal. If LP relaxation gives an integer optimal solution, then it is optimal. Otherwise, divide the feasible solution set of IP into two (or more) disjoint subsets and continue to solve the LP relaxation for each subset. Linear Programming 2009

Then solve LP relaxation for S1 and S2 respectively. Ex: 0-1 IP Let S be the set of feasible solutions to IP. If xj* = , 0 <  <1, in an optimal solution x* to LP relaxation, divide S into two sets S1, S2, where S1 = S  { x: xj = 0}, S2 = S  { x: xj = 1}. Then solve LP relaxation for S1 and S2 respectively. Important tool to facilitate search: If we have current best integer solution (incumbent) with value z0 and LP relaxation of a subproblem gives value z’ with z0  z’, then the subproblem does not include a better solution than the current incumbent and the subproblem can be pruned. Therefore, if we can find a strong lower bound (high value), it is more likely that the subproblem can be pruned earlier in the search procedure, hence mitigating the need to seach the subproblem further (prevent the explosion of the search tree). Linear Programming 2009

Why strong formulation? Let P1, P2 be the set of feasible solutions to the LP relaxation of two different formulations. ( P1  P2, S  P1, S  P2 ) Then the optimal values of the LP relaxations of the two formulations give zP2  zp1 . Thus formulation for P1 gives stronger bound. ex) formulations for the facility location problem P2 P1 Linear Programming 2009

10.3 Modeling with exponentially many constraints Minimum spanning tree problem Graph G = (N, E), |N| = n, |E| = m Edge weight ce , e  E Find min weight spanning tree. (Recall Kruskal, Prim’s alg.) Tree : connected acyclic graph (spanning tree: all nodes of G included in the tree) Forest: acyclic graph A spanning tree has n – 1 edges and connected ( iff condition) Linear Programming 2009

Linear Programming 2009

(S) = { ( i, j )  E : i  S, j  S} ( cut defined by S) For S  N, define E(S) = { ( i, j )  E : i, j  S } (S) = { ( i, j )  E : i  S, j  S} ( cut defined by S) Subtour elimination formulation (no subtour allowed) minimize e  E ce xe subject to e  E xe = n – 1 e  E(S) xe  |S| - 1 , S  N, S  , N, xe  { 0, 1 }. Has exponential number of constraints ( 2n – 1 ) Linear programming relaxation can be obtained by replacing xe  { 0, 1 } with 0  xe  1. If not have e  E xe = n – 1, obtain formulation for min weight forest (negative arc costs may be allowed) Linear Programming 2009

Cutset formulation ( use connectedness) Use e  (S) xe  1 , S  N, S  , N. Comparing two formulations: The polyhedron of LP relaxation of subtour elim. formul. is properly contained in the one for cutset formulation, hence it is better. It can be shown that LP relaxation of subtour elim. formul. gives integer optimal solutions. Why consider IP formulation although there exist good algorithms? Algorithms fail if problem structure changed a little bit: degree constrained spanning tree problem, Shortest total path length spanning tree problem, Steiner tree problem, capacitated spanning tree problem, … Linear Programming 2009

Traveling salesman problem (undirected) Undirected graph G = (N, E), edge cost ce , e  E Find a minimum cost tour Cutset formulation minimize e  E ce xe subject to e   ({i}) xe = 2 , i  N e  (S) xe  2 , S  N, S  , N, xe  { 0, 1 }. Linear Programming 2009

Subtour elimination formulation minimize e  E ce xe subject to e   ({i}) xe = 2 , i  N e  E(S) xe  |S| - 1 , S  N, S  , N, xe  { 0, 1 }. LP relaxations of both formulations give the same solution set. Linear Programming 2009

Remarks For directed version of the TSP problem, the following formulation is possible, which is smaller in size. But it is a bad formulation. (refer exercise 10.15 in text page 477) ui – uj + nyij  n – 1 , ( i, j )  A, i, j  1, { i : ( i, j )  A} yij = 1 , j  N { j : ( i, j )  A} yij = 1 , i  N yij  { 0, 1 }, i, j  N Note that, uj’s are continuous variables in the above formulation. Undirected TSP is a special case of directed case, we may replace each edge by two directed arcs with opposite direction and having the same costs as the edge. Linear Programming 2009

Is the formulation correct? The formulation has u, y variables. If (u*, y*) feasible, we only read y* values ( projection of (u*, y*) to y space) We need to show that (1) any tour solution y* satisfies the constraints and (2) any non-tour solution does not satisfy the constraints. (1) For any tour y*, if node i is k-th node in the tour, assign ui = k. (2) If y* is 0,1 and satisfies degree constraints, it is either a tour or consists of subtours. If subtours exist, there is one that does not include node 1. Add the constraints ui – uj + nyij  n – 1 along the arcs in the subtour. Linear Programming 2009

Comparing the LP relaxation of the cutset formulation (say A) (in directed case version) and the LP relaxation of the previous formulation (B): It can be shown that the projection of the polyhedron B onto y space gives a polyhedron which completely contains A (the inclusion can be strict), hence cutset formulation (or subtour elimination formulation) is stronger. ( Let P = {(x, y)Rn  Rp : Ax + Gy  b}, then Projx (P) = { x: (x, y)  P for some y} ) Although the previous formulation is not strong, it can be an alternative to use if you only have a generic IP software to use, not the sophisticated one to handle the subtour elimination constraints. Linear Programming 2009

How to Solve the LP relaxation of the Cut-Set Formulation? (many constr.) Solve LP relaxation (w/o cut-set constraints) If y* tour, stop. O/w find violated cut-set Solve LP after adding the Cut-set constraint.  violated cut-set? Y N Stop Linear Programming 2009

Branch-and-cut approach ( cutting plane alg.) If the obtained solution is not a tour, branch and apply the same procedure again. Choose the best solution Branching : If xij*  0, 1, solve two subproblems after setting xij = 0, and xij = 1. Branch-and-cut approach ( cutting plane alg.) Ideas for TSP formulation can be used for various routing, sequencing problems. Branch-and-cut Ideas useful to solve many difficult IP problems. What can we do for the LP with many variables? For the LP with many vars. and constraints? TSP site: http://www.tsp.gatech.edu/ Linear Programming 2009