Introduction to Network Design – Optimization Background ECE 780 Introduction to Network Design – Optimization Background Includes slide materials developed by Wayne D. Grover, John Doucette, Dave Morley © Wayne D. Grover 2002, 2003, 2004

Outline Optimization Mesh-Restoration Concept Mathematical Programming Linear Programming (LP) Formulating LP Problems Solving LP Problems Integer Programming Solving MIP Problems Algebraic Expression of LP/IP Problems Mesh-Restoration Concept Terminology Spare Capacity Sharing Spare Capacity Placement (SCP) SCP Integer Programming problem

Mathematical Programming Definition “A Mathematical Programming Model is a mathematical decision model for planning (programming) decisions that optimize an objective function and satisfy limitations imposed by mathematical constraints.” 1 General Symbolic Model Maximize (or minimize): Objective Subject to: Constraints … … where are the decision variables. 1 T.W. Knowles, Management Science: Building and Using Models, Irwin, 1989.

Mathematical Programming Types of Mathematical Programs: Linear Programs (LP): the objective and constraint functions are linear and the decision variables are continuous. Integer (Linear) Programs (IP): one or more of the decision variables are restricted to integer values only and the functions are linear. Pure IP: all decision variables are integer. Mixed IP (MIP): some decision variables are integer, others are continuous. 1/0 MIP: some or all decision variables are further restricted to be valued either “1” or “0”. Nonlinear Programs: one or more of the functions is not linear.

Linear Programming General Symbolic Form Maximize: Subject to: Objective Subject to: Constraints … Bounds … where are the model parameters.

Linear Programming General Restrictions All decision variables must be nonnegative, Constant terms cannot appear on the LHS of a constraint. No variable can appear on the RHS of a constraint. No variable can appear more than once in a function, i.e. objective or constraint. Steps for Formulating LP Models Construct a verbal model. Define the decision variables. Construct the symbolic model.

Formulating LP Problems An example2 A steel company must decide how to allocate production time on a rolling mill. The mill takes unfinished slabs of steel as input and can produce either of two products: bands and coils. The products come off the mill at different rates and also have different profitabilities: Tons/ Profit/ hour ton Bands 200 $25 Coils 140 $30 The weekly production that can be justified based on current and forecast orders are: Maximum tons: Bands 6,000 Coils 4,000 2 from, R. Fourer, D. Gay, B. Kernighan, AMPL, Boyd & Fraser, 1993, pp. 2-10.

Formulating LP Problems An example (cont’d) The question facing the company is: If 40 hours of production time are available, how many tons of bands and coils should be produced to bring in the greatest total profit? Constructing the Verbal model Put the objective and constraints into words. For constraints, use the form {a verbal description of the LHS} {a relationship} {an RHS constant} Maximize: total profit Subject to: total number of production hours  40 tons of bands produced  6,000 tons of coils produced  4,000

Formulating LP Problems Defining the Decision Variables XB number of tons of bands produced. XC number of tons of coils produced. Construct the Symbolic Model Maximize: Subject to:

Solving LP Problems Graphical Solution Method Constraints Profit 6000 Coils Coils Constraints 220K Profit 6000 6000 Hours 192K 4000 4000 Optimal solution Feasible region 2000 2000 120K Bands Bands 2000 4000 6000 8000 2000 4000 6000 8000

Solving LP Problems 4 Possible Outcomes Unique Optimal Solution Alternate Optimal Solutions Unbounded Optimal Solution No Feasible Solution

Solving LP Problems Simplex method AMPL and CPLEX Efficient algorithm to solve LP problems by performing matrix operations on the LP Tableau. Developed by George Dantzig (1947). Can be used to solve small LP problems by hand. AMPL and CPLEX AMPL: modeling language (and software) for designing large and complex LP/IP problems. CPLEX: software package (“solver”) to solve large and complex LP/IP problems. Sub-Optimal Algorithms (Heuristics) Simulated annealing. Genetic algorithms. Tabu search. Many others, often very specific to the type of problem.

Integer Programming Convert Example to Integer Program Assume that orders for bands and coils are placed (and filled) in 1,000s of pounds only. Although feasible region is greatly reduced, problem becomes much more difficult. New Symbolic Model Let the new decision variables be the number of 1000 pound “units” or orders of bands and coils. Maximize: Subject to: integer integer

Integer Programming Graphical Solution Method 2 4 6 8 2 4 6 8 Feasible integer solutions Bands Coils $185K Optimal integer solution ($185K)

Solving MIP Problems Branch-and-Bound Procedure The solution space consists of a tree of LP subproblems, in which each integer variable is either fixed or its integrality constraint is “relaxed.” The root node of the tree is the LP relaxation of the problem, i.e. all integer variables are relaxed. The relaxation can result in an all integer solution, or a fractional solution (some decision variables are non-integer). If the solution of the relaxation has fractional-valued integer variables, a fractional variable is selected for branching and two new subproblems are generated, each with more restrictive bounds on the branching variable. The subproblems can result in an all integer solution, an infeasible problem or another fractional solution. If the solution is fractional, the process is repeated. Branches are “fathomed” if the subproblem is infeasible, the objective value is worse than the current best integer solution or the subproblem gives an integer solution.

Solving MIP Problems Branch-and-Bound Tree for the Bands and Coils IP Problem Supplement: see “Appendix” slides at end for detailed working of this example 1 2 3 6 4 5 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 6<=XB<=6 2<=XC<=4 *Infeasible 0<=XB<=5 3<=XC<=4 Obj. = 183K XB = 3.00 XC = 3.71 0<=XC<=4 Obj. = 192K XC = 1.40 Obj. = 189K XB = 5.14 XC = 2.00 XB = 5.00 XC = 2.10 2<=XC<=2 Obj. = 185K 8 7 9 10

Algebraic Expression of LP/IP Problems Why use it? Most LP/IP problems are quite large and it becomes very cumbersome to describe them by explicitly giving each linear function, equality, and inequality in full. It is desirable to model problems in a more general fashion (e.g. give an IP for optimally designing a mesh-restorable network in general as opposed to doing so for a specific network).

Algebraic Expression of LP/IP Problems Basic Production Model (Revisited) Original Model Algebraic Model Maximize: Subject to: Given: a set of products hours available at the mill tons per hour of product j, for each profit per ton of product j, for each maximum tons of product j, for each Define variables: tons of product j to be made, for each \Problem name: prob.lp Maximize 25 XB + 30 XC Subject To 0.005 XB + 0.007143 XC <= 40 Bounds 0 <= XB <= 6000 0 <= XC <= 4000 End

Solving Network Flow Problems with LP/IP Models

Network flow LP formulations Example: LP to find max flow between nodes 1-5 1 5 2 4 3 20 10 8 Edge capacities: Define directional flow variables: x42 1 5 2 4 3 x12 x13 x32 x23 x45 x24 x35 “trans-shipment nodes” source sink

Network flow LP formulations (2) “transportation problem” or “arc-flow” approach To maximize (1  5) flow : /* using lp_solve syntax */ max: x12 + x13 ; /* (or x35 + x45) */ subject to constraints: c1: x12 + x13 = x45 + x35 ; /* source = sink */ c2: x12 + x32 - x23 + x42 - x24 = 0 ; /* node 2 trans-shipment*/ c3: x13 + x23 - x32 - x35 = 0 ; /* node 3 trans-shipment*/ c4: x24 - x45 - x42 = 0 ; /* node 4 trans-shipment*/ x42 1 5 2 4 3 x12 x13 x32 x23 x45 x24 x35

Network flow LP formulations (2) “transportation problem” or “arc-flow” approach Continued.... Also subject to (span capacity constraints): x12 <= 20 ; x13 <= 8 ; x24 <= 10 ; x42 <= 10 ; x23 <= 2 ; x32 <= 2 ; x35 <= 20 ; x45 <= 5 ; 1 5 2 4 3 20 10 8 x42 1 5 2 4 3 x12 x13 x32 x23 x45 x24 x35

Network flow LP formulations (3) “transportation problem” or “arc-flow” approach Symbolically.... s.t. Where: E = set of edges that exist N = set of nodes sij = spare capacity on edge ij (= sji)

Network flow LP formulations (4) Alternate approach: “flow assignment to routes” or “arc-path” approach Symbolically.... s.t. Where: E = set of edges that exist (indexed by k) P 15= set of “eligible” distinct routes between nodes 1 and 5 (source-sink) sk = spare capacity on edge k if the i th distinct route crosses span k. Zero otherwise

Network flow LP formulations (5) “flow assignment to routes” or “arc-path” approach - example Identify all distinct routes between source- sink (set P15) .... f1 f4 2 4 2 4 1 1 f2 3 5 3 5 f3 Route associated flow variable 1-2-4-5 f1 1-2-3-5 f2 Route associated flow variable 1-3-5 f3 1-3-2-4-5 f2

Network flow LP formulations (6) “flow assignment to routes” or “arc-path” approach - example (2) To maximize (1 -> 5) flow : max: f1 + f2 + f3 + f4 ; subject to constraints: c1: f1 + f2 <= 20 ; /*span 12 capacity */ c2: f4 + f3 <= 8 ; /* span 13 capacity */ c3: f4 + f2 <= 2 ; /* span 23 (discuss this ) */ c4: f1 + f4 <= 10 ; /* span 24 */ f1 f4 20 10 5 8 2 4 2 4 20 10 5 8 1 1 2 2 f2 3 5 3 5 f3

Network flow LP formulations (6) “flow assignment to routes” or “arc-path” approach - example (2) What are the remaining constraints ? : - for span 3-5 ... ?: - for span 4-5 ... ? c5: f3 + f2 <= 20; /*span 35 capacity */ c6: f4 + f1 <= 5; /*span 45 capacity */ (note this makes prior constraint f4 + f1<=10 redundant ) f1 f4 20 2 10 4 2 20 10 4 1 1 2 2 5 5 f2 8 3 5 8 3 5 20 20 f3

Network flow LP formulations (7) “flow assignment to routes” or “arc-path” approach - example (3) Note that the “indicator” parameters do not appear explicitly in the executable model. Really they just represent our knowledge of the topology and the routes being considered. Implicitly above, we only wrote the flow variables that had non-zero coefficients. Examples: Hence f1 is in the first constraint Hence f3 is in the fifth constraint, etc.

LP/IP for Mesh-Restoration Design Networks are Inherently Mesh-Like Level(3) North American Network Distributed mesh-restoration exploits network connectivity to allow sharing of redundancy.

Spare Capacity Sharing Consider 2 different failure scenarios X X Restoration is allowed to follow multiple distinct routes. Restoration route for both failure scenarios have several spans in common. Spare capacity on each span contributes to restorability of many spans.

Spare Capacity Placement (SCP) Optimal Design Objective is to find least costly way to place sufficient spare capacity on a network such that all spans are fully restorable. Can we use LP/IP? Reference: M. Herzberg and S. J. Bye, “An Optimal Spare-Capacity Assignment Model for Survivable Networks with Hop Limits”, IEEE Globecom’94, 1994 Integer Programming Approach Objective Function: Minimize Cost of Spare Capacity Placement Constraints: Each possible span failure has enough restoration flow for full restoration. Enough spare capacity exists on each span to accommodate restoration flows.

SCP Integer Programming Problem Parameters (Inputs) Cj: Cost of each unit of capacity on span j Li: Target Restoration level for span i (Li = 1 assumed) S: Set of spans in the network Pi: Set of eligible routes for restoration of span i wj: Number of working links (capacity units) on span j i,jp: Equal to 1 if pth eligible route for span i uses span j Variables (Outputs) fip: Restoration flow assigned to pth route for span i sj: Number of spare capacity units placed on span j

SCP Integer Programming Problem Objective Function: Subject To: 1. 2.

Appendix Added October 5, 2004 to better explain the branch-and-bound procedure

Solving MIP Problems Initial completely relaxed LP problem and solution Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 1 3 2 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 1 2 4 10 Down-forcing branch direction Up-forcing branch direction 9 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 Bounds 6<=XB<=6 2<=XC<=4 Solution *Infeasible 3 6 5 7 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 8 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 5

XC is fractional so “branch” on it by adding bounds… Coils 6 Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 4 2 XC is fractional so “branch” on it by adding bounds… Bands 2 4 6 8

This bound is changed to force XC “down” Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 This bound is changed to force XC “down” 1 3 2 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 1 2 4 10 9 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 Bounds 6<=XB<=6 2<=XC<=4 Solution *Infeasible 3 6 5 7 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 8 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 5

Found an integer solution – best so far too Coils 6 4 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 2 Found an integer solution – best so far too Bands 2 4 6 8

Now test forcing XC “up” to at least two Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 Now test forcing XC “up” to at least two 1 3 2 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 1 2 4 10 9 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 Bounds 6<=XB<=6 2<=XC<=4 Solution *Infeasible 3 6 5 7 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 8 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 5

Coils 6 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 4 Dang ! , now the solution has gone fractional on XB..ok…So now we will branch on XB 2 Bands 2 4 6 8

Xb became 5.14, so now test forcing XB “down” to at least five Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 1 3 2 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 1 2 Xb became 5.14, so now test forcing XB “down” to at least five 4 10 9 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 Bounds 6<=XB<=6 2<=XC<=4 Solution *Infeasible 3 6 5 7 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 8 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 5

Coils 6 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 4 Ok now XC is fractional again on us. so now we will branch on XC again…force it down first… 2 Bands 2 4 6 8

XC became 2.10, so now test forcing XC “down” to at least two Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 1 3 2 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 1 2 4 10 9 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 Bounds 6<=XB<=6 2<=XC<=4 Solution *Infeasible XC became 2.10, so now test forcing XC “down” to at least two 3 6 5 7 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 8 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 5

Coils 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 4 Found another fully integer solution..This one takes over as best so far (185k$) In fact this is the optimum, but we don’t know it yet…have to go on to prove this.. 2 Bands 2 4 6 8

To continue, now force XC from last node “up” to at least three Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 1 3 2 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 1 2 4 10 9 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 Bounds 6<=XB<=6 2<=XC<=4 Solution *Infeasible 3 6 To continue, now force XC from last node “up” to at least three 5 7 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 8 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 5

Fractional solution with lower objective value. Go back to last node… Coils 6 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 Fractional solution with lower objective value. Go back to last node… 2 Bands 2 4 6 8

Result is no feasible solution. Bounds 0<=XB<=6 0<=XC<=4 Solution Obj. = 192K XB = 6.00 XC = 1.40 1 3 This brings us back to this node, where XB was fractional and we now to force it “up” to at least 6. Result is no feasible solution. Tree is exhausted, so the integer solution at node 4 is now “proven” optimal. 2 Bounds 0<=XB<=6 0<=XC<=1 Solution Obj. = 180K XB = 6.00 XC = 1.00 Bounds 0<=XB<=6 2<=XC<=4 Solution Obj. = 189K XB = 5.14 XC = 2.00 1 2 4 10 9 Bounds 0<=XB<=5 2<=XC<=4 Solution Obj. = 189K XB = 5.00 XC = 2.10 Bounds 6<=XB<=6 2<=XC<=4 Solution *Infeasible 3 6 5 7 6 Bounds 0<=XB<=5 2<=XC<=2 Solution Obj. = 185K XB = 5.00 XC = 2.00 8 Bounds 0<=XB<=5 3<=XC<=4 Solution Obj. = 183K XB = 3.00 XC = 3.71 4 5

The Overall Search trajectory Coils 6 4 2 Bands 2 4 6 8