1. TRLabs & University of Alberta © Wayne D. Grover 2002, 2003
E E Module 6
Lp_solve example, LP tactics, “Network reliability”, Min cost network flow, multi-commodity max flow, more on combinatorial optimization...
W. D. Grover
TRLabs & University of Alberta © Wayne D. Grover 2002, 2003

2. Revisit mesh SCP problem
Min sum of spare capacity i P p w f = å 1 ÷ ø ö ç è æ × j s , d
S.t.
Restoration flows for each failure = failed working capacity
Restoration flows must be feasible under spare capacity
How would this be expressed as an lp_solve model... ?

3. Sample network for example of mesh “SCP”7 B 4 C
SCP = spare capacity placement
here, spans all have equal length or cost
span working capacity quantities are shown
Problem is to place spare capacity for 100% restorability via span restoration. 8 E 10 7 2 4 12 F G Z 6 9

4. Sample network for example of mesh “SCP”7 B 4 C
as a first step the “eligible restoration routes” for each failure are identified.
For example, three distinct eligible routes for restoration of span A-B under 4 hops might be as shown…(examples only) 8 E 10 7 2 4 12 F G Z 6 9
Flow variables defined for A-B as a failure span f1 f2 f3
Illustration explaining the first “restorability” constraint to follow...

5. Lp_solve example for mesh SCP
/* Objective function */
min: Sab+Sbe+Seg+Sag+Saf+Sgf+Sbc+Scz+Sez+Sfz;
/* The sum of all the flows on the different routes for restoration */
/* of span i must be greater or equal to wi */
f1 + f2 + f3 >= 7; /* span A-B */ (see prior slide for this one example)
f4 + f5 + f6 + f7 >= 8; /* span B-E */
f8 + f9 + f10 + f11 >= 12; /* span E-G */
f12 + f13 + f14 >= 5; /* span A-G */
f15 + f16 + f17 + f18 >= 10; /* span A-F */
f19 + f20 >= 6; /* span G-F */
f21 + f22 >= 4; /* span B-C */
f23 + f24 >= 6; /* span C-Z */
f25 + f26 + f27 + f28 >= 4; /* span E-Z */
f29 + f30 + f31 + f32 >= 9; /* span F-Z */
One line per span failure The details come from the route identified for restoration, and the associated flow variable for defined for each route

6. Lp_solve example for mesh SCP
/* The spare capacity on each span must be >= to the sum of the flows /* on routes that cross this span for every failure scenario */
/* Span A-B */
Sab - f4 - f5 - f6 >= 0; /* failure of B-E */
Sab - f8 - f9 >= 0; /* failure of E-G */
Sab - f14 >= 0; /* failure of A-G */
Sab - f17 - f18 >= 0; /* failure of A-F*/
Sab - f22 >= 0; /* failure of B-C */
Sab - f24 >= 0; /* failure of C-Z */
Sab - f26 >= 0; /* failure of E-Z */
Sab - f31 - f32 >= 0; /* failure of F-Z */
/* Span B-E */
Sbe - f1 - f2 >= 0; /* failure of A-B */
Sbe - f8 - f9 >= 0; /* failure of E-G */
Sbe - f14 >= 0; /* failure of A-G */
....etc... (produces S sets of (S-1) additional constraints)
The exact definition of eligible restoration routes and corresponding flow variables (not shown) is needed to produce this level of detail.
This shows an example of the logical form that is taken

7. Illustration: explaining the first spare-capacity generating constraint
/* Span A-B */
Sab - f4 - f5 - f6 >= 0; /* failure of B-E */
……. A 7 B 4 C
as a first step the “eligible restoration routes” for each failure are identified.
For example, three distinct eligible routes for restoration of span A-B under 4 hops might be as shown…(examples only) f4 8 f1 E 10 f5 7 2 f6 4 f3 12 F f2 G Z 6 9
F1, f2, f3 do not cross span A-B when B-E failure is considered
f4, f5, f6 do cross span A-B when B-E failure is considered

8. Tips on formulating LP / IP’s
Mutual exclusion: - e.g. Each working demand flow may have only one route - use a “radio button” constraint: where is a decision variable to use route i or not.
Peak minimizing: - e.g. Minimize the worst case “oversubscription” on all spans j over all failures, i cannot write min { max {function}} in an LP / IP - can write : min Z Z - zij >= 0 for every i,j

9. Bi-objective LP / IP’s
Sometimes our notion of optimality depends on more than one criterion (with different units or measures) - can form a two term objective function - “blended” by some weighting factor e.g.: perhaps: (Z1 ~ capacity Z2~ restoration paths length) (Z1 ~ cost Z2~ uncertainty)
Solve family of problem instances with varying “alpha” result is the “Pareto optimal” solution set - shows the shape of the “trade-off” of one goal versus the other - discrete (as shown) if IP - continuous Pareto curve if LP

10. Handling Modular Capacity
Sometimes capacity can only be placed in discrete modules, with associated cost e.g. OC-48, OC but demand flows are still integer or even real valued change objective function from Min (capacity) to Min (sum of all modules used * module cost) s.t. Sum of (modules placed * module capacity) >= capacity needed (see book p. 249)

11. “Network Reliability” (book p. 163)
A specialized field of graph theory that attempts to solve the computational problem posed by the question:
“If all each edge of a graph is in a working state with probability (1-p) and failed state with probability p, what is the probability that a route exists between “:
- nodes s,t -existence of a spanning tree
- all node pairs, - etc.
Example; p ~ 0.32
yes no
yes
yes
Network reliability = How likely is it that at least one route exists between nodes? (228 link state combinations to consider here)

12. “Network Reliability”
“Reliability polynomial”:
Prob.of exactly i failed and (m-i) working edges
Where:
G = graph
{s,t} = a specific node pair
p = edge failure probability
Ni() = number of edge state combinations containing exactly i routes {s,t}
m = number of edges in graph
The real problem: how to enumerate or estimate these numbers of states that contain a path between {s,t} when i edges are failed

13. “Network Reliability”
(relevance to survivable transport network design )
two-terminal (s,t} reliability is a bit like a path availability assessment
However we also need to consider specific structures (e,g rings), not just the network topology graph.
- we need to consider capacity and exact restoration mechanism
- we don’t need to consider high link failure probabilities nor high numbers of simultaneous failures
- we can accept the numerical approximation that “unavailibilities add for elements in series” (i.e., Ni reduces to ~ number of elements in normal path plus restoration considerations)
Graph reduction, factoring and decomposition methods (p. 33) can however be applied directly to problems of availability analysis.

14. Min-cost Network flow (MCNF)
so far with the LP method we have looked at finding maximum flow, subject to constraining capacities.
A related problem is to support a specified flow, between source and target nodes at min cost for the edges used.
Two - terminal MCNF :
S.t.
Where Cij = edge capacities
Optionally also :

15. “Unimodularity” Maximum flow and Min cost network flow
are problems know to have “special structure”
also called “network structure” .....
If all the inputs are integer, ... the solution will be integer even if you solve it as a Linear Program not as an Integer Program.
The mathematical property is called “unimodularity” and it is an important computational advantage whenever a problem has this special mathematical structure.
Intuitively, the solution is “trapped” onto discrete integer solution values...no operations in the formulation can fractionate the flow variables

16. “Unimodularity”
More formally, unimodularity requires that the matrix A formed by the constraint system of an IP formulation:
Ax < c
has the properties *:
(i) every ai,j is {-1,0,1}.
(ii) each column contains at most 2 non-zero coefficients.
(iii) there exists a partition (M1,M2) of the set M rows such that the sum of coefficients in columns of M1 is equal and opposite to the corresponding sum of coefficients M2
* Possible project : investigate this criterion further, write program to detect the “unimodular part” of any given problem

17. Related network flow formulations
(Chapter 4)
Multisource-sink Min cost network flow - same objective - every node has source or sink b_i quantity of “commodity” i - this canonical “transportation problem” is not quite the same as the communications version…. - It describes net flows from several factories to several distribution sites.
Multi-commodity max flow (MCMF) - numerous source-sink pairs (each a “commodity”) are simultaneously trying to maximize flow through common set of finite edge capacities > Closely related to later “path restorable mesh” problem. (a) pure max sum version (b) maximize smallest proportion of a requirement

18. Classical min-cost “transportation problem”
Routing costs A 7 B 4 C
stores 8 E 10
All one commodity 7 2 4 12 F
factories G Z 6 9
Question is: what assignment of flows to edges supplies all stores from available factories at minimum cost?

19. Communications version: min-cost multi-commodity flow
Routing costs
City A (sink ~store) A 7 B 4 C
stores 8
Each city pair is a “commodity”
only one factory and one store for each commodity E 10 7 2 4 12 F
City Z (source ~factory) G Z 6 9
A-Z commodity
C-G commodity
Question is: what assignment of flows to each edge for each commodity meets all requirements at minimum routing cost?

20. Communications version: min-cost multi-commodity flow
Two cases of MCMF:
A. no limits on capacities of edges….
B. finite capacities on edges.
---> shortest path (least cost) routing for each commodity is optimal (trivial problem)
---> shortest path (least cost) routing for each commodity is not optimal
becomes “NP- hard” problem
the difficulty is the “mutual capacity” issue
will be at the heart of our later study of the path-restoration problem

21. Quick guide to computational complexity

22. Why theoretical complexity matters
Integer programming (IP) is NP-hard.. and exhibits this behaviour in practise
Linear programming (LP) (by simplex algorithm) is theoretically NP-hard but polynomial time in most cases.
Some newer LP algorithms are provably polynomial time.
Some Integer Programming problems are “uni-modular” - solve as an LP without losing integrality
Other IP s or combinatorial optimization problems in general require heuristic (or meta-heuristic) solutions.

23. Other approaches to combinatorial optimization
“Meta-heuristics” (book: p )
(1) Simulated Annealing (SA) (2) Genetic Algorithms (3) Tabu Search
A “meta-heuristic” is an approximate, but highly general optimization technique that can be used as an alternative to solving LP / IP formulations.
It is a “meta- heuristic” because the basic search / improvement tactic can be explained in a way that is not specific to any one problem.
Advantages may be:
- handling larger size problems
- representing problem detail not amenable to an LP / IP
- non-linear constraints or objectives
Results not provably optimal but often this is OK in practice.
All need quantifiable / calculable objective function
Central issue each is trying to deal with is effecting a suitably broad search of the solution space without getting trapped in local optima

24. Meta-heuristics: Simulated Annealing (SA)
Basic ideas / concept:
Inspired by mathematical analogy to strain energy minimization in the slow cooling (annealing) of metals.
Changes to the current design are made randomly.
Improvements are kept, steps backward are normally rejected, except...
A step “backward” (i.e., a degrading of the objective function) may be kept with a random probability.
The probability of keeping such a change slowly decreases.
Where:
DE = the magnitude of degradation
t = time (elapsed simulation time)
T(t) = the “cooling schedule”
k = Boltzmanns constant

25. Meta-heuristics: Simulated Annealing (SA)
Define a cooling function or schedule T(t) and set: t = 0, temperature T(0) := Tmax ;
while T > Tmin do
For n = 1 to Nc do
propose a change in the solution;
evaluate the change in the objective function DE ;
if DE > 0 , accept the move
otherwise, if rand() < Paccept[T(t), DE], accept the move
otherwise reject the move;
End for
t := t+1 ( or often: T :=  T )
Also: keep note of global best ever found at any stage, not just current solution

26. Meta-heuristics: Simulated Annealing (SA)
An additional resource on Simulated Annealing is at the web site:
Simulated Annealing.htm
on the course web-site (related reading section for this lecture).
From “Overview of Modern Heuristic Methods” at
(where meta-heuristics are being studied for radio frequency assignment problems).

27. Meta-heuristics: Genetic Algorithms (GA)
Basic ideas / concept:
Inspired by concepts of natural selection and evolutionary improvement of an evolving population.
Concepts / terms: - a population, an individual, a generation - genome or chromosome, encoding scheme or “schemata” - selection, fitness function - cross-over (‘breeding’) - mutation
invented / developed by John Holland

28. Meta-heuristics: GA Algorithms
A key to use of GA is finding a suitable schemata - must encode all relevant aspects of the design - should permit easy evaluation of fitness function - must lend itself to a suitable “cross-over” operator
Crossover example: -uniform random cross-over point -cross-over may be followed by “chromosome repair”
Mutation: - a small random probability of (~ 0.1% typ.) of flipping a chromosome value (reason ? - search diversification)

29. Meta-heuristics: Basic GA Algorithm
initialize a (usually random) population of individuals, P (t=0);
evaluate fitness of all individuals F{ P (t=0) };
repeat
t := t + 1;
P(t) := selectparents {F {P (t-1)}}; (selects sub-population for offspring production “roulette wheel” is a common approach)
crossover on P(t); (recombine the "genes" of selected parents)
mutate P(t); (perturb the mated population stochastically)
evaluate fitness of all new individuals F{ P(t) };
select the survivors P(t) := survive (F{P(t)});
until {test for termination criterion (time, fitness, etc.)}.

30. Meta-heuristics: GA Algorithms
Additional resource on Genetic Algorithms:
Overview_of_GAs.pdf (white paper by Beasley et. al)
on course web-site (related reading section).
GA s also covered in “Overview of Modern Heuristic Methods” at

31. Meta-heuristics: Tabu Search
Basic ideas / concept:
uses ideas of conventional “hill climbing” / gradient- type search but with addition of temporary prohibition of moves that lead back to recently discovered local optimum.
Tabu search is forced to accept “uphill moves” after reaching a local optimum because the return move is tabu for some time following discovery of the local optimum.
Main proponent and book: Tabu Search by Fred Glover and M. Laguna

32. Meta-heuristics: Tabu Search
Key concepts / terms: - a “move” a “neighbourhood” - tabu moves, tabu tenure rules - recency and frequency lists - aspiration criteria - search diversification
Key to a good TS algorithm is usually quite problem-specific policies and memory structures for deciding what a “move” is and how long moves are “tabu” after leading to a local minimum.
“Aspiration criterion” are exceptions to the tabu tenure rules for moves that happen to lead directly to large solution improvements.

33. Meta-heuristics: Basic TS algorithm
i := 0; generate initial solution, so;
repeat
generate neighbour set, N(si), and objective function values for each neighbour
identify taboo neighbor set T(si), (from current list of tabu moves);
identify aspirant subset of taboo neighbors, A(si);
move to best neighbour amongst {N(si) - T(si) + A(si)}; i := i+1 ;
update recency or frequency memories and aspiration criteria (count down tenure of any currently tabu moves)
Until {test for termination criterion (time, objective value, etc.)}.

34. A recent application of Tabu Search
Application to Ring network design optimization
part of PhD studies by D. Morley (U of A 2001).
See " Tabu Search Optimization of Optical Ring Transport Networks," Globecom 2001, San Antonio, Nov , 2001, vol.4, pp
Two types of “move”:
- add a ring (from candidate set) - delete a ring
Short-term tabu memory:
- if ring r1 is added, we discourage it from being dropped for a specified number of iterations, called the tabu add tenure. - if ring r2 is dropped, we discourage it from being added back to the solution for the tabu drop tenure.
Search diversification strategy: a entirely new starting point design is generated when:
- a prior solution is revisited
- % improvement in best solution over last n iterations is too low
- absolute improvement in best solution over last n iterations is too low

35. A recent application of Tabu Search(2)
Enter with a RingBuilderTM starting design
Anatomy of a tabu search algorithm
Feasibility checks: Is a routing through rings possible that serves all demand?
Progress checks - could generate diversification re-start here
Basic “move” strategy - Drop ring each iteration - Only add ring if feasibility lost
Tabu policy is in here - Deciding which ring add / drop moves are not allowed (discouraged) at present

36. Search diversification re-starts
Tabu Search
Understanding Tabu search : typical TS trajectory (sample result from D. Morley)
notice how each “restart” is followed first by a climb in cost (probably due to initial bad drop moves, requiring adds for feasibility) then by settling into a new local cost-minima
Search diversification re-starts
For more on Tabu Search see: Tabu Search.htm from the “Overview of Modern Heuristic Methods” at