1. MIT and James Orlin © 2003 1 Introduction to Networks Eulerian Tours Hamiltonian Tours The Shortest Path Problem Dijkstra’s Algorithm for Solving the Shortest Path Problem

2. MIT and James Orlin © 2003 2 Network Models Optimization models that exhibit a very special structure For special cases, this structure to dramatically reduce computational complexity First widespread application of LP to problems of industrial logistics Addresses huge number of diverse applications Today’s lecture (first of 3): introductory material, Eulerian tours, Hamiltonian Tours, the Shortest Path Problem

3. MIT and James Orlin © 2003 3 Notation and Terminology Note: Network terminology is not (and never will be) standardized. The same concept may be denoted in many different ways. Called: NETWORK directed graph digraph graph 2 1 4 3 2 1 4 3 Class Handouts (Ahuja, Magnanti, Orlin) Node set N = {1, 2, 3, 4} Arc Set Network G = (N, A) {(1,2), (1,3), (3,2), (3,4), (2,4)} A={1-2,1-3,3-2,3-4,2-4} Graph G = (V,E) Edge set: Also Seen Vertex set V = {1,2,3,4}

4. MIT and James Orlin © 2003 4 Directed and Undirected Networks 2 34 1 a b c d e An Undirected Graph 2 34 1 a b c d e A Directed Graph  The field of Network Optimization concerns optimization problems on networks  Networks are used to transport commodities physical goods (products, liquids) communication electricity, etc.

5. MIT and James Orlin © 2003 5 Networks are Everywhere Physical Networks –Road Networks –Railway Networks –Airline traffic Networks –Electrical networks, e.g., the power grid Abstract networks –organizational charts –precedence relationships in projects Others?

6. MIT and James Orlin © 2003 6 Overview: Networks and graphs are powerful modeling tools. Most OR models have networks or graphs as a major aspect Next three lectures: we will develop models that are efficiently solvable. –Help form a toolkit for those problems that are harder to solve Next: representations of networks –A great insight from computer scientists: how data is represented is important to how it is used

7. MIT and James Orlin © 2003 7 The Adjacency Matrix (for directed graphs) 2 34 1 a b c d e A Directed Graph Have a row for each node 1 2 3 4 12341234 Have a column for each node Put a 1 in row i- column j if (i,j) is an arc What would happen if (4,2) became (2,4)?

8. MIT and James Orlin © 2003 8 The Adjacency Matrix (for undirected graphs) Have a row for each node 1 2 3 4 12341234 Have a column for each node Put a 1 in row i- column j if (i,j) is an arc 2 34 1 a b c d e An Undirected Graph The degree of a node is the number of incident arcs degree 23232323

9. MIT and James Orlin © 2003 9 The Node-Arc Incidence Matrix (for directed graphs) 2 34 1 a b c d e A Directed Graph Have a row for each node a b c d e 12341234 Have a column for each arc Put a 1 in row i- column j if arc j starts at node i. Put a -1 in row i- column j if arc j ends at node i. What would happen if arc (4,2) became arc (2,4)?

10. MIT and James Orlin © 2003 10 Representation of arc lists (for directed graphs) 2 34 1 a b c d e A Directed Graph Create a list arcs for each node 1: (1,2), (1,4) 2: (2,3) 3:  4: (4,2), (4,3) There are lots of very similar variants of this type of representation

11. MIT and James Orlin © 2003 11 On network representations Each representation has its advantages –Major purpose of a representation efficiency in algorithms ease of use Node arc incidence matrix shows up in Linear Programs Next: definitions for networks

12. MIT and James Orlin © 2003 12 Directed Path. Example: 1, 2, 5, 3, 4 (or 1, a, 2, c, 5, d, 3, e, 4) No node is repeated. Directions are important. Cycle (or circuit or loop) 1, 2, 3, 1. (or 1, a, 2, b, 3, e) A path with 2 or more nodes, except that the first node is the last node. Directions are ignored. Path: Example: 5, 2, 3, 4. (or 5, c, 2, b, 3, e, 4) No node is repeated. Directions are ignored. Directed Cycle: (1, 2, 3, 4, 1) or 1, a, 2, b, 3, c, 4, d, 1 No node is repeated. Directions are important. 2 3 4 ab c 1 5 d e 2 3 4 a b c d 1 e 2 3 4 ab c 1 5 d e 2 3 4 a b c d 1 e

13. MIT and James Orlin © 2003 13 Walks 2 34 1 a b c d e 5 2 34 1 a b c d e 5 Walks are paths that can repeat nodes and arcs Example of a directed walk: 1-2-3-5-4-2-3-5 A walk is closed if its first and last nodes are the same. A closed walk is a cycle except that it can repeat nodes and arcs.

14. MIT and James Orlin © 2003 14 More terminology An undirected network is connected if every node can be reached from every other node by a path 2 1 4 3 5 2 1 4 3 5 A directed network is connected if it’s undirected version is connected. This directed graph is connected, even though there is no directed path between 2 and 5.

15. MIT and James Orlin © 2003 15 On connectivity 6 1 4 3 7 2 10 8 9 5 There are simple efficient procedures for determining if a graph is connected. Here is a graph with two components, that is maximally connected subgraphs. 4 7 109 We will not describe these algorithms, but will do a more general algorithm later in this lecture

16. MIT and James Orlin © 2003 16 The Bridges of Koenigsberg: Euler 1736 “Graph Theory” began in 1736 Leonard Euler –Visited Koenigsberg –People wondered whether it is possible to take a walk, end up where you started from, and cross each bridge in Koenigsberg exactly once –Generally it was believed to be impossible

17. MIT and James Orlin © 2003 17 The Bridges of Koenigsberg: Euler 1736 A D C B 12 4 3 7 6 5 Is it possible to start in A, cross over each bridge exactly once, and end up back in A?

18. MIT and James Orlin © 2003 18 The Bridges of Koenigsberg: Euler 1736 A D C B 12 4 3 7 6 5 Conceptualization: Land masses are nodes

19. MIT and James Orlin © 2003 19 The Bridges of Koenigsberg: Euler 1736 12 4 3 7 6 5 Conceptualization: Bridges are arcs A C D B

20. MIT and James Orlin © 2003 20 The Bridges of Koenigsberg: Euler 1736 12 4 3 7 6 5 Translation to graphs or networks: Is there a walk starting at A and ending at A and passing through each arc exactly once? Why isn’t there such a walk? A C D B

21. MIT and James Orlin © 2003 21 Adding two bridges creates such a walk A, 1, B, 5, D, 6, B, 4, C, 8, A, 3, C, 7, D, 9, B, 2, A 12 4 3 7 6 5 A C D B 8 9 Here is the walk. Note: the number of arcs incident to B is twice the number of times that B appears on the walk.

22. MIT and James Orlin © 2003 22 Eulerian cycle: a closed walk that passes through each arc exactly once Degree of a node = number of arcs incident to the node Necessary condition: each node has an even degree. Why necessary? The degree of a node j is twice the number of times j appears on the walk (except for the initial and final node of the walk.) This condition is also nearly sufficient. If every node has even degree, and if the graph is connected, then there is an eulerian cycle.

23. MIT and James Orlin © 2003 23 Eulerian cycles Eulerian cycles and extensions are used in practice Mail Carrier routes: –visit each city block at least once –minimize travel time –other constraints in practice? Trash pickup routes –visit each city block at least once –minimize travel time –other constraints in practice?

24. MIT and James Orlin © 2003 24 More on cycles A hamiltonian cycle is a cycle that passes through each node of the graph exactly once.

25. MIT and James Orlin © 2003 25 Hamilton’s Around the World Game In 1857, the Irish mathematician, Sir William Rowan Hamilton invented a puzzle that he hoped would be very popular. The objective was to make what we just called a hamiltonian cycle. The game was not a commercial success, especially the 3D version. But the mathematics of hamiltonian cycles is very popular today.

26. MIT and James Orlin © 2003 26 Hamilton’s Around the World Game We will see this problem again when we generalize it to be the traveling salesman problem.

27. MIT and James Orlin © 2003 27 The knight’s tour problem Can a knight visit all squares of a chessboard exactly once, starting at some square, and by making 63 legitimate moves? The knight’s tour problem is a special case of the hamiltonian tour problem. The answer is yes!

28. MIT and James Orlin © 2003 28 More Definitions 2 1 4 3 A network is connected if every node can be reached from every other node by following a sequence of arcs in which direction is ignored. 5 A spanning tree is a connected subset of a network including all nodes, but containing no cycles. 2 1 4 3 5 2 1 4 3 5 2 1 4 3 5

29. MIT and James Orlin © 2003 29 More on Trees An out-tree is a spanning tree in which every node has exactly one incoming arc except for the root. Theorem. In an out-tree, there is a directed path from the root to all other nodes. (All paths come out of the root). 2 1 4 3 5 2 1 4 3 5 789 6 1011 1213

30. MIT and James Orlin © 2003 30 The Shortest Path Problem 1 2 3 4 5 6 2 4 2 1 3 4 2 3 2 What is the shortest path from a source node (often denoted as s) to a sink node, (often denoted as t)? What is the shortest path from node 1 to node 6? Assumptions for this lecture: 1. There is a path from the source to all other nodes. 2. All arc lengths are non-negative

31. MIT and James Orlin © 2003 31 Shortest Path Problem Where does it arise in practice? –Common applications shortest paths in a vehicle shortest paths in internet routing shortest paths around MIT –Less obvious: close connection to dynamic programming which we will see in later lectures How will we solve the shortest path problem? –Dijkstra’s algorithm

32. MIT and James Orlin © 2003 32 Dijkstra’s Algorithm for the Shortest Path Problem Exercise: find the shortest path from node 1 to all other nodes. Keep track of distances using labels, d(i) and each node’s immediate predecessor, pred(i). d(1)= 0, pred(1)=0; d(2) = 2, pred(2)=1 Find the other distances, in order of increasing distance from node 1. 1 2 3 4 5 6 2 4 2 1 3 4 2 3 2 Exercise with your partner. Find the shortest paths by inspection.

33. MIT and James Orlin © 2003 33 A Key Step in Shortest Path Algorithms Let d( ) denote a vector of temporary distance labels. d(j) is the length of some path from the origin node 1 to node j. 78 ij 62 10 1 Path P Up to this point, the best path from 1 to j has length 78 But P, (i,j) is a path from 1 to j of length 72. 72 Procedure Update(i ) for each (i,j)  A(i) do if d(j) > d(i) + c ij then d(j) : = d(i) + c ij and pred(j) : = i;

34. MIT and James Orlin © 2003 34 Dijkstra’s Algorithm begin d(s) : = 0 and pred(s) : = 0; d(j) : =  for each j  N - {s}; LIST : = {s}; while LIST   do begin let d(i) : = min {d(j) : j  LIST}; remove node i from LIST; update(i) if d(j) decreases, place j in LIST end Initialize distances. LIST = set of temporary nodes Select the node i on LIST with minimum distance label, and then update(i)

35. 1 2 3 4 5 6 2 4 2 1 3 4 2 3 2 d(2) =  pred(2) = LIST = {1, d(1) = 0 pred(1) = 0 d(4) =  pred(4) = d(3) =  pred(3) = d(5) =  pred(5) = d(6) =  pred(6) = \ 2, 3, \ 2 1 \ 4 1 \ 6 2 \ 4 2 \ 4, 5, \ \ 6 5 \ 6} \\ An Example \ 3 \ 2 Initialize the distances and LIST. Find the node i on LIST with minimum distance. Scan the arcs out of i, and update d( ), pred( ), and LIST 1 Find the node i on LIST with minimum distance. 2 Scan the arcs out of i, and update d( ), pred( ), and LIST Find the node i on LIST with minimum distance. 3 Scan the arcs out of i, and update d( ), pred( ), and LIST Find the node i on LIST with minimum distance. 5 Scan the arcs out of i, and update d( ), pred( ), and LIST Find the node i on LIST with minimum distance. 4 Scan the arcs out of i, and update d( ), pred( ), and LIST Find the node i on LIST with minimum distance. 6 The End

36. MIT and James Orlin © 2003 36 The Output from Dijkstra’s Algorithm 1 2 3 4 5 6 2 4 2 1 3 4 2 3 2 0 2 3 6 4 6 Dijkstra provides a shortest path from node 1 to all other nodes. It provides a shortest path tree. Note that this tree is an out-tree. Each node has one incoming arc (except for the source) To find the shortest path from node j, trace back from the node to the source.

37. MIT and James Orlin © 2003 37 Comments on Dijkstra’s Algorithm Dijkstra’s algorithm makes nodes permanent in increasing order of distance from the origin node. Dijkstra’s algorithm is efficient in its current form. The running time grows as n 2, where n is the number of nodes It can be made much more efficient In practice it runs in time linear in the number of arcs (or almost so). How can we recognize a shortest path tree? We will see that next lecture.

38. MIT and James Orlin © 2003 38 Representation as an integer program An integer program is a linear program in which some or all of the variables are required to be integer We will formulate the shortest path problem as an integer program. –Find the shortest path from node 1 to node 6 Decision variables: –x ij = 1 if arc (i,j) is in the path. –x ij = 0 if arc (i,j) is not in the path

39. MIT and James Orlin © 2003 39 Constraints There is one arc out of node 1 There is one arc into node 6 For every other node i, the number of arcs on the path entering node i is the number of arcs leaving node i. Other constraints can be added, but this turns out to be enough.

40. MIT and James Orlin © 2003 40 1 2 3 4 5 6 The constraint matrix is the node arc incidence matrix 1100000001=111000000=1 0 000000=1 0000000=11 000000= 0000000 =x 12 x 13 x 23 x 25 x 24 x 35 x 46 x 54 x 56

41. MIT and James Orlin © 2003 41 On Incidence Matrices If the constraint matrix for a linear program is a node-arc incidence matrix (at most one 1 and at most one –1 per column), then the linear program solves in integer optima. Thus, we can solve the shortest path problem as an LP, and get the optimum path.

42. MIT and James Orlin © 2003 42 The string solution Imagine replacing each arc by a string of the same length. Thus arc (1,3) would be replaced by a string of length 4 inches joining node 1 to node 3. Now hold node 1 in one hand and node 6 in the other, and pull until the string is tight. 1 2 3 4 5 6 2 4 2 1 3 4 2 3 2

43. MIT and James Orlin © 2003 43 The string solution 6 2 4 1 34 3 2 2 2 Does one get the shortest path from node 1 to node 6? If so, why? Note: In some sense we are maximizing the physical distance from node 1 to node 6. 1 2 3 4 5

44. MIT and James Orlin © 2003 44 Summary The eulerian cycle problem The hamiltonian path problem The shortest path problem Dijkstra’s algorithm finds the shortest path from node 1 to all other nodes in increasing order of distance from the source node. The bottleneck operation is identifying the minimum distance label. One can speed this up, and get an incredibly efficient algorithm Formulation of shortest path as an LP The string solution

45. MIT and James Orlin © 2003 45 Some final comments The shortest path problem shows up again and again in network optimization There is an interesting connection with dynamic programming There are other solution techniques as well. We’ll see one in a later lecture.

46. MIT and James Orlin © 2003 46 An illustration of an eulerian cycle 2 56 13 47 1-2-3-4-5-6-7-3-5-2-6-1 2 56 13 4 7 For every appearance of node j on the cycle, there is one entering arc and one leaving arc. Number of arcs into a node = number of arcs out of a node