1. Algorithms for Network Optimization Problems This handout: Minimum Spanning Tree Problem Approximation Algorithms Traveling Salesman Problem

2. Terminology of Graphs A graph (or network) consists of – a set of points – a set of lines connecting certain pairs of the points. The points are called nodes (or vertices). The lines are called arcs (or edges or links). Example:

3. Terminology of Graphs: Paths A path between two nodes is a sequence of distinct nodes and edges connecting these nodes. Example: a b

4. Terminology of Graphs: Cycles, Connectivity and Trees A path that begins and ends at the same node is called a cycle. Example: Two nodes are connected if there is a path between them. A graph is connected if every pair of its nodes is connected. A graph is acyclic if it doesn’t have any cycle. A graph is called a tree if it is connected and acyclic. Example:

5. Minimum Spanning Tree Problem Given:Graph G=(V, E), |V|=n Cost function c: E  R. Goal:Find a minimum-cost spanning tree for V i.e., find a subset of arcs E*  E which connects any two nodes of V with minimum possible cost. Example: 2 3 3 4 4 5 7 8 e b c d a 2 3 3 4 4 5 7 8 e b c d a G=(V,E) Min. span. tree: G*=(V,E*) Red bold arcs are in E*

6. Algorithm for solving the Minimum Spanning Tree Problem Initialization: Select any node arbitrarily, connect to its nearest node. Repeat –Identify the unconnected node which is closest to a connected node –Connect these two nodes Until all nodes are connected Note: Ties for the closest node are broken arbitrarily.

7. The algorithm applied to our example Initialization: Select node a to start. Its closest node is node b. Connect nodes a and b. Iteration 1: There are two unconnected node closest to a connected node: nodes c and d (both are 3 units far from node b). Break the tie arbitrarily by connecting node c to node b. 2 3 3 4 4 5 7 8 e b c d a 2 3 3 4 4 5 7 8 e b c d a Red bold arcs are in E*; thin arcs represent potential links.

8. The algorithm applied to our example Iteration 2 : The unconnected node closest to a connected node is node d (3 far from node b). Connect nodes b and d. Iteration 3: The only unconnected node left is node e. Its closest connected node is node c (distance between c and e is 4). Connect node e to node c. All nodes are connected. The bold arcs give a min. spanning tree. 2 3 3 4 4 5 7 8 e b c d a 2 3 3 4 4 5 7 8 e b c d a

9. Recall Classes of discrete optimization problems:  Class 1 problems have polynomial-time algorithms for solving the problems optimally. Ex.: Min. Spanning Tree problem Assignment Problem  For Class 2 problems (NP-hard problems) No polynomial-time algorithm is known; And more likely there is no one. Ex.: Traveling Salesman Problem Coloring problem

10. Three main directions to solve NP-hard discrete optimization problems: Integer programming techniques Heuristics Approximation algorithms We gave examples of the first two methods for TSP. In this handout, an approximation algorithm for TSP.

11. Definition of Approximation Algorithms Definition: An α-approximation algorithm is a polynomial-time algorithm which always produces a solution of value within α times the value of an optimal solution. That is, for any instance of the problem Z algo / Z opt  α, (for a minimization problem) where Z algo is the cost of the algorithm output, Z opt is the cost of an optimal solution. α is called the approximation guarantee (or factor) of the algorithm.

12. Some Characteristics of Approximation Algorithms Time-efficient (sometimes not as efficient as heuristics) Don’t guarantee optimal solution Guarantee good solution within some factor of the optimum Rigorous mathematical analysis to prove the approximation guarantee Often use algorithms for related problems as subroutines Next we will give an approximation algorithm for TSP.

13. An approximation algorithm for TSP Given an instance for TSP problem, 1.Find a minimum spanning tree (MST) for that instance. (using the algorithm of the previous handout) 2.To get a tour, start from any node and traverse the arcs of MST by taking shortcuts when necessary. Example: Stage 1Stage 2 start from this node red bold arcs form a tour

14. Approximation guarantee for the algorithm In many situations, it is reasonable to assume that triangle inequality holds for the cost function c: E  R defined on the arcs of network G=(V,E) : c uw  c uv + c vw for any u, v, w  V Theorem: If the cost function satisfies the triangle ineqality, then the algorithm for TSP is a 2-approximation algorithm. w v u

15. Approximation guarantee for the algorithm (proof)  First let’s compare the optimal solutions of MST and TSP for any problem instance G=(V,E), c: E  R. Idea: Get a tour from Minimum spanning tree without increasing its cost too much (at most twice in our case). Cost (Opt. TSP sol-n)Cost (of this tree)Cost (Opt. MST sol-n) ≥≥ Optimal TSP sol-n Optimal MST sol-n A tree obtained from the tour (*)

16. Approximation guarantee for the algorithm (proof)  The algorithm takes a minimum spanning tree starts from any node traverse the MST arcs by taking shortcuts when necessary to get a tour.  What is the cost of the tour compared to the cost of MST? Each tour (bold) arc e is a shortcut for a set of tree (thin) arcs f 1, …, f k (or simply coincides with a tree arc) start from this node red bold arcs form a tour 1 2 3 4 5 6

17. Approximation guarantee for the algorithm (proof) Based on triangle inequality, c(e)  c(f 1 )+…+c(f k ) E.g, c 15  c 13 + c 35 c 23  c 23 But each tree (thin) arc is shortcut exactly twice. (**) E.g., tree arc 3-5 is shortcut by tour arcs 1-5 and 5-6.  The following chain of inequalities concludes the proof, by using the facts we obtained so far: start from this node red bold arcs form a tour 1 2 3 4 5 6

18. Performance of TSP algorithms in practice A more sophisticated algorithm (which again uses the MST algorithm as a subroutine) guarantees a solution within factor of 1.5 of the optimum (Christofides). For many discrete optimization problems, there are benchmarks of instances on which algorithms are tested. For TSP, such a benchmark is TSPLIB. On TSPLIB instances, the Christofides’ algorithm outputs solutions which are on average 1.09 times the optimum. For comparison, the nearest neighbor algorithm outputs solutions which are on average 1.26 times the optimum. A good approximation factor often leads to good performance in practice.