Approximation Algorithms and Heuristics

Approximation Algorithms Many interesting problems involve sifting through a large amount of data to make an informed decision. Most of the problems of interest (that is, real-world problems) are discrete optimization problems which are NP-hard (i.e. at least as difficult as NP). This means that the number of operations required to find an optimal solution grows exponentially with problem size. Typical exponential growth rates involves doubling the number of operations for each additional element added to the problem set. Unless it turns out that P = NP, we will not be able to find optimal solutions for all members of a particular problem class in polynomial time. In other words we must give up on one of the following three requirements. The algorithm will. . . (1) always find optimal solutions (2) in polynomial time (3) for any problem instance

A Subset of Problems in NP may have Polynomial Solutions Omitting Requirement # 3 - A subset of problems for TSP in which all points in the problem set are either (1) on the convex hull or (2) points on a line inside the convex hull is polynomial-time solvable. Given that there are n points on the convex hull and m interior points on a line, the complexity of the solution is O(mn) time and O(n) space. http://www.kurims.kyoto-u.ac.jp/~kyodo/kokyuroku/contents/pdf/0853-05.pdf

TSP is NP-Hard but (so far) has not been Proved to be NP-Complete A problem is NP if it can be shown (in deterministic polynomial time) that a given candidate solution is, in fact, a solution to the problem. A problem is NP-Hard if it is at least as hard as any NP problem of equal size. A problem is NP-complete if it is both NP and NP-Hard Say wha??? Every problem in P is clearly also in NP because, if it can be solved deterministically in polynomial time, then it can be solve non-deterministically in polynomial time. We can prove (proof not provided here) that TSP is NP-Hard (trust me on this, or go ahead and prove it to yourself). To be an actual solution, a candidate solution to TSP is NP if we can, in polynomial time show that (1) each city is visited only once AND (2) there is no shorter solution. So far there is no known method to achieve the second condition. So TSP has not been proved to be NP-Complete.

Some NP-Hard Algorithms Efficiently Solve Many Real-World Problems Omitting Requirement # 2 - The Simplex Algorithm, for example, has a worst-case run time of O(2n), however it provides low-order polynomial runtime performance for many real-world problems.

Approximation Algorithms Omitting Requirement #1 - Giving up optimality is by far the most common approach to dealing with the NP-Hard problems when you need a good answer in a reasonable time. a is called the performance guarantee of the algorithm. Sometime it is called the approximation ratio of the algorithm. An a-approximation algorithm for an optimization problem is a polynomial-time algorithm that, for all instances of the problem, produces a solution whose value is within a factor of a of the value of an optimal solution.

Scheduling Jobs with Deadlines

The K-Center Problem

Scheduling Jobs on Identical Parallel Machines

Analysis of Scheduling on Identical Parallel Machines

The Traveling Salesperson Problem

No a-approximation Algorithm Exists for TSP

The Triangle Inequality So what? The triangle inequality is important because it is used to define a class of problems called metric problems. So the distance directly between nodes i and j is always shorter (i.e. straight line) than any path involving an intermediate node. ckj cik cij

TSP Nearest Addition Algorithm (Heuristic)

Analysis of Nearest Addition Algorithm TSP Greedy Tour A Spanning Tree Minimum Spanning Tree

Double-Tree TSP Algorithm Minimum Spanning Tree Multigraph Eulerian Traversal Double-Tree TSP create a minimum spanning tree (e.g. Prim's Algorithm) double up on all the edges (i.e. create a multigraph) generate Eulerian Tour of the nodes remove all but the first instance of each node in tour

Christofides' Algorithm for TSP Minimum Spanning Tree Set of Nodes of Odd Degree Find the minimum spanning tree for complete graph Build a set containing all nodes of odd degree Perfect Matching to Make an Eulerian Graph Shortcut Eulerian Graph Pair up the odd degree nodes so that the total distance between matched pairs is minimal. Adding these edges to the MST produces an Eulerian graph. As in the Double Tree algorithm, shortcut the Eulerian path such that only the first encountering of each node is preserved

Is This As Good As It Gets?

The Euclidean Traveling Salesman Problem