UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2006 Lecture 7 Monday, 4/3/06 Approximation Algorithms

How to Treat NP-Complete Problems?? ä Approximation Algorithms ä Heuristic Upper or Lower Bounds ä Greedy, Simulated Annealing, Genetic “Alg”, AI ä Mathematical Programming ä Linear Programming for part of problem ä Integer Programming ä Quadratic Programming... ä Search Space Exploration: ä Gradient Descent, Local Search, Pruning, Subdivision ä Randomization, Derandomization ä Leverage/Impose Problem Structure ä Leverage Similar Problems

Approximation Algorithms Chapter 35

Basic Concepts Definitions

Definitions ä Approximation Algorithm ä produces “near-optimal” solution  Algorithm has approximation ratio  (n) if: ä Approximation Scheme  (1+  )-approximation algorithm for fixed    > 0 is an input ä Polynomial-Time Approximation Scheme ä time is polynomial in n ä Fully Polynomial-Time Approximation Scheme  time is also polynomial in (1/  )  constant-factor decrease in  causes only constant-factor running-time increase source: textbook Cormen et al. C = cost of algorithm’s solution C* = cost of optimal solution n = number of inputs = size of instance

Overview ä VERTEX-COVER ä Polynomial-time 2-approximation algorithm ä TSP ä TSP with triangle inequality ä Polynomial-time 2-approximation algorithm ä TSP without triangle inequality  Negative result on polynomial-time  (n)-approximation algorithm ä SET-COVER  polynomial-time  (n)-approximation algorithm   (n) is a logarithmic function of set size ä SUBSET-SUM ä Fully polynomial-time approximation scheme ä MAX-3-CNF Satisfiability  Randomized  (n)-approximation algorithm

Vertex Cover Polynomial-Time 2-Approximation Algorithm

Vertex Cover source: Garey & Johnson B E C F D A Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 2 NP-Complete

Vertex Cover source: textbook Cormen et al. b a c d e f ga d e f g a d g a a a c f g

Vertex Cover Theorem: APPROX-VERTEX-COVER is a polynomial-time 2-approximation algorithm. polynomial-time 2-approximation algorithm. Proof: Observe: no 2 edges of A share any vertices due to removal of incident edges Any vertex cover must include >= 1 endpoint of each edge in A APPROX-VERTEX-COVER adds both endpoints of each edge of A transitivity Algorithm runs in time polynomial in n.

Traveling Salesman TSP with triangle inequality Polynomial-time 2-approximation algorithm TSP without triangle inequality Negative result on polynomial-time  (n)-approximation algorithm

Hamiltonian Cycle source: textbook Cormen et al. Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. NP-Complete 34.2

Traveling Salesman Problem (TSP) source: textbook Cormen et al. uv x w TSP Tour of a complete, undirected graph G=(V,E) is a Hamiltonian Cycle with a designated starting/ending vertex. TSP Decision Problem: cost function NP-Complete

Minimum Spanning Tree: Greedy Algorithms A B C D E F G source: textbook Cormen et al. for Undirected, Connected, Weighted Graph G=(V,E) Produces minimum weight tree of edges that includes every vertex. Invariant: Minimum weight spanning forest Becomes single tree at end Invariant: Minimum weight tree Spans all vertices at end Time: O(|E|lg|E|) given fast FIND-SET, UNION Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue

TSP with Triangle Inequality source: textbook Cormen et al. uv x w Triangle Inequality Cost Function Satisfies final approximate tour optimal tour “full walk” = abcbhbadefegeda

TSP with Triangle Inequality Theorem: APPROX-TSP-TOUR is a polynomial-time 2-approximation algorithm polynomial-time 2-approximation algorithm for TSP with triangle inequality. for TSP with triangle inequality. Proof: Algorithm runs in time polynomial in n. (lists vertices when they are first visited and when returned to after visiting subtree) (since deleting 1 edge from a tour creates a spanning tree) (since full walk traverses each edge of T twice) Now make W into a tour H using triangle inequality. (New inequality holds since H is formed by deleting duplicate vertices from W.)

TSP without Triangle Inequality Theorem: If, then for any constant polynomial-time approximation algorithm with ratio  for TSP without triangle inequality. Proof: (by contradiction) Suppose there is one --- call it A Showing how to use A to solve NP-complete Hamiltonian Cycle problemcontradiction! Convert instance G of Hamiltonian Cycle into instance of TSP (in polynomial time): (G’=(V,E’) is complete graph on V) (assign integer cost to each edge in E’) For TSP problem (G’,c): G has Hamiltonian Cycle G does not have Hamiltonian Cycle (G’,c) contains tour of cost |V| Tour of G’ must use some edge not in E Cost of that tour of G’ Can use A on (G’,c)! G has Hamiltonian Cycle A finds tour of cost at most  |V| cost = |V| G does not have Hamiltonian Cycle A finds tour of cost >  |V| A finds tour of cost at most  (length of optimal tour) If, solving NP-complete hamiltonian cycle in polynomial time is a contradiction!

Set Cover Greedy Approximation Algorithm Greedy Approximation Algorithm polynomial-time  (n)-approximation algorithm  (n) is a logarithmic function of set size

Set Cover Problem source: textbook Cormen et al. NP-Complete

Greedy Set Covering Algorithm source: textbook Cormen et al. Greedy: select set that covers the most uncovered elements

Set Cover Theorem: GREEDY-SET-COVER is a polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm for for Proof: Algorithm runs in time polynomial in n. paid only when x is covered for the first time assume x is covered for the first time by S i (spread cost evenly across all elements covered for first time by S i ) Number of elements covered for first time by S i

Set Cover (proof continued) Theorem: GREEDY-SET-COVER is a polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm for for Proof: (continued) Cost assigned to optimal cover: Each x is in >= 1 S in C* 1 unit is charged at each stage of algorithm

Set Cover (proof continued) Theorem: GREEDY-SET-COVER is a polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm for for Proof: (continued) How does this relate to harmonic numbers?? We’ll show that: And then conclude that: which will finish the proof

Set Cover (proof continued) Proof of: For some set S: Number of elements of S remaining uncovered after S 1 …S i selected k = least index for which u k =0. Since, due to greedy nature of algorithm: since j <= u i-1 telescoping sum

Subset-Sum Exponential-Time Exact Algorithm Exponential-Time Exact Algorithm Fully Polynomial-Time Approximation Scheme

Subset-Sum Problem source: textbook Cormen et al. positive integers DecisionProblem: NP-Complete Optimization Problem seeks subset with largest sum <= t

Exponential-Time Exact Algorithm source: textbook Cormen et al. MERGE-LISTS( L,L’ ) returns sorted list = merge of sorted L, L’ with duplicates removed. Identity: L i is sorted list of every element of P i <= t

Fully Polynomial-Time Approximation Scheme source: textbook Cormen et al. Theorem: APPROX- SUBSET-SUM is a fully polynomial- time approximation scheme for subset-sum. Proof: see textbook

MAX-3-CNF Satisfiability 3-CNF Satisfiability Background 3-CNF Satisfiability Background Randomized Algorithms Randomized MAX-3-CNF SAT Approximation Algorithm

MAX-3-CNF Satisfiability ä Background on Boolean Formula Satisfiability  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: ä n boolean variables: x 1, x 2,..., x n ä m boolean connectives: boolean function with 1 or 2 inputs and 1 output ä e.g. AND, OR, NOT, implication, iff ä parentheses ä truth, satisfying assignments notions apply source: textbook Cormen et al. NP-Complete

MAX-3-CNF Satisfiability (continued) ä Background on 3-CNF-Satisfiability  Instance of language SAT is a boolean formula  consisting of: ä literal: variable or its negation ä CNF = conjunctive normal form ä conjunction: AND of clauses ä clause: OR of literal(s) ä 3-CNF: each clause has exactly 3 distinct literals source: textbook Cormen et al. NP-Complete MAX-3-CNF Satisfiability: optimization version of 3-CNF-SAT - Maximization: satisfy as many clauses as possible - Input Restrictions: - exactly 3 literals/clause - no clause contains both variable and its negation

Definition  Randomized Algorithm has approximation ratio  (n) if, for expected cost C of solution produced by randomized algorithm: source: textbook Cormen et al. size of input = n

Randomized Approximation Algorithm for MAX-3-CNF SAT source: textbook Cormen et al. Theorem: Given an instance of MAX-3-CNF satisfiability with n variables x 1, x 2,..., x n with m clauses, the randomized algorithm that independently sets each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm. Proof:

For More Information.... ä “Approximation Algorithms for NP-Hard Problems”, editor Dorit Hochbaum, PWS Publishing, Co., ä “Approximation Algorithms” by Vazirani. ä “Computers and Intractability” by Garey & Johnson.