UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lecture 6 Tuesday, 10/30/01 Approximation Algorithms

How to Treat NP-Complete Problems?? ä Approximation Algorithms ä Heuristic Upper or Lower Bounds ä Greedy, Simulated Annealing, Genetic “Alg”, AI ä Mathematical Programming ä Linear Programming for part of problem ä Integer Programming ä Quadratic Programming... ä Search Space Exploration: ä Gradient Descent, Local Search, Pruning, Subdivision ä Randomization, Derandomization ä Leverage/Impose Problem Structure ä Leverage Similar Problems

Overview ä Approximation Algorithms ä Highlights from Textbook ä Highlights from Garey & Johnson ä Literature Case Study ä Homework #4 ä Follow-up on NP-Completeness Literature Case Study

Approximation Algorithms Chapter 37

Basic Concepts Definitions

Definitions ä Approximation Algorithm ä produces “near-optimal” solution  Algorithm has approximation ratio  (n) if: ä Approximation Scheme  (1+  )-approximation algorithm for fixed    > 0 is an input ä Polynomial-Time Approximation Scheme ä time is polynomial in n ä Fully Polynomial-Time Approximation Scheme  time is also polynomial in (1/  )  constant-factor decrease in  causes only constant-factor running-time increase source: textbook Cormen et al. C = cost of algorithm’s solution C* = cost of optimal solution n = number of inputs = size of instance

Overview ä VERTEX-COVER ä Polynomial-time 2-approximation algorithm ä TSP ä TSP with triangle inequality ä Polynomial-time 2-approximation algorithm ä TSP without triangle inequality  Negative result on polynomial-time  (n)-approximation algorithm ä SET-COVER  polynomial-time  (n)-approximation algorithm   (n) is a logarithmic function of set size ä SUBSET-SUM ä Fully polynomial-time approximation scheme ä MAX-3-CNF Satisfiability  Randomized  (n)-approximation algorithm

Vertex Cover Polynomial-Time 2-Approximation Algorithm

Vertex Cover source: Garey & Johnson B E C F D A Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 3 NP-Complete

Vertex Cover source: textbook Cormen et al. b a c d e f ga d e f g a d g a a a c f g

Vertex Cover Theorem: APPROX-VERTEX-COVER is a polynomial-time 2-approximation algorithm. polynomial-time 2-approximation algorithm. Proof: Observe: no 2 edges of A share any vertices due to removal of incident edges Any vertex cover must include >= 1 endpoint of each edge in A APPROX-VERTEX-COVER adds both endpoints of each edge of A transitivity Algorithm runs in time polynomial in n.

Homework #4 ä APPROX-VERTEX-COVER instance ä Textbook: Exercise , p. 968

Traveling Salesman TSP with triangle inequality Polynomial-time 2-approximation algorithm TSP without triangle inequality Negative result on polynomial-time  (n)-approximation algorithm

Hamiltonian Cycle source: textbook Cormen et al. Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. NP-Complete

Traveling Salesman Problem (TSP) source: textbook Cormen et al. uv x w TSP Tour of a complete, undirected graph G=(V,E) is a Hamiltonian Cycle with a designated starting/ending vertex. TSP Decision Problem: cost function NP-Complete

Minimum Spanning Tree: Greedy Algorithms A B C D E F G source: textbook Cormen et al. for Undirected, Connected, Weighted Graph G=(V,E) Produces minimum weight tree of edges that includes every vertex. Invariant: Minimum weight spanning forest Becomes single tree at end Invariant: Minimum weight tree Spans all vertices at end Time: O(|E|lg|E|) given fast FIND-SET, UNION Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue

TSP with Triangle Inequality source: textbook Cormen et al. uv x w Triangle Inequality Cost Function Satisfies

TSP with Triangle Inequality Theorem: APPROX-TSP-TOUR is a polynomial-time 2-approximation algorithm polynomial-time 2-approximation algorithm for TSP with triangle inequality. for TSP with triangle inequality. Proof: Algorithm runs in time polynomial in n. (lists vertices when they are first visited and when returned to after visiting subtree) (since deleting 1 edge from a tour creates a spanning tree) (since full walk traverses each edge of T twice) Now make W into a tour using triangle inequality (since H was formed by deleting vertices from W)

TSP without Triangle Inequality Theorem: If, then for any constant polynomial-time approximation algorithm with ratio  for TSP without triangle inequality. Proof: (by contradiction) Suppose there is one --- call it A Showing how to use A to solve NP-complete Hamiltonian Cycle problemcontradiction! Convert instance G of Hamiltonian Cycle into instance of TSP (in polynomial time): (G’=(V,E’) is complete graph on V) (assign integer cost to each edge in E’) For TSP problem (G’,c): G has Hamiltonian Cycle G does not have Hamiltonian Cycle (G’,c) contains tour of cost |V| Tour of G’ must use some edge not in E Cost of that tour of G’ Can use A on (G’,c)! G has Hamiltonian Cycle A finds tour of cost = |V| G does not have Hamiltonian Cycle A finds tour of cost >  |V|

Homework #4 ä TSP Closest-Point Heuristic proof ä Textbook: Exercise , p. 973

Set Cover Greedy Approximation Algorithm Greedy Approximation Algorithm polynomial-time  (n)-approximation algorithm  (n) is a logarithmic function of set size

Set Cover Problem source: textbook Cormen et al. NP-Complete

Greedy Set Covering Algorithm source: textbook Cormen et al. Greedy: select set that covers the most uncovered elements

Set Cover Theorem: GREEDY-SET-COVER is a polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm for for Proof: Algorithm runs in time polynomial in n. paid only when x is covered for the first time assume x is covered for the first time by S i (spread cost evenly across all elements covered for first time by S i ) Number of elements covered for first time by S i

Set Cover (proof continued) Theorem: GREEDY-SET-COVER is a polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm for for Proof: (continued) Cost assigned to optimal cover: Each x is in >= 1 S: 1 unit is charged at each stage of algorithm

Set Cover (proof continued) Theorem: GREEDY-SET-COVER is a polynomial-time  (n)-approximation algorithm polynomial-time  (n)-approximation algorithm for for Proof: (continued) How does this relate to harmonic numbers?? We’ll show that: And then conclude that: which will finish the proof

Set Cover (proof continued) Proof of: For some set S: Number of elements of S remaining uncovered after S 1 …S i selected k = least index for which u k =0. Since, due to greedy nature of algorithm: since j <= u i-1 telescoping sum

Homework #4 ä GREEDY-SET-COVER output ä Textbook: Exercise , p. 978

Subset-Sum Exponential-Time Exact Algorithm Exponential-Time Exact Algorithm Fully Polynomial-Time Approximation Scheme

Subset-Sum Problem source: textbook Cormen et al. positive integers DecisionProblem: NP-Complete Optimization Problem seeks subset with largest sum <= t

Exponential-Time Exact Algorithm source: textbook Cormen et al. MERGE-LISTS( L,L’ ) returns sorted list = merge of sorted L, L’ with duplicates removed. Identity: L i is sorted list of every element of P i <= t

Fully Polynomial-Time Approximation Scheme source: textbook Cormen et al. Theorem: APPROX- SUBSET-SUM is a fully polynomial- time approximation scheme for subset-sum. Proof: see textbook

Homework #4 ä APPROX-SUBSET-SUM proof ä Textbook: Exercise , p. 983 ä (eq only)

MAX-3-CNF Satisfiability 3-CNF Satisfiability Background 3-CNF Satisfiability Background Randomized Algorithms Randomized MAX-3-CNF SAT Approximation Algorithm

MAX-3-CNF Satisfiability ä Background on Boolean Formula Satisfiability  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: ä n boolean variables: x 1, x 2,..., x n ä m boolean connectives: boolean function with 1 or 2 inputs and 1 output ä e.g. AND, OR, NOT, implication, iff ä parentheses ä truth, satisfying assignments notions apply source: textbook Cormen et al. NP-Complete

MAX-3-CNF Satisfiability (continued) ä Background on 3-CNF-Satisfiability  Instance of language SAT is a boolean formula  consisting of: ä literal: variable or its negation ä CNF = conjunctive normal form ä conjunction: AND of clauses ä clause: OR of literal(s) ä 3-CNF: each clause has exactly 3 distinct literals source: textbook Cormen et al. NP-Complete MAX-3-CNF Satisfiability: optimization version of 3-CNF-SAT - Maximization: satisfy as many clauses as possible - Input Restrictions: - exactly 3 literals/clause - no clause contains both variable and its negation

Definition  Randomized Algorithm has approximation ratio  (n) if, for expected cost C of solution produced by randomized algorithm: source: textbook Cormen et al.

Randomized Approximation Algorithm for MAX-3-CNF SAT source: textbook Cormen et al. Theorem: Given an instance of MAX-3-CNF satisfiability with n variables x 1, x 2,..., x n with m clauses, the randomized algorithm that independently sets each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm. Proof:

Homework #4 ä MAX-CNF 2-approximation algorithm ä New Textbook: Exercise , p ä EXTRA-CREDIT: ä Randomized Algorithm Probability Analysis ä New Textbook: Exercise 5.2-1, p. 98

Approximation Algorithms Garey & Johnson Bin-Packing

Bin-Packing Problems Bin Packing Input: A set of n items with sizes d_1,...,d_n. A set of m bins with capacity c_1,...,c_m. Problem: How do you store the set of items using the fewest number of bins?

Bin-Packing Heuristics source: Garey & Johnson

Bin-Packing Heuristics source: Garey & Johnson

Bin-Packing Heuristics source: Garey & Johnson

Bin-Packing Heuristics source: Garey & Johnson

Approximation Algorithms Literature Case Study “Approximation Schemes for Covering and Packing Problems in Image Processing and VLSI” [Hochbaum and Maass, Journal of ACM, 1985]

Disk Covering Problem |N| = n = 25 How many disks are needed to cover the set of points? Given a set of points N and many copies of a disk... Here 11 disks suffice. Is this optimal?

Notation Area I |N| = n = 25 Disk Diameter = D strip width = D strip includes left boundary strip does not include right boundary strip 1 strip 2 strip 3 strip 4 strip 5 strip 6

Notation (continued) strip 1 strip 2 strip 3 strip 4 strip 5 strip 6 Partition 1 = S 1 = {(strip 1, strip 2, strip 3), (strip 4, strip 5, strip 6)} Partition 2 = S 2 = {(strip 2, strip 3, strip 4), (strip 5, strip 6, strip 1)} Partition 3 = S 3 = {(strip 3, strip 4, strip 5), (strip 6, strip 1, strip 2)}

Shifting Strategy Total Area = I |N| = 25 Disk Diameter = D The paper uses a shifting strategy to develop Polynomial Approximation Scheme for Covering Problems. Strategy bounds error in generic divide-and- conquer. For one partition, apply divide-and-conquer down to vertical strip level and return union of disks in cover. This produces a globally feasible solution. Repeat for each partition. Shift algorithm chooses “best partition.”

Approximation Algorithms Homework #4

Chapter Dependencies Ch 1-6 Math Review: Asymptotics, Recurrences, Summations, Sets, Graphs, Counting, Probability, Calculus, Proofs Techniques (e.g. Inductive) Logarithms Ch 7-10 Sorting Ch 11-14, 15 Data Structures Ch 16, 17, 18 Advanced Design & Analysis Techniques Ch Advanced Data Structures Ch 23-25,26,27 Graph Algorithms Ch 28, 29 Parallel Comparison-Based Sorting Networks, Arithmetic Circuits Ch 30 Parallel Comparison-Based Sorting Networks Ch 31 Matrix Operations Math: Linear Algebra Ch 35 Computational Geometry Math: Geometry (High School Level) Ch 36 NP-Completeness Ch 37 Approximation Algorithms You’re responsible for all sections of this chapter + extra material from Garey & Johnson + supplement from new edition of text + literature paper

Homework #4 ä APPROX-VERTEX-COVER instance ä TSP Closest-Point Heuristic proof ä GREEDY-SET-COVER output ä APPROX-SUBSET-SUM proof ä MAX-CNF 2-approximation algorithm ä EXTRA-CREDIT: ä Randomized Algorithm Probability Analysis

NP-Completeness Follow-up on Literature Case Study

State-of-the-Art: Covering Problems covering P: finite point sets geometric covering 2D translational covering non-geometric covering P: shapes cover partition : VERTEX-COVER, SET-COVER, EDGE-COVER, VLSI logic minimization, facility location Polynomial-time algorithms for triangulation and some tilings Q: convex Q: nonconvex BOX-COVER - -Thin coverings of the plane with congruent convex shapes - -Translational covering of a convex set by a sequence of convex shapes - -Translational covering of nonconvex set with nonidentical covering shapes - -NP-hard/complete polygon problems - -polynomial-time results for restricted orthogonal polygon covering and horizontally convex polygons - -approximation algorithms for boundary, corner covers of orthogonal polygons..... covering decomposition: Q: identical... 1D interval covered by annuli

“Optimal Packing and Covering in the Plane are NP-Complete” [Fowler et al.] source: Fowler et al.

“Optimal Packing and Covering in the Plane are NP-Complete” [Fowler et al.] source: Fowler et al.

“Covering Polygons is Hard” [Culberson, Reckhow] source: Culberson, Reckhow ä Journal (prepublication) version of the paper is on our web site, along with all figures

For More Information.... ä “Approximation Algorithms for NP-Hard Problems”, editor Dorit Hochbaum, PWS Publishing, Co., 1997.