Approximation Schemes Meera Mohan Sudhir Koduri Xiaolu Liu

Why Approximation Algorithms Problems that we cannot find an optimal solution in a polynomial time E.g. Set Cover, Bin Packing Need to find a near-optimal solution: Heuristic Approximation algorithms: to find approximate solutions to optimization problems

Approximation Algorithms to find approximate solutions to optimization problems guarantees an approximation ratio often associated with NP-hard problems unlikely that there can ever be efficient polynomial time exact algorithms solving NP-hard problems settle for polynomial time non-optimal solutions the approximation is optimal up to a small constant factor (for instance within 5% of the optimal solution) increasingly being used for problems where exact polynomial algorithms are known but are too expensive due to the sizes of the data sets

Computationally Hard Problems You are given a computationally hard problem. Here are two scenarios: No knowledge about approximation: Spend a few months looking for an optimal solution Confess that you cannot do it Get fired!!! Knowledge about approximation: Show that this is a NP-complete (NP-hard) problem Propose a good algorithm (either heuristic or approximation) to find a near-optimal solution Calculate the approximation ratio

Application of Approximation Algorithms Shortest path: Given a graph with edge costs and a pair of nodes, find the shortest path (least costs) between them Traveling salesman: Given a complete graph with nonnegative edge costs, find a minimum cost cycle visiting every vertex exactly once

Approximation Algorithms An algorithm to solve an optimization problem that runs in polynomial time in the length of the input and outputs a solution that is guaranteed to be close to the optimal solution. "Close" has some well-defined sense called the performance guarantee. guaranteed to find a solution at most (or at least, as appropriate) ρ times the optimum ratio ρ is the performance ratio or relative performance guarantee of the algorithm Approximation Ratio ρ(n): Define: C* as a optimal solution and C as the solution produced by the approximation algorithm max (C/C*, C*/C) <= ρ(n) Maximization problem: 0 < C <= C*, thus C*/C shows that C* is larger than C by ρ(n) Minimization problem: 0 < C* <= C, thus C/C* shows that C is larger than C* by ρ(n)

Approximation Algorithms (cont) PTAS (Polynomial-Time Approximation Scheme): (1 + ε)-approximation algorithm, A for a NP-hard optimization П where its running time is bounded by a polynomial in the size of instance I ε is the precision requirement A is the approximation algorithm Family {Ae}e>0 of (1+e)-approximation algorithms running time polynomial in |I| ok: exponential in 1/e : e.g. O(|I|1/e)

Approximation Algorithms (cont) FPTAS (Fully PTAS): The same as PTAS + time is bounded by a polynomial in both the size of instance I and 1/ε FPTAS: Fully PTAS running time also polynomial in 1/e : e.g. O(|I|/e3)

Catch-22 We cannot find C*, how can we compare C to C*? How can we design an algorithm so that we can compare C to C* This is the objective of this presentation!!!

Optimization Problem Instance: I Length of description: |I|=n Goal: minimize objective function ==> Opt(I)

Complexity Classes Relationships NPO APX PTAS FPTAS PO PO C FPTAS C PTAS C APX C NPO

Strongly and Weakly NP-hard If a problem is NP-hard even if the input is encoded in unary, then it is called strongly NP-hard = NP-hard in the strong sense = unary NP-hard If a problem is polynomially solvable under a unary encoding, then it is solvable in pseudo-polynomial time. weak sense strong sense

p-approximable class Let π be an optimization problem; if its decision version is strongly NP-complete, then π is not fully p-approximable. Theorem 8.14 FPTAS: Strong NP-hardness ==> no FPTAS

Knapsack Problem Knapsack is p-approximable Fully p-approximable Knapsack problem is solvable in pseudo-polynomial time

Properties of Optimization Problem Let π be an optimization problem with the following properties: f(I) and max(I) are polynomially related through len(I); that is, there exist bivariate polynomials p and q such that we have both: f(I) ≤ p(len(I), max(I)) and max(I) ≤ q(len(I), f(I)); The objective value of any feasible solution varies linearly with the parameters of the instance; and, π can be solved in pseudo-polynomial time. Then π is fully p-approximable.

Simple and p-simple Problems An optimization problem is simple if, for each fixed bound B, the set of instances with optimal values not exceeding B is decidable in polynomial time. For example, Clique, Vertex Cover and Set Cover are simple problems An optimization problem is p-simple if, there exists a fixed bi-variate polynomial q, such that the set of instances I with optimal values not exceeding B is decidable in q(|I|, B) time. For example, Partition is a p-simple problem

simple : p-simple :: PTAS : FPTAS Analogy simple : p-simple :: PTAS : FPTAS A problem is p-simple if it is simple + the uniformity condition, much like our fully p-approximable Simplicity is necessary but, not a sufficient condition for membership in PTAS. For example, Clique, while simple cannot be in PTAS unless P=NP

Analogy (cont’d) Let π be an optimization problem. If π is p-approximable(π Є PTAS), then it is simple If π is fully p-approximable(π Є FPTAS), then it is p-simple PTAS ==> simple problem FPTAS ==> p-simple problem Simple problem + P=NP ==> PTAS

p-simple If π is an NP problem with an NP complete decision version, and for each instance I of π, f (I) and max (I) are polynomially related through len (I), then π is p-simple if and only if, it can be solved in pseudo-polynomial time. ==> class PTAS is much richer than class FPTAS

Maximum Independent Subset Problem An instance of a maximum independent subset problem is given by a collection of items, each with a value. The feasible solutions of the instance form an independence system; that is, every subset of a feasible solution is also a feasible solution. The goal is to maximize the sum of the values of the items included in the solution.

Summary Approximability has its own hierarchy of complexity classes Problems in NP have very different approximability properties: … some are impossible to approximate (k-center, TSP) … some are hard, with a bound depending on the input size (set cover) … some can be approximated with some constant ratio (vertex cover, k-center with triangle inequality, TSP with triangle inequality) … and some can be approximated as closely as you like (knapsack) Approximability has its own hierarchy of complexity classes

References The Theory of Computation, Bernard M.Moret. Addison-Wesley, 1998 Theory of Computation, Dexter Kozen, Springer, 2006 M.X. Geomans, "The Knapsack Problem and Fully Polynomial Time Approximation Schemes (FPTAS)," Seminar Series in Theoretical Computer Science, March 10, 2006 H. Sachnai and T. Tamir, "Polynomial Time Approximation Schemes- A Survey" V. Vazirani, "Approximation Algorithms," Springer, pp.68-72, 2003.

Thank You !!!

Approximation Schemes Sudhir Koduri

Problem – Proving completion algorithm is indeed well behaved. Theorem 8.18 A Maximum independent subset problem is in PTAS if and only if, for any k, it admits a polynomial-time k-completion algorithm. Problem – Proving completion algorithm is indeed well behaved. Shifting Technique Decomposes a problem into suitably sized “adjacent” subpieces and create subproblems by grouping a number of adjacent subpieces.

Example A linear array of some kl subpieces in which the subpieces end up in l groups of k consecutive subpieces each. Disk Covering Problem - N points in the plane and disks of fixed diameter D, cover all points with the smallest number of disks. Approx. algo. divides the area in which the n points reside into vertical strips of width D.

Theorem 8.19 If algorithm A has absolute approximation ratio RA, then the shifting algorithm has absolute approximation ratio (kRA + 1/k+1). Since A yields RA approximations, the number of approximations for partition is bounded by 1/RA Σ Nj , where Nj is the optimal number of disks needed to cover the points.

A disk cannot cover points in two elementary strips that are not adjacent. We find the locally optimal solutions within each strip Union of all would result in exceeding N by at most the number of the disks that cover in two adjacent strips.

Divide and conquer Strategy New problem To minimize the number of disks of diameter D needed cover a collection of points placed in a vertical strip of width kD Divide and conquer Strategy

Fixed-Ratio Approximations There are a very large number of problems that have - fixed-ratio approximations schemes - belong to Apx but do not appear to PTAS Examples Vertex Cover, Maximum Cut, Maximum 3SAT

Theorem 8.21 MAXkSAT has a 2¯k approximation. Consider the following simple algorithm. Assign to each remaining clause ci weight 2-ci; thus every unassigned literal left in a clause halves the weight of that clause. Pick any variable X that appears in some remaining clause and set X to true if the sum of the weights of the clauses in which X appears as an uncomplemented literal exceeds the sum of the clauses in which it appears as a complemented literal; set it to false otherwise.

Update the clauses and their weights and repeat until all clauses have been satisfied or reduced to a falsehood.

Classification of problems within classes NPO, Apx and PTAS?? Reduction – type of reduction Correspondence between solutions and instances The main reason for these requirements is that we need to be able to retrieve a good approximation for problem ¶1 from a reduction to a problem ¶2 for which we already have an approximate solution algorithm.

Fixed-Ratio Approximations Xiaolu Liu Mississippi State University Nov 12, 2007 CSEET

Outline Introduce Fixed-Ratio Approximations APX, PTAS,NPO Optimization Problems Approximation Algorithms Fixed-Ratio Approximations APX, PTAS,NPO Classify NPO, APX, PTAS Scheme of the reduction PTAS-reduces Gap-preserving reducing

Optimization Problems Many hard problems (especially NP-hard) are optimization problems e.g. find shortest TSP tour e.g. find smallest vertex cover e.g. find largest clique Minimization or maximization problem

Approximation Algorithms Often happy with approximately optimal solution we want approximation algorithm with guaranteed approximation ratio of r meaning: on every input x, output is guaranteed to have value at most r*opt for minimization at least opt/r for maximization “opt” = value of optimal solution

Fixed-ration Approximation C-approximation algorithm for some constant c if it can be proven that the solution is at most c times worse than the optimal solution. C is called the approximation ratio Within 5% of the optimal solution Fixed-ration approximation-Ideally, the approximation is optimal up to a small constant factor (Eg. within 5% of the optimal solution).

APX and PTAS PTAS If there is a polynomial-time algorithm to solve a problem within every fixed percentage polynomial-time approximation scheme APX the set of NPO problems that can be approximated within some constant factor, but not every constant factor. Unless P=NP, there are problems in APX but not in PTAS To say a problem is APX-hard is generally bad news, because it denies the existence of a PTAS, which are the most useful sort of approximation algorithm.

Classify NPO, APX, PTAS Scheme of the reduction The requisite style of reduction between approximation problems

Scheme of the reduction Establish a correspondence between solutions as well as instances Preserve approximation ratios Map f between instances Map g between solutions along with know algorithm A New approximation algorithm A’ Call in succession the routines map f->A->map g map g between solutions along with know algorithm A, using A we have been able to get a good approximation for our original problem

Scheme of the reduction Sufficient generality to prove the separation of NPO, PTAS, and APX 3rd map function In order to get Sufficient generality, we introduce 3rd map function, maps precision requirements for TT1 onto precision requirement for TT2

Scheme of the reduction Difference come from the requirements on handling the precision requirement PTAS-reduction Separate APX and PTAS Gap-preserving reducing Separate NPO and PTAS

PTAS-reduction Let and be two problems in NPO. PTAS-reduces Three functions f, g, and h Instance x of , f(x) is an instance of , and computable in time polynomial in |x| Instance x of , solution y for instance f(x) of , and precision requirement , g(x,y, ) is a solution for x and computable in time polynomial in |x| and |y| h is computable injective function on the set of rationals in [0,1) If the value y obeys h( ), the value of g(x,y, ) obeys Both f and g must be computable in a polynomail time map g between solutions along with know algorithm A, using A we have been able to get a good approximation for our original problem

Proposition of PTAS-reduces Reflexive and transitive If belongs to APX (respectively ,PTAS), then belongs to APX (respectively, PTAS) Complete for NPO An optimization problem belongs to NPO (respectively, APX) Every Problem in NPO (respectively, APX) PTAS-reduces to it

Max3SAT 2-k approximation Most fundamental NP-complete problem Key problem in APX due to its nature

OptNp The class of problems that PTAS-reduce to Max3SAT OptNp APX Not see nature problems that are complete for NPO and APX OptNp has at lest one Max3SAT The standard complete problems for NPO and APX are generalizations of Max3SAT

Why OptNp Interesting Include many natural problems Bounded-degree Vertex Cover Bounded-degree Independent Set Maximum Cut Many of them are OptNp-Hard OptNp-Hard can not be in PTAS unless P=NP Many of them are OptNp-Hard, we can use PTAS-reduction from Max3SAT to show that

Gap-preserving reducing To prove NPO not belong to PTAS unless p=NP Create a gap: Decision Problem  Opimization Problem “yes” instance  instance with optimal value on one side of the gap “no”instance  on the other side of the gap

Gap-preserving reducing Two maximization problems Polynomial-time map Two pair of functions (C1,r1) and (C2,r2) opt(x) c1(x) opt(f(x)) c2(f(x)) Opt(x)  c1(x)/r1(x) opt(f(x))  c2(f(x))/r2(f(x)) yes yes C2,r2 C1,r1 f r2 r1 no Gap-preserving reduction from TT1 to TT2 is a polynomail-time mapping They should hold this implication no

Use of Gap-preserving reducing Combine it with a gap-creating reduction Be of limited interest The problem for which we had a gap-creating reduction were few Not been used much in further transformation OptNP-hard problems can not be in PTAS, unless P=NP It has become possible to prove any of the OptNP-hard problems can not be in PTAS, unless P=NP

Conclusion- NP-Hardness of Approximation Scheme The problem is not p-simple Its decision version is strongly NP-complete Then it is not in FPTAS, unless P=NP

Conclusion- NP-Hardness of Approximation Scheme The problem is not p-simple It is OptNP-hard Then it is not in PTAS, unless P=NP

Conclusion No OptNP-Hard problem can be in PTAS unless P=NP OptNP=APX

Thanks Any Questions?