1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2002 Lecture 6 Tuesday, 10/8/02 NP-Completeness

2. Overview ä NP-Completeness ä Motivation ä Reduction to Establish Lower Bounds ä Basics ä Literature ä How to Treat NP- Hard or Complete Problems ä Midterm Exam

3. Motivation

4. Why do we care?? ä When faced with a problem in practice, need to know whether or not one can expect produce a “fast” algorithm (i.e. O(n k )) to solve it ä “Intractable” problems occur frequently in practice ä Surface differences between tractable and intractable problems are often hard to see

5. UMass Lowell Algorithms Lab Telecommunications Data Mining & Visualization Manufacturing Design Analyze Apply PhD research Undergraduate & Masters research OS 305 Prof. Daniels

6. Industrial Motivation Supporting Apparel Manufacturing OrderedContainment Geometric Restriction Distance-BasedSubdivision MaximumRectangle Limited Gaps MinimalEnclosure Column-Based Layout Two-Phase Layout LatticePacking Maximal Cover Containment NP-complete

7. Dynamic Covering Motivation sensor coverage sensor coverage clustering clustering shape recognition shape recognition channel channel assignment assignment production allocation production allocation facility location facility location layout layout

8. New Covering Research: Objective ä Practical solutions to dynamic hard core covering problems ä dynamic instance = sequence of static instances ä hard : even static problem is computationally “intractable” ä core : useful in multiple applications ä covering ~ assignment ~ allocation ä Practical : solve static instance i quickly if it is “easy” ä combinatorially “small” or ä clearly infeasible (for decision problem) or ä clearly feasible (for decision problem) or ä a “small” change from instance i-1 static instance 0 static instance 1 static instance i...

9. State-of-the-Art: Dynamic Algorithms & Data Structures ä Polynomial-time problems: ä Data structures supporting fast updates ä Self-adjusting (e.g. splay trees) [Sleator, CMU] ä Kinetic Data Structures [Guibas, Stanford] ä Hard problems: ä Polynomial-time approximation schemes for optimization problems : “Greedy” can be sensitive to small changes [Gao, et al.] ä Meta-heuristics: no theoretical guarantee ä Practical algorithms for small and/or easy instances: ä Containment decision problem: [Daniels, Milenkovic] ä successful building block for layout problems ä leverages strong combinatorial optimization techniques ä Distributed heuristics ä Primarily for dynamic network routing or channel assignment ä [Wu et al.] connected dominating sets for network routing ä no theoretical guarantee; some empirical support ä Randomized, distributed: ä [Gao, et al.] discrete mobile centers (clustering) ä theoretical framework; assumes knowledge of change trajectories static

10. State-of-the-Art: Covering Problems covering P: finite point sets geometric covering 2D translational covering non-geometric covering P: shapes cover partition : VERTEX-COVER, SET-COVER, EDGE-COVER, VLSI logic minimization, facility location Polynomial-time algorithms for triangulation and some tilings Q: convex Q: nonconvex BOX-COVER - -Thin coverings of the plane with congruent convex shapes - -Translational covering of a convex set by a sequence of convex shapes - -Translational covering of nonconvex set with nonidentical covering shapes - -NP-hard/complete polygon problems - -polynomial-time results for restricted orthogonal polygon covering and horizontally convex polygons - -approximation algorithms for boundary, corner covers of orthogonal polygons..... covering decomposition: Q: identical... 1D interval covered by annuli

11. New Covering Research Example 1: Translational 2D Covering [recent work with grad student Rajasekhar Inkulu (CCCG conference paper)] Q3Q3 Q1Q1 Q2Q2 Sample P and Q P1P1 P2P2 Translated Q Covers P P1P1 Q1Q1 Q2Q2 Q3Q3 P2P2 Translational 2D Polygon Covering ä Input: ä Covering polygons Q = {Q 1, Q 2,..., Q m } ä Target polygons (or point-sets) P = {P 1, P 2,..., P n } ä Output:  Translations  = {  1,  2,...,  m } such that STATIC NP-COMPLETE

12. New Covering Research Example 2: 2D Channel Assignment [recent work with Masters student Sa Liu (GLOBECOM conference paper) ] ä Input: ä r x c cell grid ä q channels (frequencies) ä D(i,j) = cell(i,j) traffic demand ä # cell-phone calls ä Co-channel interference constraints based on interference threshold value ä Output: ä Feasible assignment of channels to cells that ä satisfies demand and co-channel interference constraints ä uses minimum number of channels e.g. no channel repetition within any 2 x 2 square STATIC NP-COMPLETE

13. Reduction to Establish Lower Bound Tightening a Lower Bound for Maximum-Area Axis-Parallel Rectangle Problem from  (n) to  (nlgn) from  (n) to  (nlgn)

14. Approach ä Go around lower bound brick wall by: ä examining strong lower bounds for some similar problems ä transforming a similar problem to our problem [this process is similar to what we do when we prove problems NP- complete] n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2 n log 2 n

15. Lower Bounds for Related Problems n1 n log(n) n log 2 (n) 2n2n2n2n n5n5n5n5 SmallestOuterRectangle:   (n) SmallestOuterCircle:   (n) LargestInnerRectangle:   (n) LargestInnerCircle:   (n log n) point set, polygon point set polygon LargestInnerRectangle:   (n log 2 (n))polygon worst-case bounds on our problem

16. Lower Bound of  (n log n) by Transforming a ( seemingly unrelated ) Problem MAX-GAP instance: given n real numbers { x 1, x 2,... x n } find the maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation specialized polygon x2x2x2x2 x4x4x4x4 x3x3x3x3 x1x1x1x1 Rectangle area is a solution to the MAX-GAP instance Rectangle algorithm must take as least as much time as MAX-GAP. MAX-GAP is known to be in  (n log n). Rectangle algorithm must take  (n log n) time for specialized polygons. [Transforming yet another different problem yields bound for unspecialized polygons.]

17. NP-Completeness Chapter 34

18. Basic Concepts Polynomial Time & Problem Formalization Polynomial-Time Verification NP-Completeness & Reducibility NP-Completeness Proofs Expanding List of Hard Problems via Reduction

19. Polynomial Time & Problem Formalization ä Formalize notions of abstract and concrete decision problems ä Encoding a problem ä Define class of polynomial-time solvable decision problems ä Define complexity class ä Express relationship between decision problems and algorithms that solve them using framework of formal language theory

20. Polynomial Time & Problem Formalization (continued) ä Solvable in Polynomial Time  i.e.  (n k ) for some positive constant k ä Tractable: k is typically small (e.g. 1, 2, 3) ä Typically holds across computation models ä Closure properties (chaining/composition) ä Abstract Problem ä Decision Problem: yes/no answer ä impose bound on optimization problem to recast

21. Polynomial Time & Problem Formalization (continued) ä Encoding of set S of abstract objects is mapping e from S to set of binary strings ä Concrete Problem: instance set is set of binary strings  Concrete Problem is Polynomial-Time Solvable if there exists an algorithm to solve it in time  (n k ) for some positive constant k ä Complexity Class P = set of polynomial-time solvable concrete decision problems ä Given abstract decision problem Q that maps instance set I to {0,1} can induce a concrete decision problem e(Q) ä If solution to abstract-problem instance ä then solution to concrete-problem instance is also Q(i)

22. Polynomial Time & Problem Formalization (continued) ä is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input, produces f(x) as output. ä For set I of problem instances, encodings are polynomially related if there exist 2 polynomial-time computable functions f 12, f 21 such that ä If 2 encodings e 1, e 2 of abstract problem are polynomially related, then polynomial-time solvability of problem is encoding-independent: Lemma 34.1: Let Q be an abstract decision problem on an instance set I and let e 1, e 2 be polynomially related encodings on I. Then, Goal: Extend definition of polynomial-time solvability from concrete to abstract using encodings as bridge

23. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework  Alphabet  is finite set of symbols  Language L over  is any set of strings consisting of symbols from  ä Set of instances for decision problem Q is  View Q as language L over  = {0,1} ä Algorithm A accepts string if, given input x, output A(x) is 1 ä Language accepted by A is set of strings A accepts

24. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework (continued) ä Language L is decided by algorithm A if every binary string in L is accepted by A and every binary string not in L is rejected by A ä Language L is accepted in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string, A accepts x in time O(n k ) ä Language L is decidable in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string, A correctly decides in time O(n k ) if

25. Polynomial Time & Problem Formalization (continued) ä Formal Language Framework (continued) ä Complexity Class is a set of languages whose membership is determined by a complexity measure (i.e. running time) of algorithm that determines whether a given string belongs to a language. ä We already defined: Complexity Class P = set of polynomial- time solvable concrete decision problems ä Language Framework gives other definitions: P = { : there exists an algorithm A that decides L in polynomial time} decides L in polynomial time} = { L : L is accepted by a polynomial-time algorithm} = { L : L is accepted by a polynomial-time algorithm}

26. Polynomial-Time Verification ä Examine algorithms that “verify” membership in languages ä Define class of decision problems whose solutions can be verified in polynomial time

27. Polynomial-Time Verification (continued) ä Verification Algorithms ä algorithm A verifies language L if for any string certificate y that A can use to prove that ä also, for any string certificate y that A can use to prove that ä language verified by verification algorithm A is

28. Polynomial-Time Verification (continued) ä The Complexity Class NP ä languages that can be verified by a polynomial- time algorithm ä A verifies language L in polynomial time ä The Complexity Class co-NP ä set of languages such that

29. Polynomial-Time Verification (continued) source: 91.503 textbook Cormen et al. P P P 34.3

30. NP-Completeness & Reducibility ä Polynomial-Time Reducibility ä Define complexity class of NP-complete problems ä Relationships among complexity classes ä Circuit satisfiability as an NP-complete problem

31. NP-Completeness & Reducibility (continued) ä Q can be reduced to Q’ if any instance of Q can be “easily rephrased as an instance of Q’, the solution to which provides solution to instance of Q ä L 1 is polynomial-time reducible to L 2 if exists polynomial-time computable function source: 91.503 textbook Cormen et al. 34.4

32. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al. 34.5 34.3

33. NP-Completeness & Reducibility (continued) source: 91.503 textbook Cormen et al. Language is NP-complete if and without this condition, language is only NP-hard NP set of all NP- complete languages popular theoretical computer science view Theorem: - If any NP-complete problem is polynomial-time solvable, then P=NP. - Equivalently, if any problem in NP is not polynomial-time solvable, then no NP-complete problem is polynomial-time solvable.

34. NP-Completeness & Reducibility (continued): Circuit Satisfiability ä Satisfying Assignment: truth assignment inducing output = 1 ä GOAL: Show Circuit Satisfiability is NP-complete source: 91.503 textbook Cormen et al. Boolean Combinational Gate Types: AND, NOT, OR single circuit output circuit inputs Language Def: CIRCUIT-SAT = { :C is a satisfiable boolean combinational circuit.} 34.8

35. NP-Completeness & Reducibility (continued) Circuit Satisfiability ä CIRCUIT-SAT is in NP: ä Construct 2-input polynomial-time algorithm A that verifies CIRCUIT-SAT ä Input 1: encoding of circuit C ä Input 2: certificate = assignment of boolean values to all wires of C ä Algorithm A : ä For each logic gate A checks if certificate value for gate’s output wire correctly computes gate’s function based on input wire values ä If output of entire circuit is 1, algorithm outputs 1 ä Otherwise, algorithm outputs 0 ä For satisfiable circuit, there is a (polynomial-length) certificate that causes A to output 1 ä For unsatisfiable circuit, no certificate can cause A to output 1 ä Algorithm A runs in polynomial time source: 91.503 textbook Cormen et al.

36. NP-Completeness & Reducibility (continued) Circuit Satisfiability ä CIRCUIT-SAT is NP-hard: ä L represents some language in NP ä Create polynomial-time algorithm F computing reduction function f that maps every binary string x to a circuit C=f(x) such that ä ä Must exist algorithm A verifying L in polynomial time. (A from NP) ä F uses A to compute f. ä Represent computation of A as sequence of configurations ä F constructs single circuit that computes all configurations produced by initial configuration source: 91.503 textbook Cormen et al.

37. NP-Completeness Proofs (continued) ä Proving a Language NP-Complete ä Proving a Language NP-Hard ä all steps except (1) source: 91.503 textbook Cormen et al.

38. NP-Completeness Proofs (continued) ä Reducing Boolean Circuit Satisfiability to Boolean Formula Satisfiability  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  consisting of: ä n boolean variables: x 1, x 2,..., x n ä m boolean connectives: boolean function with 1 or 2 inputs and 1 output ä e.g. AND, OR, NOT, implication, iff ä parentheses ä truth, satisfying assignments notions apply source: 91.503 textbook Cormen et al.

39. NP-Completeness Proofs (continued) ä Show: SAT is NP-Complete ä SAT is in NP via argument similar to circuit case ä Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability ä Careful! Naive approach might have shared subformulas and cause formula size to grow exponentially! source: 91.503 textbook Cormen et al.

40. NP-Completeness Proofs (continued) ä Show: SAT is NP-Complete (continued) ä Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability (continued)  Approach: For each wire x i in circuit C, formula  has variable x i. Express gate operation. source: 91.503 textbook Cormen et al. 34.10

41. NP-Completeness Proofs (continued) ä Reducing Formula Satisfiability to 3-CNF- Satisfiability  Boolean Formula Satisfiability: Instance of language SAT is a boolean formula  ä CNF = conjunctive normal form ä conjunction: AND of clauses ä clause: OR of literal(s) ä 3-CNF: each clause has exactly 3 distinct literals ä Show: 3-CNF is NP-Complete source: 91.503 textbook Cormen et al.

42. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF ä 1) Similar to reduction source: 91.503 textbook Cormen et al. Binary “ParseTree” 34.11

43. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF  2) Convert each clause  i ’ into conjunctive normal form  construct truth table for  i ’ ä using entries for 0, build DNF formula for  convert DNF into CNF formula  i ” by applying DeMorgan’s laws ä each clause has at most 3 literals source: 91.503 textbook Cormen et al.

44. NP-Completeness Proofs (continued) ä Show: 3-CNF is NP-Complete (continued) ä 3CNF is in NP via argument similar to circuit case ä Reduce Formula Satisfiability to 3-CNF-Satisfiability  3 steps progressively transforming formula  closer to 3CNF  3) Convert each clause  i ” so it has exactly 3 distinct literals ä add “padding variables”  create  i ”’ ä Size of resulting formula is polynomial in length of original formula and reduction takes only polynomial time and 3-CNF formula source: 91.503 textbook Cormen et al.

45. Expanding List of Hard Problems via Reduction ä Relationships among some NP-complete problems source: 91.503 textbook Cormen et al. source: Garey & Johnson Circuit-SAT

46. Vertex Cover source: Garey & Johnson B E C F D A Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 2

47. Dominating Set source: Garey & Johnson

48. Clique source: Garey & Johnson B E C F D A Clique of an undirected graph G=(V,E) is a complete subgraph of G. clique of size 3

49. Hamiltonian Cycle source: 91.503 textbook Cormen et al. Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. 34.2

50. Reducing 3-CNF-SAT to Clique x1x1x1x1 x1x1x1x1 x2x2x2x2 x2x2x2x2 x1x1x1x1 x3x3x3x3 Construct graph G such that  is satisfiable iff G has clique of size k For each clause place triple of vertices v 1 r, v 2 r, v 3 r into V. Put edge for v i r, v j s if corresponding literals consistent and r not= s. source: 91.503 textbook Cormen et al. 34.14

51. Reducing Clique to VertexCover source: 91.503 textbook Cormen et al. Construct complement G =(V, E ) of graph G such that G has clique of size k iff graph G has a vertex cover of size |V|-k. z w u y v x 34.15

52. Literature

53. Resources - Recommended: ä Computers & Intractability ä by Garey & Johnson ä W.H.Freeman ä 1990 ä ISBN 0716710455.

54. How to Treat NP-Hard or Complete Problems

55. How to Treat NP-Complete Problems?? ä Approximation Algorithms ä Heuristic Upper or Lower Bounds ä Greedy, Simulated Annealing, Genetic “Alg”, AI ä Mathematical Programming ä Linear Programming for part of problem ä Integer Programming ä Quadratic Programming... ä Search Space Exploration: ä Gradient Descent, Local Search, Pruning, Subdivision ä Randomization, Derandomization ä Leverage/Impose Problem Structure ä Leverage Similar Problems