Advanced Algorithms Analysis and Design By Dr. Nazir Ahmad Zafar Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Lecture No. 44 NP Completeness Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NP Completeness Polynomial Time Algorithms On any inputs of size n, if worst-case running time of algorithm is O(nk), for constant k, then it is called polynomial time algorithm Every problem can not be solved in polynomial time There exists some problems which can not be solved by any computer, in any amount of time, e.g. Turing’s Halting Problem Such problems are called un-decidable Some problems can be solved but not in O(nk) Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NP Completeness Polynomial Time (Class P) These problems are solvable in polynomial time Problems in class P are also called tractable Intractable Problems Problems not in P are called intractable These problems can be solved in reasonable amount of time only for small input size Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Decision & Optimization Problems Decision problems The problems which return yes or no for a given input and a question regarding the same problem Optimization problems Find a solution with best value, it can be maximum or minimum. There can be more than one solutions for it Optimization problems can be considered as decision problems which are easier to study. Finding a path between u and v using fewest edges in an un-weighted directed graph is O. P. but does a path exist from u to v consisting of at most k edges is D. P.? Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NP: Nondeterministic Problems Class NP Problems which are verifiable in polynomial time. Whether there exists or not any polynomial time algorithm for solving such problems, we do not know. Can be solved by nondeterministic polynomial However if we are given a certificate of a solution, we could verify that certificate is correct in polynomial time P = NP? Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NP: Nondeterministic Problems Nondeterministic algorithm: break in two steps Nondeterministic step generate a candidate solution called a certificate Deterministic (verification) Step It takes certificate and an instance of problem as input, returns yes if certificate represents solution In NP problems, verification step is polynomial Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Example: Hamiltonian Cycle Given: a directed graph G = (V, E), determine a simple cycle that contains each vertex in V, where each vertex can only be visited once Certificate: Sequence: v1, v2, v3, …, vn Generating certificates Verification: 1) (vi, vi+1) E for i = 1, …, n-1 (vn, v1) E It takes polynomial time hamiltonian not hamiltonian Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Reduction in Polynomial Time Algorithm Given two problems A, B, we say that A is reducible to B in polynomial time (A p B) if There exists a function f that converts the input of A to inputs of B in polynomial time A(x) = YES B(f(x)) = YES where x is input for A and f(x) is input for B Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Reduction in Polynomial Time Algorithm Solving a decision problem A in polynomial time Use a polynomial time reduction algorithm to transform A into B Run a known polynomial time algorithm for B Use the answer for B as the answer for A Polynomial time algorithm to decide A f Polynomial time algorithm to decide B yes no Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NP Complete A problem A is NP-complete if 1) A NP 2) B p A for all B NP If A satisfies only property No. 2 then B is NP-hard No polynomial time algorithm has been discovered for an NP-Complete problem No one has ever proven that no polynomial time algorithm can exist for any NP-Complete problem Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Reduction and NP Completeness Let A and B are two problems, and also suppose that we are given No polynomial time algorithm exists for problem A If we have a polynomial reduction f from A to B Then no polynomial time algorithm exists for B f Problem B yes no Problem A Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Relation in Between P, NP, NPC P NP (Researchers Believe) NPC NP (Researchers Believe) P = NP (or P NP, or P NP) ??? NPC = NP (or NPC NP, or NPC NP) ??? P NP One of the deepest, most perplexing open research problems in theoretical computer science since 1971 NPC P NP Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Problem Definitions: Circuit Satisfiability Boolean Combinational Circuit Boolean combinational elements wired together Each element takes a set of inputs and produces a set of outputs in constant number, assume binary Limit the number of outputs to 1 Logic gates: NOT, AND, OR Satisfying assignment: a true assignment causing the output to be 1. A circuit is satisfiable if it has a satisfying assignment. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Two Instances: Satisfiable and Un-satisfiable Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Problem: Circuit Satisfiability Statement Given a boolean combinational circuit composed of AND, OR, and NOT, is it stisfiable? Intuitive solution For each possible assignment, check whether it generates 1. Suppose the number of inputs is k, then the total possible assignments are 2k. So the running time is (2k). When the size of the problem is (k), then the running time is not polynomial Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Lemma 2: CIRCUIT-SAT is NP Hard Proof: Suppose X is any problem in NP Construct polynomial time algorithm F that maps every instance x in X to a circuit C = f(x) such that x is YES C CIRCUIT-SAT (is satisfiable). Since X NP, there is a polynomial time algorithm A which verifies X. Suppose the input length is n and Let T(n) denote the worst-case running time. Let k be the constant such that T(n) = O(nk) and the length of the certificate is O(nk). Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Circuit Satisfiability Problem is NP-complete Represent computation of A as a sequence of configurations, c0, c1,…,ci,ci+1,…,cT(n), each ci can be broken into various components ci is mapped to ci+1 by the combinational circuit M implementing the computer hardware. It is to be noted that A(x, y) = 1 or 0. Paste together all T(n) copies of the circuit M. Call this as, F, the resultant algorithm Please see the overall structure in the next slide Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Circuit Satisfiability Problem is NP-complete Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Circuit Satisfiability Problem is NP-complete Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Circuit Satisfiability Problem is NP-complete Now it can be proved that: F correctly constructs reduction, i.e., C is satisfiable if and only if there exists a certificate y, such that A(x, y) = 1. F runs in polynomial time (left as an assignment) Construction of C takes O(nk) steps, a step takes polynomial time F takes polynomial time to construct C from x. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NP-completeness Proof Basis Lemma 3 If X is problem such that P‘ p X for some P'NPC, then X is NP-hard. Moreover, X NP X NPC. Proof: Since P’ is NPC hence for all P” in NP, we have P” p P’ (1) And P‘ p X given (2) By (1) and (2) P” p P’ p X P” p X hence X is NP-hard Now if X NP X NPC Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Formula Satisfiability: Notations and Definitions SAT Definition n boolean variables: x1, x2,…, xn. m boolean connectives: ,,,,, and Parentheses. A SAT is satisfiable if there exists a true assignment which causes to evaluate to 1. In Formal Language SAT={< >: is a satifiable boolean formula}. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
SAT is NP Complete Theorem: SAT is NP-complete. Proof: SAT belongs to NP. Given a satisfying assignment Verifying algorithm replaces each variable with its value, and evaluates formula in polynomial time. SAT is NP-hard Sufficient to show that CIRCUIT-SATp SAT Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
SAT is NP Complete CIRCUIT-SATp SAT, i.e., any instance of circuit satisfiability can be reduced in polynomial time to an instance of formula satisfiability. Intuitive induction: Look at the gate that produces the circuit output. Inductively express each of gate’s inputs as formulas. Formula for circuit is obtained by writing an expression that applies gate’s function to its input formulas. Unfortunately, this is not a polynomial time reduction This is because the gate whose output is fed to 2 or more inputs of other gates, cause size to grow exponentially. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Example of Reduction of CIRCUIT-SAT to SAT = x10(x10(x7 x8 x9)) (x9(x6 x7)) (x8(x5 x6)) (x7(x1 x2 x4)) (x6 x4)) (x5(x1 x2)) (x4x3) INCORRECT REDUCTION: = x10= x7 x8 x9=(x1 x2 x4) (x5 x6) (x6 x7) =(x1 x2 x4) ((x1 x2) x4) (x4 (x1 x2 x4))=…. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NPC Proof: 3 CNF Satisfiability Definitions: A literal in a boolean formula is an occurrence of a variable or its negation. Clause, OR of one or more literals. CNF (Conjunctive Nornal Form) is a boolean formula expressed as AND of clauses. 3-CNF is a CNF in which each clause has exactly 3 distinct literals. (a literal and its negation are distinct) 3-CNF-SAT: whether a given 3-CNF is satiafiable? Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
3-CNF-SAT is NP Complete Proof: 3-CNF-SAT NP 3-CNF-SAT is NP-hard. SAT p3-CNF-SAT? Suppose is any boolean formula, Construct a binary ‘parse’ tree, with literals as leaves and connectives as internal nodes. Introduce yi for output of each internal node. Rewrite formula to ‘: AND of root and conjunction of clauses describing operation of each node. In ', each clause has at most three literals. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
3-CNF-SAT is NP Complete Change each clause into conjunctive normal form as: Construct a true table, (at most 8 by 4) Write disjunctive normal form for items evaluating 0 Using DeMorgan law to change to CNF. Result: '' in CNF but each clause has 3 or less literals. Change 1 or 2-literal clause into 3-literal clause as: Two literals: (l1 l2), change it to (l1 l2 p) (l1 l2 p). If a clause has one literal l, change it to (lpq)(lpq) (lpq) (lpq). Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Binary parse tree for =((x1 x2) ((x1 x3) x4))x2 '= y1(y1 (y2x2)) (y2 (y3 y4)) (y4 y5) (y3 (x1 x2)) (y5 (y6 x4)) (y6 (x1 x3)) Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
Example of Converting a 3-literal clause to CNF format Disjunctive Normal Form: i'=(y1y2x2)(y1y2x2) (y1y2x2) (y1y2x2) Conjunctive Normal Form: i''=(y1y2x2)(y1y2x2) (y1y2x2)(y1y2x2) Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
CLIQUE: NPC Proof Definition: A clique in an undirected graph G = (V, E) is a subset V‘ V, each pair of V’ is connected by an edge in E, i.e., clique is a complete subgraph of G. Size of a clique is number of vertices in the clique. Optimization problem: Find maximum size clique. Decision problem: whether a clique of given size k exists in the graph? CLIQUE = {<G, k>: G is a graph with a clique of size k.} Intuitive solution: ??? Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
CLIQUE is NP Complete Theorem CLIQUE problem is NP-complete. Proof: CLIUEQE NP: given G = (V, E) and a set V‘ V as a certificate for G. The verifying algorithm checks for each pair of u, v V', whether <u, v> E. time: O(|V'|2|E|). CLIQUE is NP-hard: Show 3-CNF-SAT pCLUQUE. Surprise: from boolean formula to graph. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
CLIQUE is NP Complete Reduction from 3-CNF-SAT to CLUQUE. Suppose = C1 C2… Ck be a boolean formula in 3-CNF with k clauses. We construct a graph G = (V, E) as follows: For each clause Cr =(l1r l2r l3r), place triple of v1r, v2r, v3r into V Put edge between vertices vir and vjs when: r s, i.e. vir and vjs are in different triples, and corresponding literals are consistent, i.e, lir is not negation of ljs . Then is satisfiable G has a clique of size k. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
=(x1x2x3)(x1x2x3)(x1x2x3) and its reduced graph G C1=x1x2 x3 Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design
NP-completeness proof structure