1. Advanced Algorithms Analysis and DesignBy
Dr. Nazir Ahmad Zafar
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

2. Lecture No. 44 NP Completeness
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

3. NP Completeness Polynomial Time Algorithms
On any inputs of size n, if worst-case running time of algorithm is O(nk), for constant k, then it is called polynomial time algorithm
Every problem can not be solved in polynomial time
There exists some problems which can not be solved by any computer, in any amount of time, e.g. Turing’s Halting Problem
Such problems are called un-decidable
Some problems can be solved but not in O(nk)
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

4. NP Completeness Polynomial Time (Class P)
These problems are solvable in polynomial time
Problems in class P are also called tractable
Intractable Problems
Problems not in P are called intractable
These problems can be solved in reasonable amount of time only for small input size
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

5. Decision & Optimization Problems
Decision problems
The problems which return yes or no for a given input and a question regarding the same problem
Optimization problems
Find a solution with best value, it can be maximum or minimum. There can be more than one solutions for it
Optimization problems can be considered as decision problems which are easier to study.
Finding a path between u and v using fewest edges in an un-weighted directed graph is O. P. but does a path exist from u to v consisting of at most k edges is D. P.?
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

6. NP: Nondeterministic Problems
Class NP
Problems which are verifiable in polynomial time.
Whether there exists or not any polynomial time algorithm for solving such problems, we do not know.
Can be solved by nondeterministic polynomial
However if we are given a certificate of a solution, we could verify that certificate is correct in polynomial time
P = NP?
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

7. NP: Nondeterministic Problems
Nondeterministic algorithm: break in two steps
Nondeterministic step
generate a candidate solution called a certificate
Deterministic (verification) Step
It takes certificate and an instance of problem as input, returns yes if certificate represents solution
In NP problems, verification step is polynomial
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

8. Example: Hamiltonian Cycle
Given: a directed graph G = (V, E), determine a simple cycle that contains each vertex in V, where each vertex can only be visited once
Certificate:
Sequence: v1, v2, v3, …, vn
Generating certificates
Verification:
1) (vi, vi+1)  E for i = 1, …, n-1
(vn, v1)  E
It takes polynomial time
hamiltonian
not hamiltonian
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

9. Reduction in Polynomial Time Algorithm
Given two problems A, B, we say that A is reducible to B in polynomial time (A p B) if
There exists a function f that converts the input of A to inputs of B in polynomial time
A(x) = YES  B(f(x)) = YES
where x is input for A and f(x) is input for B
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

10. Reduction in Polynomial Time Algorithm
Solving a decision problem A in polynomial time
Use a polynomial time reduction algorithm to transform A into B
Run a known polynomial time algorithm for B
Use the answer for B as the answer for A
Polynomial time algorithm to decide A f
Polynomial time
algorithm to decide B  
yes no
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

11. NP Complete A problem A is NP-complete if 1) A  NP
2) B p A for all B  NP
If A satisfies only property No. 2 then B is NP-hard
No polynomial time algorithm has been discovered for an NP-Complete problem
No one has ever proven that no polynomial time algorithm can exist for any NP-Complete problem
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

12. Reduction and NP Completeness
Let A and B are two problems, and also suppose that we are given
No polynomial time algorithm exists for problem A
If we have a polynomial reduction f from A to B
Then no polynomial time algorithm exists for B f
Problem B  
yes no
Problem A
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

13. Relation in Between P, NP, NPC
P  NP (Researchers Believe)
NPC  NP (Researchers Believe)
P = NP (or P  NP, or P  NP) ???
NPC = NP (or NPC  NP, or NPC  NP) ???
P  NP
One of the deepest, most perplexing open research problems in theoretical computer science since 1971
NPC P NP
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

14. Problem Definitions: Circuit Satisfiability
Boolean Combinational Circuit
Boolean combinational elements wired together
Each element takes a set of inputs and produces a set of outputs in constant number, assume binary
Limit the number of outputs to 1
Logic gates: NOT, AND, OR
Satisfying assignment: a true assignment causing the output to be 1.
A circuit is satisfiable if it has a satisfying assignment.
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

15. Two Instances: Satisfiable and Un-satisfiable
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

16. Problem: Circuit Satisfiability
Statement
Given a boolean combinational circuit composed of AND, OR, and NOT, is it stisfiable?
Intuitive solution
For each possible assignment, check whether it generates 1.
Suppose the number of inputs is k, then the total possible assignments are 2k.
So the running time is (2k).
When the size of the problem is (k), then the running time is not polynomial
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

17. Lemma 2: CIRCUIT-SAT is NP Hard
Proof:
Suppose X is any problem in NP
Construct polynomial time algorithm F that maps every instance x in X to a circuit C = f(x) such that x is YES  C  CIRCUIT-SAT (is satisfiable).
Since X  NP, there is a polynomial time algorithm A which verifies X.
Suppose the input length is n and Let T(n) denote the worst-case running time.
Let k be the constant such that T(n) = O(nk) and the length of the certificate is O(nk).
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

18. Circuit Satisfiability Problem is NP-complete
Represent computation of A as a sequence of configurations, c0, c1,…,ci,ci+1,…,cT(n), each ci can be broken into various components
ci is mapped to ci+1 by the combinational circuit M implementing the computer hardware.
It is to be noted that A(x, y) = 1 or 0.
Paste together all T(n) copies of the circuit M. Call this as, F, the resultant algorithm
Please see the overall structure in the next slide
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

19. Circuit Satisfiability Problem is NP-complete
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Circuit Satisfiability Problem is NP-complete
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

20. Circuit Satisfiability Problem is NP-complete
Now it can be proved that:
F correctly constructs reduction, i.e., C is satisfiable if and only if there exists a certificate y, such that A(x, y) = 1.
F runs in polynomial time
(left as an assignment)
Construction of C takes O(nk) steps, a step takes polynomial time
F takes polynomial time to construct C from x.
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

21. NP-completeness Proof Basis
Lemma 3
If X is problem such that P‘ p X for some P'NPC, then X is NP-hard. Moreover, X  NP X NPC.
Proof:
Since P’ is NPC hence for all P” in NP, we have
P” p P’ (1)
And P‘ p X given (2)
By (1) and (2)
P” p P’ p X  P” p X hence X is NP-hard
Now if X  NP X NPC
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

22. Formula Satisfiability: Notations and Definitions
SAT Definition
n boolean variables: x1, x2,…, xn.
m boolean connectives: ,,,,, and
Parentheses.
A SAT  is satisfiable if there exists a true assignment which causes  to evaluate to 1.
In Formal Language
SAT={< >:  is a satifiable boolean formula}.
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

23. SAT is NP Complete Theorem: SAT is NP-complete. Proof:
SAT belongs to NP.
Given a satisfying assignment
Verifying algorithm replaces each variable with its value, and evaluates formula in polynomial time.
SAT is NP-hard
Sufficient to show that CIRCUIT-SATp SAT
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

24. SAT is NP Complete
CIRCUIT-SATp SAT, i.e., any instance of circuit satisfiability can be reduced in polynomial time to an instance of formula satisfiability.
Intuitive induction:
Look at the gate that produces the circuit output.
Inductively express each of gate’s inputs as formulas.
Formula for circuit is obtained by writing an expression that applies gate’s function to its input formulas.
Unfortunately, this is not a polynomial time reduction
This is because the gate whose output is fed to 2 or more inputs
of other gates, cause size to grow exponentially.
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

25. Example of Reduction of CIRCUIT-SAT to SAT
= x10(x10(x7 x8 x9))
(x9(x6  x7))
(x8(x5  x6))
(x7(x1 x2 x4))
(x6 x4))
(x5(x1  x2))
(x4x3)
INCORRECT REDUCTION: = x10= x7 x8 x9=(x1 x2 x4)  (x5  x6) (x6  x7)
=(x1 x2 x4)  ((x1  x2)  x4) (x4  (x1 x2 x4))=….
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

26. NPC Proof: 3 CNF Satisfiability
Definitions:
A literal in a boolean formula is an occurrence of a variable or its negation.
Clause, OR of one or more literals.
CNF (Conjunctive Nornal Form) is a boolean formula expressed as AND of clauses.
3-CNF is a CNF in which each clause has exactly 3 distinct literals.
(a literal and its negation are distinct)
3-CNF-SAT: whether a given 3-CNF is satiafiable?
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

27. 3-CNF-SAT is NP Complete
Proof:
3-CNF-SAT  NP
3-CNF-SAT is NP-hard.
SAT p3-CNF-SAT?
Suppose  is any boolean formula, Construct a binary ‘parse’ tree, with literals as leaves and connectives as internal nodes.
Introduce yi for output of each internal node.
Rewrite formula to ‘: AND of root and conjunction of clauses describing operation of each node.
In ', each clause has at most three literals.
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

28. 3-CNF-SAT is NP Complete
Change each clause into conjunctive normal form as:
Construct a true table, (at most 8 by 4)
Write disjunctive normal form for items evaluating 0
Using DeMorgan law to change to CNF.
Result: '' in CNF but each clause has 3 or less literals.
Change 1 or 2-literal clause into 3-literal clause as:
Two literals:
(l1 l2), change it to (l1 l2 p)  (l1 l2 p).
If a clause has one literal l, change it to (lpq)(lpq) (lpq) (lpq).
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

29. Binary parse tree for =((x1 x2) ((x1 x3)  x4))x2
'= y1(y1  (y2x2))
(y2  (y3  y4))
(y4  y5)
(y3  (x1  x2))
(y5  (y6  x4))
(y6  (x1  x3))
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

30. Example of Converting a 3-literal clause to CNF format
Disjunctive Normal Form:
i'=(y1y2x2)(y1y2x2)
(y1y2x2) (y1y2x2)
Conjunctive Normal Form:
i''=(y1y2x2)(y1y2x2)
(y1y2x2)(y1y2x2)
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

31. CLIQUE: NPC Proof Definition:
A clique in an undirected graph G = (V, E) is a subset V‘  V, each pair of V’ is connected by an edge in E, i.e., clique is a complete subgraph of G.
Size of a clique is number of vertices in the clique.
Optimization problem: Find maximum size clique.
Decision problem: whether a clique of given size k exists in the graph?
CLIQUE = {<G, k>: G is a graph with a clique of size k.}
Intuitive solution: ???
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

32. CLIQUE is NP Complete Theorem CLIQUE problem is NP-complete. Proof:
CLIUEQE NP: given G = (V, E) and a set V‘  V as a certificate for G. The verifying algorithm checks for each pair of u, v  V', whether <u, v>  E. time: O(|V'|2|E|).
CLIQUE is NP-hard:
Show 3-CNF-SAT pCLUQUE.
Surprise: from boolean formula to graph.
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

33. CLIQUE is NP Complete Reduction from 3-CNF-SAT to CLUQUE.
Suppose  = C1 C2… Ck be a boolean formula in 3-CNF with k clauses.
We construct a graph G = (V, E) as follows:
For each clause Cr =(l1r l2r l3r), place triple of v1r, v2r, v3r into V
Put edge between vertices vir and vjs when:
r  s, i.e. vir and vjs are in different triples, and
corresponding literals are consistent, i.e, lir is not negation of ljs .
Then  is satisfiable  G has a clique of size k.
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

34. =(x1x2x3)(x1x2x3)(x1x2x3) and its reduced graph G
C1=x1x2 x3
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

35. NP-completeness proof structure