Lecture 6: Computational Complexity 虞台文 大同大學資工所 智慧型多媒體研究室

Content Time-Bounded Turing Machine Rate of Growth Functions Time-Bound Simulations P and NP NP –Completeness Bounded Halting Problems Some Important NP-Complete Problems

Lecture 6: Computational Complexity Time-Bounded Turing Machine 大同大學資工所 智慧型多媒體研究室

Computational Effectiveness for Solvable Problems Not Solvable Solvable Can such a problem, e.g., TSP, be solved in any practical sense?

Yields in t Steps ├ ├ iff ├ iff such that ├ and ├

Time-Bounded TM Let T: N  N be a function. Let L  be a language, and let be a k-tape TM with 0  . We say that M decides L in time T if the following holds: w  L  w  L  ├ for some t  T(|w|); ├ for some t  T(|w|);

TIME(T) We say that L is decidable in time T if there is some k-tape TM (k > 0) that decides L in time T. The class of all languages decidable in time T is denoted by

Absolute Time Bound (2n + 4) # w Head Y/N # Head |w|+1 writing steps |w|+3 head-moving steps >L # RYR >L # RNR erase w (write ‘#’) write ‘Y’ or ‘N’

Lecture 6: Computational Complexity Rate of Growth Functions 大同大學資工所 智慧型多媒體研究室

Time-Bound Notations Upper Bound Lower Bound Exact Bound

Time-Bound Notations Upper Bound Lower Bound Exact Bound Fact:

Lemma

Lemma Pf) c For a, we need to prove and . Therefore, we have to find c (exercise) such that

Lemma Pf) Since f = O(nd), we only need to show that nd = O(nd+1), which is trivially satisfied because nd  nd+1 for all n  0. Suppose that nd+1 = O(nd), Then, nnd = nd+1  cnd for n  n0. But for any c, when n > c, nd+1  cnd. 

Theorem

Theorem Pf) Because f = O(nd), we only need to prove that nd = O(rn), i.e., k terms

Theorem Pf) Hence, nd = O(rn). Because f = O(nd), we only need to prove that nd = O(rn), i.e., Hence, nd = O(rn). Let

Lecture 6: Computational Complexity Time-Bound Simulations 大同大學資工所 智慧型多媒體研究室

Time-Complexities on TMs Not Solvable Solvable The complexities to solve problems on different TMs are different.

Theorem 1 where

Theorem 2 where

Conclusions A polynomial algorithm of any TM can be simulated using some polynomial algorithms of other TM’s. In the following, we consider two classes of problems: Polynomial algorithms (P) Exponential algorithms (NP) ?

Lecture 6: Computational Complexity P and NP 大同大學資工所 智慧型多媒體研究室

Definition (P ) That is, P denotes the class of all languages that are decided by TM’s in some polynomial time bounds.

Example: (Integer Programming) 0-1 Does have solution in 0 and 1? Matrix formulation: Given A and b, is there any binary vector x such that

Example: (Integer Programming) Given A and b, is there any binary vector x such that Example: (Integer Programming) A b x Encode: Language No answer!

Example: (Integer Programming) Given A and b, is there any binary vector x such that Example: (Integer Programming) Language No answer! Language Yes, it is. Fact:

Definition M accepts L in nondeterministic time T if the following holds ├ For all , for some u, v *,  , and t  T(|w|). We say that L is acceptable in nondeterministic time T if there is an NTM that accepts L in nondeterministic time T.

What is the difference between TIME(T) and NTIME(T)? Definition The class of languages acceptable in nondeterministic time T.

Definition (NP ) That is, NP denotes the class of all languages that are accepted by NTM’s in some nondeterministic polynomial time bounds.

Definition (Step Counting Function) f : NN is a step-counting function if  a k-tape Turing machine M such that for all w*, M halts in exactly f(|w|) steps on input w, i.e., ├ for some u1, …, uk, v1, …, vk * and a1, …, ak.

Exercise Given d >0, there exists a polynomial p of degree d being a step counting function.

Theorem 3 See text for the proof. Let L be a language accepted in time T1 by a NTM M1=(K1, 1, 1, s1) where T1 is a step counting function.  A k2-tape DTM M2=(K2, 2, 2, s2) decides L in time T2 where

Polynomial Balanced Languages Let L  *$* be a language, where $. Then, L is said to be polynomial balanced if there exists a polynomial p such that

Theorem 4 Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a polynomial balanced language.

Theorem 4 M: >$(1) M’: M’’: >RNR >RNR M’’ Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “” L’ is a p.b.l.  Polynomial p s.t. x$yL’ only if |y|  p(|x|). L’P k–tape DTM M’ that decides L’ in time q (a polynomial). To prove L  NP, we construct an NTM M based on M’ to accept L as follows: M: >$(1) . . . M’’ M’: >RNR M’’: >RNR

Theorem 4 M: >$(1) M’: M’’: >RNR >RNR M’’ Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 M can be accept L in nondeterministic time p + q. Hence, L  NP. Pf) “” L’ is a p.b.l.  Polynomial p s.t. x$yL’ only if |y|  p(|x|). L’P k–tape DTM M’ that decides L’ in time q (a polynomial). To prove L  NP, we construct an NTM M based on M’ to accept L as follows: M: >$(1) . . . M’’ M’: >RNR M’’: >RNR

Theorem 4 Y(x)=? M’ =? Pf) “” We shall show that Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “” We shall show that L’ = {x$Y(x) : x  L, |Y(x)|  p(|x|)}, where p is a polynomial, such that L’P, i.e.,  a DTM M’ can decides L’ in polynomial time. Y(x)=? M’ =?

Theorem 4 . . . Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “” Since LNP,  an NTM M that accept L in nondeterministic time q (a polynomial). Let  be the encoding function that encodes TM’s and their memory configurations in {I, c}*. That is, given xL, we have such that ├M . . . Let Y(x) be the set of all strings where M is the NTM that accepts L.

Theorem 4 Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “”  q(|x|) constant to be discussed accept in time q input length

Theorem 4 Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “”  q(|x|)

Theorem 4 Polynomial of |x| Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “” Polynomial of |x|

Theorem 4 Let L’ is polynomial balanced. Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “” Let L’ is polynomial balanced.

Theorem 4 ? Let L’ is polynomial balanced. Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “” ? Let L’ is polynomial balanced.

Theorem 4 Pf) “” Let L  * be a language, $ and ||2. L  NP iff  L’  *$* such that where L’ is a p.b.l. Theorem 4 Pf) “” We now show that L’P by designing a DTM M’ which decides L’ as follows: Check the syntax of the input string To verify the following things: Is Cj a configuration of M? C0 = (s, #x#)? Ct = (h, uv)? If rejected, print out ‘N’ then halt. This steps requires O(n) time. Check Cj├M Cj+1, j= 0, …, t1 by consulting (M). If rejected print out ‘N’ and then halt. This steps requires O(n2). Therefore, L’P.

Example (Integer Programming BNP )

Example (Integer Programming BNP ) No answer

Is P = NP ? P NP P  NP NP  P How to show? Show and Trivial (why?) Difficult No answer, now. P  NP Show NP  P and NP P

Example (TSPNP ) 1 2 5 3 4 6 7 8 TSPF Find the shortest tour t.

Example (TSPNP ) t is a bijection TSPF Find the shortest tour t. E.g., Example (TSPNP ) TSPF 1 2 5 3 4 6 7 8 Find the shortest tour t. It is not language decision problem. Rather, it is a problem of function evaluation.

Example (TSPNP ) TSPF Find the shortest tour t. TSP Denote the cost of t as D(t). TSP’ Polynomial balanced

Exercises Show that TSPNP. TSPP   a polynomial algorithm for TSPF.

Remark TSPP P = NP

Lecture 6: Computational Complexity NP-Completeness 大同大學資工所 智慧型多媒體研究室

Definition: Polynomial-Time Computable is said to be computed in time T if  a k-tape DTM s.t. ├ with t  T(|x|). Furthermore, if T(|x|) is a polynomial, then f is said to be polynomial-time computable.

Definition: Polynomial-Time Reduction Let and be languages, be polynomial-time computable. Then L1 is said to be polynomial-time reducible to L2 iff

More on Polynomial Reduction L2  P  L1  P. Transitivity Time to decide p.r.

Definition: NP-Completeness is called NP-complete iff

More on NP-Completeness p.r. p.r. P NP-Complete

NP-Hard Not Solvable Solvable NP p.r. p.r. P NP-Complete NP-Hard

Important Theorem Let L be an NP-complete language. Then, Pf) “” “” trivial “” trivial

Remark If one can decide any NP-complete language in polynomial time, then all languages in NP can be decided effectively. Surely, the guy will win Turing Award.

Lecture 6: Computational Complexity Bounded Halting Problems 大同大學資工所 智慧型多媒體研究室

Bounded Halting Problem N0 Fact: Halting problem is not solvable. Is Bounded Halting problem is solvable?

Theorem: N0 is NP-Complete

Theorem: N0 is NP-Complete Define

Theorem: N0 is NP-Complete Define Exercise

Theorem: N0 is NP-Complete For any LNP, find L. LNP Let Fact: L(w) is polynomial computable. w  L  L(w)  N0.

The Language

Theorem: is NP-Complete Exercise Find .

Theorem: is NP-Complete Find

Theorem: is NP-Complete Find Case 1. # t + 2  2|u| + 2 + t = 2t + 4  2|u| =2|u’|

Theorem: is NP-Complete Find Halt within 2|u’| steps. Halt within t steps. Case 1. # t + 2  2|u| + 2 + t = 2t + 4  2|u| =2|u’|

Theorem: is NP-Complete Find Case 2. t +2|u| t =2|u| =2|u’|

Theorem: is NP-Complete Find Halt within 2|u’| steps. Halt within t steps. Case 2. t +2|u| t =2|u| =2|u’|

Theorem: is NP-Complete Find Hence, See previous slides

Theorem: is NP-Complete Summary of the proof: Known NP-complete

Special Bounded Halting Problem N2 ├

Theorem: N2 is NP-Complete ├ Pf) obvious. p.r. p.r. p.r. ? ? ?

Theorem: N2 is NP-Complete ├ Pf) obvious. p.r.  = ?

Theorem: N2 is NP-Complete  :

Theorem: N2 is NP-Complete  : 2|u| Let 1 t

Theorem: N2 is NP-Complete  : M ’ halts on  in t ’ steps M halts on u in t steps 2|u| Let 1 t

Theorem: N2 is NP-Complete  :

Summary p.r.

Lecture 6: Computational Complexity Some Important NP-Complete Problems 大同大學資工所 智慧型多媒體研究室

Bounded Tiling Problem (BT) Tiling System: D: a set of finite number of tiles. d0D: the tile place at the origin. HDD: the Horizontal tiling rule. VDD: the Vertical tiling rule.

Bounded Tiling Problem (BT) Given D and s > 0, an s  s tiling by D is a function 1 2 3 4 5 such that

Bounded Tiling Problem (BT) BT is NP-Complete

Theorem: BT is NP-Complete Pf) a p.b.l. p.r.

Theorem: BT is NP-Complete  Theorem: BT is NP-Complete  :

Theorem: BT is NP-Complete 1 t t+1

Theorem: BT is NP-Complete 1 t t+1 1) For each a, create tiles (a, k) (a, k+1)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 (a, 6) (a, 7) 1) For each a, create tiles (a, 5) (a, 6) (a, 4) (a, 5) (a, k) (a, k+1) (a, 3) (a, 4) (a, 2) (a, 3) (a, 1) (a, 2) (a, 0) (a, 1)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 Can’t reach the highest layer if q  h. 2) For each (q, a, p, b)  where b and q  h, create tiles (q, a, 5) (p, b, 6) (q, a, 4) (p, b, 5) (q, a, k) (p, b, k+1) (q, a, 3) (p, b, 4) (q, a, 2) (p, b, 3) (q, a, 1) (p, b, 2) (q, a, 0) (p, b, 1)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 3) For each (q, a, p, L) , create tiles (b, k) (p, b, k+1) p (q, a, k) (a, k+1) p (b, 2) (p, b, 3) (q, a, 2) (a, 3) (?, 0) (?, 1) (?, 2) (b, 0) (b, 1) (b, 2) (a, 0) (a, 1) (q, a, 2)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 (?, 6) (?, 7) (a, 6) (a, 7) 3) For each (q, a, p, L) , create tiles (?, 5) (?, 6) (a, 5) (a, 6) (?, 4) (?, 5) (a, 4) (a, 5) (b, k) (p, b, k+1) p (q, a, k) (a, k+1) p (?, 3) (?, 4) (a, 3) (a, 4) (?, 2) (?, 3) (b, 2) (p, b, 3) (q, a, 2) (a, 3) (?, 0) (?, 1) (?, 2) (b, 0) (b, 1) (b, 2) (a, 0) (a, 1) (q, a, 2)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 4) For each (q, a, p, R) , create tiles (q, a, k) (a, k+1) p (b, k) (p, b, k+1) p (q, a, 2) (a, 3) (b, 2) (p, b, 3) (?, 0) (?, 1) (?, 2) (a, 0) (a, 1) (q, a, 2) (b, 0) (b, 1) (b, 2)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 (?, 6) (?, 7) (a, 6) (a, 7) 4) For each (q, a, p, R) , create tiles (?, 5) (?, 6) (a, 5) (a, 6) (?, 4) (?, 5) (a, 4) (a, 5) (q, a, k) (a, k+1) p (b, k) (p, b, k+1) p (?, 3) (?, 4) (a, 3) (a, 4) (?, 2) (?, 3) (q, a, 2) (a, 3) (b, 2) (p, b, 3) (?, 0) (?, 1) (?, 2) (a, 0) (a, 1) (q, a, 2) (b, 0) (b, 1) (b, 2)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 (?, 6) (?, 7) (a, 6) (a, 7) 5) For each a, create tiles (?, 5) (?, 6) (a, 5) (a, 6) (?, 4) (?, 5) (a, 4) (a, 5) (h, a, k) (h, a, k+1) (?, 3) (?, 4) (a, 3) (a, 4) (h, a, k+1) (?, 2) (?, 3) (q, a, 2) (a, 3) (b, 2) (h, b, 3) (?, 0) (?, 1) (?, 2) (a, 0) (a, 1) (q, a, 2) (b, 0) (b, 1) (b, 2)

Theorem: BT is NP-Complete 1 6 7 2 3 4 5 (s, #, 0) d0 = # 6) (#, 0) # 7) Base tile (s, #, 0) # (#, 0) # (#, 0) # (#, 0) # (#, 0) # (#, 0) # (#, 0) # (#, 0) #

Theorem: BT is NP-Complete 1) For each a, create tiles 4) For each (q, a, p, R) , create tiles (a, k) (a, k+1) (q, a, k) (a, k+1) p (b, k) (p, b, k+1) p 2) For each (q, a, p, b)  where b and q  h, create tiles 5) For each a, create tiles (h, a, k) (h, a, k+1) (q, a, k) (p, b, k+1) 6) 7) Base tile 3) For each (q, a, p, L) , create tiles (s, #, 0) d0 = # (#, 0) # (b, k) (p, b, k+1) p (q, a, k) (a, k+1) p

Theorem: BT is NP-Complete Pf) a p.b.l. p.r.

Integer Programming already known Is B NP-complete?

Theorem: B is NP-Complete p.r.

Theorem: B is NP-Complete  : p.r.

Theorem: B is NP-Complete  : Define variables where

Theorem: B is NP-Complete Every place has exactly one tile Horizontal rules Vertical rules Place d0 at (0, 0)

Theorem: B is NP-Complete Every place has exactly one tile Horizontal rules Vertical rules Place d0 at (0, 0)

Theorem: B is NP-Complete Introducing slack variables Theorem: B is NP-Complete Every place has exactly one tile Horizontal rules Vertical rules Place d0 at (0, 0)

Theorem: B is NP-Complete Introducing slag variables Theorem: B is NP-Complete Every place has exactly one tile Horizontal rules Vertical rules Place d0 at (0, 0)

Theorem: B is NP-Complete Introducing slag variables The coefficient of each variable is 0 or 1. Theorem: B is NP-Complete The right-hand side of each equality is 1. Every place has exactly one tile Horizontal rules Vertical rules Place d0 at (0, 0)

Language Br is NP-Complete

Hamilton Cycle

Theorem: H is NP-Complete Pf) p.r.

Theorem: H is NP-Complete x y z a b c d

Theorem: H is NP-Complete x y z a b c d A Hamilton cycle entering the box from a, it must leave the box from c. A Hamilton cycle entering the box from d, it must leave the box from b.

Theorem: H is NP-Complete x y z a b c d

Theorem: H is NP-Complete x y z a b c d a b c d

Theorem: H is NP-Complete a b c d

Theorem: H is NP-Complete  :

Theorem: H is NP-Complete

Theorem: H is NP-Complete

Theorem: H is NP-Complete

Theorem: H is NP-Complete  :

Traveling Salesman Problem (TSP) already known Is TSP NP-complete?

Theorem: TSP is NP-Complete p.r. p.r.

Theorem: TSP is NP-Complete  : p.r. p.r.

Theorem: TSP is NP-Complete  :

Theorem: TSP is NP-Complete G = (V, E)  :

Theorem: TSP is NP-Complete G = (V, E)  : There exists a Hamilton cycle in graph G=(V, E) if and only if there exists a zero-cost tour for the following TSP .

Satisfiability Problem (SAT) Example: Satisfiability Problem (SAT) : Boolean Variables; literals : the negation of xi; A formula in the propositional calculus is an expression that can be constructed using literals and the operations  (and) and  (or). A satisfiability problem is to determine if a formula is true for any truth assignment of variables.

Normal Forms CNF DNF

Cook’s Theorem SAT is NP-complete SATP iff P =NP

SAT is NP-complete Cook’s Theorem Pf) skip p.r. p.r.

Cook’s Theorem Define literals:

Cook’s Theorem

True if and only if there is no different tiles being placed at each cell. Cook’s Theorem

True if and only if horizontal rules are satisfied at each cell. Cook’s Theorem True if and only if horizontal rules are satisfied at each cell.

True if and only if vertical rules are satisfied at each cell. Cook’s Theorem True if and only if vertical rules are satisfied at each cell.

True if an only if d0 is place at (0, 0). Cook’s Theorem True if an only if d0 is place at (0, 0).

Cook’s Theorem

Summary p.r. p.r.