1. Lecture 6: Computational Complexity
虞台文
大同大學資工所
智慧型多媒體研究室

2. Content Time-Bounded Turing Machine Rate of Growth Functions
Time-Bound Simulations
P and NP
NP –Completeness
Bounded Halting Problems
Some Important NP-Complete Problems

3. Lecture 6: Computational Complexity
Time-Bounded Turing Machine
大同大學資工所
智慧型多媒體研究室

4. Computational Effectiveness for Solvable Problems
Not Solvable
Solvable
Can such a problem, e.g., TSP,
be solved in any practical sense?

5. Yields in t Steps ├ ├
iff ├
iff
such that ├
and ├

6. Time-Bounded TM
Let T: N  N be a function. Let L  be a language, and let be a k-tape TM with 0  . We say that M decides L in time T if the following holds:
w  L 
w  L  ├
for some t  T(|w|); ├
for some t  T(|w|);

7. TIME(T)
We say that L is decidable in time T if there is some k-tape TM (k > 0) that decides L in time T.
The class of all languages decidable in time T is denoted by

8. Absolute Time Bound (2n + 4)# w
Head
Y/N #
Head
|w|+1 writing steps
|w|+3 head-moving steps
>L #
RYR
>L #
RNR
erase w (write ‘#’)
write ‘Y’ or ‘N’

9. Lecture 6: Computational Complexity
Rate of
Growth Functions
大同大學資工所
智慧型多媒體研究室

10. Time-Bound Notations
Upper Bound
Lower Bound
Exact Bound

11. Time-Bound Notations
Upper Bound
Lower Bound
Exact Bound
Fact:

12. Lemma

13. Lemma Pf) c For a, we need to prove and .
Therefore, we have to find c (exercise) such that

14. Lemma
Pf)
Since f = O(nd), we only need to show that nd = O(nd+1), which is trivially satisfied because nd  nd+1 for all n  0.
Suppose that nd+1 = O(nd), Then, nnd = nd+1  cnd for n  n0.
But for any c, when n > c, nd+1  cnd. 

15. Theorem

16. Theorem
Pf)
Because f = O(nd), we only need to prove that nd = O(rn), i.e.,
k terms

17. Theorem Pf) Hence, nd = O(rn).
Because f = O(nd), we only need to prove that nd = O(rn), i.e.,
Hence, nd = O(rn).
Let

18. Lecture 6: Computational Complexity
Time-Bound Simulations
大同大學資工所
智慧型多媒體研究室

19. Time-Complexities on TMs
Not Solvable
Solvable
The complexities to solve problems
on different TMs are different.

20. Theorem 1
where

21. Theorem 2
where

22. Conclusions
A polynomial algorithm of any TM can be simulated using some polynomial algorithms of other TM’s.
In the following, we consider two classes of problems:
Polynomial algorithms (P)
Exponential algorithms (NP) ?

23. Lecture 6: Computational Complexity
P and NP
大同大學資工所
智慧型多媒體研究室

24. Definition (P )
That is, P denotes the class of all languages that are decided by TM’s in some polynomial time bounds.

25. Example: (Integer Programming)
0-1
Does
have solution in 0 and 1?
Matrix formulation:
Given A and b, is there any binary vector x such that

26. Example: (Integer Programming)
Given A and b, is there any binary vector x such that
Example: (Integer Programming) A b x
Encode:
Language
No answer!

27. Example: (Integer Programming)
Given A and b, is there any binary vector x such that
Example: (Integer Programming)
Language
No answer!
Language
Yes, it is.
Fact:

28. Definition M accepts L in nondeterministic
time T if the following holds ├
For all ,
for some u, v *,  , and t  T(|w|).
We say that L is acceptable in nondeterministic time T if there is an NTM that accepts L in nondeterministic time T.

29. What is the difference between TIME(T) and NTIME(T)?
Definition
The class of languages acceptable in nondeterministic time T.

30. Definition (NP )
That is, NP denotes the class of all languages that are accepted by NTM’s in some nondeterministic polynomial time bounds.

31. Definition (Step Counting Function)
f : NN is a step-counting function if  a k-tape Turing machine M such that for all w*, M halts in exactly f(|w|) steps on input w, i.e., ├
for some u1, …, uk, v1, …, vk * and a1, …, ak.

32. Exercise
Given d >0, there exists a polynomial p of degree d being a step counting function.

33. Theorem 3 See text for the proof.
Let L be a language accepted in time T1 by a NTM M1=(K1, 1, 1, s1) where T1 is a step counting function.
 A k2-tape DTM M2=(K2, 2, 2, s2) decides L in time T2 where

34. Polynomial Balanced Languages
Let L  *$* be a language, where $. Then, L is said to be polynomial balanced if there exists a polynomial p such that

35. Theorem 4 Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a polynomial balanced language.

36. Theorem 4 M: >$(1) M’: M’’: >RNR >RNR M’’ Pf) “”
Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
L’ is a p.b.l.
 Polynomial p s.t. x$yL’ only if |y|  p(|x|).
L’P
k–tape DTM M’ that decides L’ in time q (a polynomial).
To prove L  NP, we construct an NTM M based on M’ to accept L as follows: M:
>$(1)
. . .
M’’
M’:
>RNR
M’’:
>RNR

37. Theorem 4 M: >$(1) M’: M’’: >RNR >RNR M’’
Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
M can be accept L in nondeterministic time p + q.
Hence, L  NP.
Pf)
“”
L’ is a p.b.l.
 Polynomial p s.t. x$yL’ only if |y|  p(|x|).
L’P
k–tape DTM M’ that decides L’ in time q (a polynomial).
To prove L  NP, we construct an NTM M based on M’ to accept L as follows: M:
>$(1)
. . .
M’’
M’:
>RNR
M’’:
>RNR

38. Theorem 4 Y(x)=? M’ =? Pf) “” We shall show that
Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
We shall show that
L’ = {x$Y(x) : x  L, |Y(x)|  p(|x|)},
where p is a polynomial, such that L’P, i.e.,
 a DTM M’ can decides L’ in polynomial time.
Y(x)=?
M’ =?

39. Theorem 4 . . . Pf) “” Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
Since LNP,  an NTM M that accept L in nondeterministic time q (a polynomial).
Let  be the encoding function that encodes TM’s and their memory configurations in {I, c}*.
That is, given xL, we have
such that ├M
. . .
Let Y(x) be the set of all strings
where M is the NTM that accepts L.

40. Theorem 4 Pf) “” Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
 q(|x|)
constant
to be discussed
accept in time q
input length

41. Theorem 4 Pf) “” Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
 q(|x|)

42. Theorem 4 Polynomial of |x| Pf) “”
Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
Polynomial of |x|

43. Theorem 4 Let L’ is polynomial balanced. Pf) “”
Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
Let
L’ is polynomial balanced.

44. Theorem 4 ? Let L’ is polynomial balanced. Pf) “”
Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“” ?
Let
L’ is polynomial balanced.

45. Theorem 4 Pf) “” Let L  * be a language, $ and ||2.
L  NP iff  L’  *$* such that
where L’ is a p.b.l.
Theorem 4
Pf)
“”
We now show that L’P by designing a DTM M’ which decides L’ as follows:
Check the syntax of the input string
To verify the following things:
Is Cj a configuration of M?
C0 = (s, #x#)?
Ct = (h, uv)?
If rejected, print out ‘N’ then halt. This steps requires O(n) time.
Check Cj├M Cj+1, j= 0, …, t1 by consulting (M). If rejected print out ‘N’ and then halt. This steps requires O(n2).
Therefore, L’P.

46. Example (Integer Programming BNP )

47. Example (Integer Programming BNP )
No answer

48. Is P = NP ? P NP P  NP NP  P How to show? Show and Trivial (why?)
Difficult
No answer, now.
P  NP
Show
NP  P
and NP P

49. Example (TSPNP ) 1 2 5 3 4 6 7 8
TSPF
Find the shortest tour t.

50. Example (TSPNP ) t is a bijection TSPF Find the shortest tour t.
E.g.,
Example (TSPNP )
TSPF 1 2 5 3 4 6 7 8
Find the shortest tour t.
It is not language decision problem. Rather, it is a problem of function evaluation.

51. Example (TSPNP ) TSPF Find the shortest tour t. TSP
Denote the cost of t as D(t).
TSP’
Polynomial balanced

52. Exercises Show that TSPNP. TSPP   a polynomial algorithm for TSPF.

53. Remark
TSPP
P = NP

54. Lecture 6: Computational Complexity
NP-Completeness
大同大學資工所
智慧型多媒體研究室

55. Definition: Polynomial-Time Computable
is said to be computed in time T if
 a k-tape DTM s.t. ├
with t  T(|x|).
Furthermore, if T(|x|) is a polynomial, then f is said to be polynomial-time computable.

56. Definition: Polynomial-Time Reduction
Let and be languages,
be polynomial-time computable.
Then L1 is said to be polynomial-time reducible to L2 iff

57. More on Polynomial Reduction
L2  P  L1  P.
Transitivity
Time to decide
p.r.

58. Definition: NP-Completeness
is called NP-complete iff

59. More on NP-Completeness
p.r.
p.r. P
NP-Complete

60. NP-Hard
Not Solvable
Solvable NP
p.r.
p.r. P
NP-Complete
NP-Hard

61. Important Theorem Let L be an NP-complete language. Then, Pf) “” “”
trivial
“”
trivial

62. Remark
If one can decide any NP-complete language in polynomial time, then all languages in NP can be decided effectively.
Surely, the guy will win Turing Award.

63. Lecture 6: Computational Complexity
Bounded Halting Problems
大同大學資工所
智慧型多媒體研究室

64. Bounded Halting Problem N0
Fact:
Halting problem is not solvable.
Is Bounded Halting problem is solvable?

65. Theorem: N0 is NP-Complete

66. Theorem: N0 is NP-Complete
Define

67. Theorem: N0 is NP-Complete
Define
Exercise

68. Theorem: N0 is NP-Complete
For any LNP, find L.
LNP
Let
Fact: L(w) is polynomial computable.
w  L  L(w)  N0.

69. The Language

70. Theorem: is NP-Complete
Exercise
Find .

71. Theorem: is NP-Complete
Find

72. Theorem: is NP-Complete
Find
Case 1. #
t + 2  2|u| + 2
+ t
= 2t + 4  2|u|
=2|u’|

73. Theorem: is NP-Complete
Find
Halt within 2|u’| steps.
Halt within t steps.
Case 1. #
t + 2  2|u| + 2
+ t
= 2t + 4  2|u|
=2|u’|

74. Theorem: is NP-Complete
Find
Case 2. t
+2|u| t
=2|u|
=2|u’|

75. Theorem: is NP-Complete
Find
Halt within 2|u’| steps.
Halt within t steps.
Case 2. t
+2|u| t
=2|u|
=2|u’|

76. Theorem: is NP-Complete
Find
Hence,
See previous slides

77. Theorem: is NP-Complete
Summary of the proof:
Known NP-complete

78. Special Bounded Halting Problem N2├

79. Theorem: N2 is NP-Complete├
Pf)
obvious.
p.r.
p.r.
p.r. ? ? ?

80. Theorem: N2 is NP-Complete├
Pf)
obvious.
p.r.
 = ?

81. Theorem: N2 is NP-Complete
 :

82. Theorem: N2 is NP-Complete
 :
2|u|
Let 1 t

83. Theorem: N2 is NP-Complete
 :
M ’ halts on  in t ’ steps
M halts on u in t steps
2|u|
Let 1 t

84. Theorem: N2 is NP-Complete
 :

85. Summary
p.r.

86. Lecture 6: Computational Complexity
Some Important NP-Complete Problems
大同大學資工所
智慧型多媒體研究室

87. Bounded Tiling Problem (BT)
Tiling System:
D: a set of finite number of tiles.
d0D: the tile place at the origin.
HDD: the Horizontal tiling rule.
VDD: the Vertical tiling rule.

88. Bounded Tiling Problem (BT)
Given D and s > 0, an s  s tiling by D is a function 1 2 3 4 5
such that

89. Bounded Tiling Problem (BT)
BT is NP-Complete

90. Theorem: BT is NP-Complete
Pf)
a p.b.l.
p.r.

91. Theorem: BT is NP-Complete
Theorem: BT is NP-Complete
 :

92. Theorem: BT is NP-Complete1 t
t+1

93. Theorem: BT is NP-Complete1 t
t+1
1) For each a, create tiles
(a, k)
(a, k+1)

94. Theorem: BT is NP-Complete1 6 7 2 3 4 5
(a, 6)
(a, 7)
1) For each a, create tiles
(a, 5)
(a, 6)
(a, 4)
(a, 5)
(a, k)
(a, k+1)
(a, 3)
(a, 4)
(a, 2)
(a, 3)
(a, 1)
(a, 2)
(a, 0)
(a, 1)

95. Theorem: BT is NP-Complete1 6 7 2 3 4 5
Can’t reach the highest layer if q  h.
2) For each (q, a, p, b)  where
b and q  h, create tiles
(q, a, 5)
(p, b, 6)
(q, a, 4)
(p, b, 5)
(q, a, k)
(p, b, k+1)
(q, a, 3)
(p, b, 4)
(q, a, 2)
(p, b, 3)
(q, a, 1)
(p, b, 2)
(q, a, 0)
(p, b, 1)

96. Theorem: BT is NP-Complete1 6 7 2 3 4 5
3) For each (q, a, p, L) ,
create tiles
(b, k)
(p, b, k+1) p
(q, a, k)
(a, k+1) p
(b, 2)
(p, b, 3)
(q, a, 2)
(a, 3)
(?, 0)
(?, 1)
(?, 2)
(b, 0)
(b, 1)
(b, 2)
(a, 0)
(a, 1)
(q, a, 2)

97. Theorem: BT is NP-Complete1 6 7 2 3 4 5
(?, 6)
(?, 7)
(a, 6)
(a, 7)
3) For each (q, a, p, L) ,
create tiles
(?, 5)
(?, 6)
(a, 5)
(a, 6)
(?, 4)
(?, 5)
(a, 4)
(a, 5)
(b, k)
(p, b, k+1) p
(q, a, k)
(a, k+1) p
(?, 3)
(?, 4)
(a, 3)
(a, 4)
(?, 2)
(?, 3)
(b, 2)
(p, b, 3)
(q, a, 2)
(a, 3)
(?, 0)
(?, 1)
(?, 2)
(b, 0)
(b, 1)
(b, 2)
(a, 0)
(a, 1)
(q, a, 2)

98. Theorem: BT is NP-Complete1 6 7 2 3 4 5
4) For each (q, a, p, R) ,
create tiles
(q, a, k)
(a, k+1) p
(b, k)
(p, b, k+1) p
(q, a, 2)
(a, 3)
(b, 2)
(p, b, 3)
(?, 0)
(?, 1)
(?, 2)
(a, 0)
(a, 1)
(q, a, 2)
(b, 0)
(b, 1)
(b, 2)

99. Theorem: BT is NP-Complete1 6 7 2 3 4 5
(?, 6)
(?, 7)
(a, 6)
(a, 7)
4) For each (q, a, p, R) ,
create tiles
(?, 5)
(?, 6)
(a, 5)
(a, 6)
(?, 4)
(?, 5)
(a, 4)
(a, 5)
(q, a, k)
(a, k+1) p
(b, k)
(p, b, k+1) p
(?, 3)
(?, 4)
(a, 3)
(a, 4)
(?, 2)
(?, 3)
(q, a, 2)
(a, 3)
(b, 2)
(p, b, 3)
(?, 0)
(?, 1)
(?, 2)
(a, 0)
(a, 1)
(q, a, 2)
(b, 0)
(b, 1)
(b, 2)

100. Theorem: BT is NP-Complete1 6 7 2 3 4 5
(?, 6)
(?, 7)
(a, 6)
(a, 7)
5) For each a, create tiles
(?, 5)
(?, 6)
(a, 5)
(a, 6)
(?, 4)
(?, 5)
(a, 4)
(a, 5)
(h, a, k)
(h, a, k+1)
(?, 3)
(?, 4)
(a, 3)
(a, 4)
(h, a, k+1)
(?, 2)
(?, 3)
(q, a, 2)
(a, 3)
(b, 2)
(h, b, 3)
(?, 0)
(?, 1)
(?, 2)
(a, 0)
(a, 1)
(q, a, 2)
(b, 0)
(b, 1)
(b, 2)

101. Theorem: BT is NP-Complete1 6 7 2 3 4 5
(s, #, 0)
d0 = # 6)
(#, 0) #
7) Base tile
(s, #, 0) #
(#, 0) #
(#, 0) #
(#, 0) #
(#, 0) #
(#, 0) #
(#, 0) #
(#, 0) #

102. Theorem: BT is NP-Complete
1) For each a, create tiles
4) For each (q, a, p, R) , create tiles
(a, k)
(a, k+1)
(q, a, k)
(a, k+1) p
(b, k)
(p, b, k+1) p
2) For each (q, a, p, b)  where
b and q  h, create tiles
5) For each a, create tiles
(h, a, k)
(h, a, k+1)
(q, a, k)
(p, b, k+1) 6)
7) Base tile
3) For each (q, a, p, L) , create tiles
(s, #, 0)
d0 = #
(#, 0) #
(b, k)
(p, b, k+1) p
(q, a, k)
(a, k+1) p

103. Theorem: BT is NP-Complete
Pf)
a p.b.l.
p.r.

104. Integer Programming
already known
Is B NP-complete?

105. Theorem: B is NP-Complete
p.r.

106. Theorem: B is NP-Complete
 :
p.r.

107. Theorem: B is NP-Complete
 :
Define variables
where

108. Theorem: B is NP-Complete
Every place has exactly one tile
Horizontal rules
Vertical rules
Place d0 at (0, 0)

109. Theorem: B is NP-Complete
Every place has exactly one tile
Horizontal rules
Vertical rules
Place d0 at (0, 0)

110. Theorem: B is NP-Complete
Introducing slack variables
Theorem: B is NP-Complete
Every place has exactly one tile
Horizontal rules
Vertical rules
Place d0 at (0, 0)

111. Theorem: B is NP-Complete
Introducing slag variables
Theorem: B is NP-Complete
Every place has exactly one tile
Horizontal rules
Vertical rules
Place d0 at (0, 0)

112. Theorem: B is NP-Complete
Introducing slag variables
The coefficient of each variable is 0 or 1.
Theorem: B is NP-Complete
The right-hand side of each equality is 1.
Every place has exactly one tile
Horizontal rules
Vertical rules
Place d0 at (0, 0)

113. Language Br is NP-Complete

114. Hamilton Cycle

115. Theorem: H is NP-Complete
Pf)
p.r.

116. Theorem: H is NP-Completex y z a b c d

117. Theorem: H is NP-Completex y z a b c d
A Hamilton cycle entering the box from a, it must leave the box from c.
A Hamilton cycle entering the box from d, it must leave the box from b.

118. Theorem: H is NP-Completex y z a b c d

119. Theorem: H is NP-Completex y z a b c d a b c d

120. Theorem: H is NP-Completea b c d

121. Theorem: H is NP-Complete
 :

122. Theorem: H is NP-Complete

123. Theorem: H is NP-Complete

124. Theorem: H is NP-Complete

125. Theorem: H is NP-Complete
 :

126. Traveling Salesman Problem (TSP)
already known
Is TSP NP-complete?

127. Theorem: TSP is NP-Complete
p.r.
p.r.

128. Theorem: TSP is NP-Complete
 :
p.r.
p.r.

129. Theorem: TSP is NP-Complete
 :

130. Theorem: TSP is NP-Complete
G = (V, E)
 :

131. Theorem: TSP is NP-Complete
G = (V, E)
 :
There exists a Hamilton cycle in graph G=(V, E) if and only if there exists a zero-cost tour for the following TSP .

132. Satisfiability Problem (SAT)
Example:
Satisfiability Problem (SAT)
: Boolean Variables;
literals
: the negation of xi;
A formula in the propositional calculus is an expression that can be constructed using literals and the operations  (and) and  (or).
A satisfiability problem is to determine if a formula is true for any truth assignment of variables.

133. Normal Forms
CNF
DNF

134. Cook’s Theorem
SAT is NP-complete
SATP iff P =NP

135. SAT is NP-complete
Cook’s Theorem
Pf)
skip
p.r.
p.r.

136. Cook’s Theorem
Define literals:

137. Cook’s Theorem

138. True if and only if there is no different tiles being placed at each cell.
Cook’s Theorem

139. True if and only if horizontal rules are satisfied at each cell.
Cook’s Theorem
True if and only if horizontal rules are satisfied at each cell.

140. True if and only if vertical rules are satisfied at each cell.
Cook’s Theorem
True if and only if vertical rules are satisfied at each cell.

141. True if an only if d0 is place at (0, 0).
Cook’s Theorem
True if an only if d0 is place at (0, 0).

142. Cook’s Theorem

143. Summary
p.r.
p.r.