1. CS154, Lecture 7:
Turing Machines

2. Turing Machine (1936) INFINITE REWRITABLE TAPE FINITE STATE CONTROL A…
INFINITE REWRITABLE TAPE

3. Turing Machine (1936)

5. Turing Machines versus DFAs
TM can both write to and read from the tape
The head can move left and right
The input doesn’t have to be read entirely,
and the computation can continue further (even, forever) after all input has been read
Accept and Reject take immediate effect

6. This Turing machine decides the language {0}
Σ = {0}
read
write
move
0 → 0, R
 → , R
qaccept
0 → 0, R
 → , R
qreject
This Turing machine decides the language {0}

7. This Turing machine recognizes the language {0}
Σ = {0}
read
write
move
0 → 0, R
 → , R
qaccept
0 → 0, R
 → , R
0 → 0, R
 → , L
This Turing machine recognizes the language {0}

8. Deciding the language L = { w#w | w  {0,1}* }
STATE
STATE
STATE
STATE
STATE
STATE
STATE
STATE
STATE
STATE
STATE
STATE
STATE
q0, FIND #
qGO LEFT
q1, FIND #
q0, FIND #
q1, FIND #
q#, FIND 
q1, FIND #
q#, FIND 
q0, FIND 
q1, FIND 
q0, FIND 
q1, FIND 
qGO LEFT
q1, FIND 
and so on… X 1 1 1 X # # X 1 1 1
If there’s no # on the tape (or more than one #), reject.
While there is a bit to the left of #, Replace the first bit with X, and check if the first bit b to the right of the # is identical. (If not, reject.) Replace that bit b with an X too.
If there’s a bit to the right of #, then reject else accept

9. Definition: A Turing Machine is a 7-tuple
T = (Q, Σ, Γ, , q0, qaccept, qreject), where:
Q is a finite set of states
Σ is the input alphabet, where   Σ
Γ is the tape alphabet, where   Γ and Σ  Γ
 : Q  Γ → Q  Γ  {L, R}
q0  Q is the start state
qaccept  Q is the accept state
qreject  Q is the reject state, and qreject  qaccept

10. Turing Machine Configurations

11. Defining Acceptance and Rejection for TMs
Let C1 and C2 be configurations of a TM M Definition: C1 yields C2 if M is in configuration C2 after running M in configuration C1 for one step
Suppose (q1, b) = (q2, c, L) Then aaq1bb yields aq2acb
Suppose (q1, a) = (q2, c, R) Then cabq1a yields cabcq2
Let w  Σ* and M be a Turing machine
M accepts w if there are configs C0, C1, ..., Ck, s.t.
C0 = q0w
Ci yields Ci+1 for i = 0, ..., k-1, and
Ck contains the accept state qaccept

12. A TM M recognizes a language L if M accepts exactly those strings in L
A language L is recognizable (a.k.a. recursively enumerable)
if some TM recognizes L
A TM M decides a language L if M accepts all strings in L and rejects all strings not in L
A language L is decidable (a.k.a. recursive) if some TM decides L

14. A Turing machine for deciding { 0 | n ≥ 0 }
Turing Machine PSEUDOCODE:
Sweep from left to right, cross out every other 0
If in step 1, the tape had only one 0, accept
If in step 1, the tape had an odd number of 0’s, reject
Move the head back to the first input symbol.
Go to step 1.
Why does this work?
Idea: Every time we return to stage 1, the number of 0’s on the tape has been halved.

15. Step 4 2n
{ 0 | n ≥ 0 }
Step 1
Step 2
Step 3

16. { 0 | n ≥ 0 } q00000 2n q1000 xq300 x0q40 x0xq3 x0q2x xq20x…

17. Multitape Turing Machines
 : Q  Γk → Q  Γk  {L,R}k

18. Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine

19. Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine

20. Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine

21. Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine

22. Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine

23. Theorem: L is decidable iff both L and L are recognizable

24. Recall: Given L  Σ*, define L := Σ* \ L
Theorem: L is decidable iff both L and L are recognizable
Given:
a TM M1 that recognizes L and
a TM M2 that recognizes L,
want to build a new machine M that decides L
How? Any ideas?
M1 always accepts x, when x is in L
M2 always accepts x, when x isn’t in L

25. Recall: Given L  Σ*, define L := Σ* \ L
Theorem: L is decidable iff both L and L are recognizable
Given:
a TM M1 that recognizes L and
a TM M2 that recognizes L,
want to build a new machine M that decides L
Simulate M1 (x) on one tape, M2 (x) on another. Exactly one of the two will accept If M1 accepts then accept If M2 accepts then reject

26. Nondeterministic Turing Machines
Have multiple transitions for a state, symbol pair
Theorem: Every nondeterministic Turing machine N can be transformed into a Turing Machine M that accepts precisely the same strings as N.
Proof Idea (more details in Sipser) Pick a natural ordering on all strings in {Q  Γ  #}*
M(w): For all strings D  {Q  Γ  #}* in the ordering,
Check if D = C0#  #Ck where C0, …,Ck is some accepting computation history for N on w. If so, accept.

27. It is all Zeros and Ones
One of the most popular and overemphasized clichés about computer scientists. Proxy for: genius / nerd / insensitive / anti-social / …
(still bits are quite fundamental, “more than atoms”)

28. Bit Strings Encoding
Encode a finite string in Σ* as a bit string: encode each character as log | Σ | bits.
For x  Σ* define bΣ(x) to be its binary encoding
For x, y  Σ*, to encode the pair of x and y can add as x, y over Σ’ = Σ{,}.
Or sometimes better: (x, y) := 0|bΣ(x)|1 bΣ(x) bΣ(y)

29. TM Encoding ( (p, i), (q, j, L) ), ( (p, i), (q, j, R) ) , …
Can encode a TM as a bit string: n (states), m (tape symbols), (first) k (are input symbols), s (start state), t (accept state), r (reject state), u (blank symbol), transition1, transition2, …
( (p, i), (q, j, L) ), ( (p, i), (q, j, R) ) , …
Similarly, we can encode DFAs and NFAs as bit strings

30. 0n10m10k10s10t10r10u1 … ( (p, i), (q, j, L) ) = 0p10i10q10j10
Other ways to encode a TM exist:
start state
reject state
n states
0n10m10k10s10t10r10u1 …
m tape symbols
(first k are input symbols)
blank symbol
accept state
( (p, i), (q, j, L) ) = 0p10i10q10j10
( (p, i), (q, j, R) ) = 0p10i10q10j100

31. Binary languages about computations
Define the following languages over {0,1}:
ADFA = { (B, w) | B encodes a DFA over some Σ, and B accepts w  Σ* }
ANFA = { (B, w) | B encodes an NFA, B accepts w }
ATM = { (M, w) | M encodes a TM, M accepts w }

32. ATM = { (M, w) | M encodes a TM over some Σ,
ATM = { (M, w) | M encodes a TM over some Σ, w encodes a string over Σ and M accepts w}
Technical Note:
We’ll use an decoding of pairs, TMs, and strings so that every binary string decodes to some pair (M, w)
If x  {0,1}* doesn’t decode to (M,w) in the usual way, then we define that x decodes to the pair (D, ε) where D is a “dummy” TM that accepts nothing.
Then, we can define the complement of ATM very simply:
ATM = { (M, w) | M does not accept w }

33. Universal Turing Machines
Theorem: There is a Turing machine U which takes as input:
(1) the code of an arbitrary TM M
(2) an input string w
such that U accepts (M, w)  M accepts w.
This is a fundamental property of TMs: There is a Turing Machine that can run arbitrary Turing Machine code!
Note that DFAs/NFAs do not have this property: ADFA and ANFA are not regular.

34. The Church-Turing Thesis
Everyone’s Intuitive Notion of Algorithms
= Turing Machines
This is not a theorem – it is a falsifiable scientific hypothesis.
And it has and is still been tested

35. Theorem: ADFA is decidable
ADFA = { (D, w) | D is a DFA that accepts string w }
Theorem: ADFA is decidable
Proof: A DFA is a special case of a TM. Run the universal U on (D, w) and output its answer.
ANFA = { (N, w) | N is an NFA that accepts string w }
Theorem: ANFA is decidable. (Why?)
ATM = { (M, w) | M is a TM that accepts string w }
Theorem: ATM is recognizable but not decidable!

36. There are non-recognizable languages
Assuming the Church-Turing Thesis, this means there are problems that NO computing device can solve!
We can prove this using a counting argument: We will show there is no onto function from the set of all Turing Machines to the set of all languages over {0,1} (works for any finite Σ)
That is, every mapping from Turing machines to languages fails to cover all possible languages

37. “There are more problems to solve than there are programs to solve them.”
Languages over {0,1}

38. f : A  B is not onto  (9 b 2 B)(8 a 2 A)[f(a)  b]
Let L be any set and 2L be the power set of L
Theorem: There is no onto function from L to 2L
Proof: Assume, for a contradiction, there is an onto function f : L  2L
Define S = { x  L | x  f(x) }  2L
If f is onto, then there is a y  L with f(y) = S
Suppose y  S. By definition of S, y  f(y) = S.
Suppose y  S. By definition of S, y  f(y) = S.
Contradiction!

39. f : A  B is not onto  (9 b 2 B)(8 a 2 A)[f(a)  b]
Let L be any set and 2L be the power set of L
Theorem: There is no onto function from L to 2L
Let f : L  2L be an arbitrary function
Define S = { x  L | x  f(x) } 2 2L
For all x  L,
If x  S then x  f(x) [by definition of S]
If x  S then x  f(x)
In either case, we have f(x)  S. (Why?)
Therefore f is not onto!

40. What does this mean?
No function from L to 2L can “cover” all the elements in 2L
No matter what the set L is,
the power set 2L always has strictly larger cardinality than L

41. 2M Thm:There are non-recognizable languages ⊆
Proof: Suppose all languages are recognizable. Then for all L, there’s a Turing machine M for recognizing L. Hence there is an onto R: {Turing Machines}  {Languages}
{Languages over {0,1}}
{Turing Machines} ⊆
{Sets of strings of 0s and 1s}
{0,1}*
Set M 2M
But there is no onto function from {Turing Machines} ⊆ M to 2^M. Contradiction!

42. Russell’s Paradox in Set Theory
In the early 1900’s, logicians were trying to
define consistent foundations for mathematics.
Suppose X = “Universe of all possible sets”
Frege’s Axiom: Let f : X  {0,1}
Then {S  X | f(S) = 1} is a set.
Define F = { S  X | S  S }
Suppose F  F. Then by definition, F  F. So F  F and by definition F  F.
This logical system is inconsistent!