1. Turing Machines
Complexity ©D.Moshkovitz

2. Motivation
Our main goal in this course is to analyze problems and categorize them according to their complexity.
Complexity ©D.Moshkovitz

3. Motivation
We ask question such as “how much time it takes to compute something?”
Complexity ©D.Moshkovitz

4. Motivation
But in order to answer them we must first have a computational model to relate to.
Complexity ©D.Moshkovitz

5. Introduction Objectives:
To introduce the computational model called “Turing Machine”.
Overview:
Deterministic Turing machines
Multi-tape Turing machines
Non-deterministic Turing machines
The Church-Turing thesis
Complexity classes as bounds on resources required by TMs.
Complexity ©D.Moshkovitz

6. Schematic of a Turing Machine
Read/Write head
There’s b here!
moves: left/right a
infinite tape
Complexity ©D.Moshkovitz

7. Formal Definition of a TM
SIP
Formal Definition of a TM
A deterministic Turing Machine is a tuple consisting of several objects.
Complexity ©D.Moshkovitz

8. Formal Definition of a TM
1. Q - a finite set of states.
Complexity ©D.Moshkovitz

9. Formal Definition of a TM
2.  - the input alphabet, finite set not containing the blank symbol _. b a c d
Complexity ©D.Moshkovitz

10. Formal Definition of a TM
3.  - the tape alphabet, where  and _. B _ A b a C c d
Complexity ©D.Moshkovitz

11. Formal Definition of a TM
4. :QQ{L,R} - the transition function. q0 q0 a
Complexity ©D.Moshkovitz

12. Formal Definition of a TM
5. q0 - the start state
Complexity ©D.Moshkovitz

13. Formal Definition of a TM
6. qacceptQ - the accept state.
Complexity ©D.Moshkovitz

14. Formal Definition of a TM
7. qrejectQ - the reject state. qrejectqaccept.
Complexity ©D.Moshkovitz

15. Formal Definition of a TM Summary
1. Q - the set of states.
2.  - the input alphabet.
3.  - the tape alphabet
4. :QQ{L,R} - the transition function.
5. q0 - the start state.
6. qacceptQ - the accept state.
7. qrejectQ - the reject state.
Complexity ©D.Moshkovitz

16. Computations The Start Configuration
start state q0
head: on the leftmost square
the input: starting from left
Complexity ©D.Moshkovitz

17. Note: the head cannot move to the left of this square!
Computations Example
(q0,a)=(q0,b,R) q0 q0 b
Note: the head cannot move to the left of this square!
Complexity ©D.Moshkovitz

18. Computations Accepting Configuration
If the computation ever enters the accept state, it halts.
qaccept
Complexity ©D.Moshkovitz

19. Computations Rejecting Configuration
If the computation ever enters the reject state, it also halts.
qreject
Note: the machine may loop and not reach any of these two!
Complexity ©D.Moshkovitz

20. The Language a TM Accepts
A Turing Machine accepts its input, if it reaches an accepting configuration.
The set of inputs it accepts is called its language.
Complexity ©D.Moshkovitz

21. the position of the head
Configurations
How many distinct configurations may a Turing machine which uses N cells have?  
||N N
|Q|
the content of the tape
the position of the head
the state
Complexity ©D.Moshkovitz

22. Building a TM for a Simple Language
L = { anbncn | n0 }
aaabbbccc
Examples:
Member of L:
Non-Member of L:
aaabbcccc
Complexity ©D.Moshkovitz

23. The Turing Machine 1. Q = {q0,q1,q2,q3,q4,qaccept,qreject}
2.  = {a,b,c}
3.  = {a,b,c,_,X,Y,Z}
4.  will be specified shortly.
5. q0 - the start state.
6. qacceptQ - the accept state.
7. qrejectQ - the reject state.
Complexity ©D.Moshkovitz

24. The Transitions Functionq0 q3 q2 q1 q4
qac
aa, R
cZ,L
bY,R
bb, R
ZZ, L
XX, R
YY, R ZZ, R
__, R
YY, R
aX,R
ZZ, R
bb, L aa, L YY, L
aa, R
bb, R
YY, R
ZZ, R q1
bY,R
transitions not specified here yield qreject
aX,R q2 q0
cZ,L
XX, R
__, R q3
bb, L aa, L YY, L
YY, R
qac q4
ZZ, L
__, R
YY, R ZZ, R
Complexity ©D.Moshkovitz

25. Demonstration . . . q1 q1 q1 q2 q2 q2 q0 q0 q0 q0 q0 q3 q3 q3 qac qac
aa, R YY, R
Demonstration
bb, R ZZ, R q1 q1 q1
bY,R
aX,R q2 q2 q2 q0 q0 q0 q0 q0
XX, R
cZ,L
__, R q3 q3 q3
ZZ, L bb, L YY, L aa, L
YY, R
qac
qac
__, R q4 q4 q4
YY, R ZZ, R
. . . X a b Y Z c _ _
Complexity ©D.Moshkovitz

26. Equivalent Models
Deterministic Turing machines are extremely powerful.
We can simulate many other models by them and vice-versa with only a polynomial loss of efficiency.
Next we’ll see an example for such model.
Complexity ©D.Moshkovitz

27. Multi-Tape Turing Machines
SIP
Multi-Tape Turing Machines
The input is written on the first tape a b _
. . . b _
. . . b a _
. . .
Complexity ©D.Moshkovitz

28. Multi-Tape Turing Machines
1. Q - the set of states.
2.  - the input alphabet.
3.  - the tape alphabet
4. :QkQ({L,R})k - the transition function, where k (the number of tapes) is some constant.
5. q0 - the start state.
6. qacceptQ - the accept state.
7. qrejectQ - the reject state.
Complexity ©D.Moshkovitz

29. Robustness
Multi-tape machines are polynomially equivalent to single-tape machines.
We can state a much stronger claim concerning the robustness of the Turing machine model:
Complexity ©D.Moshkovitz

30. The Church-Turing Thesis
Intuitive notion of algorithms
Turing machine algorithms
Complexity ©D.Moshkovitz

31. What’s Next? We proceed with a less realistic computational model,
Which can be simulated by DTMs
However, with an exponential loss of efficiency.
Complexity ©D.Moshkovitz

32. Non-deterministic Turing Machines
1. Q - the set of states.
2.  - the input alphabet.
3.  - the tape alphabet
4. :QP(Q{L,R}) - the transition function.
5. q0 - the start state.
6. qacceptQ - the accept state.
7. qrejectQ - the reject state.
power set P(A)={B | BA}
Complexity ©D.Moshkovitz

33. Computations deterministic computation
non-deterministic computation tree .
accepts if some branch reaches an accepting configuration
time
Note: the size of the tree is exponential in its height
Complexity ©D.Moshkovitz

34. Alternative Description
A non-deterministic machine always guesses correctly the ultimate choice.
Complexity ©D.Moshkovitz

35. Example
A non-deterministic TM which checks if two vertices are connected in a graph may simply guess a path between them.
Now it only needs to verify this is a valid path.
Complexity ©D.Moshkovitz

36. Simulating a Non-deterministic machine by a Deterministic One
SIP
Simulating a Non-deterministic machine by a Deterministic One
We’ll describe a deterministic 3-tapes Turing machine which simulates a given non-deterministic machine.
Complexity ©D.Moshkovitz

37. Simulating a Non-deterministic machine by a Deterministic One
. . .
input tape
. . .
simulation tape
. . .
address tape
Complexity ©D.Moshkovitz

38. non-deterministic computation
Addresses 1
non-deterministic computation 1 2 3
111 1 1 2 1
1312 1 1 2 1 2 1 1 1 2
Complexity ©D.Moshkovitz

39. Simulation Write 111…1 on the address tape.
Copy the input to the simulation tape.
Simulate the NTM: use the choices dictated by the address tape (if valid).
If it accepted – accept.
Replace the address string with the lexicographically next string. If there is no such – reject.
Go to step 2.
Complexity ©D.Moshkovitz

40. Complexity Classes Now that we have a formal computational model
We may begin to categorize problems
According to the resources TMs which compute them require.
Complexity ©D.Moshkovitz

41. Time Complexity Definition: Let t:NN be a function.
TIME(t(n))={L | L is a language decidable by a O(t(n)) deterministic TM}
NTIME(t(n))={L | L is a language decidable by a O(t(n)) non-deterministic TM}
Complexity ©D.Moshkovitz

42. Polynomial Time Definition: { anbncn | n0 }  P Example:
Complexity ©D.Moshkovitz

43. Which are in P and Which aren’t?
Minimum Spanning Tree
The Towers of Hanoi
The Halting Problem   
Complexity ©D.Moshkovitz

44. Non-Deterministic Polynomial Time
Definition:
Examples:
the tour problem
the seating problem
Complexity ©D.Moshkovitz

45. Observation Claim: PNP
Proof: A deterministic Turing machine is a special case of non-deterministic Turing machines. 
Complexity ©D.Moshkovitz

46. Exponential Time
Definition:
Complexity ©D.Moshkovitz

47. Space Complexity Definition: Let f:NN be a function.
SPACE(f(n))={L | L is a language decidable by an O(f(n)) space deterministic TM}
NSPACE(f(n))={L | L is a language decidable by an O(f(n)) space non-deterministic TM}
Complexity ©D.Moshkovitz

48. Logarithmic Space
Definition:
Complexity ©D.Moshkovitz

49. Polynomial Space Definition: { anbncn | n0 }  PSPACE Example:
Complexity ©D.Moshkovitz

50. Observation Claim: PPSPACE
Proof: A TM which runs in time t(n) can use at most t(n) space. 
Complexity ©D.Moshkovitz

51. Observation Claim: PSPACEEXPTIME Proof:
A machine that uses polynomial space has at most exponential number of configurations (remember? ).
As deterministic machine that halts may not repeat a configuration
Its running time is bounded by the number of possible configurations. 
Complexity ©D.Moshkovitz

52. Conjectured Relations Among Deterministic Classes
EXPTIME
PSPACE P
Complexity ©D.Moshkovitz

53. Halting Problem is Undecidable
Assume a TM M that, given a TM M’ and input x, decides if M’ halts on x
Construct M”
Run M” on the representation of M”  contradiction M” M M’ x
Yes No
<M”>
Yes
Complexity ©D.Moshkovitz

54. Diagonalization Thm: P  EXPTIME
Proof: We construct a language L  EXPTIME however, L is not accepted by any TM running in polynomial time.
Let
L = {x | x= <M>#1c#1e#(01)*, M doesn’t accept x within c|x|e time }
Complexity ©D.Moshkovitz

55. P vs EXPTIME
L = {x | x= <M>#1c#1e#(01)*, M doesn’t accept x within c|x|e time }
Lemma: L EXPTIME
Proof: in fact in |x|·|x||x| time
Lemma: L  P
Proof: assume, by way of contradiction, a TM M that accepts every x L in time c|x|e  run it on the string <M>#1c#1e# to arrive at contradiction
Complexity ©D.Moshkovitz

56. Conjectured Relation Between P, NP and co-NP
Def: Co-NP = {*-L | LNP}
Complexity ©D.Moshkovitz

57. 
Summary
We presented two main computational models deterministic Turing machines and non-deterministic Turing machines.
We simulated NTM by DTM with an exponential loss of efficiency.
Complexity ©D.Moshkovitz

58. 
Summary
The Church-Turing thesis: Deterministic Turing Machines are equivalent to our intuitive notion of algorithms.
Keeping it in mind, we’ll usually describe algorithms in pseudo-code rather than as TMs.
Complexity ©D.Moshkovitz

59. 
Summary
Using Turing machines we’ve defined various complexity classes:
P – Polynomial time
NP – Non-deterministic Polynomial time
EXPTIME – Exponential time
L – Logarithmic space
NL – Non-deterministic Logarithmic space
PSPACE – Polynomial Space
And discussed the relations between them.
Complexity ©D.Moshkovitz