Turing Machines Complexity ©D.Moshkovitz

Motivation Our main goal in this course is to analyze problems and categorize them according to their complexity. Complexity ©D.Moshkovitz

Motivation We ask question such as “how much time it takes to compute something?” Complexity ©D.Moshkovitz

Motivation But in order to answer them we must first have a computational model to relate to. Complexity ©D.Moshkovitz

Introduction Objectives: To introduce the computational model called “Turing Machine”. Overview: Deterministic Turing machines Multi-tape Turing machines Non-deterministic Turing machines The Church-Turing thesis Complexity classes as bounds on resources required by TMs. Complexity ©D.Moshkovitz

Schematic of a Turing Machine Read/Write head There’s b here! moves: left/right a infinite tape Complexity ©D.Moshkovitz

Formal Definition of a TM SIP 128-129 Formal Definition of a TM A deterministic Turing Machine is a tuple consisting of several objects. Complexity ©D.Moshkovitz

Formal Definition of a TM 1. Q - a finite set of states. Complexity ©D.Moshkovitz

Formal Definition of a TM 2.  - the input alphabet, finite set not containing the blank symbol _. b a c d Complexity ©D.Moshkovitz

Formal Definition of a TM 3.  - the tape alphabet, where  and _. B _ A b a C c d Complexity ©D.Moshkovitz

Formal Definition of a TM 4. :QQ{L,R} - the transition function. q0 q0 a Complexity ©D.Moshkovitz

Formal Definition of a TM 5. q0 - the start state Complexity ©D.Moshkovitz

Formal Definition of a TM 6. qacceptQ - the accept state. Complexity ©D.Moshkovitz

Formal Definition of a TM 7. qrejectQ - the reject state. qrejectqaccept. Complexity ©D.Moshkovitz

Formal Definition of a TM Summary 1. Q - the set of states. 2.  - the input alphabet. 3.  - the tape alphabet 4. :QQ{L,R} - the transition function. 5. q0 - the start state. 6. qacceptQ - the accept state. 7. qrejectQ - the reject state. Complexity ©D.Moshkovitz

Computations The Start Configuration start state q0 head: on the leftmost square the input: starting from left Complexity ©D.Moshkovitz

Note: the head cannot move to the left of this square! Computations Example (q0,a)=(q0,b,R) q0 q0 b Note: the head cannot move to the left of this square! Complexity ©D.Moshkovitz

Computations Accepting Configuration If the computation ever enters the accept state, it halts. qaccept Complexity ©D.Moshkovitz

Computations Rejecting Configuration If the computation ever enters the reject state, it also halts. qreject Note: the machine may loop and not reach any of these two! Complexity ©D.Moshkovitz

The Language a TM Accepts A Turing Machine accepts its input, if it reaches an accepting configuration. The set of inputs it accepts is called its language. Complexity ©D.Moshkovitz

the position of the head Configurations How many distinct configurations may a Turing machine which uses N cells have?   ||N N |Q| the content of the tape the position of the head the state Complexity ©D.Moshkovitz

Building a TM for a Simple Language L = { anbncn | n0 } aaabbbccc Examples: Member of L: Non-Member of L: aaabbcccc Complexity ©D.Moshkovitz

The Turing Machine 1. Q = {q0,q1,q2,q3,q4,qaccept,qreject} 2.  = {a,b,c} 3.  = {a,b,c,_,X,Y,Z} 4.  will be specified shortly. 5. q0 - the start state. 6. qacceptQ - the accept state. 7. qrejectQ - the reject state. Complexity ©D.Moshkovitz

The Transitions Function q0 q3 q2 q1 q4 qac aa, R cZ,L bY,R bb, R ZZ, L XX, R YY, R ZZ, R __, R YY, R aX,R ZZ, R bb, L aa, L YY, L aa, R bb, R YY, R ZZ, R q1 bY,R transitions not specified here yield qreject aX,R q2 q0 cZ,L XX, R __, R q3 bb, L aa, L YY, L YY, R qac q4 ZZ, L __, R YY, R ZZ, R Complexity ©D.Moshkovitz

Demonstration . . . q1 q1 q1 q2 q2 q2 q0 q0 q0 q0 q0 q3 q3 q3 qac qac aa, R YY, R Demonstration bb, R ZZ, R q1 q1 q1 bY,R aX,R q2 q2 q2 q0 q0 q0 q0 q0 XX, R cZ,L __, R q3 q3 q3 ZZ, L bb, L YY, L aa, L YY, R qac qac __, R q4 q4 q4 YY, R ZZ, R . . . X a b Y Z c _ _ Complexity ©D.Moshkovitz

Equivalent Models Deterministic Turing machines are extremely powerful. We can simulate many other models by them and vice-versa with only a polynomial loss of efficiency. Next we’ll see an example for such model. Complexity ©D.Moshkovitz

Multi-Tape Turing Machines SIP 136-138 Multi-Tape Turing Machines The input is written on the first tape a b _ . . . b _ . . . b a _ . . . Complexity ©D.Moshkovitz

Multi-Tape Turing Machines 1. Q - the set of states. 2.  - the input alphabet. 3.  - the tape alphabet 4. :QkQ({L,R})k - the transition function, where k (the number of tapes) is some constant. 5. q0 - the start state. 6. qacceptQ - the accept state. 7. qrejectQ - the reject state. Complexity ©D.Moshkovitz

Robustness Multi-tape machines are polynomially equivalent to single-tape machines. We can state a much stronger claim concerning the robustness of the Turing machine model: Complexity ©D.Moshkovitz

The Church-Turing Thesis  Intuitive notion of algorithms Turing machine algorithms Complexity ©D.Moshkovitz

What’s Next? We proceed with a less realistic computational model, Which can be simulated by DTMs However, with an exponential loss of efficiency. Complexity ©D.Moshkovitz

Non-deterministic Turing Machines 1. Q - the set of states. 2.  - the input alphabet. 3.  - the tape alphabet 4. :QP(Q{L,R}) - the transition function. 5. q0 - the start state. 6. qacceptQ - the accept state. 7. qrejectQ - the reject state. power set P(A)={B | BA} Complexity ©D.Moshkovitz

Computations deterministic computation non-deterministic computation tree . accepts if some branch reaches an accepting configuration time Note: the size of the tree is exponential in its height Complexity ©D.Moshkovitz

Alternative Description A non-deterministic machine always guesses correctly the ultimate choice. Complexity ©D.Moshkovitz

Example A non-deterministic TM which checks if two vertices are connected in a graph may simply guess a path between them. Now it only needs to verify this is a valid path. Complexity ©D.Moshkovitz

Simulating a Non-deterministic machine by a Deterministic One SIP 138-140 Simulating a Non-deterministic machine by a Deterministic One We’ll describe a deterministic 3-tapes Turing machine which simulates a given non-deterministic machine. Complexity ©D.Moshkovitz

Simulating a Non-deterministic machine by a Deterministic One . . . input tape . . . simulation tape . . . address tape Complexity ©D.Moshkovitz

non-deterministic computation Addresses 1 non-deterministic computation 1 2 3 111 1 1 2 1 1312 1 1 2 1 2 1 1 1 2 Complexity ©D.Moshkovitz

Simulation Write 111…1 on the address tape. Copy the input to the simulation tape. Simulate the NTM: use the choices dictated by the address tape (if valid). If it accepted – accept. Replace the address string with the lexicographically next string. If there is no such – reject. Go to step 2. Complexity ©D.Moshkovitz

Complexity Classes Now that we have a formal computational model We may begin to categorize problems According to the resources TMs which compute them require. Complexity ©D.Moshkovitz

Time Complexity Definition: Let t:NN be a function. TIME(t(n))={L | L is a language decidable by a O(t(n)) deterministic TM} NTIME(t(n))={L | L is a language decidable by a O(t(n)) non-deterministic TM} Complexity ©D.Moshkovitz

Polynomial Time Definition: { anbncn | n0 }  P Example: Complexity ©D.Moshkovitz

Which are in P and Which aren’t? Minimum Spanning Tree The Towers of Hanoi The Halting Problem    Complexity ©D.Moshkovitz

Non-Deterministic Polynomial Time Definition: Examples: the tour problem the seating problem Complexity ©D.Moshkovitz

Observation Claim: PNP Proof: A deterministic Turing machine is a special case of non-deterministic Turing machines.  Complexity ©D.Moshkovitz

Exponential Time Definition: Complexity ©D.Moshkovitz

Space Complexity Definition: Let f:NN be a function. SPACE(f(n))={L | L is a language decidable by an O(f(n)) space deterministic TM} NSPACE(f(n))={L | L is a language decidable by an O(f(n)) space non-deterministic TM} Complexity ©D.Moshkovitz

Logarithmic Space Definition: Complexity ©D.Moshkovitz

Polynomial Space Definition: { anbncn | n0 }  PSPACE Example: Complexity ©D.Moshkovitz

Observation Claim: PPSPACE Proof: A TM which runs in time t(n) can use at most t(n) space.  Complexity ©D.Moshkovitz

Observation Claim: PSPACEEXPTIME Proof: A machine that uses polynomial space has at most exponential number of configurations (remember? ). As deterministic machine that halts may not repeat a configuration Its running time is bounded by the number of possible configurations.  Complexity ©D.Moshkovitz

Conjectured Relations Among Deterministic Classes EXPTIME PSPACE P Complexity ©D.Moshkovitz

Halting Problem is Undecidable Assume a TM M that, given a TM M’ and input x, decides if M’ halts on x Construct M” Run M” on the representation of M”  contradiction M” M M’ x Yes No <M”> Yes Complexity ©D.Moshkovitz

Diagonalization Thm: P  EXPTIME Proof: We construct a language L  EXPTIME however, L is not accepted by any TM running in polynomial time. Let L = {x | x= <M>#1c#1e#(01)*, M doesn’t accept x within c|x|e time } Complexity ©D.Moshkovitz

P vs EXPTIME L = {x | x= <M>#1c#1e#(01)*, M doesn’t accept x within c|x|e time } Lemma: L EXPTIME Proof: in fact in |x|·|x||x| time Lemma: L  P Proof: assume, by way of contradiction, a TM M that accepts every x L in time c|x|e  run it on the string <M>#1c#1e# to arrive at contradiction Complexity ©D.Moshkovitz

Conjectured Relation Between P, NP and co-NP Def: Co-NP = {*-L | LNP} Complexity ©D.Moshkovitz

 Summary We presented two main computational models deterministic Turing machines and non-deterministic Turing machines. We simulated NTM by DTM with an exponential loss of efficiency. Complexity ©D.Moshkovitz

 Summary The Church-Turing thesis: Deterministic Turing Machines are equivalent to our intuitive notion of algorithms. Keeping it in mind, we’ll usually describe algorithms in pseudo-code rather than as TMs. Complexity ©D.Moshkovitz

 Summary Using Turing machines we’ve defined various complexity classes: P – Polynomial time NP – Non-deterministic Polynomial time EXPTIME – Exponential time L – Logarithmic space NL – Non-deterministic Logarithmic space PSPACE – Polynomial Space And discussed the relations between them. Complexity ©D.Moshkovitz