Complexity ©D.Moshkovitz 1 Turing Machines

Complexity ©D.Moshkovitz 2 Motivation Our main goal in this course is to analyze problems and categorize them according to their complexity.

Complexity ©D.Moshkovitz 3 Motivation We ask question such as “how much time it takes to compute something?”

Complexity ©D.Moshkovitz 4 Motivation But in order to answer them we must first have a computational model to relate to.

Complexity ©D.Moshkovitz 5 Introduction Objectives: –To introduce the computational model called “Turing Machine”. Overview: –Deterministic Turing machines –Multi-tape Turing machines –Non-deterministic Turing machines –The Church-Turing thesis –Complexity classes as bounds on resources required by TMs.

Complexity ©D.Moshkovitz 6 Schematic of a Turing Machine Read/Write head infinite tape moves: left/right There’s b here! a

Complexity ©D.Moshkovitz 7 Formal Definition of a TM A deterministic Turing Machine is a tuple consisting of several objects. SIP

Complexity ©D.Moshkovitz 8 Formal Definition of a TM 1. Q - a finite set of states.

Complexity ©D.Moshkovitz 9 Formal Definition of a TM 2.  - the input alphabet, finite set not containing the blank symbol _.

Complexity ©D.Moshkovitz 10 Formal Definition of a TM 3.  - the tape alphabet, where  and _ .

Complexity ©D.Moshkovitz 11 Formal Definition of a TM 4.  :Q  Q  {L,R} - the transition function. q0q0 a q0q0

Complexity ©D.Moshkovitz 12 Formal Definition of a TM 5. q 0 - the start state

Complexity ©D.Moshkovitz 13 Formal Definition of a TM 6. q accept  Q - the accept state.

Complexity ©D.Moshkovitz 14 Formal Definition of a TM 7. q reject  Q - the reject state. q reject  q accept.

Complexity ©D.Moshkovitz 15 Formal Definition of a TM Summary 1. Q - the set of states. 2.  - the input alphabet. 3.  - the tape alphabet 4.  :Q  Q  {L,R} - the transition function. 5. q 0 - the start state. 6. q accept  Q - the accept state. 7. q reject  Q - the reject state.

Complexity ©D.Moshkovitz 16 Computations The Start Configuration q0q0 head: on the leftmost square the input: starting from left start state

Complexity ©D.Moshkovitz 17 Computations Example q0q0  (q 0,a)=(q 0,b,R) q0q0 b Note: the head cannot move to the left of this square!

Complexity ©D.Moshkovitz 18 Computations Accepting Configuration q accept If the computation ever enters the accept state, it halts.

Complexity ©D.Moshkovitz 19 Computations Rejecting Configuration q reject If the computation ever enters the reject state, it also halts. Note: the machine may loop and not reach any of these two!

Complexity ©D.Moshkovitz 20 The Language a TM Accepts A Turing Machine accepts its input, if it reaches an accepting configuration. The set of inputs it accepts is called its language.

Complexity ©D.Moshkovitz 21 Configurations How many distinct configurations may a Turing machine which uses N cells have? ||N||N  |Q||Q|  N the content of the tape the state the position of the head

Complexity ©D.Moshkovitz 22 Building a TM for a Simple Language L = { a n b n c n | n  0 } aaabbbccc Examples: Member of L: Non-Member of L:aaabbcccc

Complexity ©D.Moshkovitz 23 The Turing Machine 1. Q = {q 0,q 1,q 2,q 3,q 4,q accept,q reject } 2.  = {a,b,c} 3.  = {a,b,c,_,X,Y,Z} 4.  will be specified shortly. 5. q 0 - the start state. 6. q accept  Q - the accept state. 7. q reject  Q - the reject state.

Complexity ©D.Moshkovitz 24 The Transitions Function q0q0 q3q3 q2q2 q1q1 q4q4 q ac a  a, R c  Z,L b  Y,R b  b, R Z  Z, L X  X, R Y  Y, R Z  Z, R _  _, R Y  Y, R transitions not specified here yield q reject a  X,R Y  Y, R Z  Z, R b  b, L a  a, L Y  Y, L q0q0 q3q3 q2q2 q1q1 q4q4 q ac a  a, R c  Z,L b  Y,R b  b, R Z  Z, L X  X, R Y  Y, R Z  Z, R _  _, R Y  Y, R a  X,R Y  Y, R Z  Z, R b  b, L a  a, L Y  Y, L

Complexity ©D.Moshkovitz 25 Demonstration q0q0 q3q3 q2q2 q1q1 q4q4 q ac a  X,R a  a, R Y  Y, R c  Z,L b  Y,R b  b, R Z  Z, R Z  Z, L b  b, L Y  Y, L a  a, L X  X, R Y  Y, R Z  Z, R _  _, R Y  Y, R abc_... q0q0 q3q3 q0q0 q1q1 X q2q2 q1q1 YZ q2q2 q3q3 q0q0 q0q0 q4q4 _ q4q4 q ac

Complexity ©D.Moshkovitz 26 Equivalent Models Deterministic Turing machines are extremely powerful. We can simulate many other models by them and vice-versa with only a polynomial loss of efficiency. Next we’ll see an example for such model.

Complexity ©D.Moshkovitz 27 Multi-Tape Turing Machines aababb_... bbbbb__ baaba__ SIP The input is written on the first tape

Complexity ©D.Moshkovitz 28 Multi-Tape Turing Machines 1. Q - the set of states. 2.  - the input alphabet. 3.  - the tape alphabet 4.  :Q  k  Q  (  {L,R}) k - the transition function, where k (the number of tapes) is some constant. 5. q 0 - the start state. 6. q accept  Q - the accept state. 7. q reject  Q - the reject state.

Complexity ©D.Moshkovitz 29 Robustness Multi-tape machines are polynomially equivalent to single-tape machines. We can state a much stronger claim concerning the robustness of the Turing machine model:

Complexity ©D.Moshkovitz 30 The Church-Turing Thesis Intuitive notion of algorithms Turing machine algorithms 

Complexity ©D.Moshkovitz 31 What’s Next? We proceed with a less realistic computational model, Which can be simulated by DTMs However, with an exponential loss of efficiency.

Complexity ©D.Moshkovitz 32 Non-deterministic Turing Machines 1. Q - the set of states. 2.  - the input alphabet. 3.  - the tape alphabet 4.  :Q  P(Q  {L,R}) - the transition function. 5. q 0 - the start state. 6. q accept  Q - the accept state. 7. q reject  Q - the reject state. power set P(A)={B | B  A}

Complexity ©D.Moshkovitz 33 Computations deterministic computation non-deterministic computation tree Note: the size of the tree is exponential in its height time accepts if some branch reaches an accepting configuration

Complexity ©D.Moshkovitz 34 Alternative Description A non-deterministic machine always guesses correctly the ultimate choice.

Complexity ©D.Moshkovitz 35 Example A non-deterministic TM which checks if two vertices are connected in a graph may simply guess a path between them. Now it only needs to verify this is a valid path.

Complexity ©D.Moshkovitz 36 Simulating a Non-deterministic machine by a Deterministic One We’ll describe a deterministic 3- tapes Turing machine which simulates a given non-deterministic machine.3- tapes SIP

Complexity ©D.Moshkovitz 37 Simulating a Non-deterministic machine by a Deterministic One... input tape simulation tape address tape

Complexity ©D.Moshkovitz 38 Addresses non-deterministic computation

Complexity ©D.Moshkovitz 39 Simulation 1.Write 111…1 on the address tape. 2.Copy the input to the simulation tape. 3.Simulate the NTM: use the choices dictated by the address tape (if valid). 4.If it accepted – accept. 5.Replace the address string with the lexicographically next string. If there is no such – reject. 6.Go to step 2.

Complexity ©D.Moshkovitz 40 Complexity Classes Now that we have a formal computational model We may begin to categorize problems According to the resources TMs which compute them require.

Complexity ©D.Moshkovitz 41 Time Complexity Definition: Let t:N  N be a function. TIME(t(n))={L | L is a language decidable by a O(t(n)) deterministic TM} NTIME(t(n))={L | L is a language decidable by a O(t(n)) non-deterministic TM}

Complexity ©D.Moshkovitz 42 Polynomial Time Definition: { a n b n c n | n  0 }  P Example:

Complexity ©D.Moshkovitz 43 Which are in P and Which aren’t? Minimum Spanning Tree The Towers of Hanoi The Halting Problem 

Complexity ©D.Moshkovitz 44 Non-Deterministic Polynomial Time Definition: the tour problem the seating problem the tour problem the seating problem Examples:

Complexity ©D.Moshkovitz 45 Observation Claim: P  NP Proof: A deterministic Turing machine is a special case of non-deterministic Turing machines.

Complexity ©D.Moshkovitz 46 Definition: Exponential Time

Complexity ©D.Moshkovitz 47 Space Complexity Definition: Let f:N  N be a function. SPACE(f(n))={L | L is a language decidable by an O(f(n)) space deterministic TM} NSPACE(f(n))={L | L is a language decidable by an O(f(n)) space non-deterministic TM}

Complexity ©D.Moshkovitz 48 Logarithmic Space Definition:

Complexity ©D.Moshkovitz 49 Polynomial Space Definition: { a n b n c n | n  0 }  PSPACE Example:

Complexity ©D.Moshkovitz 50 Observation Claim: P  PSPACE Proof: A TM which runs in time t(n) can use at most t(n) space.

Complexity ©D.Moshkovitz 51 Observation Claim: PSPACE  EXPTIME Proof: –A machine that uses polynomial space has at most exponential number of configurations (remember? ).remember? –As deterministic machine that halts may not repeat a configuration –Its running time is bounded by the number of possible configurations.

Complexity ©D.Moshkovitz 52 EXPTIME PSPACE Conjectured Relations Among Deterministic Classes P

Complexity ©D.Moshkovitz 53 Halting Problem is Undecidable Assume a TM M that, given a TM M’ and input x, decides if M’ halts on x Construct M” Run M” on the representation of M”  contradiction M M’ x Yes No M” Yes

Complexity ©D.Moshkovitz 54 Diagonalization Thm: P  EXPTIME Proof: We construct a language L  EXPTIME however, L is not accepted by any TM running in polynomial time. Let L = {x | x= #1 c #1 e #(01)*, M doesn’t accept x within c|x| e time }

Complexity ©D.Moshkovitz 55 P vs EXPTIME L = {x | x= #1 c #1 e #(01)*, M doesn’t accept x within c|x| e time } Lemma: L  EXPTIME Proof: in fact in |x|·|x| |x| time Lemma: L  P Proof: assume, by way of contradiction, a TM M that accepts every x  L in time c|x| e  run it on the string #1 c #1 e # to arrive at contradiction

Complexity ©D.Moshkovitz 56 Conjectured Relation Between P, NP and co-NP Co-NPNP P Def: Co-NP = {  *-L | L  NP}

Complexity ©D.Moshkovitz 57 Summary We presented two main computational models deterministic Turing machines and non-deterministic Turing machines. We simulated NTM by DTM with an exponential loss of efficiency. 

Complexity ©D.Moshkovitz 58 Summary The Church-Turing thesis: Deterministic Turing Machines are equivalent to our intuitive notion of algorithms. Keeping it in mind, we’ll usually describe algorithms in pseudo-code rather than as TMs. 

Complexity ©D.Moshkovitz 59 Summary Using Turing machines we’ve defined various complexity classes: –P – Polynomial time –NP – Non-deterministic Polynomial time –EXPTIME – Exponential time –L – Logarithmic space –NL – Non-deterministic Logarithmic space –PSPACE – Polynomial Space And discussed the relations between them. 