CSCI 3130: Formal languages and automata theory Andrej Bogdanov The Chinese University of Hong Kong Turing Machines Fall 2011

What is a computer? A computer is a machine that manipulates data according to a list of instructions. computer input program output

What is a computer? world void hello( String name) { print( “Hello, “, name); } Hello, world E2-E4 computer deep blue ? ? ?

Turing Machines … abb ☐☐ inputblanks control head Can both read from and write to the tape Head can move both left and right Tape is infinite Has two special states accept and reject

Example L 1 = {w#w: w ∈ { a, b }*} Strategy: Read and remember the first symbol Cross it off ( x ) Read the first symbol past # If they don’t match, reject abbaa#abbaa xbbaa#abbaa If they do, cross it off xbbaa#xbbaa

Example L 1 = {w#w: w ∈ { a, b }*} Strategy: Look for the first uncrossed symbol Cross it off ( x ) Read the first uncrossed symbol past # xbbaa#xbbaa xxbaa#xbbaa If they match, cross it off, else reject xxbaa#xxbaa At the end, there should be only x s and # s xxxxx#xxxxx If not, reject; otherwise, accept.

How Turing Machines operate … aba ☐ q1q1 q2q2 a/bLa/bL Replace a with b, and move head left current state … abb ☐ q1q1 q2q2 a/bLa/bL new state

Formal Definition A Turing Machine is (Q, , , , q 0, q acc, q rej ) : –Q is a finite set of states; –  is the input alphabet not containing the blank symbol ☐ –  is the tape alphabet (    ) including ☐ –q 0 in Q is the start state; –q acc, q rej in Q are the accepting and rejecting state –  is the transition function  : (Q – {q acc, q rej })   → Q   {L, R} Turing Machines are deterministic

Configurations A configuration consists of the current state, the head position, and tape contents … aba ☐ q1q1 ab q 1 a q1q1 q acc a/bRa/bR abb q acc … abb ☐ q acc configuration

Configurations We say configuration C yields C’ if the TM can go from C to C’ in one step The start configuration of the TM on input w is q 0 w An accepting configuration is one that contains q acc ; A rejecting configuration is one that contains q rej ab q 1 aabb q acc yields

The language of a Turing Machine We say M accepts x if there exists a sequence of configurations C 0, C 1,..., C k where C 0 is starting C i yields C i+1 C k is accepting The language recognized by M is the set of all strings that M accepts

Looping Something strange can happen in a Turing Machine: Inputs can be divided into three types q0q0 q acc q rej  = { 0, 1 } input:  ☐/☐R☐/☐R 0/0R0/0R 1/1R1/1R This machine never halts q acc q rej accept rejectloop

Halting We say M halts on x if there exists a sequence of configurations C 0, C 1,..., C k where C 0 is starting C i yields C i+1 C k is accepting or rejecting A TM M is a decider if it halts on every input Language L is decidable if it is recognized by a TM that halts on every input

Programming Turing Machines L 1 = {w#w: w ∈ { a, b }*} Description of Turing Machine: Until you reach # Read and remember entry Move right past # and past all x s If this entry is different, reject Otherwise Move left past # and to right of first x If you see only x s followed by ☐, accept xbbaa#xbbaa xxbaa#xxbaa Write x xxbaa#xbbaa Write x xxbaa#xxbaa

Programming Turing Machines L 1 = {w#w: w ∈ {a, b}*} q0q0 q a1 q a2 q2q2 q b1 q b2 q3q3 q rej q1q1 q acc a/xR #/#R b/xL b/xR a/xL a/aR b/bR a/aR b/bR x/xR a/aL b/bL x/xL ☐/☐R☐/☐R #/#R #/#L a/aL b/bL x/xR everything else

Programming Turing Machines q0q0 q a1 q a2 q2q2 q b1 q b2 q3q3 q1q1 q acc a/xR #/#R b/xL b/xR a/xL a/aR b/bR a/aR b/bR x/xR a/aL b/bL x/xL ☐/☐R☐/☐R #/#R #/#L a/aL b/bL x/xR input: aab#aab configurations: q 0 aab#aab x q a1 ab#aab xa q a1 b#aab xab q a1 #aab xab# q a2 aab xab q 2 #xab xa q 3 b#xab x q 3 ab#xab q 3 xab#xab x q 0 ab#xab

Programming Turing Machines L 2 = { a i b j c k : i  j = k and i, j, k > 0 } High-level description of TM: For every a : Cross off the same number of b s and c s Uncross the crossed b s (but not the c s) Cross off this a If all a s and c s are crossed off, accept. aabbcccc  = { a, b, c }  = { a, b, c, a, b, c, ☐ }

Programming Turing Machines L 2 = { a i b j c k : i  j = k and i, j, k > 0 } Low-level description of TM: Scan input from left to right to check it looks like aa * bb * cc * Move the head to the first symbol of the tape For every a : Cross off the same number of b s and c s Restore the crossed of b s (but not the c s) Cross off this a If all a s and c s are crossed off, accept. how do we know? how to do this?

Programming Turing Machines Implementation details: Cross off the same number of b s and c s: Replace b by b Move right until you see a c Move left just past the last b If any b s are left, repeat Replace c by c Put a special marker on top of first a Move the head to the first symbol of the tape aabbcccc  aa q bbcccc  aab q bcccc  aabb q cccc  aab q bcccc  aabb q cccc  aabbc q ccc  aabb q cccc   = { a, b, c }  = { a, b, c, a, b, c, a, a, ☐ } 

Programming Turing Machines L 3 = { # x 1 # x 2... # x l : x i ∈ {0, 1}* and x i ≠ x j for each i ≠ j} #01#0011#1 High-level description of TM: On input w, For every pair of blocks x i and x j in w, Compare the blocks x i and x j If they are the same, reject. Otherwise, accept.

Programming Turing Machines L 3 = { # x 1 # x 2... # x l : x i ∈ {0, 1}* and x i ≠ x j for each i ≠ j} #01#0011#1 Low-level description: Place a mark on the leftmost # (i.e. replace # by # ) and move right  If input is , or has exactly one #, accept. #01#0011#1  Place another mark on next unmarked # (If there are no more #, accept) 2. #01#0011#1  

Programming Turing Machines L 3 = { # x 1 # x 2... # x l : x i ∈ {0, 1}* and x i ≠ x j for each i ≠ j} current state: Repeat Step 3 5. #01#0011#1         Compare the two strings to the right of marked #. If there are same, reject 3.  4.Move the right # to the right If not possible, move the left # to the next # and put the right # on the next If not possible, accept   

Programming Turing Machines L 3 = { # x 1 # x 2... # x l : x i ∈ {0, 1}* and x i ≠ x j for each i ≠ j} #01#0011#1     accept 4.Move the right # to the right If not possible, move the left # to the next # and put the right # on the next If not possible, accept   Compare the two strings to the right of marked #. If there are same, reject 3.  

How to describe Turing Machines? Unlike for DFAs, NFAs, PDAs, we rarely give complete state diagrams of Turing Machines Usually we give a high-level description: A recipe about the workings of the Turing Machine Just like in cooking, practice makes perfect!

Programming Turing Machines L 4 = { 〈 G 〉 : G is a connected undirected graph} Q: How do we feed a graph into a Turing Machine? A: We represent it by a string, e.g (1,2,3,4)((1,4),(2,3),(3,4)(4,2)) Convention for describing graphs: ( nodes )( edges ) no node must repeat edges are pairs ( node 1, node 2 )

Programming Turing Machines L 4 = { 〈 G 〉 : G is a connected undirected graph} On input 〈 G 〉, 0. Verify that 〈 G 〉 is the description of a graph (no vertex repeats; edges only go between nodes) 1.Mark the first node of G 2.Repeat until no new nodes are marked: For each node, mark it if it is attached to an already marked node 3.If all nodes are marked accept, otherwise reject x x x x

Programming Turing Machines L 4 = { 〈 G 〉 : G is a connected undirected graph} (1,2,3,4)((1,4)(2,3)(3,4)(4,2))               etc (1,2,3,4)((1,4)(2,3)(3,4)(4,2))    

What’s so great about Turing Machines?

Hilbert’s list of 23 problems Leading mathematician in the 1900s At the 1900 International Congress of Mathematicians, proposed a list of 23 problems for the 20 th century Hilbert’s 10 th problem: David Hilbert ( ) Find a method to tell if an equation like xyz – 3xy + 6xz + 2 = 0 has integer solutions

A brief history of computing devices Z3 (Germany, 1941)ENIAC (Pennsylvania, 1945) PC (1980) MacBook Air (2008)

Computation is universal In principle, all these have the same problem-solving ability

The Church-Turing Thesis Turing Machine If an algorithm can be implemented on any realistic computer, then it can be implemented on a Turing Machine.

The Church-Turing Thesis Turing Machine quantum computing DNA computing cosmic computing

Alan Turing Invented the Turing Test to tell humans and computers apart Broke German encryption machines in World War II The Turing Award is the “Nobel prize of Computer Science” Alan Turing ( ) Turing’s motivation: By studying Turing Machines, we can understand the limitations of real computers.

Undecidability What Hilbert meant to ask was: Building on work of Gödel, Church, Turing, Davis, Putnam, Robinson, in 1970 Matijasievič showed: Find a Turing Machine to tell if an equation like xyz – 3xy + 6xz + 2 = 0 has integer solutions There is no such Turing Machine!

Computer program analysis public static void main(String args[]) { System.out.println("Hello World!"); } What does this program do? How about this one? public static void main(String args[]) { int i = 0; for (j = 1; j < 10; j++) { i += j; if (i == 28) { System.out.println("Hello World!"); }

Computers cannot analyze programs! No Turing Machine can do this: input: The code of a java program P Accept if P prints “hello, world” Reject if not Significance: It is impossible for computer to predict what a computer program will do!

How do you argue things like that? To argue what computers cannot do, we need to have a precise definition of what a computer is. “On Computable Numbers, with an Application to the Entscheidungsproblem” 1936: Section 1. Computing Machines Turing’s answer: A computer is just a Turing Machine.

The Church-Turing Thesis “On Computable Numbers, with an Application to the Entscheidungsproblem” 1936: Section 9. The extent of the computable numbers All arguments [for the CT Thesis] which can be given are bound to be, fundamentally, appeals to intuition, and for this reason rather unsatisfactory mathematically. The arguments which I shall use are of three kinds: 1. A direct appeal to intuition 2. A proof of the equivalence of two definitions (In case the new definition has greater intuitive appeal) 3. Giving examples of large classes of numbers which are computable.