1. Recursively Enumerable Languages
Chapter 8
Recursively Enumerable Languages 1
Copyright © 2011 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

2. Recursively Enumerable Languages
Recursively enumerable (r.e.) languages are those that can be accepted by a TM
Recursive languages are those that can be decided by a TM
Only in the second case are we guaranteed an answer to the question: Given a string x, is x an element of the language?
We will study the relationships between these two kinds of languages, and we will see that there are languages that are not r.e., as well as languages that are r.e. but not recursive
Introduction to Computation 2

3. Recursively Enumerable and Recursive
Definition 8.1: A TM T with input alphabet  accepts a language L  * if it accepts the strings in L and no others
T decides L if T computes the characteristic function L : *  {0,1} that has the value 1 at strings in L and the value 0 otherwise
In both cases, the issue is whether the input string is an element of L. The second approach may be more informative, however, because a TM accepting L may not return an answer if the string is not in L
Introduction to Computation 3

4. Recursively Enumerable and Recursive (cont’d.)
Theorem 8.2: Every recursive language is recursively enumerable
Theorem 8.3: If L  * is accepted by a TM T that halts on every input string, then L is recursive
Theorem 8.4: If L1 and L2 are both recursively
enumerable languages over , then L1 ∪ L2 and L1 ∩ L2 are also recursively enumerable
Theorem 8.5: If L1 and L2 are both recursive
languages over , then L1 ∪ L2 and L1 ∩ L2 are also recursive
Introduction to Computation 4

5. Recursively Enumerable and Recursive (cont’d.)
Theorem 8.6: If L is a recursive language over , then its complement L’ is also recursive
Theorem 8.7: If L is a recursively enumerable language, and its complement L’ is also recursively enumerable, then L is recursive
Introduction to Computation 5

6. Enumerating a Language
Definition 8.8: Let T be a k-tape Turing machine for some k  1, and let L  *. We say T enumerates L if it operates such that the following conditions are satisfied:
The tape head on the first tape never moves to the left, and no nonblank symbol printed on tape 1 is subsequently modified or erased
For every x  L, there is some point during the operation of T when tape 1 has contents x1 # x2 # … # xn # x # for some n  0, where the xi’s are also elements of L and x1, x2, … , xn, x are distinct
If L is finite, then nothing is printed after the # following the last element of L
Introduction to Computation 6

7. Enumerating a Language (cont’d.)
Theorem 8.9: For every language L  *, L is recursively enumerable if and only if there is a TM enumerating L, and L is recursive if and only if there is a TM that enumerates the strings in L in canonical order
Introduction to Computation 7

8. Enumerating a Language (cont’d.)
Proof: we need to show these four things
If there is a TM that accepts L then there’s one that enumerates L
If there is a TM that enumerates L then there’s one that accepts it
If there is a TM that decides L, then there is a TM that enumerates L in canonical order
If there is a TM that enumerates L in canonical order then there is a TM that decides L
Introduction to Computation 8

9. Enumerating a Language (cont’d.)
To prove these four statements, we’ll make use of the Church-Turing sequence and just describe an algorithm, rather than specifying a TM in detail
We start with the third statement
If T decides L, then for any string x, we can give x to T and wait for it to give an answer
The algorithm for enumerating L is to consider the strings of * in canonical order, and for each one, add it to the output only if T says “yes”
Introduction to Computation 9

10. Enumerating a Language (cont’d.)
For statement 1, we consider the strings in canonical order, but “waiting for T to give an answer” might mean waiting forever, so we make repeated passes
On each pass, we examine one more string, and for each string that we’re still unsure about, we consider one more step in T’s processing of that string
Statement 2 is easy
If T enumerates L then an algorithm to accept L is: watch the computation of T and accept x precisely if T lists x (this algorithm never returns an answer if x  L)
Statement 4 is almost as easy:
Watch the computation until either x or a string greater than x is listed; in the first case, x is in L, and in the second case it isn’t
Introduction to Computation 10

11. More General Grammars
Definition 8.10: An unrestricted grammar is a 4-tuple G=(V, , S, P), where V and  are disjoint sets of variables and terminals, respectively
Just as in a CFG, S is the start variable and P is a set of productions. Here, however, a production can have
the form   , where ,  (V ∪ )* and  is any string containing at least one variable
It will turn out that unrestricted grammars correspond to recursively enumerable languages, just as CFGs correspond to PDAs and regular grammars to FAs
Introduction to Computation 11

12. More General Grammars (cont’d.)
We can continue to use much of the notation developed for CFGs, but one important difference is that the assumption S * xAy * z no longer implies that z = xwy for some string w
Theorem 8.13: For every unrestricted grammar G, there is a Turing machine T with L(T) = L(G)
Proof: construct a TM that accepts L(G)
It will be simpler to build a nondeterministic TM
Introduction to Computation 12

13. More General Grammars (cont’d.)
The TM will contain all the variables and terminals in G and works as follows
It moves the tape head to the blank square following the input string
During the second phase of its operation, T treats this blank square as if it were the beginning of the tape, and the input string is undisturbed
T simulates a derivation in G nondeterministically as follows:
Introduction to Computation 13

14. More General Grammars (cont’d.)
First the symbol S is written in the square following the blank
Each subsequent step involves
Choosing a production    in the grammar
Selecting an occurrence of , if there is one, in the string currently on the tape
Replacing the occurrence of  by , which may mean moving the rest of the string to the right or left
The final phase is to compare the two strings on the tape and to accept if and only if they are equal
Introduction to Computation 14

15. More General Grammars (cont’d.)
Theorem 8.14: For every TM T with input alphabet , there is an unrestricted grammar generating the language L(T)  *
Proof: for simplicity assume that  = {a, b}. The grammar will have three types of productions:
Those of type 1 are
S  S() | T T  T(aa) | T(bb) | q0()
which generate strings q0()(11)(22)…(kk) followed by zero or more copies of (). Each pair of ’s is thought of as two copies of a symbol in x *
Introduction to Computation 15

16. More General Grammars (cont’d.)
Proof (cont’d.)
Productions of type 2 allow the moves of T to be simulated on the second copy x2 of x, while keeping the first copy x1 unchanged. (The additional variables in the string, which include state names, , and parentheses, prevent the derivation from producing a string of terminals before it has been determined whether T accepts x.)
Productions of type 3 allow everything in the string except x1 to be erased, provided that the computation of T reaches ha
Introduction to Computation 16

17. More General Grammars (cont’d.)
To illustrate the productions of types 2 and 3, consider the sample transition diagram of T below
Productions for the first transition, for example, are q0()  ()q q0(a)  (a)q q0(b)  (b)q1
Productions for the third transition look like
( ρ) q1( a)  q2 ( ρ) ( $), where ,   {a, b, } and ρ is any tape symbol of T. These are more complex because the transition involves a move to the left
Introduction to Computation 17

18. More General Grammars (cont’d.)
We show the moves corresponding to the simulation of T on the input ba. At each step, the underlined portion is the part used in the next production.
q0()(bb)(aa)() ⊢ ()q1(bb)(aa)()
⊢ ()(bb)q1(aa)() ⊢ ()q2(bb)(a$)()
⊢ ()(bb)q2(a$)() ⊢ ()(bb)(a$)q3()
⊢ ()(bb)(a$)ha()
The appearance of ha in the derivation is what will allow everything except x1, the original input string, to disappear
Introduction to Computation

19. More General Grammars (cont’d.)
Productions like (1 2) ha  ha (1 2) ha and
ha (1 2)  ha (1 2) ha allow the copies of ha to “propagate”
Productions like
ha (a 2)  a ha (b 2)  b ha ( 2)  
allow us to eliminate everything but ba, which is accepted
Here is the rest of the derivation:
()(bb)(a$)ha() ⊢ ()(bb)ha(a$)ha()
⊢ () ha(bb) ha(a$)ha() ⊢ ha ()ha(bb)ha(a$)ha()
⊢ ha(bb)ha(a$)ha() ⊢ b ha (a$)ha()
⊢ baha() ⊢ ba
Introduction to Computation

20. Context-Sensitive Languages and the Chomsky Hierarchy
Definition: A context-sensitive grammar (CSG) is an unrestricted grammar in which no production is length-decreasing
In other words, every production is of the form   , where ||  ||
A language is a context-sensitive language (CSL) if it can be generated by a CSG
CSGs cannot have -productions, and CSLs cannot include 
We think of CSLs as a generalization of CFLs
Introduction to Computation 20

21. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
Definition 8.18: A linear-bounded automaton (LBA) is a nondeterministic TM with this exception:
There are two extra tape symbols, [ and ]
The initial configuration of M corresponding to input x is q0[x]
During its computation, M is not permitted to replace either of these brackets or to move its tape head to the left of the [ or to the right of the ]
Introduction to Computation 21

22. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
Theorem 8.19: If L  * is a CSL, then there is an LBA that accepts L
We can follow the proof of Theorem 8.13, except that instead of being able to use the space to the right of the input string on the tape, the LBA must use the space between the two brackets
It can do this by converting individual symbols into symbol pairs so as to simulate two tape “tracks”
Introduction to Computation 22

23. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
If L  * is accepted by a LBA M, then there is a CSG generating L - {}
The proof is similar to that of Theorem 8.14
For details, see book
The four levels of language we have seen so far correspond to the Chomsky Hierarchy
Introduction to Computation 23

24. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
Type
Languages (Grammars)
Form of Productions
Accepting Device 3
Regular
A  aB, A  
Finite
Automaton 2
Context-free
A  
Pushdown Automaton 1
Context-sensitive
  
with ||  ||
LBA
Unrestricted
 
Turing machine
Introduction to Computation 24

25. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
Theorem 8.22: Every CSL is recursive
Proof: Let G be a CSG generating L
Theorem 8.19 says that there is an LBA M accepting L
It suffices to show that there is a nondeterministic TM T1 accepting L such that no input string can possibly cause T1 to loop forever
We may consider M to be a nondeterministic TM T, which begins by inserting the markers [ and ] in the squares where they would be already if we were thinking of T as an LBA
Introduction to Computation 25

26. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
T1 is constructed as a modification of T
T1 also begins by placing the markers on the tape
Just like T, T1 performs the first iteration in a simulated derivation in G by writing S in the first position of the second track
Before the second iteration, however, it moves to the blank portion of the tape after the right marker and records the string S obtained in the first iteration
Introduction to Computation 26

27. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
In each subsequent iteration it performs the following four steps (the first three the same as T)
It selects a production   
It attempts to select an occurrence of  in the current string; if it is unable to, it compares the current string to the input x and accepts if they’re equal, rejects if they’re not
Introduction to Computation 27

28. Context-Sensitive Languages and the Chomsky Hierarchy (cont’d.)
Four steps (cont’d.)
If it can select an occurrence of , it replaces it by , and rejects if this would result in a string longer than the input
If it successfully replaces  by , it compares the new current string with the strings it has written in the portion of the tape to the right of ]; it rejects if there’s a match, and otherwise writes the new string after the most recent entry
Introduction to Computation 28

29. Not Every Language is Recursively Enumerable
We will now consider languages over an alphabet  and TMs with input alphabet 
We will show that there are more languages than TMs to accept them. It follows that there must be many languages not accepted by any TM
The first step is to explain how it makes sense to talk about one infinite set being larger than another
We’ll formulate two definitions
What it means for two sets to be the same size
What it means for one to be larger than another
Introduction to Computation 29

30. Not Every Language is Recursively Enumerable (cont’d.)
For finite sets, this is easy, because we know how to say that one number is equal to or bigger than another.
Definition 8.23: In general, two sets A and B are the same size if there is a bijection f : A  B. A is larger than B if some subset of A is the same size as B but A itself is not
Definition 8.24: A set A is countably infinite if there is a
bijection f : ℕ  A, or a list a0, a1, … of elements of A such that every element of A appears exactly once in the list
A is countable if A is either finite or countably infinite
Introduction to Computation 30

31. Not Every Language is Recursively Enumerable (cont’d.)
Theorem 8.25: Every infinite set has a countably infinite subset, and every subset of a countable set is countable
For proof, see book
Even though the set ℕ  ℕ seems much larger than ℕ, it is countable and therefore the same size as ℕ
We can see this by forming a two-dimensional array with all the ordered pairs (i, j) and starting a spiral path at (0,0) that hits each ordered pair. This is a
way of listing the elements of ℕ  ℕ
Introduction to Computation 31

32. Not Every Language is Recursively Enumerable (cont’d.)
The set 2ℕ is uncountable. The proof uses a diagonalization argument
For every list A0, A1, … of subsets of ℕ, we can define the set A as follows:
A = {n  ℕ | n ∉ An}
Then for every n  ℕ, A ≠ An, because n is an element of A if and only if it is not an element of An
The conclusion is that A cannot be one of the subsets in the list, which means that there can be no list
containing all the subsets of ℕ
Introduction to Computation

33. Not Every Language is Recursively Enumerable (cont’d.)
Here’s the reason we call this argument a diagonal argument:
Subsets of ℕ can be represented by infinite sequences of 0’s and 1’s. For example,
 ↔ …
ℕ ↔ …
{0, 3, 4} ↔ …
{1, 3, 5, …} ↔ …
Let each set Ai in the list be represented as follows:
Ai ↔ ai, 0 ai, 1 ai, 2 ai, 3 …
Introduction to Computation

34. Not Every Language is Recursively Enumerable (cont’d.)
Then the set A was constructed by looking at the diagonal entries (underlined)
A0 ↔ a0,0 a0,1 a0,2 a0,3 a0,4 …
A1 ↔ a1,0 a1,1 a1,2 a1,3 a1,4 …
A2 ↔ a2,0 a2,1 a2,2 a2,3 a2,4 …
A3 ↔ a3,0 a3,1 a3,2 a3,3 a3,4 …
and reversing them. For example, A contains 0 if and
only if A0 doesn’t; i.e., A ↔ a0 a1 a2 a3 …
where a0 is the opposite of a0,0, and in general, ai is the opposite of ai,i
Introduction to Computation

35. Not Every Language is Recursively Enumerable (cont’d.)
Theorem 8.32: Not all languages are recursively enumerable
In fact, the set of languages over {0,1} that are not recursively enumerable is uncountable
We showed that the set of subsets of ℕ is uncountable, and we observed that because {0,1}* is the same size as ℕ, it follows that the set of languages over {0,1} is uncountable
But the set of recursively enumerable languages over {0,1} is countable: each one can be accepted by a TM, and we can list the TMs T by listing the strings e(T) that represent them
Introduction to Computation 35