INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011

Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, , q 0, q accept, q reject ), where: Q is a finite set of states Γ is the tape alphabet, where   Γ and Σ  Γ q 0  Q is the start state Σ is the input alphabet, where   Σ  : Q  Γ → Q  Γ  {L,R} q accept  Q is the accept state q reject  Q is the reject state, and q reject  q accept

The language L is Turing-recognizable or recursively enumerable if some TM accepts all and only the strings in L. A language is called decidable or recursive if some TM accepts all the strings in L and rejects all the strings not in L. recursive languages r.e. languages

THE CHURCH-TURING THESIS L is recognized by a program for some computer* ⇔ L is recognized by a TM * The computer must be “reasonable”

The Church-Turing Thesis is “consistent” with all known “reasonable” computers. SCRATCH: R1: R2:  RAM: 1111… … … #1011# # #...# Programs for a computer have instructions like ADD R1, R2, R3; LOAD R1, R2; STORE R1,R2; MUL R1, R2, R2; BRANCH R1, X;…

Programming languages like Java, Python, Scheme, C, … are equivalent to TMs We call such languages Turing complete. Corollary. If two programming languages are Turing complete, then they can recognize exactly the same set of languages.

We can encode a TM as a string of 0s and 1s… 0 n 10 m 10 k 10 s 10 t 10 r 10 u 1… n states m tape symbols (first k are input symbols) start state accept state reject state blank symbol 〈 (p,a), (q,b,L) 〉 = 0 p 10 a 10 q 10 b 10

Since TMs and other languages are equivalent we can also express TMs as programs. Since programs are strings, we can consider languages whose elements are programs. We let 〈 M 〉 denote the encoding of TM M. Theorem. We can make a Universal TM, a TM that takes any 〈 M 〉 and any string w as input and simulates the computation of M on w.

Similarly, we can encode DFAs, NFAs, CFGs, etc into strings of 0s and 1s A DFA = { 〈 B,w 〉 | B is a DFA that accepts string w } A NFA = { 〈 N,w 〉 | N is an NFA that accepts string w } A CFG = { 〈 G,w 〉 | G is a CFG that generates string w } So we can define the following languages:

A DFA = { 〈 B,w 〉 | B is a DFA that accepts string w } Theorem: A DFA is decidable Proof Idea: Simulate B on w A CFG = { 〈 G,w 〉 | G is a CFG that generates string w } Theorem: A CFG is decidable Proof Idea: Transform G into CNF. Try all derivations of length 2|w|-1 Corollary: A NFA is decidable

E DFA = { 〈 B 〉 | B is a DFA and L(B) = Ø} Theorem: E DFA is decidable Proof Idea: Let B = (Q, ,δ,q 0,F) and L(B)=Ø. 1.Set MARK := {q 0 } 2.Repeat until MARK is unchanged: MARK := MARK ∪ δ(MARK,Σ) 3.If F ∩ MARK = Ø, accept, else reject. ALL DFA = { 〈 B 〉 | B is a DFA and L(B) = Σ*} Theorem: ALL DFA is decidable Proof Idea: If L(B) = Σ* then L(¬B) = Ø

INFINITE DFA = { 〈 B 〉 | B is a DFA and L(B) is infinite} Theorem: INFINITE DFA is decidable Proof Idea: Let B = (Q, ,δ,q 0,F). Suppose ∃ w ∈ L(B) : |w| ≥ |Q|. 1.Let C be a DFA for {w : |w| ≥ |Q|}. 2.Build a DFA D for L(C) ∩ L(B). 3.If D ∈ E DFA then reject, else accept. FINITE DFA = { 〈 B 〉 | B is a DFA and L(B) is finite} Corollary: FINITE DFA is decidable

The language L is undecidable if there is no TM that decides L. If L is undecidable, then every TM must either: 1. Accept (infinitely many) strings s ∉ L. 2. Reject (infinitely many) strings s ∈ L. 3. Loop forever on (infinitely many) strings.

UN DECIDABILITY We will prove that there are some undecidable languages – i.e., problems that a program can’t solve no matter how long it computes. The proof is “simple:” There are more languages than there are Turing Machines.

POP QUIZ Let ℕ = {0,1,2,…} be the Natural Numbers. Let = {0,2,4,6,…} be the Even Numbers. Let ℤ = {…,-2,-1,0,1,2,…} be the Integers. Which is biggest, | ℕ |, ||, or | ℤ |?

Are there more blue or yellow dots? ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

SET THEORY 101 A function ƒ : A → B is: 1-1 (or injective) if onto (or surjective) if bijective if it is 1-1 and onto. ∀ x ∈ A : ƒ(x) is unique. ∀ y ∈ B : y ∈ ƒ(A) 1-1 then |A| |B| onto then |A| |B| bijective then |B| |A|. ƒ can help us count. If ƒ is: ≤ ≥ =

COUNTABLE SETS We say a set Y is countable if |Y| ≤ | ℕ |. Some countable sets: = {0,2,4,6,…} = {1,3,5,7,…} SQUARES = {0,1,4,9,16,25…} |POWERS| = |SQUARES| = || = || = | ℕ | POWERS = {1,2,4,8,16,32…} Ø, {0}, {0,1}, {0,1, …, 255}

There exists a bijection between ℕ and ℕ ℕ. (0,0) (0,1) (0,2) (0,3) (0,4) … (1,0) (1,1) (1,2) (1,3) (1,4) … (2,0) (2,1) (2,2) (2,3) (2,4) … (3,0) (3,1) (3,2) (3,3) (3,4) … (4,0) (4,1) (4,2) (4,3) (4,4) …

{0,1}* is countable { 〈 M 〉 | M is a TM } is countable ℚ + = { p/q | p,q ∈ ℤ + } is countable! Is any set uncountable?

Theorem: There is no onto function from the positive integers to the real interval (0,1) 12345:12345: … … … … … : Proof:Suppose ƒ is such a function: [ n-th digit of r ] = if [ n-th digit of f(n) ]  1 2otherwise For all n, ƒ(n)  r

The process of constructing a counterexample by “contradicting the diagonal” is called DIAGONALIZATION

Prove that |(-1,1)| = |(0,1)| = | ℝ | Prove that |ℝ ℝ| = ℝ Is there a set bigger than ℝ ?

Let L be any set and (L) be the power set of L Theorem: There is no onto map from L to (L) Proof: Assume, for a contradiction, that there is an onto map ƒ : L  (L) Let D ƒ = { x  L | x  ƒ(x) } If D ƒ = ƒ(y) then y  D ƒ if and only if y  D ƒ We construct a set D ƒ that cannot be the image, ƒ(y) for any y  L. D ƒ is called the diagonal set for ƒ.

How is that diagonalization? x y 1 ∈ f(x)?y 2 ∈ f(x)?y 3 ∈ f(x)?y 4 ∈ f(x)? … y1y1 YNYY y2y2 NYNY y3y3 NNNN y4y4 YNNY  Y Y Y N (y i ∈ D) = Y iff (y i ∈ f(y i )) = N

EXAMPLE Let L = {0,1,2}. Then (L) = {Ø, {0}, {1}, {2}, {0,1}, {0,2}, {1,2}, {0,1,2}} Let ƒ(0) = {1}, ƒ(1) = Ø, ƒ(2) = {0,2}. Then: x 0 ∈ ƒ(x)?1 ∈ ƒ(x)?2 ∈ ƒ(x)? 0NYN 1NNN 2YNY D ƒ ={0,1} Characteristic sequence for ƒ(2)={0,2}

No matter what, (L) always has more elements than L

Not all languages over {0,1} are decidable Turing Machines Strings of 0s and 1s Sets of strings of 0s and 1s Languages over {0,1} L(L)