1. Module 5 Topics Proof of the existence of unsolvable problems
Proof Technique
There are more problems/languages than there are programs/algorithms
Countable and uncountable infinities

2. Overview We will show that there are more problems than programs
Actually more problems than programs in any computational model (programming language)
Implication
Some problems are not solvable

3. Define set of problems Observation about programs
Preliminaries
Define set of problems
Observation about programs

4. Define set of problems
We will restrict the set of problems to be the set of language recognition problems over the alphabet {a}.
That is
Universe: {a}*
Yes Inputs: Some language L subset of {a}*
No Inputs: {a}* - L

5. Set of Problems *
The number of distinct problems is given by the number of languages L subset of {a}*
2{a}* is our shorthand for this set of subset languages
Examples of languages L subset of {a}*
0 elements: { }
1 element: {/\}, {a}, {aa}, {aaa}, {aaaa}, …
2 elements: {/\, a}, {/\, aa}, {a, aa}, …
Infinite # of elements: {an | n is even}, {an | n is prime}, {an | n is a perfect square}

6. Infinity and {a}* All strings in {a}* have finite length
The number of strings in {a}* is infinite
The number of languages L in 2{a}* is infinite
The number of strings in a language L in 2{a}* may be finite or infinite

7. Define set of programs
The set of programs we will consider are the set of legal C++ programs as defined in earlier lectures
Key Observation
Each C++ program can be thought of as a finite length string over alphabet SP
SP = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

8. Example * int main(int A[], int n){ {26 characters including newline}
int i, max; {13 characters including initial tab}
{1 character: newline}
if (n < 1) {12 characters}
return (“Illegal Input”); {28 characters including 2 tabs}
max = A[0]; {13 characters}
for (i = 1; i < n; i++) {25 characters}
if (A[i] > max) {18 characters}
max = A[i]; {15 characters}
return (max); {15 characters}
} {2 characters including newline}

9. Number of programs The set of legal C++ programs is clearly infinite
It is also no more than |SP*|
SP = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

10. Goal
Show that the number of languages L in 2{a}* is greater than the number of strings in SP*
SP = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}
Problem
Both are infinite

11. How do we compare the relative sizes of infinite sets?
Bijection (yes)
Proper subset (no)

12. Bijections
Two sets have EQUAL size if there exists a bijection between them
bijection is a 1-1 and onto function between two sets
Examples
Set {1, 2, 3} and Set {A, B, C}
Positive even numbers and positive integers

13. Bijection Example Positive Integers Positive Even Integers 1 2 2 4 3 6
1 2
2 4
3 6
i 2i …

14. Proper subset Finite sets Infinite sets
S1 proper subset of S2 implies S2 is strictly bigger than S1
Example
women proper subset of people
number of women less than number of people
Infinite sets
Counterexample
even numbers and integers

15. Countable Uncountable
Two sizes of infinity
Countable
Uncountable

16. Countably infinite set S *
Definition 1
S is equal in size (bijection) to N
N is the set of natural numbers {1, 2, 3, …}
Definition 2 (Key property)
There exists a way to list all the elements of set S (enumerate S) such that the following is true
Every element appears at a finite position in the infinite list

17. Uncountable infinity *
Any set which is not countably infinite
Examples
Set of real numbers
2{a}*, the set of all languages L which are a subset of {a}*
Further gradations within this set, but we ignore them

18. Proof

19. (1) The set of all legal C++ programs is countably infinite
Every C++ program is a finite string
Thus, the set of all legal C++ programs is a language LC
This language LC is a subset of SP*

20. For any alphabet S, * is countably infinite
Enumeration ordering
All length 0 strings
|S|0 = 1 string: l
All length 1 strings
|S| strings
All length 2 strings
|S|2 strings …
Thus, SP* is countably infinite

21. Example with alphabet {a,b} *
Length 0 strings
0 and l
Length 1 strings
1 and a, 2 and b
Length 2 strings
3 and aa, 4 and ab, 5 and ba, 6 and bb, ...
Question
write a program that takes a number as input and computes the corresponding string as output

22. (2) The set of languages in 2{a}* is uncountably infinite
Diagonalization proof technique
“Algorithmic” proof
Typically presented as a proof by contradiction

23. Algorithm Overview *
To prove this set is uncountably infinite, we construct an algorithm D that behaves as follows:
Input
A countably infinite list of languages L[] subset of {a}*
Output
A language D(L[]) which is a subset of {a}* that is not on list L[]
Declaration of algorithm D?

24. Visualizing D List L[] L[0] L[1] L[2] L[3] Language D(L[]) not in
...
Language
D(L[]) not in
list L[]
Algorithm D

25. Why existence of D implies result
If the number of languages in 2{a}* is countably infinite, there exists a list L[] s.t.
L[] is complete
it contains every language in 2{a}*
L[] is countably infinite
The existence of algorithm D implies that no list of languages in 2{a}* is both complete and countably infinite
Specifically, the existence of D shows that any countably infinite list of languages is not complete

26. Visualizing One Possible L[ ] *aa
aaa
aaaa
...
#Rows is countably infinite
Given
#Cols is countably infinite
{a}* is countably infinite
L[0] IN
L[1]
OUT IN
L[2]
OUT
L[3] IN
OUT
L[4] IN
OUT
...
Consider each string to be a feature
A set contains or does not contain each string

27. Constructing D(L[ ]) *
We construct D(L[]) by using a unique feature (string) to differentiate D(L[]) from L[i]
Typically use ith string for language L[i]
Thus the name diagonalization
D(L[]) l a aa
aaa
aaaa
...
L[0]
OUT IN IN IN IN IN
L[1] IN
OUT IN IN
OUT IN
L[2]
OUT
OUT
OUT IN
OUT
OUT
L[3]
OUT IN IN
OUT IN
OUT
L[4]
OUT IN
OUT
OUT
OUT IN
...

28. Questions * Do we need to use the diagonal?aa
aaa
aaaa
...
L[0] IN IN IN IN IN
L[1] IN
OUT IN
OUT IN
L[2]
OUT
OUT
OUT
OUT
OUT
L[3]
OUT IN IN
OUT
OUT
L[4]
OUT IN
OUT
OUT IN
...
Do we need to use the diagonal?
Every other column and every row?
Every other row and every column?
What properties are needed to construct D(L[])?

29. Visualization All problems Solvable Problems
The set of solvable problems is a proper subset of the set of all problems.

30. Summary Equal size infinite sets: bijections
Countable and uncountable infinities
More languages than algorithms
Number of algorithms countably infinite
Number of languages uncountably infinite
Diagonalization technique
Construct D(L[]) using infinite set of features
The set of solvable problems is a proper subset of the set of all problems