1. 1 Lecture 6 Topics –Proof of the existence of unsolvable problems Problems/languages not in REC Proof Technique –There are more problems/languages than programs/algorithms –Countable and uncountable infinities

2. 2 Overview We will show that there are more languages than algorithms –Actually more languages than programs in any computational model (programming language) Implication –Some problems are not solvable

3. 3 How do we compare the relative sizes of infinite sets? Bijection (yes) Proper subset (no)

4. 4 Bijections Two sets have EQUAL size if there exists a bijection between them –bijection is a 1-1 and onto function between two sets Examples –Set {1, 2, 3} and Set {A, B, C} –Positive even numbers and positive integers

5. 5 Bijection Example Positive Integers Positive Even Integers 12 24 36...... i2i …...

6. 6 Proper subset Finite sets –S1 proper subset of S2 implies S2 is strictly bigger than S1 Example –women proper subset of people –number of women less than number of people Infinite sets –Counterexample even numbers and integers

7. 7 Two sizes of infinity Countable Uncountable

8. 8 Countably infinite set S Definition 1 –S is equal in size (bijection) to N N is the set of natural numbers {1, 2, 3, …} Definition 2 (Key property) –There exists a way to list all the elements of set S (enumerate S) such that the following is true Every element appears at a finite position in the infinite list

9. 9 Uncountable infinity Any set which is not countably infinite Examples –Set of real numbers –Set of all languages Further gradations within this set, but we ignore them

10. 10 The set of all algorithms is countably infinite Every algorithm can be represented as a C ++ program –Church’s Thesis Every C ++ program is a finite string Thus, the set of all legal C ++ programs is a language L C This language L C is a subset of  

11. 11  * is countably infinite Enumeration ordering –All length 0 strings |  | 0 = 1 string: –All length 1 strings |  | strings –All length 2 strings |  | 2 strings –...

12. 12 The set of all languages is uncountably infinite Diagonalization proof technique –Proof by contradiction

13. 13 High Level Proof Assume that the set is countably infinite This implies there exists a list L where –Every language is L i for some number i –No other assumptions about L Using L, we construct a new language D which is not on L –Contradiction: L cannot exist Thus the uncountably infinite result follows

14. 14 Representing L L0L0 L1L1 L2L2 L3L3 L4L4... 010001... IN OUTIN OUT IN OUT IN OUT #Rows is countably infinite By assumption #Cols is countably infinite  * is countably infinite Consider each string to be a feature –A set contains or does not contain each string

15. 15 Constructing D We construct D by using a unique feature (string) to differentiate D from L i –Typically use ith string for language L i –Thus the name diagonalization L0L0 L1L1 L2L2 L3L3 L4L4... 010001... IN OUTIN OUT IN OUT IN OUT IN OUT D

16. 16 Questions Do we need to use the diagonal? –Every other column and every row? –Every other row and every column? What properties are needed to construct D? L0L0 L1L1 L2L2 L3L3 L4L4... 010001... IN OUTIN OUT IN OUT IN OUT

17. 17 Languages REC REC is a proper subset of the set of all languages.

18. 18 Summary Equal size infinite sets: bijections –Countable and uncountable infinities More languages than algorithms –Number of algorithms countably infinite –Number of languages uncountably infinite –Diagonalization technique Construct D using infinite set of features REC is a proper subset of the set of all languages