Algorithmic Complexity and Random Strings Manfred Denker Wojbor A. Woyczyński Presentation for STAT433 Lawrence Leinweber

Algorithmic Complexity ran•dom adj. Having no specific pattern or objective Statistics Does not produce the same outcome every time Relative frequency approaches a stable limit as the number of events increases Each member has an equal chance 1. Is related to algorithmic complexity, 2. (a) is easy (b) equipartitioning (c) too narrow 4/18/07 Algorithmic Complexity

Algorithmic Complexity Examples from Chapter 1 1111111111111111111111111 10101010101010101010101010 10010011100100111001001110010011 011011100101110111100010011010 10111001011111001000000110101001 (a) Is obviously bad (b) is not good (c,d,e) look promising 4/18/07 Algorithmic Complexity

Algorithmic Complexity Examples in Blocks 1111111111111111111111111 10 10 10 10 10 10 10 10 10 10 10 10 10 10010011 10010011 10010011 10010011 0 1 10 11 100 101 110 111 1000 1001 1010 10111001011111001000000110101001 (b,c) are repetitive blocks, (d) looks suspicious, (e) is really random, by computer 4/18/07 Algorithmic Complexity

Mathematical Randomness Martin-Löf Random No features that make the string stand out from typical strings Von Mises Random Frequencies of 0s and 1s are stable Kolmogorov Random The string has a complex minimal description 1. Is very broad, 2. Equipartitioning, 3. Gets to the issue of descriptions 4/18/07 Algorithmic Complexity

Examples in Descriptions 1111111111111111111111111 Always “1” 10 10 10 10 10 10 10 10 10 10 10 10 10 “10” Repeated Forever 10010011 10010011 10010011 10010011 “10010011” Repeated Forever 0 1 10 11 100 101 110 111 1000 1001 1010 Whole Numbers Expressed in Binary 10111001011111001000000110101001 “10111001011111001000000110101001” a) Describes most succinctly, (b,c) are the same but (c) has a longer block description, (d) is interesting: short but sophisticated (e) has no shorter description than itself, but then no description requires much more than that 4/18/07 Algorithmic Complexity

Algorithmic Complexity The Minimum Cost Algorithm to Solve a Problem or Express a Result A Measure of the Difficulty in Time and Storage Space Not Practical to Measure: 1, n, n2, en Leads to Paradoxes 4/18/07 Algorithmic Complexity

Algorithmic Complexity Algorithms as Data John von Neumann Von Neumann Architecture Programs and Data in the Same Kind of Memory Compilers Translate Programs Treat Programs as Data A Suitable Machine Can Read a Text Description of an Algorithm then Perform the Algorithm’s Actions 4/18/07 Algorithmic Complexity

Algorithmic Complexity Turing Machine Machine Consists of: A Tape and a Current Position on the Tape A Program and a Current Step in the Program Machine Can Run Each Step of the Program Depending on the Program Step and Reading the Tape: Move the Position on the Tape Write the Tape Change the Step in the Program Halt Alan Turing 4/18/07 Algorithmic Complexity

Elements of a Turing Machine Tape Storage Device Input / Output Program Executes: In Discrete Steps Loops Decisions 4/18/07 Algorithmic Complexity

Computational Complexity Not Algorithmic Complexity Measure of the Execution Time per n Input Items Big-O Notation: Sorting Requires O(n log n) Time t: t < k n log n, k, n P Class – Polynomial Time: O(nm), m NP Class – Non-Polynomial: O(en), “Intractable” 4/18/07 Algorithmic Complexity

Universal Turing Machine A Turing Machine, Running Any Particular Program, Can Be Simulated on a Universal Turing Machine, by Inputting an Encoding of That Program Model of a General Purpose Computer Programs Can Run on Any Suitable “Platform” 4/18/07 Algorithmic Complexity

Halting Problem (Simplified) Does Program p Halt or Does It Loop Forever? H(p) Begin If p halts then H = true else H = false End Try This Program: Z(p) Begin If H(p) then loop forever else Z = false End If Z(Z) loops, H(Z) = true,  Z halts If Z(Z) halts, H(Z) = false,  Z loops 4/18/07 Algorithmic Complexity

Algorithmic Complexity Gödel Numbering Function That Assigns a Unique Natural Number to Each Symbol and Formula A = { A, … , Z, 0, … , 9, , , , , … } Code(n) Begin While n > 0 Begin n = n – 1; Code = Code + A(n mod |A| + 1); n = n  |A| End End Gödel Numbering of Code(n) is n Kurt Gödel 4/18/07 Algorithmic Complexity

Kolmogorov Complexity Andrey Kolmogorov Kolmogorov Compexity of a Number is the Length of the Shortest Description of the Number Shortdef(i) Begin n = 0; While UTM(Shortdef)  i Begin n = n + 1; ShortDef = Code(n) End End Kolmogorov Complexity of i is |Shortdef(i)| 4/18/07 Algorithmic Complexity

Richard-Berry Paradox “The smallest number that cannot be defined in less than twenty words.” = 12 words Lilbigdef(i) Begin Lilbigdef = 0; While |Shortdef(Lilbigdef)| < i Begin Lilbigdef = Lilbigdef + 1 End End Lilbigdef(1000) Is Defined in < 1000 Symbols 4/18/07 Algorithmic Complexity

Gödel Incompleteness Theorem Turing’s Halting Problem – 1936 Some Problems Are Undecidable Gödel’s Incompleteness Theorem – 1931 In a Mathematical System, Some True Statements about the System That Cannot Be Proven from the System’s Axioms 4/18/07 Algorithmic Complexity

The Fall of Determinism Heisenberg Uncertainty Principle – 1927 Can’t Observe Electron’s Position, Momentum Simultaneously Einstein’s Theory of Special Relativity – 1905 No Simultaneity of Frames of Reference Michelson & Morley – 1887 No Ether Medium for Electromagnetic Waves 4/18/07 Algorithmic Complexity

Algorithmic Complexity Summary Random Phenomena Can Be Characterized by Their Algorithmic Complexity The Minimum Length Algorithm to Describe a Number or Result Leads to Paradoxes Algorithmic Complexity is Related to Important Developments in the History of Computers 4/18/07 Algorithmic Complexity