1. Algorithmic Complexity and Random Strings
Manfred Denker
Wojbor A. Woyczyński
Presentation for STAT433
Lawrence Leinweber

2. Algorithmic Complexity
ran•dom adj.
Having no specific pattern or objective
Statistics
Does not produce the same outcome every time
Relative frequency approaches a stable limit as the number of events increases
Each member has an equal chance
1. Is related to algorithmic complexity, 2. (a) is easy (b) equipartitioning (c) too narrow
4/18/07
Algorithmic Complexity

3. Algorithmic Complexity
Examples from Chapter 1
(a) Is obviously bad (b) is not good (c,d,e) look promising
4/18/07
Algorithmic Complexity

4. Algorithmic Complexity
Examples in Blocks
(b,c) are repetitive blocks, (d) looks suspicious, (e) is really random, by computer
4/18/07
Algorithmic Complexity

5. Mathematical Randomness
Martin-Löf Random
No features that make the string stand out from typical strings
Von Mises Random
Frequencies of 0s and 1s are stable
Kolmogorov Random
The string has a complex minimal description
1. Is very broad, 2. Equipartitioning, 3. Gets to the issue of descriptions
4/18/07
Algorithmic Complexity

6. Examples in Descriptions
Always “1”
“10” Repeated Forever
“ ” Repeated Forever
Whole Numbers Expressed in Binary
“ ”
a) Describes most succinctly, (b,c) are the same but (c) has a longer block description, (d) is interesting: short but sophisticated (e) has no shorter description than itself, but then no description requires much more than that
4/18/07
Algorithmic Complexity

7. Algorithmic Complexity
The Minimum Cost Algorithm to Solve a Problem or Express a Result
A Measure of the Difficulty in Time and Storage Space
Not Practical to Measure: 1, n, n2, en
Leads to Paradoxes
4/18/07
Algorithmic Complexity

8. Algorithmic Complexity
Algorithms as Data
John von Neumann
Von Neumann Architecture
Programs and Data in the Same Kind of Memory
Compilers Translate Programs
Treat Programs as Data
A Suitable Machine Can Read a Text Description of an Algorithm then Perform the Algorithm’s Actions
4/18/07
Algorithmic Complexity

9. Algorithmic Complexity
Turing Machine
Machine Consists of:
A Tape and a Current Position on the Tape
A Program and a Current Step in the Program
Machine Can Run Each Step of the Program
Depending on the Program Step and Reading the Tape:
Move the Position on the Tape
Write the Tape
Change the Step in the Program
Halt
Alan Turing
4/18/07
Algorithmic Complexity

10. Elements of a Turing Machine
Tape
Storage Device
Input / Output
Program Executes:
In Discrete Steps
Loops
Decisions
4/18/07
Algorithmic Complexity

11. Computational Complexity
Not Algorithmic Complexity
Measure of the Execution Time per n Input Items
Big-O Notation: Sorting Requires O(n log n) Time
t: t < k n log n, k, n
P Class – Polynomial Time: O(nm), m
NP Class – Non-Polynomial: O(en), “Intractable”
4/18/07
Algorithmic Complexity

12. Universal Turing Machine
A Turing Machine, Running Any Particular Program, Can Be Simulated on a Universal Turing Machine, by Inputting an Encoding of That Program
Model of a General Purpose Computer
Programs Can Run on Any Suitable “Platform”
4/18/07
Algorithmic Complexity

13. Halting Problem (Simplified)
Does Program p Halt or Does It Loop Forever?
H(p) Begin If p halts then H = true else H = false End
Try This Program:
Z(p) Begin If H(p) then loop forever else Z = false End
If Z(Z) loops, H(Z) = true,  Z halts
If Z(Z) halts, H(Z) = false,  Z loops
4/18/07
Algorithmic Complexity

14. Algorithmic Complexity
Gödel Numbering
Function That Assigns a Unique Natural Number to Each Symbol and Formula
A = { A, … , Z, 0, … , 9, , , , , … }
Code(n) Begin While n > 0 Begin n = n – 1; Code = Code + A(n mod |A| + 1); n = n  |A| End End
Gödel Numbering of Code(n) is n
Kurt Gödel
4/18/07
Algorithmic Complexity

15. Kolmogorov Complexity
Andrey Kolmogorov
Kolmogorov Compexity of a Number is the Length of the Shortest Description of the Number
Shortdef(i) Begin n = 0; While UTM(Shortdef)  i Begin n = n + 1; ShortDef = Code(n) End End
Kolmogorov Complexity of i is |Shortdef(i)|
4/18/07
Algorithmic Complexity

16. Richard-Berry Paradox
“The smallest number that cannot be defined in less than twenty words.” = 12 words
Lilbigdef(i) Begin Lilbigdef = 0; While |Shortdef(Lilbigdef)| < i Begin Lilbigdef = Lilbigdef + 1 End End
Lilbigdef(1000) Is Defined in < 1000 Symbols
4/18/07
Algorithmic Complexity

17. Gödel Incompleteness Theorem
Turing’s Halting Problem – 1936
Some Problems Are Undecidable
Gödel’s Incompleteness Theorem – 1931
In a Mathematical System, Some True Statements about the System That Cannot Be Proven from the System’s Axioms
4/18/07
Algorithmic Complexity

18. The Fall of Determinism
Heisenberg Uncertainty Principle – 1927
Can’t Observe Electron’s Position, Momentum Simultaneously
Einstein’s Theory of Special Relativity – 1905
No Simultaneity of Frames of Reference
Michelson & Morley – 1887
No Ether Medium for Electromagnetic Waves
4/18/07
Algorithmic Complexity

19. Algorithmic Complexity
Summary
Random Phenomena Can Be Characterized by Their Algorithmic Complexity
The Minimum Length Algorithm to Describe a Number or Result
Leads to Paradoxes
Algorithmic Complexity is Related to Important Developments in the History of Computers
4/18/07
Algorithmic Complexity