1. Computational Irreducibility & Emergence
Hervé ZWIRN
CNRS & Paris Diderot University (LIED)
ENS Cachan (CMLA)
IHPST
Histoire et Philosophie de l’Informatique
IHPST le 22 juin 2017
Hervé ZWIRN

2. Computational Irreducibility
Determinism and Impredictibility
A few examples
Cellular automata : general framework (discret)
Hervé ZWIRN

3. Determinism and Impredictibility
Langton’s ant
Rules:
1. If the ant is on a black square, it turns right   and moves forward one unit.
2. If the ant is on a white square, it turns left   and moves forward one unit.
3. When the ant leaves a square, it inverts the color.
Hervé ZWIRN

4. Determinism and Impredictibility
Langton’s ant
Hervé ZWIRN

5. Determinism and Impredictibility
Hervé ZWIRN

6. Determinism and Impredictibility
This is impredictible and nobody knows how to prove that. 
The only way to know what happens is to run a simulation of the steps of the ant and see what happens
Hervé ZWIRN

7. Determinism and Impredictibility
Hervé ZWIRN

8. Determinism and Impredictibility
Rule 30
Hervé ZWIRN

9. Determinism and Impredictibility
Rule 110
Hervé ZWIRN

10. Determinism and Impredictibility
Rule 110
If you want to know what is the 1 000th line, you have to run the simulation and go through the 999 previous steps.
There is no other way, no shortcut allowing to compute the nth line without going through the (n-1) previous steps.
Hervé ZWIRN

11. Determinism and Impredictibility
Conway’s game of life
Any live cell with fewer than two or more than three live neighbours dies.
Any live cell with two or three live neighbours lives on to the next generation.
Any dead cell with exactly three live neighbours becomes a live cell.
Hervé ZWIRN

12. Determinism and Impredictibility
Conway’s game of life
Hervé ZWIRN

13. Determinism and Impredictibility
Conway’s game of life
Glider gun
Hervé ZWIRN

14. Determinism and Impredictibility
Conway’s game of life
Hervé ZWIRN

15. Determinism and Impredictibility
Computational Irreducibility
CIR
The behavior of the system can be found only by direct simulation or observation: No general predictive procedure is possible.
Wolfram S., Undecidability and intractability in theoretical physics, Vol 54, N 8. Phys. Rev. Letters, 1985
Hervé ZWIRN

16. Computational Irreducibility
Predicting :
Knowing the result before the system
computing faster than the system
finding shortcuts
Intuitively a system will be CIR if there is no other way to reach the nth state than to go successively through the (n-1) previous ones. There is no shortcut.
This is not a robust definition since we would like a process to be CIR even if it is possible to reach the nth state by going through (n-1) states that are “near” of the (n-1) previous states of the system if it is still impossible to go directly to it.
Hervé ZWIRN

17. Computational Irreducibility
Giving a rigorous and robust definition is not easy because it is necessary to precise formally which path of (n-1) steps is acceptable as being near the real path followed by the system.
H. Zwirn & J.P. Delahaye , Irreducibility and Computational Equivalence: Wolfram Science 10 Years After the Publication of A New Kind of Science, H. Zenil (Ed), Springer, 2013 
H. Zwirn, Computational Irredudicibility and Computational Analogy, Complex Systems, Vol 24, Issue 2, 2015
H. Zwirn "Les systèmes déterministes simples sont-ils toujours prédictibles"  in Complexité et désordre, Grenoble Sciences Ed., 2015
H. Zwirn “Emergence et irréductibilité computationnelle”, à paraitre dans « Complexité ​et ​ désordre : adaptation, localisation, dynamique », Editions Matériologiques, 2017
Hervé ZWIRN

18. Computational irreducibility
Why is Computational irreducibility (CIR) interesting?
From the algorithmic point of view :
Is there any robust definition?
Do any really CIR processes exist?
Philosophical reasons:
Understanding the behaviour of Complex Systems
Understanding emergent phenomena
Hervé ZWIRN

19. Computational irreducibility
CIR means impossible to speed-up
Is every algorithm ’’speed able’’ ?
The computation model matters
Turing machines with k tapes (k ≥ 2)
Hervé ZWIRN

20. Computational irreducibility
Some speed-up theorems:
The problem of deciding if a string is a palindrome which is O(n2) in the 1-tape Turing machines model and O(n) in the 2-tape Turing machines model.
For any k-tapes Turing machine M operating in time f(n) there exists a k'-tapes Turing machine M' operating in time f'(n)=f(n)+n (where  is an arbitrary small positive constant) which simulates M.
Given any k-tape Turing machine M operating within time f(n), it's possible to construct a 1-tape Turing machine M' operating within time O(f(n)2) and such that for any input x, M(x)=M'(x).
Hervé ZWIRN

21. Computational irreducibility
The computation model
Turing machines with 3 symbols (0, 1, #) k ≥ 2 tapes and one way write only tape which is used for output. We suppose that when the computation ends, the result is the number written at the right end of the output tape.
Given a Turing machine M computing f(n) in time T(M(n)), let's denote by Rn,1, …, Rn,i, …, Rn,T(M(n)) the content of the output tape of M during the computation of f(n) after 1 step of computation, …, i steps of computation and T(M(n)) steps of computation.
Hervé ZWIRN

22. Computational irreducibility
(E-Turing machine): A Turing machine Mf will be called a
E-Turing machine for f if:
(i) Mf computes every f(n)
(ii) during the computation of f(n), there exist increasing kn(i) for i=1 to n-1, such that f(i) is written on the output tape Rn,kn(i)
at the right of the last symbol #
Hervé ZWIRN

23. Computational irreducibility
E-Turing machine for f
f(1)
n-1
f(2)
f(n-2)
f(1) n
f(2)
f(n-1)
kn-1 (1)
kn (1)
kn-1 (2)
kn (2)
f(n-2)
kn (n-2)
kn-1 (n-2)
kn (n-1)
kn-1 (n-1)
f(n-1)
f(n)
kn (n)
Hervé ZWIRN

24. Computational irreducibility
Tentative definition (CIR):
A function f will be said CIR if and only if any Turing machine computing every f(n), is a E-Turing machine for f.
Not a Robust Definition
Need for more sophisticated concepts
Hervé ZWIRN

25. Computational irreducibility
(Asymptotically optimal Turing machine): We will say that a Turing machine Mf* for f is an asymptotically optimal Turing machine for f if for any other Turing machine M computing f:
T(Mf*(n)) = O(T(M(n)))
i.e. there are constants c > 0, n0 > 0 such that n > n0, T(Mf* (n))  cT(M(n)).
Asymptotically, no other Turing machine computing f computes faster than Mf*
We assume that it is the case
Hervé ZWIRN

26. Computational irreducibility
(Efficient E-Turing machine): We will say that a E-Turing machine Mfeff for f is an efficient E-Turing machine for f if for any other E-Turing machine Mf for f:
T(Mfeff(n)) = O(T(Mf(n)))
i.e. there are constants c > 0, n0 > 0 such that n > n0, T(Mfeff (n))  cT(Mf(n)).
Asymptotically, no other E-Turing machine for f computes faster than Mfeff
We assume that it is the case
Hervé ZWIRN

27. Computational irreducibility
A Turing Machine M will be said to be a P‑approximation of a E‑Turing machine for f if and only if there are a function F such that F(n)=O(T(Mf* (n)/n)) and a Turing machine P such that:
(i) on input n, M computes a result rn such that P computes f(n) from rn in a number of steps F(n) and halts.
(ii) during the computation, there exist increasing kn(i) for i=1 to n-1, such that P computes f(i) from i and Rn,kn(i) in a number of steps F(i) and halts.
Hervé ZWIRN

28. Computational irreducibility
P‑approximation of a E‑Turing machine for f
f(1) n
f(2)
f(n-1) P r1 n r2
rn-1
kn (1)
kn (2)
kn (n-1)
O(T(Mf*(n)/n))
f(n) rn
kn (n)
Hervé ZWIRN

29. Computational irreducibility
Let M be a P‑approximation of a E‑Turing machine for f.
Computation of f(n) based on the P‑approximation M:
The computation of f(n) done initially through M with input n and continued when M has computed rn, by P which computes f(n) from n and rn in a time F(n) and halts.
Hervé ZWIRN

30. Computational irreducibility
Computation based on a P‑approximation of a E‑Turing machine for f
f(1)
f(2)
f(n-1) P r1 n r2
rn-1
O(T(Mf* (n)/n))
f(n) rn
Hervé ZWIRN

31. Computational irreducibility
Strongly CIR (resp CIR) function:
A function f(n) from N to N will be said to be strongly CIR (resp CIR) if and only if for any Turing machine M computing every f(n) there is a P-approximation of a E‑Turing machine for f, M’, such that for every n (resp. for infinitely many n), the computation of f(n) by M is based on M’.
If a function is strongly CIR, for each n there is no other way to compute f(n) than to compute before all the values f(i) for i<n (or values that are near in the sense given in the definition of the approximation of a E-Turing machine). There is no shortcut allowing to get directly the value of f(n) without having computed before f(n-1) or a value that is near f(n-1) and so forth for the previous values.
Hervé ZWIRN

32. Computational irreducibility
Theorem
if f is CIR no Turing machine computing every f(n) can compute f(n) faster than an efficient E­-Turing machine for f.
More precisely, if Mf is a Turing machine computing every f(n) and if f is CIR then T(Mfeff(n)) = O(T(Mf(n))).
Hervé ZWIRN

33. Computational irreducibility
Possible candidates:
Langton’s ant
Rule 110
The number of configurations of index < n still alive after n steps in the game of life
The function f défined as :
f(1) = the first digit of 
f(n) = the digit of  after having skipped f(n-1) digits from the digit f(n-1)
Open problem:
Prove that any of the above possible candidates is CIR.
Hervé ZWIRN

34. Emergence What is necessary for Emergence? 2 levels:
individual / collectif
micro / macro
Knowledge of the rules for the low level
Apparent irreducibility of the phenomenon appearing at the upper level to the low level rules
Hervé ZWIRN

35. Emergence appears when the dynamics of the low level is CIR
Objective (weak) emergence
Objective emergence should be independant of our human capacities.
Non-epistemic criterion.
Emergence appears when the dynamics of the low level is CIR
Hervé ZWIRN