Computational Irreducibility & Emergence Hervé ZWIRN CNRS & Paris Diderot University (LIED) ENS Cachan (CMLA) IHPST Histoire et Philosophie de l’Informatique IHPST le 22 juin 2017 Hervé ZWIRN

Computational Irreducibility Determinism and Impredictibility A few examples Cellular automata : general framework (discret) Hervé ZWIRN

Determinism and Impredictibility Langton’s ant Rules: 1. If the ant is on a black square, it turns right   and moves forward one unit. 2. If the ant is on a white square, it turns left   and moves forward one unit. 3. When the ant leaves a square, it inverts the color. Hervé ZWIRN

Determinism and Impredictibility Langton’s ant Hervé ZWIRN

Determinism and Impredictibility Hervé ZWIRN

Determinism and Impredictibility This is impredictible and nobody knows how to prove that.  The only way to know what happens is to run a simulation of the 10 000 steps of the ant and see what happens Hervé ZWIRN

Determinism and Impredictibility Hervé ZWIRN

Determinism and Impredictibility Rule 30 Hervé ZWIRN

Determinism and Impredictibility Rule 110 Hervé ZWIRN

Determinism and Impredictibility Rule 110 If you want to know what is the 1 000th line, you have to run the simulation and go through the 999 previous steps. There is no other way, no shortcut allowing to compute the nth line without going through the (n-1) previous steps. Hervé ZWIRN

Determinism and Impredictibility Conway’s game of life Any live cell with fewer than two or more than three live neighbours dies. Any live cell with two or three live neighbours lives on to the next generation. Any dead cell with exactly three live neighbours becomes a live cell. Hervé ZWIRN

Determinism and Impredictibility Conway’s game of life   Hervé ZWIRN

Determinism and Impredictibility Conway’s game of life Glider gun Hervé ZWIRN

Determinism and Impredictibility Conway’s game of life Hervé ZWIRN

Determinism and Impredictibility Computational Irreducibility CIR The behavior of the system can be found only by direct simulation or observation: No general predictive procedure is possible. Wolfram S., Undecidability and intractability in theoretical physics, Vol 54, N 8. Phys. Rev. Letters, 1985 Hervé ZWIRN

Computational Irreducibility Predicting : Knowing the result before the system computing faster than the system finding shortcuts Intuitively a system will be CIR if there is no other way to reach the nth state than to go successively through the (n-1) previous ones. There is no shortcut. This is not a robust definition since we would like a process to be CIR even if it is possible to reach the nth state by going through (n-1) states that are “near” of the (n-1) previous states of the system if it is still impossible to go directly to it. Hervé ZWIRN

Computational Irreducibility Giving a rigorous and robust definition is not easy because it is necessary to precise formally which path of (n-1) steps is acceptable as being near the real path followed by the system. H. Zwirn & J.P. Delahaye , Irreducibility and Computational Equivalence: Wolfram Science 10 Years After the Publication of A New Kind of Science, H. Zenil (Ed), Springer, 2013  H. Zwirn, Computational Irredudicibility and Computational Analogy, Complex Systems, Vol 24, Issue 2, 2015 H. Zwirn "Les systèmes déterministes simples sont-ils toujours prédictibles"  in Complexité et désordre, Grenoble Sciences Ed., 2015 H. Zwirn “Emergence et irréductibilité computationnelle”, à paraitre dans « Complexité ​et ​ désordre : adaptation, localisation, dynamique », Editions Matériologiques, 2017 Hervé ZWIRN

Computational irreducibility Why is Computational irreducibility (CIR) interesting? From the algorithmic point of view : Is there any robust definition? Do any really CIR processes exist? Philosophical reasons: Understanding the behaviour of Complex Systems Understanding emergent phenomena Hervé ZWIRN

Computational irreducibility CIR means impossible to speed-up Is every algorithm ’’speed able’’ ? The computation model matters Turing machines with k tapes (k ≥ 2) Hervé ZWIRN

Computational irreducibility Some speed-up theorems: The problem of deciding if a string is a palindrome which is O(n2) in the 1-tape Turing machines model and O(n) in the 2-tape Turing machines model. For any k-tapes Turing machine M operating in time f(n) there exists a k'-tapes Turing machine M' operating in time f'(n)=f(n)+n (where  is an arbitrary small positive constant) which simulates M. Given any k-tape Turing machine M operating within time f(n), it's possible to construct a 1-tape Turing machine M' operating within time O(f(n)2) and such that for any input x, M(x)=M'(x). Hervé ZWIRN

Computational irreducibility The computation model Turing machines with 3 symbols (0, 1, #) k ≥ 2 tapes and one way write only tape which is used for output. We suppose that when the computation ends, the result is the number written at the right end of the output tape. Given a Turing machine M computing f(n) in time T(M(n)), let's denote by Rn,1, …, Rn,i, …, Rn,T(M(n)) the content of the output tape of M during the computation of f(n) after 1 step of computation, …, i steps of computation and T(M(n)) steps of computation. Hervé ZWIRN

Computational irreducibility (E-Turing machine): A Turing machine Mf will be called a E-Turing machine for f if: (i) Mf computes every f(n) (ii) during the computation of f(n), there exist increasing kn(i) for i=1 to n-1, such that f(i) is written on the output tape Rn,kn(i) at the right of the last symbol # Hervé ZWIRN

Computational irreducibility E-Turing machine for f f(1) n-1 f(2) f(n-2) f(1) n f(2) f(n-1) kn-1 (1) kn (1) kn-1 (2) kn (2) f(n-2) kn (n-2) kn-1 (n-2) kn (n-1) kn-1 (n-1) f(n-1) f(n) kn (n) Hervé ZWIRN

Computational irreducibility Tentative definition (CIR): A function f will be said CIR if and only if any Turing machine computing every f(n), is a E-Turing machine for f. Not a Robust Definition Need for more sophisticated concepts Hervé ZWIRN

Computational irreducibility (Asymptotically optimal Turing machine): We will say that a Turing machine Mf* for f is an asymptotically optimal Turing machine for f if for any other Turing machine M computing f: T(Mf*(n)) = O(T(M(n))) i.e. there are constants c > 0, n0 > 0 such that n > n0, T(Mf* (n))  cT(M(n)). Asymptotically, no other Turing machine computing f computes faster than Mf* We assume that it is the case Hervé ZWIRN

Computational irreducibility (Efficient E-Turing machine): We will say that a E-Turing machine Mfeff for f is an efficient E-Turing machine for f if for any other E-Turing machine Mf for f: T(Mfeff(n)) = O(T(Mf(n))) i.e. there are constants c > 0, n0 > 0 such that n > n0, T(Mfeff (n))  cT(Mf(n)). Asymptotically, no other E-Turing machine for f computes faster than Mfeff We assume that it is the case Hervé ZWIRN

Computational irreducibility A Turing Machine M will be said to be a P‑approximation of a E‑Turing machine for f if and only if there are a function F such that F(n)=O(T(Mf* (n)/n)) and a Turing machine P such that: (i) on input n, M computes a result rn such that P computes f(n) from rn in a number of steps F(n) and halts. (ii) during the computation, there exist increasing kn(i) for i=1 to n-1, such that P computes f(i) from i and Rn,kn(i) in a number of steps F(i) and halts. Hervé ZWIRN

Computational irreducibility P‑approximation of a E‑Turing machine for f f(1) n f(2) f(n-1) P r1 n r2 rn-1 kn (1) kn (2) kn (n-1) O(T(Mf*(n)/n)) f(n) rn kn (n) Hervé ZWIRN

Computational irreducibility Let M be a P‑approximation of a E‑Turing machine for f. Computation of f(n) based on the P‑approximation M: The computation of f(n) done initially through M with input n and continued when M has computed rn, by P which computes f(n) from n and rn in a time F(n) and halts. Hervé ZWIRN

Computational irreducibility Computation based on a P‑approximation of a E‑Turing machine for f f(1) f(2) f(n-1) P r1 n r2 rn-1 O(T(Mf* (n)/n)) f(n) rn Hervé ZWIRN

Computational irreducibility Strongly CIR (resp CIR) function: A function f(n) from N to N will be said to be strongly CIR (resp CIR) if and only if for any Turing machine M computing every f(n) there is a P-approximation of a E‑Turing machine for f, M’, such that for every n (resp. for infinitely many n), the computation of f(n) by M is based on M’. If a function is strongly CIR, for each n there is no other way to compute f(n) than to compute before all the values f(i) for i<n (or values that are near in the sense given in the definition of the approximation of a E-Turing machine). There is no shortcut allowing to get directly the value of f(n) without having computed before f(n-1) or a value that is near f(n-1) and so forth for the previous values. Hervé ZWIRN

Computational irreducibility Theorem if f is CIR no Turing machine computing every f(n) can compute f(n) faster than an efficient E­-Turing machine for f. More precisely, if Mf is a Turing machine computing every f(n) and if f is CIR then T(Mfeff(n)) = O(T(Mf(n))). Hervé ZWIRN

Computational irreducibility Possible candidates: Langton’s ant Rule 110 The number of configurations of index < n still alive after n steps in the game of life The function f défined as : f(1) = the first digit of  f(n) = the digit of  after having skipped f(n-1) digits from the digit f(n-1) Open problem: Prove that any of the above possible candidates is CIR. Hervé ZWIRN

Emergence What is necessary for Emergence? 2 levels: individual / collectif micro / macro Knowledge of the rules for the low level Apparent irreducibility of the phenomenon appearing at the upper level to the low level rules Hervé ZWIRN

Emergence appears when the dynamics of the low level is CIR Objective (weak) emergence Objective emergence should be independant of our human capacities. Non-epistemic criterion. Emergence appears when the dynamics of the low level is CIR Hervé ZWIRN