1. Cellular Automata CS 591 Complex Adaptive Systems Spring 2008 Professor Melanie Moses 2/4/08

2. Review of concepts of complexity Measures of complexity Computational measures –Computational complexity: Measures program running time, e.g., P, NP, NC –Language complexity: Classes of languages that can be computed (recognized) by different abstract machines, e.g. regular, context free, context sensitive languages Information-theoretic approaches: –Algorithmic Complexity (Komogorov, AIC): Length of (# of bits to represent) the shortest program that can produce the phenomenon; grows with unpredictability Problems: Uncomputable, machine dependent, high for random strings –Shannon entropy: Number of bits to represent a string –*Mutual information:amount of info that one random variable contains about another Logical depth (Bennett): running time of the shortest program, combines computational complexity and AIC Effective complexity (Gell-Mann): AIC of the regularities; Effective complexity is highest of non-random, non-regular strings

3. Mutual Information Measures the amount of information that one random variable contains about another random variable. –Mutual information is a measure of reduction of uncertainty due to another random variable. –That is, mutual information measures the dependence between two random variables. –It is symmetric in X and Y, and is always non-negative. Recall: Entropy of a random variable X is H(X). Conditional entropy of a random variable X given another random variable Y = H(X | Y). The mutual information of two random variables X and Y is: Gell-Mann: If a string is divided into two parts the mutual AIC is the sum of the AIC's of the parts minus the AIC of the whole.

4. Kauffman’s Boolean Networks from Lansing 2003 What is the relationship between the average connectedness of genes to the ability of organisms to evolve? Given N bulbs and K connections behavior is 1) Chaotic: If K is large, the bulbs keep twinkling chaotically 2) Frozen or periodic: If K is small (K = 1), some flip on and off, most soon stop 3) Complex: If K is around 2, complex patterns appear, in which twinkling islands of stability develop, changing shape at their borders. A network that is either frozen solid or chaotic cannot transmit information and thus cannot adapt. A Boolean Net has 2 N possible states, but many fewer basins of attraction

6. Introduction to Cellular Automata (CA) Invented by John von Neumann (circa~1950). A cellular automata consists of: –A regular arrangement (lattice) of cells. –Cells can be in one of a finite number of states at each time step. –All cells have the same synchronous update rule. –Cells have a local interaction neighborhood. Example: The game of life. Many cellular automata applications: –Hydrodynamics, fluid dynamics –Forest fire simulation –Models of ecosystems and epidemics –Ferro-magnetic modeling (ISING models)

7. One-Dimensional CA Illustration t i - r... i - 1 i i+1i+1... i+ri+r t+1t+1 i   i r

8. One-Dimensional Cellular Automaton (CA) Consists of a linear array of identical cells (called a lattice), each of which can be in a finite number of k states. The (local) state of cell i at time t is denoted: The (global) state s t at time t is the configuration of the entire array, where N is the (possibly infinite) size of the array.

9. One-Dimensional CA cont. At each time step, all cells in the array update their state simultaneously, according to a local update rule. This update rule takes as input the local neighborhood configuration of a cell. A local neighborhood configuration consists of s i and its 2r nearest neighbors (r cells on either side): r is called the radius of the CA. The local update rule, which is the same for every cell in the array, can be represented as a lookup table, which lists all possible neighborhood configurations

10. From Flake ch 15, rule 6 C i-1 (t)C i (t)C i+1 (t)C i (t+1) 0000 0011 0101 0111 1001 1011 1101 1110 Wolfram rule Sum to 0 --> 0 1 --> 1 2 --> 1 3 --> 0 0110 = ‘rule 6’ Flake: k(2r+1) rules (??) (k-1)(2r+1) +1 works

11. Example One-Dimensional CA Rule 110 The number of states, k=2. The alphabet The size of the array, N=11. The configuration space The radius r = 1. The rule table : –neighborhood : 000 001 010 011 100 101 110 111 – : 0 1 1 1 0 1 1 0 This is rule 110 (base 10) because the output states are: 01101110 (base 2). Read right to left. Known as Wolfram notation. Rule 110 supports universal computation.

12. 1 1 0 0 1 1 0 0 0 1 0 1 1 0 1 1 1 0 1 1 1 1 Neighborhood  t = 1 t = 0 Periodic boundary conditions Neighborhood: Output bit: Lattice: Rule Table  :

13. Rule 110 Space-Time Plot

14. Rule 30 current pattern 111 110 101 100 011 010 001 000 new state for center cell 0 0 0 1 1 1 1 0

15. Comments on Rule 30 Generates apparent randomness, despite being finite Wolfram proposed using the central column as a pseudo- random number generator Passes many tests for randomness, but many inputs produce regular patterns: –All zeroes –00001000111000 repeated infinitely (try separating by 6 1s) Used in Mathematica for creating random integers (Wikipedia)

16. Wolfram’s CA Classification Class I: Eventually every cell in the array settles into one state, never to change again. –Analogous to computer programs that halt after a few steps and to dynamical systems that have fixed-point attractors. Class II: Eventually the array settles into a periodic cycle of states (called a limit cycle). –Analogous to computer programs that execute infinite loops and to dynamical systems that fall into limit cycles. Class III: The array forms “aperiodic” random-like patterns. –Analogous to computer programs that are pseudo-random number generators (pass most tests for randomness, highly sensitive to seed, or initial condition). –Analogous to chaotic dynamical systems. Almost never repeat themselves, sensitive to initial conditions, embedded unstable limit cycles.

17. Wolfram’s Classification cont. Class IV: The array forms complex patterns with localized structure that move through space and time: –Difficult to describe. Not regular, not periodic, not random. –Speculate that it is interesting computation. Hypothesis: The most interesting and complex behavior occurs in Class IV CA---the edge of chaos. Example: Rule 110

18. Wolfram Class I

19. Wolfram’s Class II

20. Wolfram’s Class III

21. Wolfram’s Class IV

22. More Cellular Automata Wed. Feb 6 Langston’s lambda –Example calculations Exercise: Finding complex CAs –Flake’s CA applet Universal computation –Rule 110 –Game of Life

23. Langton’s Lambda Parameter Wolfram’s classification scheme is phenomenological (argument by visual inspection of space-time diagrams). Chris Langton (1986) quantified the classification scheme by introducing the parameter. Lambda is a statistic of the output states in the CA lookup table, defined as the fraction of non-quiescent states in this table. The quiescent state is an arbitrarily chosen state Example: For a 2-state CA ( ), and quiescent state s=0, lambda is the fraction of 1s in the output states of.

24. Lambda Parameter cont. Where n q = the number of rules that map to the quiescent state. Using the lambda parameter to study CA s: –Table walkthrough: Start with one random table and progressively perturb it through the range: –Random table: Interpret lambda as a bias on the random selection of states to fill up the table. (Get a new table for every new value of lambda). Use various statistics to measure CA “average” behavior, as a function of lambda: –Single-site entropy. –Same site mutual information across time steps. –Two-site mutual information.

25. From Flake ch 15, rule 6 C i-1 (t)C i (t)C i+1 (t)C i (t+1) 0000 0011 0101 0111 1001 1011 1101 1110 Wolfram ‘sum’ rule Sum to 0 --> 0 1 --> 1 2 --> 1 3 --> 0 0110 = ‘rule 6’ Flake: k(2r+1) rules (??) (k-1)(2r+1) +1 works Sum rule: 0110 To calculate convert sum notation to full rule table Wolfram rule: 01111110 = rule 116 = representation of all rules: = 1-1/k = 1/2 ‘Interesting cases’: 0 < < 1/2 For sum rule 0110, = 3/4

26. Lambda Space and Wolfram Classes

27. Lambda Parameter cont. Table walkthrough illustrates interesting change in dynamics as function of lambda: –Transient lengths increase dramatically in middle of the range (see next slide). –However, different traversals of lambda space using the walkthrough method make the transition at different lambda values, although there is a well defined distribution around a mean value. –Size of the array has an effect on dynamics only for intermediate values of lambda. –Transient length depends exponentially on array size at lambda = 0.5. –Overall evolutionary pattern in time is more random as lambda increases past the transition region (use entropy and mutual information). –Transition region supports both static and propagating structures.

28. Comments on the Lambda Parameter Claims: –There is a phase transition between periodic and chaotic behavior. Most complex behavior is in the vicinity of the transition: The “edge of chaos.” –CA s near the transition point correspond to Wolfram’s Class IV. –CA s capable of performing complex computations will be found near the transition point (long transients). –E.g., the game of life has lambda=0.273 (in the transition region for K=2, N=9 2D CA s). Criticisms of the Lambda parameter: –CA s with high lambda-value can still have simple behavior. Lambda describes “average” behavior. –Lambda does not take the initial state of the computation into account (see Assignment 3).

29. Exercise Find a complex rule set with 3 states and r = 1 (neighborhood size = 3) Represent in Flake (sum) rule notation To convert from sum notation –Binomial coefficient (Number of combinations without repetition) –Number of sets with exactly k of n states = n!/(k!(n-k!)) In what range should c be? How sensitive is the rule to initial conditions?

30. Entropy and Lambda

31. Critical Slowing Down, Length of Transients Depends on Cell Size

32. Phase Transitions and Lambda

33. Computation in Cellular Automata CA s as computers: –Initial configuration constitutes the data that the physical computer is processing. –Transition function implements the algorithm which is applied to the data. –Examples: Majority calculations, synchronization (Mitchell and Crutchfield). CA s as logical universes within which computers can be embedded: –Initial configuration constitutes a computer. –Transition function is the “physics” obeyed by the parts of the embedded computer. –The algorithm and data are functions of the precise state of the initial configuration of the embedded computer. –In the most general case, the initial configuration is a universal computer. –Examples: Rule 110, Game of life CA s can implement storage, counting, logical operators CA s are networks of FSAs A Turing machine is equivalent to an FSA with an infinite storage tape

34. Example One-Dimensional CA Rule 110 The number of states, k=2. The alphabet The size of the array, N=11. The configuration space The radius r = 1. The rule table : –neighborhood : 000 001 010 011 100 101 110 111 – : 0 1 1 1 0 1 1 0 This is rule 110 (base 10) because the output states are: 01101110 (base 2). Read right to left. Known as Wolfram notation. Rule 110 supports universal computation.

35. Rule 110 Space-Time Plot

36. Two-Dimensional Cellular Automata Source: Wikipedia Wire World Conus textile Cyclic Cellular Automaton

37. The Game of Life The number of states, k=2. The alphabet, –Call 0 “dead” and 1 “alive.” The Moore neighborhood: Transition rules: –Loneliness: If a live cell has less than 2 live neighbors, then it dies. –Overcrowding: If a live cell has more than 3 live neighbors, then it dies. –Birth: If an empty (dead) cell has 3 live neighbors, then it becomes alive. –Otherwise: If a cell has 2 or 3 live neighbors, then it remains unchanged. NE SW SSE EW NNW

38. Moore vs. Von Neumann Neighborhoods NE SW SSE EW NNW S EW N MooreVon Neumann

39. Example Transition Center square changes to one (birth). “Just 3 for birth, 2 or 3 for survival.” 0 0 10 10 10 0 1

40. Possible Life Histories (Dynamical behaviors that enable universal computation) Static structures: –Beehive, Loaf, Pond, etc. (Figure 15.11) Periodic structures: –Blinkers (Figure 15.12) Moving structures: –Gliders (next slide) Glider guns Logical gates (and, or, not) Self-reproducing structures.

41. Static Objects

42. Periodic Objects

43. Gliders (Moving Objects)

44. Logical Operators

45. Can we predict the dynamics of a given initial condition? Consider a straight line of n live cells: –n = 1,2 Dies out immediately. –n = 3 Blinker –n = 4 Becomes a beehive –n = 5 Traffic lights –n = 6 Dies out at t = 12 –n = 7 Interesting behavior, terminating in the honey farm. –N = 8 4 blocks and 4 beehives. –N = 9 2 sets of traffic lights. –Etc.

46. Reading for Monday Wolfram A new kind of science, chapter 11, p 674-691 Rule 110 online book at http://www.wolframscience.com/nksonline/toc.htmlhttp://www.wolframscience.com/nksonline/toc.html pdf will be posted on course site http://cs.unm.edu/~melaniem/courses/CAS08.html Background on tag systems (not required) in Wolfram Chapter 3 and http://mathworld.wolfram.com/CyclicTagSystem.html http://mathworld.wolfram.com/TagSystem.html Flake’s applets, C code also downloadable http://mitpress.mit.edu/books/FLAOH/cbnhtml/javalarge.html