1. 1 Cellular Automata

2. 2 Introduction What are Cellular Automata? CA are discrete dynamic systems. CA's are said to be discrete because they operate in finite space and time and with properties that can have only a finite number of states. CA's are said to be dynamic because they exhibit dynamic behaviors. Basic Idea: Simulate complex systems by interaction of cells following easy rules.

3. 3 Introduction To put it another way: “Not to describe a complex system with complex equations, but let the complexity emerge by interaction of simple individuals following simple rules.”

4. 4 Cellular Automata A CA is a spatial lattice of N cells, each of which is one of k states at time t. Each cell follows the same simple rule for updating its state. The cell's state s at time t+1 depends on its own state and the states of some number of neighbouring cells at t. For one-dimensional CAs, the neighbourhood of a cell consists of the cell itself and r neighbours on either side. Hence, k and r are the parameters of the CA. CAs are often described as discrete dynamical systems with the capability to model various kinds of natural discrete or continuous dynamical systems

5. 5 History Original concept of CA is most strongly associated with John von Neumann. von Neumann was interested in the connections between biology and the then new study of automata theory. Stanislaw Ulam suggested that von Neumann use a cellular automata as a framework for researching these connections.

6. 6 History The original concept of CA can be credited to Ulam, while the early development of the concept is credited to von Neumann. Although von Neumann made many contributions and developments in CA, they are commonly referred to as “non-von Neumann style”, while the standard model of computation (CPU, globally addressable memory, serial processing) is know as “von Neumann style”.

7. 7 BACKGROUND Time FrameMajor Players Contribution Early 50’s J. Von Neuman, E.F. Codd, Henrie & Moore, H Yamada & S. Amoroso Modeling biological systems - cellular models ‘60s & ‘70sA. R. Smith, Hillis, ToffoliLanguage recognizer, Image Processing ‘80 sS. Wolfram,Crisp,VichniacDiscrete Lattice,statistical systems, Physical systems ‘87 - ‘96 IIT KGP, GroupAdditive CA, characterization,applications ‘97 - ‘99B.E.C GroupGF (2 p ) CA

8. 8 Why should we study Cellular Automata? Powerful computation engines. Discrete dynamical system simulators. Conceptual vehicle for exploring pattern formation. Models of fundamental physics.

9. 9 Why should we study Cellular Automata? Powerful computation engines. Allow very efficient parallel computation Discrete dynamical system simulator. Allow for a systematic investigation of complex phenomena. Original models of fundamental physics. Instead of looking at the equations of fundamental physics, consider modelling them with CA.

10. 10 Applications Simulations Gas behavior Forest fire propagation Urban development Traffic flow Air flow Crystallization process Alternative to differential equations

11. 11 Components Cell Basic element of a CA. Cells can be thought of as memory elements that store state information. All cells are updated synchronously according to the transition rules. Lattice Spatial web of cells. Simplest lattice is one dimensional.

12. 12 Behavior Local interaction leads to global dynamics. One can think of the behavior of a cellular automata like that of a “wave” at a sports event. Each person reacts to the state of his neighbors (if they stand, he stands).

13. 13 Behavior Rule Application Next state of the core cell is related to the states of the neighborhood cells and its current state. An example rule for a one dimensional CA: 011->x0x All possible states must be described. Next state of the core cell is only dependent upon the sum of the states of the neighborhood cells. For example, if the sum of the adjacent cells is 4 the state of the core cell is 1, in all other cases the state of the core cell is 0.

14. 14 Behavior Neighborhoods von Neumann Moore Extended Moore What about border cells? Common approach is to “wrap” around.

15. 15 Behavior Neighborhoods von Neumann – Composed of the cells above, below, left and right. Has a radius of 1.

16. 16 Behavior Neighborhoods Moore – Composed of the same cells as the von Nuemann neighborhood, but also includes the diagonal cells. Has a radius of 1.

17. 17 Behavior Neighborhoods Extended Moore – Composed of the same cells as the Moore neighborhood, but the radius of neighborhood is increased. Has a radius of 2.

18. 18 Behavior Rule Systems Rule systems define the dynamic behavior of the CA. Even in a simple one dimensional system, there are many different ways to determine the next step.  Could base the next state of the cell off of the sum of the states of your neighbors (Game of Life).  Could modify the scope of the neighborhood, so the resulting neighbors could be local (touching), close (neighbor’s neighbors) or global (anywhere in the system) or possibly use random neighbors.  Could allow the cells to grow and die.

19. 19 Characteristics Discrete lattice of cells. Homogeneity – all of the cells of the lattice are equivalent. Discrete states – each cell takes on one of a finite number of possible discrete states. Local interactions – each cell interacts only with cells that are in its local neighborhood.

20. 20 Characteristics Discrete dynamics – at each discrete time unit, each cell updates its current state according to a transition rule taking into account the states of the cells in its neighbourhood.

21. 21 Example of 1-D cellular automaton For a binary input N long, are there more 1s than 0s? Set k=2 and r=1 with the following rule: 000000001001010010011011100100101101110110111111 00010111 Cell + 2 neighbours: Result: That is, the value of a cell at time t+1 will depend on its value and the values of its two immediate neighbours at time t. This is a form of ‘majority voting’ between all three cells.

22. 22 One-Dimensional CA Illustration t i - r... i - 1 i i+1i+1... i+ri+r t+1t+1 i   i r

23. 23 One-Dimensional Cellular Automata (CA) Consists of a linear array of identical cells (called a lattice), each of which can be in a finite number of k states. The (local) state of cell i at time t is denoted: The (global) state s t at time t is the configuration of the entire array, where N is the (possibly infinite) size of the array.

24. 24 One-Dimensional CA cont. At each time step, all cells in the array update their state simultaneously, according to a local update rule. This update rule takes as input the local neighborhood configuration  of a cell. A local neighborhood configuration  consists of s i and its 2r nearest neighbors (r cells on either side): r is called the radius of the CA. The local update rule , which is the same for every cell in the array, can be represented as a lookup table, which lists all possible neighborhood configurations

25. 25 rule 6 C i-1 (t)C i (t)C i+1 (t)C i (t+1) 0000 0011 0101 0111 1001 1011 1101 1110 Wolfram rule Sum to 0 --> 0 1 --> 1 2 --> 1 3 --> 0 0110 = ‘rule 6’ Flake: k(2r+1) rules (??) (k-1)(2r+1) +1 works

26. 26 Example One-Dimensional CA Rule 110 The number of states, k=2. The alphabet The size of the array, N=11. The configuration space  The radius r = 1. The rule table : neighborhood : 000 001 010 011 100 101 110 111  : 0 1 1 1 0 1 1 0 This is rule 110 (base 10) because the output states are: 01101110 (base 2). Read right to left. Known as Wolfram notation. Rule 110 supports universal computation.

27. 27 1 1 0 0 1 1 0 0 0 1 0 1 1 0 1 1 1 0 1 1 1 1 Neighborhood  t = 1 t = 0 Periodic boundary conditions Neighborhood: Output bit: Lattice: Rule Table  :

28. 28 Rule 110 Space-Time Plot

29. 29 Rule 30 current pattern 111 110 101 100 011 010 001 000 new state for center cell 0 0 0 1 1 1 1 0

30. 30 Comments on Rule 30 Generates apparent randomness, despite being finite Wolfram proposed using the central column as a pseudo- random number generator Passes many tests for randomness, but many inputs produce regular patterns: All zeroes 00001000111000 repeated infinitely (try separating by 6 1s) Used in Mathematica for creating random integers (Wikipedia)

31. 31 Variants Asynchronous CA CA rules are typically applied simultaneously across all cells in the lattice. This variant allows the state of the cells to be updated asynchronously. Probabilistic CA The deterministic state-transitions are replaced with specifications of the probabilities of the cell-value assignments.

32. 32 Variants Non-homogenous CA State transition rules are allowed to vary from cell to cell. Mobile CA Some or all lattice sites are free to move about the lattice. Essentially primitive models of mobile robots. Used to model some aspects of military engagements.

33. 33 Variants Structurally Dynamic CA The topology (the sites and connections among sites) are allowed to evolve.

34. 34 Langton ’ s Lambda Parameter Wolfram’s classification scheme is phenomenological (argument by visual inspection of space-time diagrams). Chris Langton (1986) quantified the classification scheme by introducing the parameter. Lambda is a statistic of the output states in the CA lookup table, defined as the fraction of non-quiescent states in this table. The quiescent state is an arbitrarily chosen state Example: For a 2-state CA ( ), and quiescent state s=0, lambda is the fraction of 1s in the output states of .

35. 35 Lambda Parameter cont. Where n q = the number of rules that map to the quiescent state. Using the lambda parameter to study CA s: Table walkthrough: Start with one random table and progressively perturb it through the range: Random table: Interpret lambda as a bias on the random selection of states to fill up the table. (Get a new table for every new value of lambda). Use various statistics to measure CA “average” behavior, as a function of lambda: Single-site entropy. Same site mutual information across time steps. Two-site mutual information.

36. 36 Density classification In the above example, we have assumed wrap-around, and r=1. In this case, the CA has reached a ‘limit point’ from which no escape is possible. CAs have been used for simulating fluid dynamics, chemical oscillations, crystal growth, galaxy formation, stellar accretion disks, fractal patterns on mollusc shells, parallel formal language recognition, plant growth, traffic flow, urban segregation, image processing tasks, etc …

37. 37 Types of neighbourhood Many more neighbourhood techniques exist - see http://cell-auto.com and follow the link to ‘neighbourhood survey’http://cell-auto.com

38. 38 Wolfram ’ s CA Classification Class I: Eventually every cell in the array settles into one state, never to change again. Simple Uniform final state (all black or all white) Some examples are rules 0, 32, 128, 160, 250, 254 Analogous to computer programs that halt after a few steps and to dynamical systems that have fixed-point attractors. after a finite number of time steps, the CA tends to achieve a unique state from nearly all possible starting conditions (limit points)

39. 39 Wolfram Class I

40. 40 Wolfram Class I

41. 41 Class II: Eventually the array settles into a periodic cycle of states (called a limit cycle). Set of simple structures Structures remain the same or repeat every so often Examples include rules 132, 164, 218, 222 Analogous to computer programs that execute infinite loops and to dynamical systems that fall into limit cycles. the CA creates patterns that repeat periodically or are stable (limit cycles) – probably equivalent to a regular grammar/finite state automaton Wolfram ’ s CA Classification

42. 42 Wolfram ’ s Class II

43. 43 Wolfram ’ s Class II

44. 44 Class III: The array forms “aperiodic” random-like patterns. Appears random Smaller structures can be seen some at some level Most are expected to be computationally irreducible Examples include rules 22, 30, 126 Analogous to computer programs that are pseudo-random number generators (pass most tests for randomness, highly sensitive to seed, or initial condition). Analogous to chaotic dynamical systems. Almost never repeat themselves, sensitive to initial conditions, embedded unstable limit cycles. from nearly all starting conditions, the CA leads to aperiodic-chaotic patterns, where the statistical properties of these patterns are almost identical (after a sufficient period of time) to the starting patterns (self-similar fractal curves) – computes ‘irregular problems’ Wolfram ’ s CA Classification

45. 45 Wolfram ’ s Class III

46. 46 Wolfram ’ s Class III

47. 47 Wolfram ’ s Classification cont. Class IV: The array forms complex patterns with localized structure that move through space and time : Has order and randomness Smaller scale structures interacting in complex ways Examples include codes 1815, 2007, 1659, 2043 Recall: Codes are “totalistic” CAs where new color depends on average of neighbors Class 4 emerges as an intermediate class between classes 2 and 3 Difficult to describe. Not regular, not periodic, not random. Speculate that it is interesting computation. after a finite number of steps, the CA usually dies, but there are a few stable (periodic) patterns possible (e.g. Game of Life) - Class 4 CA are believed to be capable of universal computation Hypothesis: The most interesting and complex behavior occurs in Class IV CA---the edge of chaos. Example: Rule 110

48. 48 Wolfram ’ s Class IV

49. 49 Wolfram ’ s Class IV

50. 50 Exceptions Totalistic automata that don’t seem to fit into just one class Codes 219, 438, 1380, 1632

51. 51 Initial condition sensitivity Each class responds differently to a change in its initial conditions Response types Class 1 changes always die out Changes continue on but are localized for Class 2 Uniform rate of change affecting the whole system seen in Class 3 Class 4 has nonuniform changes

52. 52 Class 1 Class 2

53. 53 Class 3 Class 4

54. 54 Claim Differences in responses of classes show each class handles information in a different way Fundamental to our understanding of nature

55. 55 Class 2 Repetitive behavior No for support long-range communication Lack of long-range communication makes systems of limited size forcing repetitiveness

56. 56 Observing systems of limited behavior Limiting the size forces repetivness Period of repetition increases with size of system With n cells, there are at most 2 n possible states (maximum period of 2 n ) Modulus

57. 57 Repetition as a function of system size Rule 90 Rule 30 Rule 110 Rule 45

58. 58 Class 3 randomness Randomness exists even without random initial conditions Different initial conditions can produce random behavior or nested pattern behavior in the same rule (rule 22) Some rules need the random initial condition to exhibit randomness (90) and some rules don’t (30)

59. 59 “Instrinsic Randomness” Do systems like rule 22 or rule 30 have intrinsic randomness? Do these examples prove that certain systems have intrinsic randomness and do not depend on initial conditions? Special initial conditions can make class 3 systems behave like a class 2 or even a class 1 system (rule 126)

60. 60 Rule 22 with different initial conditions

61. 61 Rule 22 with another set of initial conditions

62. 62 Rule 22 appearing random with different initial conditions

63. 63 Class 4 structures Certain structures will always last Any way to predict the structures of a given rule and initial conditions? One can find all structures given a period, but prediction is another matter

64. 64 Attractors Sequences of cells restricted as iterations progress, even with random initial conditions Networks examples

65. 65 Types of Networks Classes 1 and 2 Never have more than t 2 nodes after t steps Classes 3 and 4 Allowed sequences of cells becomes more complicated Number of nodes increases at least exponentially

66. 66 Class 3 and 4 Exceptions Increase in network complexity not seen in special initial conditions for rules 204, 240, 30, and 90 Onto mappings defined Any other initial conditions than “special” initial conditions rapidly increase in complexity

67. 67 Lambda Space and Wolfram Classes

68. 68 Lambda Parameter cont. Table walkthrough illustrates interesting change in dynamics as function of lambda: Transient lengths increase dramatically in middle of the range (see next slide). However, different traversals of lambda space using the walkthrough method make the transition at different lambda values, although there is a well defined distribution around a mean value. Size of the array has an effect on dynamics only for intermediate values of lambda. Transient length depends exponentially on array size at lambda = 0.5. Overall evolutionary pattern in time is more random as lambda increases past the transition region (use entropy and mutual information). Transition region supports both static and propagating structures.

69. 69 Comments on the Lambda Parameter Claims: There is a phase transition between periodic and chaotic behavior. Most complex behavior is in the vicinity of the transition: The “edge of chaos.” CA s near the transition point correspond to Wolfram’s Class IV. CA s capable of performing complex computations will be found near the transition point (long transients). E.g., the game of life has lambda=0.273 (in the transition region for K=2, N=9 2D CA s). Criticisms of the Lambda parameter: CA s with high lambda-value can still have simple behavior. Lambda describes “average” behavior. Lambda does not take the initial state of the computation into account (see Assignment 3).

70. 70 John Conway ’ s Game of Life 2D cellular automata system. Each cell has 8 neighbors - 4 adjacent orthogonally, 4 adjacent diagonally. This is called the Moore Neighborhood.

71. 71 Simple rules, executed at each time step: A live cell with 2 or 3 live neighbors survives to the next round. A live cell with 4 or more neighbors dies of overpopulation. A live cell with 1 or 0 neighbors dies of isolation. An empty cell with exactly 3 neighbors becomes a live cell in the next round.

72. 72 Assumptions Computation universality not required Characteristics 8 states, 2D Cellular automata Needed CA grid of 100 cells Self Reproduction into identical copy Input tape with data and instructions Concept of Death Significance – Could be modeled through computer programs Loops

73. 73 Langton’s Loop 0 – Background cell state3, 5, 6 – Phases of reproduction 1 – Core cell state4 – Turning arm left by 90 degrees 2 – Sheath cell state state 7 – Arm extending forward cell state

74. 74 Loop Reproduction

75. 75 Loop Death

76. 76 Langton ’ s Loops Chris Langton formulated a much simpler form of self-rep structure - Langton's loops - with only a few different states, and only small starting structures.

77. 77 Example: Modelling Sharks and Fish:

78. 78 modeled a predator/prey relationship Begins with a randomly distributed population of fish, sharks, and empty cells in a 1000x2000 cell grid (2 million cells) Initially, 50% of the cells are occupied by fish 25% are occupied by sharks 25% are empty

79. 79 Here ’ s the number 2 million Fish: red; sharks: yellow; empty: black

80. 80 Rules A dozen or so rules describe life in each cell: birth, longevity and death of a fish or shark breeding of fish and sharks over- and under-population fish/shark interaction Important: what happens in each cell is determined only by rules that apply locally, yet which often yield long-term large-scale patterns.

81. 81 Do a LOT of computation! Apply a dozen rules to each cell Do this for 2 million cells in the grid Do this for 20,000 generations Well over a trillion calculations per run! Do this as quickly as you can

82. 82 Do a LOT of computation! We used a 20-CPU cluster in the Computer Science Department (Galaxy) ‘Gal’ is the smaller of two clusters run by the Department (larger one has 64 CPUs) 15x faster than a single PC Longest runs still took about 45 minutes GO PARALLEL !!!

83. 83 Rules in detail: Initial Conditions Initially cells contain fish, sharks or are empty Empty cells = 0 (black pixel) Fish = 1 (red pixel) Sharks = –1 (yellow pixel)

84. 84 Rules in detail: Breeding Rule Breeding rule: if the current cell is empty If there are >= 4 neighbors of one species, and >= 3 of them are of breeding age,  Fish breeding age >= 2,  Shark breeding age >=3, and there are <4 of the other species: then create a species of that type  +1= baby fish (age = 1 at birth)  -1 = baby shark (age = |-1| at birth)

85. 85 Breeding Rule: Before EMPTY

86. 86 Breeding Rule: After

87. 87 Rules in Detail: Fish Rules If the current cell contains a fish: Fish live for 10 generations If >=5 neighbors are sharks, fish dies (shark food) If all 8 neighbors are fish, fish dies (overpopulation) If a fish does not die, increment age

88. 88 Rules in Detail: Shark Rules If the current cell contains a shark: Sharks live for 20 generations If >=6 neighbors are sharks and fish neighbors =0, the shark dies (starvation) A shark has a 1/32 (.031) chance of dying due to random causes If a shark does not die, increment age

89. 89 Shark Random Death: Before I Sure Hope that the random number chosen is >.031

90. 90 Shark Random Death: After YES IT IS!!! I LIVE

91. 91 Sample Code (C++): Breeding

92. 92 Results Next several screens show behavior over a span of 10,000+ generations

93. 93 Generation: 0

94. 94 Generation: 100

95. 95 Generation: 500

96. 96 Generation: 1,000

97. 97 Generation: 2,000

98. 98 Generation: 4,000

99. 99 Generation: 8,000

100. 100 Generation: 10,500

101. 101 Long-term trends Borders tended to ‘harden’ along vertical, horizontal and diagonal lines Borders of empty cells form between like species Clumps of fish tend to coalesce and form convex shapes or ‘communities’

102. 102 Variations of Initial Conditions Still using randomly distributed populations: Medium-sized population. Fish/sharks occupy: 1/16 th of total grid Fish: 62,703; Sharks: 31,301 Very small population. Fish/sharks occupy: 1/800 th of total grid Initial population: Fish: 1,298; Sharks: 609

103. 103 Generation 10020001000 40008000 Medium-sized population (1/16 of grid)

104. 104 Very Small Populations Random placement of very small populations can favor one species over another Fish favored: sharks die out Sharks favored: sharks predominate, but fish survive in stable small numbers

105. 105 Gen. 100 40006000 8000 1500 10,00012,00014,000 Ultimate welfare of sharks depends on initial random placement of fish and sharks Very Small Populations

106. 106 Very small populations Fish can live in stable isolated communities as small as 20-30 A community of less than 200 sharks tends not to be viable

107. 107 Community image This is what a community of virtual plants looks like Contrasting tones show patches of resource depletion

108. 108 This is a single propagule of a virtual plant It is about to grow in a resource-rich above- and below-ground environment

109. 109

110. 110

111. 111

112. 112

113. 113

114. 114

115. 115

116. 116

117. 117

118. 118

119. 119

120. 120

121. 121

122. 122

123. 123

124. 124

125. 125

126. 126

127. 127 The plant has produced abundant growth above- and below-ground and zones of resource depletion have appeared

128. 128 See Rod Hunt at http://www.ex.ac.uk/~rh203/ for lots more about the plant life CA and its uses http://www.ex.ac.uk/~rh203/