1. A New Kind of Science Chapter 3 Matthew Ziegler CS 851 – Bio-Inspired Computing

2. 2 Overview Wolfram’s scientific philosophy: The key to understanding complex behavior can be found in very simple discrete systems Elementary cellular automata Rule 110, a universal Turing machine Other systems in chapter 3 Wolfram’s Observations and Conclusions

3. 3 Simple Rules, Complex Behavior Overall theme is that systems with simple rules can produce complex behavior –Is this counterintuitive? Wolfram thinks so. Many other systems have simple components but complicated behavior –Nearly all topics on the course syllabus, e.g., neural nets, GAs, swarms –Human brain, very complex considering only 12 million bytes in human genome* –Many of these systems require additional inputs aside from initial conditions Wolfram considers systems with very simple rules and where output is based on initial conditions R. Kurzweil, “Reflections on Stephen Wolfram's A New Kind of Science,” http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html *

4. 4 Elementary Cellular Automata 2^8 = 256 rules –Rigid array –Parallel update

5. 5 Elementary Cellular Automata by the Numbers small persistent patterns (~170) ostationary omoving growing patterns (~85) orepetitive patterns (~45) omore complex patterns (36) ofractal, nested patterns (24) orandom patterns (10) ocomplex mixture of regular and irregular (1) D. Drysdale, “Critical Review of A New Kind of Science,” http://freespace.virgin.net/david.drysdale/wolfram/review.html

6. 6 Elementary Cellular Automata Taxonomy Class 1 – Repetition Class 2 – Nesting Class 3 – Randomness Class 4 – Localized Structures R. Kurzweil, “Reflections on Stephen Wolfram's A New Kind of Science,” http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html

7. 7 Class 1 – Repetition Degenerate, single color or checker board Nothing really interesting or surprising Rule 250

8. 8 Class 2 – Nesting Rule 90 Repeated nested patterns

9. 9 Class 3 – Randomness Rule 30 10 rules of elementary CA are random Statistical analysis show randomness Are purely random patterns less complex than localized structures?

10. 10 Class 4 – Localized Structures Rule 110 “Beyond randomness” P. 60 Neither regular nor completely random Unique rule among 256 elementary CA

11. 11 More on Rule 110 Rule 110 displays complex behavior, despite simple rules and initial conditions Rule 110 discovered by Wolfram in the 1980s –Wolfram proposed it was “Universal” e.g. could be used to simulate a universal Turing machine Proved by Matthew Cook (former Wolfram employee) in the1990s –Wolfram suing Cook for presenting results before book released Throughout chapter 3 Wolfram demonstrates adding more complex rules do not give rise to more complex behavior –However he does mention additional complexity is needed for system details

12. 12 3 Color Cellular Automata More than 7 trillion rules Totalistic rules –New color from average of previous colors –6561 rules More complexity in rules Similar Behavior to elementary CA

13. 13 Other Systems Surveyed in Chapter 3 Mobile Automata Turing Machines Substitutional Systems Sequential Substitutional Systems Tag Systems Cyclic Tag Systems Register Machines Symbolic Systems

14. 14 Mobile Automata Basic mobile automata does not show complex behavior Add another rule, update color of immediate neighbors Still 99% show repetitive behavior –Every few thousands rules has nested structure –Searching 50,000 rules randomness is found Random pattern and active cell movement Add a 2 nd new rule, active cells can split or disappear –Complexity now emerges –Correlation b/tw number of active cells and complexity Activity is reason for more complex behavior in cellular automata than mobile automata

15. 15 Turing Machines Behavior depends on state of the head and color of the cell, not on the color of neighboring cells 2 states, 2 color Turing Machines show repetitive and nested behavior 3 states: still only repetitive and nested behavior 4 states: random patterns emerge More than 4 states: randomness becomes more common, but behaviors are not more complex Complexity threshold met at 4 states

16. 16 Substitutional Systems Number of elements in a row can change Need rules that depend on both color of active element and a neighboring elements to get complexity above nesting Additional rule: allow elements to disappear, giving slow growth to the patterns –Slow growth only exhibits repeating patterns –Adding elements with 3 or more color gives complex random behavior

17. 17 Sequential Substitutional Systems Similar to search and replace in text editor 1 or 2 replacements yield only simple patterns Complex patterns are found with 3 or more replacements

18. 18 Tag Systems Fixed number of blocks removed at the beginning of the sequence, rules specify number and colors of block to be tagged onto the end of the sequence based on removed blocks 1 block removed yields repetitive or nested behavior 2 block removed produces complex behavior

19. 19 Cyclic Tag Systems Very simple rules 1 underlying rule with two cases, alternate between cases at each step Fluctuations in growth of the sequence are many times random

20. 20 Register Machines 2 register machine, 2 different instructions –increment –decrement-jump 4 or less instructions in program: repetitive behavior 5-7 instructions in program: nested behavior 8 instructions in program: random behavior when a register value turns to zero 3 register machine, random at 7 instructions in a program, but no more complexity Adding different kinds of instructions also does not increase complexity

21. 21 One Instruction Computer subz A,B,C mem[A] = mem[A] - mem[B]; if (mem[A] == 0) PC = C; else PC = PC + 1;

22. 22 Symbolic Systems Simple symbolic rules, such as: e[x_][y_] -> x[x[y]] This particular rule eventually always stabilizes But simple symbolic rules can give produce repetitive, nested, and random patterns

23. 23 Wolfram’s Observations Are there special features inherent with CA that are crucial to complexity? –Rigid array counterexample: Substitutional Systems –Parallel update counterexample: Mobile Automata None of the features in CA actually matter: Complexity is universal

24. 24 Complexity Threshold Adding complexity to rules increases complexity of behavior until complexity threshold is crossed Complexity threshold is low Adding more complex rules past the threshold does not perceivably increase complexity of behavior complexity Repetition Nesting Complexity Threshold

25. 25 References and Links Collection of Reviews –http://www.math.usf.edu/~eclark/ANKOS_reviews.htmlhttp://www.math.usf.edu/~eclark/ANKOS_reviews.html Critical Review of "A New Kind of Science" and Notes on "A New Kind of Science" –http://freespace.virgin.net/david.drysdale/wolfram/review.htmlhttp://freespace.virgin.net/david.drysdale/wolfram/review.html Reflections on Stephen Wolfram's "A New Kind of Science" –http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.htmlhttp://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html One Instruction Set Computer –http://www.cs.ucsd.edu/classes/fa96/cse30/lec2/oic.htmlhttp://www.cs.ucsd.edu/classes/fa96/cse30/lec2/oic.html Life Applet –http://hensel.lifepatterns.net/http://hensel.lifepatterns.net/