1. Towards new order Lasse Eriksson 12.3.2003 12.3.2003
AS Complex Systems

2. Towards new order - outline
Introduction
Self-Organized Criticality (SOC)
Sandpile model
Edge of Chaos (EOC)
Two approaches
Measuring Complexity
Correlation distance
Phase transition
Highly Optimized Tolerance (HOT)
Summary and conclusions
AS Complex Systems

3. Introduction From Catastrophe to Chaos? [12]
Catastrophe theory studies and classifies phenomena characterized by sudden shifts in behavior arising from small changes in circumstances
Originated by the French mathematician Rene Thom in the 1960s, catastrophe theory is a special branch of dynamical systems theory
”... the big fashion was topological defects. Everybody was ... finding exotic systems to write papers about. It was, in the end, a reasonable thing to do. The next fashion, catastrophe theory, never became important for anything.” - James P. Sethna (Cornell University, Ithaca) [11]
AS Complex Systems

4. Introduction We are moving from Chaos towards new Order
From Chaotic to Complex systems
What is the difference, just a new name? Is it really something new?
Langton's famous egg diagram (EOC, [7])
AS Complex Systems

5. Introduction [13] Chaos vs. Complex
Advances in the scientific study of chaos have been important motivators/roots of the modern study of complex systems
Chaos deals with deterministic systems whose trajectories diverge exponentially over time
Models of chaos generally describe the dynamics of one (or a few) variables which are real. Using these models some characteristic behaviors of their dynamics can be found
Complex systems do not necessarily have these behaviors. Complex systems have many degrees of freedom: many elements that are partially but not completely independent
Complex behavior = "high dimensional chaos”
AS Complex Systems

6. Introduction
Chaos is concerned with a few parameters and the dynamics of their values, while the study of complex systems is concerned with both the structure and the dynamics of systems and their interaction with their environment
Same kind of phenomena in catastrophe, chaos and complexity theory
Stable states become unstable
Sudden changes in system’s behavior
Critical points, edges...
AS Complex Systems

7. Introduction [13]
Complexity is …(the abstract notion of complexity has been captured in many different ways. Most, if not all of these, are related to each other and they fall into two classes of definitions):
1) ...the (minimal) length of a description of the system.
2) ...the (minimal) amount of time it takes to create the system
The length of a description is measured in units of information. The former definition is closely related to Shannon information theory and algorithmic complexity, and the latter is related to computational complexity
AS Complex Systems

8. Introduction Emergence is...
1) ...what parts of a system do together that they would not do by themselves: collective behavior
How behavior at a larger scale of the system arises from the detailed structure, behavior and relationships on a finer scale
2) ...what a system does by virtue of its relationship to its environment that it would not do by itself: e.g. its function
Emergence refers to all the properties that we assign to a system that are really properties of the relationship between a system and its environment
3) ...the act or process of becoming an emergent system
AS Complex Systems

9. About fractals [1]
Many objects in nature are best described geometrically as fractals, with self-similar features on all length scales
Mountain landscapes: peaks of all sizes, from kilometres down to millimetres
River networks: streams of all sizes
Earthquakes: occur on structures of faults ranging from thousands of kilometres to centimetres
Fractals are scale-free (you can’t determine the size of a picture of a part of a fractal without a yardstick)
How does nature produce fractals?
AS Complex Systems

10. About fractals
The origin of the fractals is a dynamical, not a geometrical problem
Geometrical characterization of fractals has been widely examined but it would be more interesting to gain understanding of their dynamical origin
Consider earthquakes:
Earthquakes last for a few seconds
The fault formations in the crust of the earth are built up during some millions of years and the crust seems to be static if the observation period is a human lifetime
AS Complex Systems

11. About fractals
The laws of physics are local, but fractals are organized over large distances
Large equilibrium systems operating near their ground state tend to be only locally correlated. Only at a critical point where continuous phase transition takes place are those systems fractal
AS Complex Systems

12. Example: damped spring
Consider a damped spring as shown in the figure
Let’s model (traditionally!) and simulate the system
mass m
spring constant k
damping coefficient B
position x(t) x m k B
AS Complex Systems

13. Example: damped spring
AS Complex Systems

14. Example: damped spring
Zoom:
AS Complex Systems

15. Damped spring [1]
In theory, ideal periodic motion is well approximated by sine wave
Oscillatory behavior with decreasing amplitude theoretically continues forever
In real world, the motion would stop because of the imperfections such as dust
Once the amplitude gets small enough, the emotion suddenly stops
This generally occurs at the end of an oscillation where the velocity is smallest
This is not the state of smallest energy!
In a sense, the system is most likely to settle near a ”minimally stable” state
AS Complex Systems

16. Multiple Pendulums
The same kind of behavior can be detected when analysing pendulums
Consider coupled pendulums which all are in a minimally stable state
The system is particularly sensitive to small perturbations which can ”avalanche” through the system
Small disturbances could grow and propagate through the system with little resistance despite the damping and other impediments
AS Complex Systems

17. Multiple Pendulums
Since energy is dissipated through the process, the energy must be replenished for avalanches to continue
If ”Self-Organized Criticality” i.e. ”SOC” is considered, the interest is on the systems where energy is constantly supplied and eventually dissipated in the form of avalanches
AS Complex Systems

18. Self-Organized Criticality [1],[2]
Concept was introduced by Per Bak, Chao Tang, and Kurt Wiesenfeld in 1987
SOC refers to tendency of large dissipative systems to drive themselves to a critical state with a wide range of length and time scales
The dynamics in this state is intermittent with periods of inactivity separated by well defined bursts of activity or avalanches
The critical state is an attractor for the dynamics
The idea provides a unifying concept for large-scale behavior in systems with many degrees of freedom
AS Complex Systems

19. Self-Organized Criticality
SOC complements the concept ”chaos” wherein simple systems with a small number of degrees of freedom can display quite complex behavior
Large avalanches occur rather often (there is no exponential decay of avalanche sizes, which would result in a characteristic avalanche size), and there is a variety of power laws without cutoffs in various properties of the system
The paradigm model for this type of behavior is the celebrated sandpile cellular automaton also known as the Bak-Tang-Wiesenfeld (BTW) model
AS Complex Systems

20. Sandpile model [1],[2]
Adding sand slowly to a flat pile will result only in some local rearrangement of particles
The individual grains, or degrees of freedom, do not interact over large distances
Continuing the process will result in the slope increasing to a critical value where an additional grain of sand gives rise to avalanches of any size, from a single grain falling up to the full size of the sand pile
The pile can no longer be described in terms of local degrees of freedom, but only a holistic description in terms of one sandpile will do
The distribution of avalanches follows a power law
AS Complex Systems

21. Sandpile model
If the slope were too steep one would obtain a large avalanche and a collapse to a flatter and more stable configuration
If the slope were too shallow the new sand would just accumulate to make the pile steeper
If the process is modified, for instance by using wet sand instead of dry sand, the pile will modify its slope during a transient period and return to a new critical state
Consider snow screens: if you build them to prevent avalanches, the snow pile will again respond by locally building up to steeper states, and large avalanches will resume
AS Complex Systems

22. [14]
AS Complex Systems

23. Simulation of the sandpile model [1]
2D cellular automaton with N sites
Integer variables zi on each site i represent the local sandpile height
When height exceeds critical height zcr (here 3), then 1 grain is transferred from unstable site to each 4 neighboring site
A toppling may initiate a chain reaction, where the total number of topplings is a measure of the size of an avalanche
Figure: after grains dropped on a single site (fractals?)
AS Complex Systems

24. Simulation of the sandpile model
To explore the SOC of the sandpile model, one can randomly add sand and have the system relax
The result is unpredictable and one can only simulate the resulting avalanche to see the outcome
State of a sandpile after adding pseudo-randomly a large amount of sand on a 286*184 size lattice
Figure: open boundaries
Heights: 0 = black, 1 = red, 2 = blue, 3 = green
AS Complex Systems

25. Simulation of the sandpile model
Configuration seems random, but some subtle correlations exist (e.g. never do two black cells lie adjacent to each other, nor does any site have four black neighbors)
Avalanche is triggered if a small amount of sand is added to a site near the center
To follow the avalanche, a cyan color has been given to sites that have collapsed in the following figures
AS Complex Systems

26. Simulation of the sandpile model
time
AS Complex Systems

27. Sandpile model
Figure shows a log-log plot of the distribution of the avalanche sizes s (number of topplings in an avalanche), P is the probability distribution
AS Complex Systems

28. Sandpile model
Because of the power law, the initial state was actually remarkably correlated although it seemed at first featureless
For random distribution of z’s (pile heights), one would expect the chain reaction of an avalanche to be either
Subcritical (small avalanche)
Supercritical (exploding avalanche with collapse of the entire system)
Power law indicates that the reaction is precisely critical, i.e. the probability that the activity at some site branches into more than one active site, is balanced by the probability that the activity dies
AS Complex Systems

29. Simulation of the sandpile model [3]
Sandpile Java applet
AS Complex Systems

30. The Edge of Chaos [4] Christopher Langton’s 1-D CA
States ”alive”, ”dead”: if a cell and its neighbors are dead, they will remain dead in the next generation
Some CA’s are boring since all cells either die after few generations or they quickly settle into simple repeating patterns
These are ”highly ordered” CA’s
The behavior is predictable
Other CA’s are boring because their behavior seems to be random
These are ”chaotic” CA’s
The behavior is unpredictable
AS Complex Systems

31. The Edge of Chaos
Some CA’s show interesting (complex, lifelike) behavior
These are near the border of between chaos and order
If they were more ordered, they would be predictable
If they were less ordered, they would be chaotic
This boundary is called the ”Edge of Chaos”
Langton defined a simple number that can be used to help predict whether a given CA will fall in the ordered realm, in the chaotic realm, or near the boundary
The number (0  l  1) can be computed from the rules of the CA. It is simply the fraction of rules in which the new state of the cell is living
AS Complex Systems

32. The Edge of Chaos
Remember that the number of rules (R) of a CA is determined by: ,
where K is the number of states and N is the number of neighbors
If l = 0, the cells will die immediately; if l = 1, any cell with a living neighbor will live forever
Values of l close to zero give CA's in the ordered realm and values near one give CA's in the chaotic realm. The edge of chaos is somewhere in between
Value of l does not simply represent the edge of chaos. It is more complicated
AS Complex Systems

33. The Edge of Chaos
You can start with l = 0 (death) and add randomly rules that lead to life instead of death => l > 0
You get a sequence of CA’s with values of l increasing from zero to one
In the beginning, the CA’s are highly ordered and in the end they are chaotic. Somewhere in between, at some critical value of l, there will be a transition from order to chaos
It is near this transition that the most interesting CA's tend to be found, the ones that have the most complex behavior
Critical value of l is not a universal constant
AS Complex Systems

34. Edge of Chaos CA-applet
The Edge of Chaos
Edge of Chaos CA-applet
AS Complex Systems

35. EOC – another approach [5]
Consider domino blocks in a row (stable state, minimally stable?)
Once the first block is nudged, an ”avalanche” is started
The system will become stable once all blocks are lying down
The nudge is called perturbation and the duration of the avalanche is called transient
The strength of the perturbation can be measured in terms of the effect it had i.e. the length of time the disturbance lasted (or the transient length) plus the permanent change that resulted (none in the domino case)
Strength of perturbation is a measure of stability!
AS Complex Systems

36. EOC – another approach Examples of perturbation strength:
Buildings in earthquakes: we require short transient length and return to the initial state (buildings are almost static)
Air molecules: they collide with each other continually, never settling down and never returning to exactly the same state (molecules are chaotic)
For air molecules the transient length is infinite, whereas for our best building method it would be zero. How about in the middle?
AS Complex Systems

37. EOC – another approach A room full of people:
A sentence spoken may be ignored (zero transient), may start a chain of responses which die out and are forgotten by everyone (a short transient) or may be so interesting that the participants will repeat it later to friends who will pass it on to other people until it changes the world completely (an almost infinite transient - e.g. the Communist Manifesto by Karl Marx is still reverberating around the world after over 120 years)
This kind of instability with order is called the Edge of Chaos, a system midway between stable and chaotic domains
AS Complex Systems

38. EOC – another approach
EOC is characterised by a potential to develop structure over many different scales and is an often found feature of those complex systems whose parts have some freedom to behave independently
The three responses in the room example could occur simultaneously, by affecting various group members differently
AS Complex Systems

39. EOC – another approach
The idea of transients is not restricted in any way and it applies to different type of systems
Social, inorganic, politic, psychological…
Possibility to measure totally different type of systems with the same measure
It seems that we have a quantifiable concept that can apply to any kind of system. This is the essence of the complex systems approach, ideas that are universally applicable
AS Complex Systems

40. Correlation Distance [5]
Correlation is a measure of how closely a certain state matches a neighbouring state, it can vary from 1 (identical) to -1 (opposite)
For a solid we expect to have a high correlation between adjacent areas, but the correlation is also constant with distance
For gases correlation should be zero, since there is no order within the gas because each molecule behaves independently. Again the distance isn't significant, zero should be found at all scales
AS Complex Systems

41. Correlation Distance
Each patch of gas or solid is statistically the same as the next. For this reason an alternative definition of transient length is often used for chaotic situations i.e. the number of cycles before statistical convergence has returned
When we can no longer tell anything unusual has happened, the system has returned to the steady state or equilibrium
Instant chaos would then be said to have a transient length of zero, the same as a static state - since no change is ever detectable. This form of the definition will be used from now on
AS Complex Systems

42. Measuring Complex Systems [5]
For complex systems we should expect to find neither maximum correlation (nothing is happening) nor zero (too much happening), but correlations that vary with time and average around midway
We would also expect to find strong short range correlations (local order) and weak long range ones
E.g. the behavior of people is locally correlated but not globally
Thus we have two measures of complexity
Correlations varying with distance
Long non-statistical transients
AS Complex Systems

43. Phase Transitions [5],[6]
Phase transition studies came about from the work begun by John von Neumann and carried on by Steven Wolfram in their research of cellular automata
Consider what happens if we heat and cool systems
At high temperatures systems are in gaseous state (chaotic)
At low temperatures systems are in solid state (static)
At some point between high and low temperatures the system changes its state between the two i.e. it makes a phase transition
There are two kinds of phase transitions: first order and second order
AS Complex Systems

44. Phase Transitions
First order we are familiar with when ice melts to water
Molecules are forced by a rise in temperature to choose between order and chaos right at 0° C, this is a deterministic choice
Second order phase transitions combine chaos and order
There is a balance of ordered structures that fill up the phase space
The liquid state is where complex behaviour can arise
AS Complex Systems

45. Phase Transitions [7]
Schematic drawing of CA rule space indicating relative location of periodic, chaotic, and ``complex'' transition regimes
AS Complex Systems

46. Phase Transitions
Crossing over the lines (in the egg) produces a discrete jump between behaviors (first order phase transitions)
It is also possible that the transition regime acts as a smooth transition between periodic and chaotic activity (like EOC experiments with l). This smooth change in dynamical behavior (smooth transition) is primarily second-order, also called a critical transition
AS Complex Systems

47. Phase Transitions
Schematic drawing of CA rule space showing the relationship between the Wolfram classes and the underlying phase-transition structure
AS Complex Systems

48. Wolfram’s four classes [8]
Different cellular automata seem to settle down to
A constant field (Class I)
Isolated periodic structures (Class II)
Uniformly chaotic fields (Class III)
Isolated structures showing complicated internal behavior (Class IV)
AS Complex Systems

49. Phase Transitions
Phase transition feature allows us to control complexity by external forces
Heating or perturbing system => chaotic behavior
Cooling or isolating system => static behavior
This is seen clearly in relation to brain temperature
Low = static, hypothermia
Medium = normal, organised behaviour
High = chaotic, fever
AS Complex Systems

50. Highly Optimized Tolerance [9]
HOT is a mechanism that relates evolving structure to power laws in interconnected systems
HOT systems arise, e.g. in biology and engineering where design and evolution create complex systems sharing common features
High efficiency
Performance
Robustness to designed-for uncertainties
Hypersensitivity to design flaws and unanticipated perturbations
Nongeneric, specialized, structured configurations
Power laws
AS Complex Systems

51. Highly Optimized Tolerance
Through design and evolution, HOT systems achieve rare structured states which are robust to perturbations they were designed to handle, yet fragile to unexpected perturbations and design flaws
E.g. communication and transportation systems
Systems are regularly modified to maintain high density, reliable throughput for increasing levels of user demand
As the sophistication of the systems is increased, engineers encounter a series of tradeoffs between greater productivity and the possibility of the catastrophic failure
Such robustness tradeoffs are central properties of the complex systems which arise in biology and engineering
AS Complex Systems

52. Highly Optimized Tolerance [9],[10]
Robustness tradeoffs also distinguish HOT states from the generic ensembles typically studied in statistical physics under the scenarios of the edge of chaos and self-organized criticality
Complex systems are driven by design or evolution to high-performance states which are also tolerant to uncertainty in the environment and components
This leads to specialized, modular, hierarchical structures, often with enormous “hidden” complexity with new sensitivities to unknown or neglected perturbations and design flaws
AS Complex Systems

53. Highly Optimized Tolerance [10]
Fragile
Robust, yet fragile!
Robustness of HOT systems
Fragile
(to unknown
or rare
perturbations)
Robust
(to known and
designed-for
uncertainties)
Uncertainties
Robust
AS Complex Systems

54. A simple spatial model of HOT
Square site percolation or simplified “forest fire” model
Carlson and Doyle,
PRE, Aug. 1999
AS Complex Systems

55. A simple spatial model of HOT
Assume one “spark” hits the lattice at a single site
A “spark” that hits an empty site does nothing
AS Complex Systems

56. A simple spatial model of HOT
A “spark” that hits a cluster causes loss of that cluster
AS Complex Systems

57. A simple spatial model of HOT
Yield = the density after one spark
yield
density
loss
AS Complex Systems

58. no sparks “critical point” N=100 sparks Y = (avg.) yield  = density 1
0.9
“critical point”
0.8
0.7
0.6
N=100
(size of the lattice)
sparks
0.5
0.4
0.3
0.2
0.1
0.2
0.4
0.6
0.8 1
 = density
AS Complex Systems

59. “critical point” limit N   Y = (avg.) yield c = 0.5927  = density
0.2
0.4
0.6
0.8 1
0.1
0.3
0.5
0.7
0.9
limit
N  
Y =
(avg.)
yield
c =
 = density
“critical point”
AS Complex Systems

60. Y
Fires don’t matter.
Cold 
AS Complex Systems

61. Y
Everything burns.
Burned 
AS Complex Systems

62. Critical point Y 
AS Complex Systems

63. Power laws Criticality cumulative frequency cluster size 12.3.2003
AS Complex Systems

64. Edge-of-chaos, criticality, self-organized criticality (EOC/SOC)
Essential claims:
Nature is adequately described by generic configurations (with generic sensitivity)
Interesting phenomena are at criticality (or near a bifurcation)
yield
density
AS Complex Systems

65. Highly Optimized Tolerance (HOT)
critical
Cold
Burned
AS Complex Systems

66. Optimize Yield Why power laws? Almost any distribution of sparks
of events
both analytic and numerical results.
AS Complex Systems

67. Probability distribution of sparks
High probability region
2.9529e-016
0.1902 5 10 15 20 25 30
2.8655e-011
4.4486e-026 5
AS Complex Systems 10 15 20 25 30

68. Increasing Design Degrees of Freedom
The goal is to optimize yield (push it towards the upper bound)
This is done by increasing the design degrees of freedom (DDOF)
Design parameter ρ for a percolation forest fire model
DDOF=1
AS Complex Systems

69. Increasing Design Degrees of Freedom
DDOF= 4
4 tunable densities
Each region is
characterized by
the ensemble of
random
configurations at
density ρi
AS Complex Systems

70. Increasing Design Degrees of Freedom
DDOF= 16
16 tunable
densities…
AS Complex Systems

71. Design Degrees of Freedom = Tunable Parameters
SOC = 1 DDOF
AS Complex Systems

72. Design Degrees of Freedom = Tunable Parameters
The HOT states specifically optimize yield in the presence of a constraint
A HOT state corresponds to forest which is densely planted to maximize the timber yield, with firebreaks arranged to minimize the spread damage
Blue: ρ = ρc Red: ρ = 1
HOT= many DDOF
AS Complex Systems

73. HOT: Many mechanisms grid evolved DDOF All produce: High densities
Modular structures reflecting external disturbance patterns
Efficient barriers, limiting losses in cascading failure
Power laws
AS Complex Systems

74. large events are unlikely
Small events likely
Optimized grid
density =
yield =
large events are unlikely 1
0.9
High yields
0.8
0.7
0.6
grid
0.5
0.4
random
0.3
0.2
0.1
AS Complex Systems
0.2
0.4
0.6
0.8 1

75.  Y Increasing DDOF: increases densities,
increases yields, decreases losses but
increases sensitivity Y
Robust, yet
fragile 
AS Complex Systems

76. HOT – summary
It is possible to drive a system over the critical point, where SOC systems collapse. These overcritical states are called ”highly optimized tolerance” –states
SOC is an interesting but extreme special case…
HOT may be a unifying perspective for many systems
HOT states are both robust and fragile. They are ultimately sensitive for design flaws
Complex systems in engineering and biology are dominated by robustness tradeoffs, which result in both high performance and new sensitivities to perturbations the system was not designed to handle
AS Complex Systems

77. HOT – summary The real work with HOT is in
New Internet protocol design (optimizing the throughput of a network by operating in HOT state)
Forest fire suppression, ecosystem management
Analysis of biological regulatory networks
Convergent networking protocols
AS Complex Systems

78. Summary and Conclusions
Catastrophe – Chaos – Complex
Common features, different approaches
Complexity adds dimensions to Chaos
Lack of useful applications
Self-Organized Criticality
Refers to tendency of large dissipative systems to drive themselves to a critical state
Coupled systems may collapse during an ”avalanche”
Edge of Chaos
Balancing on the egde between periodic and chaotic behavior
AS Complex Systems

79. Summary and Conclusions
Parts of a system together with the environment make it all function
Complex systems
Structure is important in complex systems
Between periodic (and static) and chaotic systems
Order and structure to chaos
Increasing the degrees of freedom
HOT
Optimizing the profit/yield/throughput of a complex system
By design one can reduce the risk of catastrophes
Yet fragile!
AS Complex Systems

80. References [1]:Bunde, Havlin (Eds.): Fractals in Science 1994
[2]:
[3]:
[4]:
[5]:
[6]:
[7]:
[8]:
AS Complex Systems

81. References
[9]:Carlson, Doyle: Highly Optimized Tolerance: Robustness and Design in Complex Systems 1999
[10]:Doyle: HOT-intro Powerpoint presentation, John Doyle www-pages:
[11]:
[12]:
[13]:
[14]:
AS Complex Systems

82. References [15]:http://pespmc1.vub.ac.be/COMPLEXI.html
[16]:
AS Complex Systems