1. Hiroki Sayama sayama@binghamton.edu
NECSI Summer School 2008 Week 3: Methods for the Study of Complex Systems Stochastic Systems
Hiroki Sayama

2. Four approaches to complexity
Nonlinear
Dynamics
Complexity =
No closed-form solution,
Chaos
Information
Complexity =
Length of description,
Entropy
Computation
Complexity =
Computational time/space,
Algorithmic complexity
Collective
Behavior
Complexity =
Multi-scale patterns,
Emergence

3. Stochastic system Deterministic system xt = F(xt-1, t)
Stochastic system (w/ discrete states)
Px(s; t) = Ss’ Px(s|s’) Px(s’; t-1)
Px(s; t): Probability for x to be s at time t
Px(s|s’): Transition probability for x in state s’ to change to s in one time step
Assumed time-independent

4. Exercise Write the model of the following system in mathematical form
Numbers on arrows are transition probabilities 1
1/4
3/4

5. Random Walk

6. Random walk
Time series in which random noise is added to the system’s previous state
xt = xt-1 + d
d : random variable (e.g., +1 or -1)
Many stochastic events that accumulate and form the system’s history

7. Exercise
Derive a mathematical representation of random walk using probability distribution, assuming:
Possible states: integers (unbounded)
Probability of moving upward and downward: 50%-50%
Px(s|s’) = ½ d(s, s’+1) + ½ d(s, s’-1)
d(a,b): Kronecker’s delta function

8. Where will you be after t steps?
Exact position of individual random walk is hard to predict, but you can still statistically describe where it is likely to go

9. Exercise
Show analytically that the expected position of a particle in random walk will not change over time
<s> = Ss s Px(s; t)

10. Exercise
Show analytically that the root square means (RMS) distance a particle travels in random walk will grow over time with power=1/2
√<(s-s0)2> = { Ss (s-s0)2 Px(s; t) }1/2

11. Dependence on time scale
Distribution of positions flattens over time (diffusion of probability)

12. After sufficiently long time
Positions of many random-walking particles follow a “normal distribution”
A.k.a. Gaussian distribution, bell curve
~ e-(x-m)2/(2s2)/(2ps2)1/2
m: mean
s: standard deviation s m

13. Relation to the central limit theorem
Sum (and hence average) of many independent random events will approximately follow a normal distribution (regardless of probability distribution of those random events)
Each step of random walk corresponds to an independent event
Position of a particle corresponds to the sum of those events

14. Self-Similar Properties of Individual Random Walk

15. Structure of a single random walk
Random walk produces self-similar trajectory

16. Fractal time series
Random walk is a very simple example of “fractal” time series
Time series whose portion has statistical properties similar to those of the whole series itself
Shows stochastic self-similarity
The word “fractal” came from the fact that these structures have fractional (non-integral) dimensions

17. Power law (scaling law)
P(w) ~ w-k
Random walk’s power spectrum shows an inverse power law distribution
For such systems, averaging system states over time fails w
Power of small fluctuation
Power of large fluctuation
The more data you collect, the larger fluctuation you find (scaling)

18. Exercise: Failure of averaging
Produce time series data using a random walk model
Average the system state for the first t steps
Plot this average as a function of t
Does the average become more accurate as you increase t?

19. Dynamics of fractal time series
They are always fluctuating, but not completely random
There are gentle correlations between past, present, and future (regularity)
The similar dynamics appear over a wide range of time scales, without any characteristic frequency

20. Fractal time series in society
Many other real-world time series generated by complex social/ economical systems (i.e., many interacting parts) have fractal characteristics
Stock prices
Currency exchange rates

21. Fractal time series in biology
Many biological/physiological time series data are found to be fractal
Heartbeat rates, breathing intervals, human gaits, body temperature, etc.
They are usually fractal if a subject is normal and healthy
If they look very regular, the subject is probably not healthy
For details see, e.g.,

22. Transition Probability Matrix and Asymptotic Probability Distribution

23. Time evolution of probabilities
If we consider probability distribution Px(si; t) as a meta-state of the system, its time evolution is linear
Px(si; t) = Sj Px(si|sj) Px(sj; t-1)
pt = A pt-1
pt: Probability vector
A: Transition probability matrix
If original state space is finite and discrete

24. Convenient properties of transition probability matrix
The product of two TPMs is also a TPM
All TPMs have eigenvalue 1
|l|  1 for all eigenvalues of any TPM
If the transition network is strongly connected, the TPM has one and only one eigenvalue 1 (no degeneration)

25. Exercise: Prove the following
The product of two TPMs is also a TPM
All TPMs have eigenvalue 1
|l|  1 for all eigenvalues of any TPM
If the transition network is strongly connected, the TPM has one and only one eigenvalue 1 (no degeneration)

26. Answer (1) All TPMs have eigenvalue 1
You can show that there exists a non- zero vector q that satisfies A q = q, i.e. (A-I) q = 0
→ |A-I| = 0
This holds when column vectors of A-I are linearly dependent with each other (i.e., A-I maps vectors to a subspace of fewer dimensions)

27. Answer (1) A-I actually looks like this:
Note that each column vector is in a subspace s1+s2+...+sn = 0
→ |A-I| = 0
a11-1
a12 …
a1n
a21
a22-1
a2n ：
an1
an2
ann-1

28. Answer (2) |l|  1 for all eigenvalues of any TPM → |l|  1
For any l, Amq = lmq (q: eigenvector)
Am is a product of TPM, therefore it must be a TPM as well whose elements are all <= 1
Amq = lmq can’t diverge
→ |l|  1

29. TPM and asymptotic probability distribution
|l|  1 for all eigenvalues of any TPM
If the transition network is strongly connected, the TPM has one and only one eigenvalue 1 (no degeneration)
→ This eigenvalue is a unique dominant eigenvalue and the probability vector will eventually converge to its corresponding eigenvector

30. An Application: Google’s “PageRank”

31. ’s “PageRank”
PageRank Explained (from
“PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages “important.” ”

32. PageRank explained mathematically
Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd, 'The PageRank Citation Ranking: Bringing Order to the Web' (1998):
Node: Web pages
link: WWW hyperlinks
State: Temporary “importance” of that node
Its coefficient matrix is a transition probability matrix that can be obtained by dividing each column of the adjacency matrix by the number of 1’s in that column.

33. Example s1 s2 s5 s3 s4
(In the actual implementation of PageRank, positive non-zero weights are forcedly assigned to all pairs of nodes in order to make the entire network strongly connected)

34. Interpreting the PageRank network as a stochastic system (1)
State of each node can be viewed as a relative population that are visiting the webpage at t
At next timestep, the population will distribute to other webpages linked from that page evenly s1 s2 s5 s3 s4

35. Interpreting the PageRank network as a stochastic system (2)
Importance of each webpage can be measured by calculating the final equilibrium state of such population distribution s1 s2 s5 s3 s4

36. Therefore...
Just one dominant eigenvector of the TPM of a strongly connected network always exists, with l = 1
This shows the equilibrium distribution of the population over WWW
So, just solve x = Ax and you will get the PageRank for all the web pages on the World Wide Web!!

37. Exercise s1 s2 s5 s3 s4
Calculate the PageRank of each node in the above network (the network is already strongly connected so you can directly calculate its dominant eigenvector)

38. Summary
Stochastic systems may be modeled using probability distributions
Random walk shows ~t1/2 RMS growth and fractal time series patterns
Time evolution of probability distributions may be modeled and analyzed as a linear system
Asymptotic distribution can be uniquely obtained if the transition network is strongly connected

39. FYI: Master equation
A continuous-time analog of the same stochastic processes
dP(s)/dt
= Ss’≠s ( R(s|s’) P(s’) – R(s’|s) P (s) )
R(a|b): Rate of transition from b to a