1. Markov Chains & Population Movements

2. Stochastic Matrices
Certain matrices, called Stochastic Matrices, are important in the study of random phenomena where the exact outcome is not known but probabilities can be determined. One example is an analysis of population movement between cities and suburbs in the United States.
Recall some basic ideas of probability.
If the outcome of an event is sure to occur, we say the probability of the outcome is 1.
On the other hand, if it will not occur we say the probability is 0.
Other probabilities are represented by fractions between 0 and 1; the larger the fraction, the greater the probability p of that outcome occurring. Thus we have the restriction 0 ≤ p ≤ 1 on a probability p.

3. If any one of n completely independent outcomes is equally likely to happen, and if m of these outcomes are of interest to us, then the probability p that one of these outcomes will occur is defined to be the fraction m/n.
An example.
Consider the event of drawing a single card from a deck of 52 playing cards. What is the probability the outcome will be an ace or a king?
We see there are 52 possible outcomes. There are 4 aces and 4 kings in the deck; there are 8 outcomes of interest.
Thus the probability of drawing an ace or a king is 8/52 or 2/13.

4. Stochastic Matrix
Definition: A stochastic matrix is a square matrix whose elements are probabilities and whose columns add up to 1.

5. Examples of Stochastic Matrices

6. Examples that aren’t Stochastic Matrices
The sum of the elements in the first column is not 1.
The 2 in the first row is not a probability since it is greater than 1.

7. A general 2 x 2 stochastic matrix can be written:
where 0 ≤ x ≤ 1 and 0 ≤ y ≤ 1.

8. Theorem:
If A and B are stochastic matrices of the same size, then AB is a stochastic matrix.
Thus if A is stochastic, then A2, A3, A4,… are all stochastic.

9. Example
It is estimated the number of people living in cities in the United States during 2012 was 80 million. The number of people living in the surrounding suburbs was 175 million. Let us represent this information by the matrix:
Consider the population flow from cities to suburbs. During 2012 the probability of a person staying in the city was Thus the probability of moving to the suburbs was 0.04 (assuming all those who moved went to the suburbs).
Consider now the reverse population flow, from suburbia to the city. The probability of a person moving to the city was 0.01; the probability of remaining in suburbia was 0.99.

10. Stochastic Matrix P
These probabilities can be written as the elements of a stochastic matrix P:
The probability of moving from location A to location B is given by the element in column A and row B.
In this context, the stochastic matrix is called the matrix of transition probabilities.

11. Now consider the population distribution in 2013, one year later:
City population in 2013 = people who remained from people who moved in from the suburbs
= (0.96 * 80) + (0.01 * 175)
= million
Suburban population in 2013 = people who moved in from the city + people who stayed from 2012
= (0.04 * 80) + (0.99 * 175)
= million

12. Note we can arrive at these numbers using matrix multiplication:
Using 2012 as the base year, let X1 be the population in 2013, one year later. We can write
X1 = PX0
Assume the population flow represented by the matrix P is unchanged over the years. The population distribution X2 after two years is given by
X2 = PX1

13. After three years the population distribution is given by
X3 = PX2
After n years we get
Xn = PXn - 1
The predictions of this model (to four decimal places) are
and so on.

14. Observe how the city population is decreasing annually, while that of the suburbs is increasing.
We will see later that the sequence X0, X1, X2,…approaches
If conditions do not change, city population will gradually approach 51 million, while the population of suburbia will approach 204 million.

15. X1 = PX0, X2 = P2X0, X3 = P3X0,…,Xn = PnX0 Xi + n = PnXi
Further, note the sequence X1, X2, X3,…,Xn can be directly computed from X0, as follows:
X1 = PX0,
X2 = P2X0,
X3 = P3X0,…,Xn = PnX0
The matrix Pn is a stochastic matrix that takes X0 into Xn, in n steps. This result can be generalized. That is, Pn can be used in this manner to predict the distribution n stages later, from any given distribution.
Xi + n = PnXi

16. Pn is called the n-step transition matrix
Pn is called the n-step transition matrix. The (i, j)th element of Pn gives the probability of going from state j to state i in n steps.
For example, it can be shown (writing to two decimal places)
Thus, for instance, the probability of living in the city in 2012 and being in the suburbs four years later is 0.15.

17. Final Notes
The probabilities in this model depend only on the current state of a person – whether the person is living in the city or in suburbia. This type of model, where the probability of going from one state to another depends only on the current state rather than on a more complete historical description, is called a Markov Chain.
A modification that allows for possible annual population growth or decrease would give improved estimates of future population distributions.
These concepts can be extended to Markov processes involving more than two states.