1. Chapman-Kolmogorov Equations
𝑃 0𝑗 (𝑛−𝑚)
𝑃 𝑖0 (𝑚)
𝑃 𝑖𝑗 (𝑛) = 𝑃 𝑖0 (𝑚) 𝑃 0𝑗 (𝑛−𝑚) + 𝑃 𝑖1 (𝑚) 𝑃 1𝑗 (𝑛−𝑚) +…+ 𝑃 𝑖𝑚 (𝑚) 𝑃 𝑚𝑗 (𝑛−𝑚)
= 𝑘=0 𝑀 𝑃 𝑖𝑘 (𝑚) 𝑃 𝑘𝑗 (𝑛−𝑚)
ex： (3 states)
𝑃 12 (2) = 𝑃 10 𝑃 12 + 𝑃 11 𝑃 12 + 𝑃 12 𝑃 22
𝑃 00 𝑃 01 𝑃 02 𝑃 10 𝑃 11 𝑃 12 𝑃 20 𝑃 21 𝑃 𝑃 00 𝑃 01 𝑃 02 𝑃 10 𝑃 11 𝑃 12 𝑃 20 𝑃 21 𝑃 22 = ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗

2. Chapman-Kolmogorov Equations (continued)

3. n-step Transition Matrices for the Weather Example

4. n-step Transition Matrices for the Weather Example (continued)

5. n-step Transition Matrices for the Inventory Example

6. Unconditional State Probabilities
𝑃 𝑋 2 =3 =𝑃 𝑋 0 =0 P 03 (2) +𝑃 𝑋 0 =1 P 13 (2) +𝑃 𝑋 0 =2 P 23 (2) +𝑃 𝑋 0 =3 P 33 (2)
=1 ‧ P 33 (2) = 0.165

7. Classification of States of a Markov Chain
Accessible => ｉ →　ｊ
2 → 3
3→2

8. Classification of States of a Markov Chain (continued)

9. Classification of States of a Markov Chain (continued)
I ↔ j , j ↔ k , I ↔ k
Weather 0,1
inventory{ 0,1,2,3} (在2 states後)
Gambling 1,

10. Recurrent States and Transient States
Gambling ：
3(X) ↗ 2 ↘ 1

11. Recurrent States and Transient States (continued)
Gambling：state 0,3

12. Recurrent States and Transient States (continued)

13. Recurrent States and Transient States (continued) 1 2 3 4

14. Periodicity Properties
$2->$1,$1->$2
Stock ↓↑

15. Periodicity Properties (continued)
Ergodic =Recurrent + aperiopdic