1. Markov Chains Ali Jalali

2. Basic Definitions Assume s as states and s as happened states. For a 3 state Markov model, we construct a transition matrix.

3. Second Order Probability Two Level Transition Probabilities would be So two level transition matrix would be

4. n th Order Markov Transition Matrix Transition matrix of n th order would be So for the probability of being in a certain state, we have

5. Some Linear Algebraic Definitions  Eigen Values  Eigen Vectors  Characteristic Polynomial  Matrix Representation

6. State Probabilities Finally, we have By means of eigen values & vectors, It is true that It could be easily proven that

7.  Thanks for Patients