1. Solutions Markov Chains 6
16.6-1) A computer is inspected at the end of every hour. It is found to be either working (up) or failed (down). If the computer is found to be up, the probability of its remaining up for the next hour is It it is down, the computer is repaired, which may require more than one hour. Whenever, the computer is down (regardlewss of how long it has been down), the probability of its still being down 1 hour later is 0.35.
a. Construct the one-step transition probability matrix.
b. Find the expected first passage time from i to j for all i, j.

2. Solutions Markov Chains 7
16.6-1) (cont.)
b. Find expected first passage times

3. Solutions Markov Chains 8
16.6-1) (cont.) Alternative solution to b.
(1)
(2)
(3)
(4)
Substituting (4) into (1) gives
And from (4),

4. Solutions Markov Chains 9
16.6-1) (cont.) Alternative solution to b. . 13 p = . 87 1 p =