1. Problems Markov Chains 2
3) Given the following one-step transition matrix of a Markov chain, determine the classes of the Markov chain and whether they are recurrent.

2. Solutions Markov Chains 6
4) A computer is inspected at the end of every hour. It is found to be either working (up) or failed (down). If the computer is found to be up, the probability of its remaining up for the next hour is It it is down, the computer is repaired, which may require more than one hour. Whenever, the computer is down (regardlewss of how long it has been down), the probability of its still being down 1 hour later is 0.35.
a. Construct the one-step transition probability matrix.
b. Find the expected first passage time from i to j for all i, j.

3. Solutions Markov Chains 10
5) A manufacturer has a machine that, when operational at the beginning of a day, has a probability of 0.1 of breaking down sometime during the day. When this happens, the repair is done the next day and completed at the end of that day.
a. Formulate the evolution of the status of the machine as a 3 state Markov Chain.
b. Fine the expected first passage times from i to j.
c. Suppose the machine has gone 20 full days without a breakdown since the last repair was completed. How many days do we expect until the next breakdown/repair?

4. Ch 12, #3a
6. A military maintenance depot overhauls tanks. There is room for three tanks in the facility and one tank in an overflow area outside. At most four tanks can be at the depot at one time. Every morning a tank arrives for an overhaul. If the depot is full, however, it is turned away, so no arrivals occur under these circumstances. When the depot is full, the entire overhaul schedule is delayed 1 day. On any given day, the following probabilities govern the completion of overhauls. Number of tanks completed Probability These values are independent of the number of tanks in the depot, but obviously no more tanks than are waiting at the start of the day can be completed. Develop a Markov chain model for this situation. Begin by defining the state to be the number of tanks in the depot at the start of each day (after the scheduled arrival). Draw the network diagram and write the state-transition matrix.