1. Limiting probabilities

4. The limiting probabilities P j exist if (a) all states of the Markov chain communicate (i.e., starting in state i, there is a positive probability of ever being in state j, for all i, j and (b) the Markov is positive recurrent (i.e, starting in any state, the mean time to return to that state is finite). When do the limiting probabilities exist?

5. The M/M/1 queue 10 2 3 

6. The M/M/1 queue

9. The expected number in the system

10. The birth and death process 10 2   3    

14. The M/M/m queue

17. A machine repair model A system with M machines and one repairman. The time between machine is exponentially distributed with mean 1/. Repair times are also exponentially distributed with mean 1/ . What is the average number of working machines? What is the fraction of time each machine is in use?

18. The machine repairman problem

25. The automated teller machine (ATM) problem Customers arrive to an ATM according to a Poisson process with rate. If the customer finds more than N other customers at the machine, he/she does not wait and goes away. Machine transaction times are exponentially distributed with mean 1/ . What is the probability that a customer goes away? What is the average number of customers at the ATM? If the machine earns $ h per customer served, what is the average revenue the machine generates per unit time?

26. The M/M/1/ N queue

30. The production inventory problem Consider a production system that manufacturers a single product. Production times are exponentially distributed with mean 1/ . The production facility can produce ahead of demand by holding finished goods inventory. Orders from customers arrives according to a Poisson process with rate. If there is inventory on-hand, the order is satisfied immediately. Otherwise, the order is backordered. The production system incurs a holding cost $ h per unit of held inventory per unit time and a backorder cost $ b per unit backordered per unit time. The production system manages its finished goods inventory using a base-stock policy with base-stock level s.

31. The production inventory problem What is the expected inventory level? What is expected backorder level? What is the expected total cost? What is the optimal base-stock level?

32. I : level of finished goods inventory B : number of backorders (backorder level) IO : inventory on order. Three basic processes

33. Under a base-stock policy, the arrival of each customer order triggers the placement of an order with the production system  s = I + IO – B  s = E[ I ] + E[ IO ] – E[ B ] Three basic processes

34. I and B cannot be positive at the same time  I = max(0, s - IO ) = ( s – IO ) +  E[ I ] = E[( s – IO ) + ]  B = max(0, IO - s ) = ( IO - s ) +  E[ B ] = E [( IO - s ) + ] Three basic processes

35. The production system behaves like an M/M/1 queue, with IO corresponding to the number of customers in the system.

36. Expected backorder level

37. Expected inventory level

38. Expected cost

39. Optimal base-stock level