Limiting probabilities

The limiting probabilities P j exist if (a) all states of the Markov chain communicate (i.e., starting in state i, there is a positive probability of ever being in state j, for all i, j and (b) the Markov is positive recurrent (i.e, starting in any state, the mean time to return to that state is finite). When do the limiting probabilities exist?

The M/M/1 queue 

The M/M/1 queue

The expected number in the system

The birth and death process 10 2   3    

The M/M/m queue

A machine repair model A system with M machines and one repairman. The time between machine is exponentially distributed with mean 1/. Repair times are also exponentially distributed with mean 1/ . What is the average number of working machines? What is the fraction of time each machine is in use?

The machine repairman problem

The automated teller machine (ATM) problem Customers arrive to an ATM according to a Poisson process with rate. If the customer finds more than N other customers at the machine, he/she does not wait and goes away. Machine transaction times are exponentially distributed with mean 1/ . What is the probability that a customer goes away? What is the average number of customers at the ATM? If the machine earns $ h per customer served, what is the average revenue the machine generates per unit time?

The M/M/1/ N queue

The production inventory problem Consider a production system that manufacturers a single product. Production times are exponentially distributed with mean 1/ . The production facility can produce ahead of demand by holding finished goods inventory. Orders from customers arrives according to a Poisson process with rate. If there is inventory on-hand, the order is satisfied immediately. Otherwise, the order is backordered. The production system incurs a holding cost $ h per unit of held inventory per unit time and a backorder cost $ b per unit backordered per unit time. The production system manages its finished goods inventory using a base-stock policy with base-stock level s.

The production inventory problem What is the expected inventory level? What is expected backorder level? What is the expected total cost? What is the optimal base-stock level?

I : level of finished goods inventory B : number of backorders (backorder level) IO : inventory on order. Three basic processes

Under a base-stock policy, the arrival of each customer order triggers the placement of an order with the production system  s = I + IO – B  s = E[ I ] + E[ IO ] – E[ B ] Three basic processes

I and B cannot be positive at the same time  I = max(0, s - IO ) = ( s – IO ) +  E[ I ] = E[( s – IO ) + ]  B = max(0, IO - s ) = ( IO - s ) +  E[ B ] = E [( IO - s ) + ] Three basic processes

The production system behaves like an M/M/1 queue, with IO corresponding to the number of customers in the system.

Expected backorder level

Expected inventory level

Expected cost

Optimal base-stock level