Many useful applications, especially in queueing systems, inventory management, and reliability analysis. A connection between discrete time Markov chains and continuous time Markov chains. Exponential distribution and the Poisson process

The exponential distribution

The memoryless property

 Exponentially distributed random variables are memoryless.

It can be shown that the exponential distribution is the only distribution that has the memoryless property.

Example 1: The amount of time one spends in a bank is exponentially distributed with mean ten minutes ( = 1/10). What is the probability that a customer spends more than 15 minutes? What is the probability that the customer spends more than 15 minutes given that she is in the bank after 10 minutes?

Example 2: Consider a branch of a bank with two agents serving customers. The time an agent takes with a customer is exponentially distributed with mean 1/. A customer enters and finds the two agents busy serving two other customers. What is the probability that the customer that just entered would be last to leave?

Suppose that X 1, X 2,..., X n are independent exponential random variables, with X i having rate  i, i =1,..., n. What is P( min( X 1, X 2,..., X n )> x )? The minimum of n exponentially distributed random variables

Suppose that X 1, X 2,..., X n are independent exponential random variables, with X i having rate  i, i =1,..., n. What is P( min( X 1, X 2,..., X n )> x )? The minimum of n exponentially distributed random variables

Suppose that X 1 and X 2 are independent exponentially distributed random variables with rates 1 and 2. What is P(X 1 < X 2 )? Comparing two exponentially distributed random variables

Suppose that X 1 and X 2 are independent exponentially distributed random variables with rates 1 and 2. What is P(X 1 < X 2 )? Comparing two exponentially distributed random variables

Suppose that X 1 and X 2 are independent exponential random variables with rates 1 = 2 =. What is the distribution of f X 1 +X 2 ( x )?)? The sum of 2 identical exponentially distributed random variables

Suppose that X 1 and X 2 are independent exponential random variables with rates 1 = 2 =. What is the distribution of f X 1 +X 2 ( x )? The sum of 2 identical exponentially distributed random variables

Suppose that X 1,..., X N are independent exponential random variables with rates i = for i =1,..., n. What is the distribution of f X X n ( x )?  X 1,..., X N has the gamma distribution with parameters n and. The sum of n identical exponentially distributed random variables

Suppose that X 1 and X 2 are independent exponential random variables with rates 1 ≠ 2. What is the distribution of f X 1 +X 2 ( x )? The sum of 2 exponentially distributed random variables

Suppose that X 1 and X 2 are independent exponential random variables with rates 1 ≠ 2. What is the distribution of f X 1 +X 2 ( x )? The sum of 2 exponentially distributed random variables

The Poisson Process

A stochastic process { N ( t ), t ≥ 0} is said to be a counting process if N ( t ) represents the total number of “events” that occur by time t (i.e., in the time interval [0, t ]). Counting Processes

A stochastic process { N ( t ), t ≥ 0} is said to be a counting process if N ( t ) represents the total number of “events” that occur by time t (i.e., in the time interval [0, t ]). Example 1: N ( t ) is the number of customers that enter a store at or prior to time t. An event corresponds to a person entering the store. Counting Processes

A stochastic process { N ( t ), t ≥ 0} is said to be a counting process if N ( t ) represents the total number of “events” that occur by time t (i.e., in the time interval [0, t ]). Example 1: N ( t ) is the number of customers that enter a store at or prior to time t. An event corresponds to a person entering the store. Example 2: N ( t ) is the number of individuals born at or prior to time t. An event occurs whenever a child is born. Counting Processes

A stochastic process { N ( t ), t ≥ 0} is said to be a counting process if N ( t ) represents the total number of “events” that occur by time t (i.e., in the time interval [0, t ]). Example 1: N ( t ) is the number of customers that enter a store at or prior to time t. An event corresponds to a person entering the store. Example 2: N ( t ) is the number of individuals born at or prior to time t. An event occurs whenever a child is born. Example 3: N ( t ) is the number of calls made to a technical help line at or prior to time t. An event occurs whenever a call is placed. Counting Processes

A counting process satisfies the following properties. (i) N ( t ) ≥ 0. (ii) N ( t ) is integer valued. (iii) If s < t, then N ( s ) ≤ N ( t ). (iv) For s < t, N ( t ) – N ( s ) equals the number of events that occurs in the time interval ( s, t ]. Properties of counting processes

A counting process is said to possess independent increments if the number of events that occur in disjoint intervals are independent.

Example 1: The number of customers N (10) that enter the store in the interval [0, 10] is independent from the number of customers N (15) – N (10) that enter the store in the interval (10, 15].

A counting process is said to possess independent increments if the number of events that occur in disjoint intervals are independent. Example 1: The number of customers N (10) that enter the store in the interval [0, 10] is independent from the number of customers N (15) – N (10) that enter the store in the interval (10, 15]. Example 2: The number of individuals N (10) born in the interval [1996, 2000] is not independent from the number of individuals N (2004) – N (2000) that enter the store in the interval (2000, 2004].

A counting process is said to possess stationary increments if the distribution of the number of events that occur in an interval depend only on the length of the interval and not the starting time of the interval.

Example 1: The number of customers N ( t ) – N ( s ) that enter the store in the interval ( s, t ] does not depend on s (this is true if there is not a particular time of day where more customers enter the store).

The Poisson processes The counting process { N ( t ) t ≥ 0} is said to be a Poisson process having rate, > 0, if (i) N ( 0 ) = 0. (ii) The process has independent increments. (iii) The number of events in any interval of length t is Poisson distributed with mean t. That is for all s, t ≥ 0

The distribution of interarrival times Let T n describe the time that elapses between ( n -1)th event and the n th event for n > 1 and let T 1 be the time of the first event. The sequence { T n, n = 1, 2,...} is called the sequence of interarrival times.

The distribution of interarrival times Let T n describe the time that elapses between ( n -1)th event and the n th event for n > 1 and let T 1 be the time of the first event. The sequence { T n, n = 1, 2,...} is called the sequence of interarrival times. Example: if T 1 = 5 and T 2 = 10  the first event arrives at time t = 5 and event 2 occurs at time t = 15.

The distribution of interarrival times P ( T 1 > t ) = P(N(t ) = 0) = e - t  T 1 has the exponential distribution.

The distribution of interarrival times P ( T 1 > t ) = P(N(t ) = 0) = e - t  T 1 has the exponential distribution. P ( T 2 > t ) = E[ P ( T 2 > t|T 1 ) ] Since P ( T 2 > t|T 1 = s ) = P (0 events in ( s, s + t ]| T 1 = s ) = P (0 events in ( s, s + t ]) = e - t Then, P ( T 2 > t ) = E[ P ( T 2 > t|T 1 ) ] = e - t  T 2 has the exponential

The distribution of interarrival times P ( T 1 > t ) = P(N(t ) = 0) = e - t  T 1 has the exponential distribution. P ( T 2 > t ) = E[ P ( T 2 > t|T 1 ) ] Since P ( T 2 > t|T 1 = s ) = P (0 events in ( s, s + t ]| T 1 = s ) = P (0 events in ( s, s + t ]) = e - t Then, P ( T 2 > t ) = E[ P ( T 2 > t|T 1 ) ] = e - t  T 2 has the exponential The same applies to other values of n  T n has the exponential distribution.

The distribution of interarrival times Let T n denote the inter-arrival time between the ( n -1)th event and the n th event of a Poisson process, then the T n ( n =1, 2,...) are independent, identically distributed exponential random variables having mean 1/.