1. Dept. of Electrical & Computer engineering
Probability and Statistics with Reliability, Queuing and Computer Science Applications: Chapter 2 on Discrete Random Variables
Dept. of Electrical & Computer engineering
Duke University
4/26/2019

2. Random Variables Sample space is often too large to deal with directly
Recall that in the sequence of Bernoulli trials, if we don’t need the detailed information about the actual pattern of 0’s and 1’s but only the number of 0’s and 1’s, we are able to reduce the sample space from size 2n to size (n+1).
Such abstractions lead to the notion of a random variable
Number : integer or real valued.
Examples of discrete RVs are, no. job or pkt arrivals in unit time, no. of rainy days in year etc.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

3. Discrete Random Variables
A random variable (rv) X is a mapping (function) from the sample space S to the set of real numbers
If image(X ) finite or countable infinite, X is a discrete rv
Inverse image of a real number x is the set of all sample points that are mapped by X into x:
It is easy to see that
Number : integer or real valued.
Examples of discrete RVs are, no. job or pkt arrivals in unit time, no. of rainy days in year etc.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

4. Probability Mass Function (pmf)
Ax : set of all sample points such that,
pmf
Pmf may also be called discrete density function.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

5. pmf Properties
Since a discrete rv X takes a finite or a countably infinite set values,
the last property above can be restated as,
All rv values satisfy p1.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

6. Distribution Function
pmf: defined for a specific rv value, i.e.,
Probability of a set
Cumulative Distribution Function (CDF)
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

7. Distribution Function properties
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

8. Discrete Random Variables
Equivalence:
Probability mass function
Discrete density function
(consider integer valued random variable)
cdf:
pmf:
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

9. Common discrete random variables
Constant
Uniform
Bernoulli
Binomial
Geometric
Poisson
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

10. Constant Random Variable
pmf
CDF
1.0 c
1.0 c
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

11. Discrete Uniform Distribution
Discrete rv X that assumes n discrete value with equal probability 1/n
Discrete uniform pmf
Discrete uniform distribution function:
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

12. Bernoulli Random Variable
RV generated by a single Bernoulli trial that has a binary valued outcome {0,1}
Such a binary valued Random variable X is called the indicator or Bernoulli random variable so that
Probability mass function:
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

13. Bernoulli Distribution
CDF
p+q=1 q x
0.0
1.0
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

14. Binomial Random Variable
Binomial rv  a fixed no. n of Bernoulli trials (BTs)
RV Yn: no. of successes in n BTs
Binomial pmf b(k;n,p)
Binomial CDF
Notes on pk: pk term signifies n-successes.
(cnk : is caused by the fact that there are these many ways in which k 1’s may appear in n-long
sequence of 1’s and 0’s e.g. (0,0,1,0,1,1,1,0,0,1,0,0,… 1)
Important: each trial is assumed to be an independent trial.
Example 2.3 notes: 3-Bernoulli trials has 8-possible outcomes,
{000, 001, 010, 100, 011, 101, 110, 111}
FX (0)  0 successes: Prob. Of event (000) = 0/125
FX (1)  at least 1 success (i.e. 0 or 1 successes): Prob. Of event ( ) = 4x0.125=0.5
FX (2)  at least 2 successes: Prob. of events (000)= FX (1) + Prob( ) =1-Prob(111)=0.75
FX (3)  1
Symmetric, +ve skewed and –ve skewed Binomial: p=0.5, < 0.5 and > 0.5
As no. of trials n increases (to infinity), B(k;n,p) can be approx. to a normal (Gaussian) distribution
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

15. Binomial Random Variable
In fact, the number of successes in n Bernoulli trials can be seen as the sum of the number of successes in each trial:
where Xi ’s are independent identically distributed Bernoulli random variables.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

16. Binomial Random Variable: pmfpk
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

17. Binomial Random Variable: CDF
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

18. Applications of the binomial
Reliability of a k out of n system
Series system:
Parallel system:
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

19. Applications of the binomial
Transmitting an LLC frame using MAC blocks
p is the prob. of correctly transmitting one block
Let pK(k) be the pmf of the rv K that is the number of LLC transmissions required to transmit n MAC blocks correctly; then
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

20. Geometric Distribution
Number of trials upto and including the 1st success.
In general, S may have countably infinite size
Z has image {1,2,3,….}. Because of independence,
1. That is, count the no of trials until the 1st success occurs. Typical output is {00001…}
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

21. Geometric Distribution (contd.)
Geometric distribution is the only discrete distribution that exhibits MEMORYLESS property.
Future outcomes are independent of the past events.
n trials completed with all failures. Y additional trials are performed before success, i.e. Z = n+Y or Y=Z-n
The last equation says that, conditioned on Z > n, the no. of trials remaining until 1st success i.e. Y=Z-n has the same pmf as Z had originally. In other words, the system does not remember how many failures it has already encountered.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

22. Geometric Distribution (contd.)
Z rv: total no. of trials upto and including the 1st success.
Modified geometric pmf: does not include the successful trial, i.e. Z=X+1. Then X is a modified geometric random variable.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

23. Applications of the geometric
The number of times the following statement is executed:
repeat S until B
is geometrically distributed assuming ….
while B do S
is modified geometrically distributed assuming ….
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

24. Negative Binomial Distribution
RV Tr: no. of trials until rth success.
Image of Tr = {r, r+1, r+2, …}. Define events:
A: Tr = n
B: Exactly r-1 successes in n-1 trials.
C: The nth trial is a success.
Clearly, since B and C are mutually independent,
We can now also define the modified –ve binomial distribution.
The resulting rv is defined as: Just the no. of failures until rth success.
Event A gets re-defined as, “Tr = n+r”  Event that there are n failures.
Event B: exactly r-1 successes in n+r-1 trials.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

25. Poisson Random Variable
RV such as “no. of arrivals in an interval (0,t)”
In a small interval, Δt, prob. of new arrival= λΔt.
pmf b(k;n, λt/n); CDF B(k;n, λt/n)=
What happens when
The problem now become similar to Bernoulli trials and Binomial distribution.
Divide the interval [0,t) into n sub-intervals, each of length t/n. For a sufficiently large n,
These n intervals can be thought as constituting a sequence of Bernoulli trials, with success
probability p= λt/n .
So now the problem can again re-defined as, finding the prob of k arrivals in a total of n intervals each of duration t/n.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

26. Poisson Random Variable (contd.)
Poisson Random Variable often occurs in situations, such as, “no. of packets (or calls) arriving in t sec.” or “no. of components failing in t hours” etc.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

27. Poisson Failure Model
Let N(t) be the number of (failure) events that occur in the time interval (0,t). Then a (homogeneous) Poisson model for N(t) assumes:
1. The probability mass function (pmf) of N(t) is:
Where l > 0 is the expected number of event occurrences per unit time
2. The number of events in two non-overlapping intervals are mutually independent
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

28. Note:
For a fixed t, N(t) is a random variable (in this case a discrete random variable known as the Poisson random variable).
The family {N(t), t  0} is a stochastic process, in this case, the homogeneous Poisson process.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

29. Poisson Failure Model (cont.)
The successive interevent times X1, X2, … in a homogenous Poisson model, are mutually independent, and have a common exponential distribution given by:
To show this:
Thus, the discrete random variable, N(t), with the Poisson distribution, is related to the continuous random variable X1, which has an exponential distribution
The mean interevent time is 1/l, which in this case is the mean time to failure
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

30. Poisson Distribution
Probability mass function (pmf) (or discrete density function):
Distribution function (CDF):
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

31. Poisson pmf pk
t=1.0
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

32. Poisson CDF t 1 2 3 4 5 6 7 8 9 10
0.5
0.1
CDF
t=1.0
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

33. Poisson pmf pk
t=4.0
t=4.0
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

34. Poisson CDF t
CDF 1 2 3 4 5 6 7 8 9 10
0.5
0.1
t=4.0
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

35. Probability Generating Function (PGF)
Helps in dealing with operations (e.g. sum) on rv’s
Letting, P(X=k)=pk , PGF of X is defined by,
One-to-one mapping: pmf (or CDF) PGF
See page 98 for PGF of some common pmfs
GX(z) is identical to the z-transform (digital filtering) of a discrete time function. If |z| < 1 (i.e. inside a unit circle), the above summation is guaranteed to converge to 1.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

36. Discrete Random Vectors
Examples:
Z=X+Y, (X and Y are random execution times)
Z = min(X, Y) or Z = max(X1, X2,…,Xk)
X:(X1, X2,…,Xk) is a k-dimensional rv defined on S
For each sample point s in S,
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

37. Discrete Random Vectors (properties)
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

38. Independent Discrete RVs
X and Y are independent iff the joint pmf satisfies:
Mutual independence also implies:
Pair wise independence vs. set-wide independence
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

39. Discrete Convolution Let Z=X+Y . Then, if X and Y are independent,
In general, then,
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University