1. EE255/CPS226 Continuous Random Variables
Dept. of Electrical & Computer engineering
Duke University
2/24/2019

2. Definitions Distribution function:
If FX(x) is a continuous function of x, then X is a continuous random variable.
FX(x): discrete in x  Discrete rv’s
FX(x): piecewise continuous  Mixed rv’s
If X is to qualify as a rv, on the space (S,F,P) then we should be able to define prob. measure for X. This in turn implies that P(X <= x) be defined for all x. This would require the existence of the event
{s| X<= s} belonging to F.
2. F(x) is actually absolutely continuous i.e. its derivative is well defined except possibly at the end points.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

3. Probability Density Function (pdf)
X : continuous rv, then,
pdf properties:
Sometimes we may have deal with mixed (discrete+continuous) type of rv’s as well. See Fig. 3.2 and understand it.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

4. Exponential Distribution
Arises commonly in reliability & queuing theory.
It exhibits memory-less (Markov) property.
Related to Poisson distribution
Inter-arrival time between two IP packets (or voice calls)
Time interval between failures, etc.
Mathematically,
No. of failure in a given interval may follow Poisson distribution.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

5. Exp Distribution: Memory-less Property
A light bulb is replaced only after it has failed.
Conversely, a critical space shuttle component is replaced after some fixed no. of hours of use. Thus exhibiting memory property.
Wait time in a queue at the check-in counter?
Exp( ) distribution exhibits the useful memory-less property, i.e. the future occurrence of random event (following Exp( ) distribution) is independent of when it occurred last.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

6. Memory-less Property (contd.)
Assuming rv X follows Exp( ) distribution,
Memory-less property: find P( ) at a future point.
X > u, is the life time, y is the residual life time
Exp( ) distribution plays a fundamental role in studying reliability theory and performance analysis. Like wise, Guassian (or normal) distribution plays a fundamental role in statistical (or random) signal processing theory, communication theory, antenna arrays processing etc. However, we defer the discussion on Guassian random variables for now.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

7. Memory-less Property (contd.)
If the component’s life time is exponentially distributed, then,
The remaining life time does not depend on how long it has already working.
If inter-arrival times (between calls) are exponentially distributed, then, time we need still wait for a new arrival is independent of how long we have already waited.
Memory-less property a.k.a Markov property
Converse is also true, i.e. if X satisfies Markov property, then it must follow Exp() distribution.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

8. Reliability & Failure Rate Theory
Reliability R(t): failure occurs after time ‘t’. Let X be the life time of a component subject to failures.
N0: total components (fixed); Ns: survived ones
f(t)Δt : unconditional prob(fail) in the interval (t, t+Δt]
conditional failure prob.?
Failure density function: f(t)t is the probability that the component will fail in the interval (t, t ].
Instantaneous Failure rate function h(t): is the conditional probability that the component will fail in the interval (t, t ], given that it survived until time t.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

9. Reliability & Failure Rate Theory (contd.)
Instantaneous failure rate: h(t) (#failures/10k hrs)
Let f(t) (failure density fn) be EXP( λ). Then,
Using simple calculus,
R(t): P(X > t). For Exp() distribution, P(-infinity < X <= t) = 1-exp(-λt). Therefore,
P(X>t) = 1- (1-exp(-λt)) = exp(-λt)
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

10. Failure Behaviors
There are other failure density functions that can be used to model DFR, IFR (or mixed) failure behavior
DFR
IFR
CFR
Failure rate
Time
DFR phase: Initial design, constant bug fixes
CFR phase: Normal operational phase
IFR phase: Aging behavior
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

11. HypoExponential HypoExp: multiple Exp stages.
2-stage HypoExp denoted as HYPO(λ1, λ2). The density, distribution and hazard rate function are:
HypoExp results in IFR: 0  min(λ1, λ2)
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

12. Erlang Distribution
Special case of HypoExp: All r stages are identical.
[X > t] = [Nt < r] (Nt : no. of stresses applied in (0,t] and Nt is Possion (param λt). This interpretation gives,
r=1 case of Erlang distribution reduces to the Exp( ) distribution case.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

13. Gamma Distribution Gamma density function is,
Gamma distribution can capture all three failure models, viz. DFR, CFR and IFR.
α = 1: CFR
α <1 : DFR
α >1 : IFR
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

14. HyperExponential Distribution
Hypo or Erlang  Sequential Exp( ) stages.
Parallel Exp( ) stages  HyperExponential.
Sum of k Exp( ) also gives k-stage HyperExp
CPU service time may be modeled as HyperExp
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

15. Weibull Distribution α is called the shape parameter.
Frequently used to model fatigue failure, ball bearing failure etc. (very long tails)
Weibull distribution is also capable of modeling DFR (α < 1), CFR (α = 1) and IFR (α >1).
α is called the shape parameter.
Weibull distribution may include the 3rd parameter sometimes. It is called the location parameter that has the effect of shifting the origin. That is, F(t) = F(t-θ) (RHS is the original Weibull distribution).
F(t) = 1 – exp{–λ(t- θ)α}
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

16. Log-logistic Distribution
Log-logistic can model DFR, CFR and IFR failure models simultaneously, unlike previous ones.
For, κ > 1, the failure rate first increases with t (IFR); after momentarily leveling off (CFR), it decreases (DFR) with time, all within the same distribution.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

17. Gaussian (Normal) Distribution
Bell shaped – intuitively pleasing!
Central Limit Theorem: mean of a large number of mutually independent rv’s (having arbitrary distributions) starts following Normal distribution as n 
μ: mean, σ: std. deviation, σ2: variance (N(μ, σ2))
μ and σ completely describe the statistics. This is significant in statistical estimation/signal processing/communication theory etc.
In these areas, very often, we have to solve optimization problems that call minimizing the variance in estimating a parameter, detecting a signal, testing a hypothesis. By assuming the distributions to be Normal, variance minimization (Least-mean-square-error (LSME) or Min. mean square error (mmse)) results are globally optimal.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

18. Normal Distribution (contd.)
N(0,1) is called normalized Guassian.
N(0,1) is symmetric i.e.
f(x)=f(-x)
F(z) = 1-F(z).
Failure rate h(t) follows IFR behavior.
Hence, N( ) is suitable for modeling long-term wear or aging related failure phenomena.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

19. Uniform Distribution U(a,b)  constant over the (a,b) interval
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

20. Defective DistributionsIf
Example:
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

21. Functions of Random Variables
Often, rv’s need to be transformed/operated upon.
Y = Φ (X) : so, what is the distribution of Y ?
Example: Y = X2
If fX(x) is N(0,1), then,
Above fY(y) is also known as the χ2 distribution (with 1-d of freedom).
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

22. Functions of R V’s (contd.)
If X is uniformly distributed, then, Y= -λ-1ln(1-X) follows Exp( ) distribution
transformations may be used to synthetically generate random numbers with desired distributions.
Computer Random No. generators may employ this method.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

23. Functions of R V’s (contd.)
Given,
A monotone differentiable function,
Above method suggests a way to get the desired CDF, given some other simple type of CDF. This allows generation of random variables with desired distribution.
Choose Φ to be F.
Since, Y=F(X), FY(y) = y and Y is U(0,1).
To generate a random variable with X having desired distribution, choose generate U(0,1) random variable Y, then transform y to x= F-1(y) .
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

24. Jointly Distributed RVs
Joint Distribution Function:
Independent rv’s: iff the following holds:
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

25. Joint Distribution Properties
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

26. Joint Distribution Properties (contd)
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

27. Order Statistics (min, max function)
Define Yk ( known as the kth order statistics)
Y1 = min{X1, X2, …, Xn}
Yn = max{X1, X2, …, Xn}
Permute {Xi} so that {Yi} are sorted (ascending order)
Y1 : life of a system with ‘series’ of components.
Yn : with ‘parallel’ (redundant) set of components.
Distribution of Yk ?
Prob. that exactly j of Xi values are in (-∞,y] and remaining (n-j) values in (y, ∞] is:
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

28. Sorted random sequence Yk
Observe that there are at least k Xi’s that are<= y. Some of the remaining
Xi’s may Also be <= y
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

29. Sorted RV’s (contd) In general,
Using FY(y), reliability may be computed as,
In general,
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

30. Sorted RV’s: min case (contd)
ith component’s life time: EXP(λi), then,
Hence, life time for such a system also has EXP() distribution with,
For the parallel case, the resulting distribution is not EXP( )
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

31. Sum of Random Variables
Z = Φ(X, Y)  ((X, Y) may not be independent)
For the special case, Z = X + Y
The resulting pdf is,
Convolution integral
If X and Y are mutually independent, the resulting pdf is just the product of two pdf’s rather than
Convolution.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

32. Sum of Random Variables (contd.)
X1, X2, .., Xk are ‘iid’ rv’c, and Xi ~ EXP(λ), then rv (X1+ X2+ ..+Xk) is k-stage Erlang with param λ.
If Xi ~ EXP(λi), then, rv (X1+ X2+ ..+Xk) is k-stage HypoExp( ) distribution. Specifically, for Z=X+Y,
In general,
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

33. Sum of Normal Random Variables
X1, X2, .., Xk are normal ‘iid’ rv’c, then, the rv Z = (X1+ X2+ ..+Xk) is also normal with,
X1, X2, .., Xk are normal. Then,
follows Gamma or the χ2 (with n-deg of freedom) distributions
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

34. Sum of RVs: Standby Redundancy
Two independent components, X and Y
Series system (Z=min(X,Y))
Parallel System (Z=max(X,Y))
Cold standby: the life time Z=X+Y
If X and Y are EXP(λ), then,
i.e., Z is Gamma distributed, and,
May be extended 1+2 cold-standbys  TMR
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

35. k-out of-n Order Statistics
Order statistics Yn-k+1 of (X1, X2, .. Xn) is:
P(Yn-k+1 ) : HYPO(nλ ,(n-1)λ , kλ )
Proof by induction:
n=2 case: k=2  Y1 = parallel; Y2 =series
F(Y1 ) = (F[y])2 or F(Yn ) = (F[y])n
Y1 distribution? : Y1 is the residual life time.
If all Xi ’s are EXP(λ)  memory-less property I.e. residual life time is independent of how long the component has already survived.
Hence, Y1 distribution is also EXP(λ).
Y1 (of Yn-k+1) that any 1-out of-n components is working (note that this is the reverse convention discussed earlier, where Y1 denotes series connection i.e. all n components are working). So, Y2 will denote 2 (both) –out of-n components working, Yn will denote n-out of-n (series) components are working.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University

36. k-out of-n Order Statistics (contd)
Assume n-parallel components. Then, Y1: 1st component failure or min{X1, X2, .. Xn}.
2nd failure would occur within Y2 = Y1 + min{X’1, X’2, .. Xn’}. Xi’s are the residual times of surving components. But due to memory-less property, Xi’s are independent of past failure behavior. Therefore, F( min{X’2, X’3, .. Xn’}) is EXP((n-1) λ). In general, for k-out of-n (k are working)
Yn-k+1 = HYPO(nλ, (n-1)λ, .., kλ)
EXP(nλ)
EXP((n-1)λ)
EXP((n-k+1)λ)
EXP(λ) Y1 Y2
Yn-k+1 Yn
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University