1. Continuous Random Variables
Probability and Statistics with Reliability, Queuing and Computer Science Applications: Chapter 3
Continuous Random Variables

2. Definitions Distribution function:
If FX(x) is a continuous function of x, then X is a continuous random variable.
FX(x): discrete in x  Discrete rv’s
FX(x): piecewise continuous  Mixed rv’s
If X is to qualify as a rv, on the space (S,F,P) then we should be able to define prob. measure for X. This in turn implies that P(X <= x) be defined for all x. This would require the existence of the event
{s| X<= s} belonging to F.
2. F(x) is actually absolutely continuous i.e. its derivative is well defined except possibly at the end points.

3. Definitions (Continued)
Equivalence:
CDF (cumulative distribution function)
PDF (probability distribution function)
Distribution function
FX(x) or FX(t) or F(t)

4. Probability Density Function (pdf)
X : continuous rv, then,
pdf properties:
Sometimes we may have deal with mixed (discrete+continuous) type of rv’s as well. See Fig. 3.2 and understand it.

5. Definitions (Continued)
Equivalence: pdf
probability density function
density function
density
f(t) =
For a non-negative
random variable

6. Exponential Distribution
Arises commonly in reliability & queuing theory.
A non-negative random variable
It exhibits memoryless (Markov) property.
Related to (the discrete) Poisson distribution
Interarrival time between two IP packets (or voice calls)
Time to failure, time to repair etc.
Mathematically (CDF and pdf, respectively):
No. of failure in a given interval may follow Poisson distribution.

7. CDF of exponentially distributed random variable with  = 0.0001
F(t) t

8. Exponential Density Function (pdf)
f(t) t

9. Memoryless property
Assume X > t. We have observed that the component has not failed until time t.
Let Y = X - t , the remaining (residual) lifetime
The distribution of the remaining life, Y, does not depend on how long the component has been operating. Distribution of Y is identical to that of X.

10. Memoryless property
Assume X > t. We have observed that the component has not failed until time t.
Let Y = X - t , the remaining (residual) lifetime

11. Memoryless property (Continued)
Thus Gt(y) is independent of t and is identical to the original exponential distribution of X.
The distribution of the remaining life does not depend on how long the component has been operating.
Its eventual breakdown is the result of some suddenly appearing failure, not of gradual deterioration.

12. Reliability as a Function of Time
Reliability R(t): failure occurs after time ‘t’. Let X be the lifetime of a component subject to failures.
Let N0: total no. of components (fixed); Ns(t): surviving ones; Nf(t): failed one by time t.
Failure density function: f(t)t is the probability that the component will fail in the interval (t, t ].
Instantaneous Failure rate function h(t): is the conditional probability that the component will fail in the interval (t, t ], given that it survived until time t.

13. Definitions (Continued)
Equivalence:
Reliability
Complementary distribution function
Survivor function
R(t) = 1 -F(t)

14. Failure Rate or Hazard Rate
Instantaneous failure rate: h(t) (#failures/10k hrs)
Let the rv X be EXP( λ). Then,
Using simple calculus the following apples to any rv,
R(t): P(X > t). For Exp() distribution, P(-infinity < X <= t) = 1-exp(-λt). Therefore,
P(X>t) = 1- (1-exp(-λt)) = exp(-λt)

15. Hazard Rate and the pdf
h(t) t = Conditional Prob. system will fail in
(t, t + t) given that it has survived until time t
f(t) t = Unconditional Prob. System will fail in
(t, t + t)
Difference between:
probability that someone will die between 90 and 91, given that he lives to 90
probability that someone will die between 90 and 91

16. Weibull Distribution
Frequently used to model fatigue failure, ball bearing failure etc. (very long tails)
Reliability:
Weibull distribution is capable of modeling DFR (α < 1), CFR (α = 1) and IFR (α >1) behavior.
α is called the shape parameter and  is the scale parameter
Weibull distribution may include the 3rd parameter sometimes. It is called the location parameter that has the effect of shifting the origin. That is, F(t) = F(t-θ) (RHS is the original Weibull distribution).
F(t) = 1 – exp{–λ(t- θ)α}

17. Failure rate of the weibull distribution with various values of  and  = 1
5.0

18. Infant Mortality Effects in System Modeling
Bathtub curves
Early-life period
Steady-state period
Wear out period
Failure rate models

19. Bathtub Curve
Until now we assumed that failure rate of equipment is time (age) independent. In real-life, variation as per the bathtub shape has been observed
Failure Rate l(t)
Infant Mortality
(Early Life Failures)
Steady State
Wear out
Operating Time

20. Early-life Period
Also called infant mortality phase or reliability growth phase
Caused by undetected hardware/software defects that are being fixed resulting in reliability growth
Can cause significant prediction errors if steady-state failure rates are used
Availability models can be constructed and solved to include this effect
Weibull Model can be used

21. Steady-state Period Failure rate much lower than in early-life period
Either constant (age independent) or slowly varying failure rate
Failures caused by environmental shocks
Arrival process of environmental shocks can be assumed to be a Poisson process
Hence time between two shocks has the exponential distribution

22. Wear out Period Failure rate increases rapidly with age
Properly qualified electronic hardware do not exhibit wear out failure during its intended service life (Motorola)
Applicable for mechanical and other systems
Weibull Failure Model can be used

23. Bathtub curve DFR phase: Initial design, constant bug fixes
CFR phase: Normal operational phase
IFR phase: Aging behavior
h(t)
(burn-in-period)
(wear-out-phase)
CFR
(useful life)
DFR
IFR t
Decreasing failure rate
Increasing fail. rate

24. Failure-Rate Multiplier
Failure Rate Models
We use a truncated Weibull Model
Infant mortality phase modeled by DFR Weibull and the steady-state phase by the exponential 7 6 5 4 3 2 1
Failure-Rate Multiplier
2,190
4,380
6,570
8,760
10,950
13,140
15,330
17,520
Operating Times (hrs)

25. Failure Rate Models (cont.)
This model has the form:
where:
steady-state failure rate
is the Weibull shape parameter
Failure rate multiplier =

26. Failure Rate Models (cont.)
There are several ways to incorporate time dependent failure rates in availability models
The easiest way is to approximate a continuous function by a decreasing step function 7 6 5 4 3 2 1
Failure-Rate Multiplier
2,190
4,380
6,570
8,760
10,950
13,140
15,330
17,520
Operating Times (hrs)

27. Failure Rate Models (cont.)
Here the discrete failure-rate model is defined by:

28. Uniform Random Variable
All (pseudo) random generators generate random deviates of U(0,1) distribution; that is, if you generate a large number of random variables and plot their empirical distribution function, it will approach this distribution in the limit.
U(a,b)  pdf constant over the (a,b) interval and CDF is the ramp function

29. Uniform density

30. { Uniform distribution The distribution function is given by:
0 , x < a,
F(x)= , a < x < b
1 , x > b. {

31. Uniform distribution (Continued)

32. HypoExponential HypoExp: multiple Exp stages in series.
2-stage HypoExp denoted as HYPO(λ1, λ2). The density, distribution and hazard rate function are:
HypoExp results in IFR: 0  min(λ1, λ2)
Disk service time may be modeled as a 3-stage Hypoexponential as the overall time is the sum of the seek, the latency and the transfer time

33. HypoExponential used in software rejuvenation models
Preventive maintenance is useful only if failure rate is increasing
Robust state
A simple and useful model of increasing failure rate:
Failure probable state
Failed state
Time to failure: Hypo-exponential distribution
Increasing failure rate aging

34. Erlang Distribution
Special case of HypoExp: All stages have same rate.
[X > t] = [Nt < r] (Nt : no. of stresses applied in (0,t]) and Nt is Possion (param λt). This interpretation gives,
r=1 case of Erlang distribution reduces to the Exp( ) distribution case.

35. Is used to approximate the deterministic one
Erlang Distribution
Is used to approximate the deterministic one
since if you keep the same mean but increase the number of stages, the pdf approaches the delta function in the limit
Can also be used to approximate the uniform distribution
r=1 case of Erlang distribution reduces to the Exp( ) distribution case.

36. probability density functions (pdf)
If we vary r keeping r/ constant, pdf of r-stage Erlang approaches an impulse function at r/ .

37. cumulative distribution functions (cdf)
And the cdf approaches a step function at r/. In other words
r-stage Erlang can approximate a deterministic variable.

38. Comparison of probability density functions (pdf)

39. Comparison of cumulative distribution functions (cdf)

40. Gamma Random Variable Gamma density function is,
Gamma distribution can capture all three failure modes, viz. DFR, CFR and IFR.
α = 1: CFR
α <1 : DFR
α >1 : IFR
Gamma with α = ½ and  = n/2 is known as the chi-square random variable with n degrees of freedom

41. HyperExponential Distribution
Hypo or Erlang  Sequential Exp( ) stages.
Alternate Exp( ) stages  HyperExponential.
CPU service time may be modeled as HyperExp
In workload based software rejuvenation model we found the sojourn times in many workload states have this distribution

42. Log-logistic Distribution
Log-logistic can model DFR, CFR and IFR failure models simultaneously, unlike previous ones.
For, κ > 1, the failure rate first increases with t (IFR); after momentarily leveling off (CFR), it decreases (DFR) with time. This is known as the inverse bath tub shape curve
Use in modeling software reliability growth

43. Hazard rate comparison

44. Defective DistributionIf
Example:
This defect (also known as the mass at infinity) could represent the probability that the program will not terminate (1-c). Continuous part can model completion time of program.
There can also be a mass at origin.

45. Pareto Random Variable
Also known as the power law or long-tailed distribution
Found to be useful in modeling
CPU time consumed by a request
Webfile sizes
Number of data bytes in FTP bursts
Thinking time of a Web browser

46. Gaussian (Normal) Distribution
Bell shaped pdf – intuitively pleasing!
Central Limit Theorem: mean of a large number of mutually independent rv’s (having arbitrary distributions) starts following Normal distribution as n 
μ: mean, σ: std. deviation, σ2: variance (N(μ, σ2))
μ and σ completely describe the statistics. This is significant in statistical estimation/signal processing/communication theory etc.
In these areas, very often, we have to solve optimization problems that call minimizing the variance in estimating a parameter, detecting a signal, testing a hypothesis. By assuming the distributions to be Normal, variance minimization (Least-mean-square-error (LSME) or Min. mean square error (mmse)) results are globally optimal.

47. Normal Distribution (contd.)
N(0,1) is called normalized Guassian.
N(0,1) is symmetric i.e.
f(x)=f(-x)
F(z) = 1-F(z).
Failure rate h(t) follows IFR behavior.
Hence, N( ) is suitable for modeling long-term wear or aging related failure phenomena.

48. Functions of Random Variables
Often, rv’s need to be transformed/operated upon.
Y = Φ (X) : so, what is the density of Y ?
Example: Y = X2
If X is N(0,1), then,
Above Y is also known as the χ2 distribution (with 1-degree of freedom).

49. Functions of RV’s (contd.)
If X is uniformly distributed, then, Y= -λ-1ln(1-X) follows Exp(λ) distribution
transformations may be used to generate random variates (or deviates) with desired distributions.

50. Functions of RV’s (contd.)
Given,
A monotone differentiable function,
Above method suggests a way to get the random variates with desired distribution.
Choose Φ to be F.
Since, Y=F(X), FY(y) = y and Y is U(0,1).
To generate a random variate with X having desired distribution, generate U(0,1) random variable Y, then transform y to x= F-1(y) .
This inversion can be done in closed-form, graphically or using a table.

51. Jointly Distributed RVs
Joint Distribution Function:
Independent rv’s: iff the following holds:

52. Joint Distribution Properties

53. Joint Distribution Properties (contd)

54. Order statistics: kofn, TMR

55. Order Statistics: KofN
X1 ,X2 ,..., Xn iid (independent and identically distributed) random variables with a common distribution function F().
Let Y1 ,Y2 ,...,Yn be random variables obtained by permuting the set X1 ,X2 ,..., Xn so as to be in
increasing order.
To be specific:
Y1 = min{X1 ,X2 ,..., Xn} and
Yn = max{X1 ,X2 ,..., Xn}

56. Order Statistics: KofN (Continued)
The random variable Yk is called the k-th ORDER STATISTIC.
If Xi is the lifetime of the i-th component in a system of n components. Then:
Y1 will be the overall series system lifetime.
Yn will denote the lifetime of a parallel system.
Yn-k+1 will be the lifetime of an k-out-of-n system.

57. Order Statistics: KofN (Continued)
To derive the distribution function of Yk, we note that the
probability that exactly j of the Xi's lie in (- ,y] and (n-j) lie in (y, ) is:

58. Applications of order statistics
Reliability of a k out of n system
Series system:
Parallel system:
Minimum of n EXP random variables is special case of Y1 = min{X1,…,Xn} where Xi~EXP(i)
Y1~EXP( i)
This is not true (that is EXP dist.) for the parallel case

59. Triple Modular Redundancy (TMR)
R(t)
Voter
R(t)
R(t)
An interesting case of order statistics occurs when we consider the Triple Modular Redundant (TMR) system (n = 3 and k = 2). Y2 then denotes the time until the second component fails. We get:

60. TMR (Continued)
Assuming that the reliability of a single component is given by,
we get:

61. TMR (Continued)
In the following figure, we have plotted RTMR(t) vs t as well as R(t) vs t.

62. TMR (Continued)

74. Cold standby (dynamic redundancy)X Y
Lifetime of
Active
EXP()
Lifetime of
Spare
EXP()
Total lifetime 2-Stage Erlang
EXP()
Assumptions: Detection & Switching perfect; spare does not fail

75. Sum of RVs: Standby Redundancy
Two independent components, X and Y
Series system (Z=min(X,Y))
Parallel System (Z=max(X,Y))
Cold standby: the life time Z=X+Y

76. Sum of Random Variables
Z = Φ(X, Y)  ((X, Y) may not be independent)
For the special case, Z = X + Y
The resulting pdf is (assuming independence),
Convolution integral (modify for the non-negative case)
If X and Y are mutually independent, the resulting pdf is just the product of two pdf’s rather than
Convolution.

77. Convolution (non-negative case)
Z = X + Y, X & Y are independent random variables (in this case, non-negative)
The above integral is often called the convolution of fX and fY. Thus the density of the sum of two non-negative independent, continuous random variables is the convolution of the individual densities.

78. Cold standby derivation
X and Y are both EXP() and independent.
Then

79. Cold standby derivation (Continued)
Z is two-stage Erlang Distributed

80. Convolution: Erlang Distribution
The general case of r-stage Erlang Distribution
When r sequential phases have independent identical exponential distributions, then the resulting density is known as r-stage (or r-phase) Erlang and is given by:

81. Convolution: Erlang (Continued)
EXP()

82. Warm standby With Warm spare, we have:
Active unit time-to-failure: EXP()
Spare unit time-to-failure: EXP()
2-stage hypoexponential distribution
EXP(+ )
EXP()

83. Warm standby derivation
First event to occur is that either the active or the spare will fail. Time to this event is min{EXP(),EXP()} which is EXP( + ).
Then due to the memoryless property of the exponential, remaining time is still EXP().
Hence system lifetime has a two-stage hypoexponential distribution with parameters
1 =  +  and 2 =  .

84. Warm standby derivation (Continued)
X is EXP(1) and Y is EXP(2) and are independent
1 = 2
Then fZ(t) is

85. Hot standby With hot spare, we have:
Active unit time-to-failure: EXP()
Spare unit time-to-failure: EXP()
2-stage hypoexponential
EXP(2)
EXP()

86. TMR and TMR/simplex as hypoexponentials

87. Hypoexponential: general case
Z = , where X1 ,X2 , … , Xr
are mutually independent and Xi is exponentially
distributed with parameter i (i = j for i = j).
Then Z is a r-stage hypoexponentially
distributed random variable.
EXP(1)
EXP(2)
EXP(r)

88. Hypoexponential: general case

89. KofN system lifetime as a hypoexponential
At least, k out of n units should be operational for the
system to be Up.
EXP(n)
EXP((n-1))
EXP(k)
EXP((k-1))
EXP()
...
... Y1 Y2
Yn-k+1
Yn-k+2 Yn

90. KofN with warm spares
At least, k out of n + s units should be operational
for the system to be Up. Initially n units are active
and s units are warm spares.
EXP(n
s)
EXP(n
+(s-1) )
EXP(n
+ )
EXP(n)
EXP(k)
...
...

91. Sum of Normal Random Variables
X1, X2, .., Xk are normal ‘iid’ rv’s, then, the rv Z = (X1+ X2+ ..+Xk) is also normal with,
X1, X2, .., Xk are normal. Then,
follows Gamma or the χ2 (with n-degrees of freedom) distribution