1. MOMENT GENERATING FUNCTION AND STATISTICAL DISTRIBUTIONS

2. MOMENT GENERATING FUNCTION
The m.g.f. of random variable X is defined as
for t Є (-h,h) for some h>0.

3. Properties of m.g.f. M(0)=E[1]=1
If a r.v. X has m.g.f. M(t), then Y=aX+b has a m.g.f.
M.g.f does not always exists (e.g. Cauchy distribution)

4. Example Suppose that X has the following p.d.f.
Find the m.g.f; expectation and variance.

5. CHARACTERISTIC FUNCTION
The c.h.f. of random variable X is defined as
for all real numbers t.
C.h.f. always exists.

6. Uniqueness
Theorem:
If two r.v.s have mg.f.s that exist and are equal, then they have the same distribution.
If two r,v,s have the same distribution, then they have the same m.g.f. (if they exist)
Similar statements are true for c.h.f.

7. STATISTICAL DISTRIBUTIONS

8. SOME DISCRETE PROBABILITY DISTRIBUTIONS
Degenerate, Uniform, Bernoulli, Binomial, Poisson, Negative Binomial, Geometric, Hypergeometric

9. DEGENERATE DISTRIBUTION
An rv X is degenerate at point k if
The cdf:

10. UNIFORM DISTRIBUTION
A finite number of equally spaced values are equally likely to be observed.
Example: throw a fair die. P(X=1)=…=P(X=6)=1/6

11. BERNOULLI DISTRIBUTION
A Bernoulli trial is an experiment with only two outcomes. An r.v. X has Bernoulli(p) distribution if

12. BINOMIAL DISTRIBUTION
Define an rv Y by
Y = total number of successes in n Bernoulli trials.
There are n trials (n is finite and fixed).
2. Each trial can result in a success or a failure.
3. The probability p of success is the same for all the trials.
4. All the trials of the experiment are independent.

13. BINOMIAL DISTRIBUTION
Example:
There are black and white balls in a box. Select and record the color of the ball. Put it back and re-pick (sampling with replacement).
n: number of independent and identical trials
p: probability of success (e.g. probability of picking a black ball)
X: number of successes in n trials

14. BINOMIAL THEOREM
For any real numbers x and y and integer n>0

15. BINOMIAL DISTRIBUTION
If Y~Bin(n,p), then

16. POISSON DISTRIBUTION
The number of occurrences in a given time interval can be modeled by the Poisson distribution.
e.g. waiting for bus, waiting for customers to arrive in a bank.
Another application is in spatial distributions.
e.g. modeling the distribution of bomb hits in an area or the distribution of fish in a lake.

17. POISSON DISTRIBUTION
If X~ Poi(λ), then E(X)= Var(X)=λ

18. Relationship between Binomial and Poisson
Let =np.
The mgf of Poisson()
The limiting distribution of Binomial rv is the Poisson distribution.

19. NEGATIVE BINOMIAL DISTRIBUTION (PASCAL OR WAITING TIME DISTRIBUTION)
X: number of Bernoulli trials required to get a fixed number of failures before the r th success; or, alternatively,
Y: number of Bernoulli trials required to get a fixed number of successes, such as r successes.

20. NEGATIVE BINOMIAL DISTRIBUTION (PASCAL OR WAITING TIME DISTRIBUTION)
X~NB(r,p)

21. NEGATIVE BINOMIAL DISTRIBUTION
An alternative form of the pdf:
Note: Y=X+r

22. GEOMETRIC DISTRIBUTION
Distribution of the number of Bernoulli trials required to get the first success.
It is the special case of the Negative Binomial Distribution r=1.
X~Geometric(p)

23. GEOMETRIC DISTRIBUTION
Example: If probability is that a light bulb will fail on any given day, then what is the probability that it will last at least 30 days?
Solution:

24. HYPERGEOMETRIC DISTRIBUTION
A box contains N marbles. Of these, M are red. Suppose that n marbles are drawn randomly from the box without replacement. The distribution of the number of red marbles, x is
X~Hypergeometric(N,M,n)
It is dealing with finite population.

25. SOME CONTINUOUS PROBABILITY DISTRIBUTIONS
Uniform, Normal, Exponential, Gamma, Chi-Square, Beta Distributions

26. Uniform Distribution
A random variable X is said to be uniformly distributed if its density function is
The expected value and the variance are

27. Uniform Distribution Example 1
The daily sale of gasoline is uniformly distributed between 2,000 and 5,000 gallons. Find the probability that sales are:
Between 2,500 and 3,000 gallons
More than 4,000 gallons
Exactly 2,500 gallons
f(x) = 1/( ) = 1/3000 for x: [2000,5000]
P(2500£X£3000) = ( )(1/3000) = .1667
1/3000 x
2000
2500
3000
5000

28. Uniform Distribution Example 1
The daily sale of gasoline is uniformly distributed between 2,000 and 5,000 gallons. Find the probability that sales are:
Between 2,500 and 3,500 gallons
More than 4,000 gallons
Exactly 2,500 gallons
f(x) = 1/( ) = 1/3000 for x: [2000,5000]
P(X³4000) = ( )(1/3000) = .333
1/3000 x
2000
4000
5000

29. Uniform Distribution Example 1
The daily sale of gasoline is uniformly distributed between 2,000 and 5,000 gallons. Find the probability that sales are:
Between 2,500 and 3,500 gallons
More than 4,000 gallons
Exactly 2,500 gallons
f(x) = 1/( ) = 1/3000 for x: [2000,5000]
P(X=2500) = ( )(1/3000) = 0
1/3000 x
2000
2500
5000

30. Normal Distribution This is the most popular continuous distribution.
Many distributions can be approximated by a normal distribution.
The normal distribution is the cornerstone distribution of statistical inference.

31. Normal Distribution
A random variable X with mean m and variance s2 is normally distributed if its probability density function is given by

32. The Shape of the Normal Distribution
The normal distribution is bell shaped, and symmetrical around m. 90 m
110
Why symmetrical? Let m = Suppose x = 110.
Now suppose x = 90

33. The Effects of m and s
How does the standard deviation affect the shape of f(x)?
s= 2
s =3
s =4
How does the expected value affect the location of f(x)?
m = 10
m = 11
m = 12

34. Finding Normal Probabilities
Two facts help calculate normal probabilities:
The normal distribution is symmetrical.
Any normal distribution can be transformed into a specific normal distribution called…
“STANDARD NORMAL DISTRIBUTION”
Example
The amount of time it takes to assemble a computer is normally distributed, with a mean of 50 minutes and a standard deviation of 10 minutes. What is the probability that a computer is assembled in a time between 45 and 60 minutes?

35. STANDARD NORMAL DISTRIBUTION
NORMAL DISTRIBUTION WITH MEAN 0 AND VARIANCE 1.
IF X~N( , 2), THEN
NOTE: Z IS KNOWN AS Z SCORES.
“ ~ “ MEANS “DISTRIBUTED AS”

36. Finding Normal Probabilities
Solution
If X denotes the assembly time of a computer, we seek the probability P(45<X<60).
This probability can be calculated by creating a new normal variable the standard normal variable.
Every normal variable
with some m and s, can
be transformed into this Z.
Therefore, once probabilities for Z
are calculated, probabilities of any
normal variable can be found.
E(Z) = m = 0
V(Z) = s2 = 1

37. Standard normal probabilities
Copied from Walck, C (2007) Handbook on Statistical Distributions for experimentalists

38. Standard normal table 1

39. Standard normal table 2

40. Standard normal table 3

41. Finding Normal Probabilities
Example - continued 45
- 50 X
- m 60
- 50
P(45<X<60) = P( < < ) 10 s 10
= P(-0.5 < Z < 1)
To complete the calculation we need to compute
the probability under the standard normal distribution

42. Using the Standard Normal Table
Standard normal probabilities have been
calculated and are provided in a table .
P(0<Z<z0)
The tabulated probabilities correspond
to the area between Z=0 and some Z = z0 >0
Z = 0
Z = z0

43. Finding Normal Probabilities
Example - continued
P(45<X<60) = P( < < ) 45 X 60
- m
- 50 s 10
= P(-.5 < Z < 1)
We need to find the shaded area
z0 = 1
z0 = -.5

44. Finding Normal Probabilities
Example - continued
P(45<X<60) = P( < < ) 45 X 60
- m
- 50 s 10
= P(-.5<Z<1) =
P(-.5<Z<0)+ P(0<Z<1)
P(0<Z<1
.3413
z=0
z0 = 1
z0 =-.5

45. Finding Normal Probabilities
The symmetry of the normal distribution makes it possible to calculate probabilities for negative values of Z using the table as follows:
-z0
+z0
P(-z0<Z<0) = P(0<Z<z0)

46. Finding Normal Probabilities
Example - continued
.1915
.3413
-.5 .5

47. Finding Normal Probabilities
Example - continued
.1915
.1915
.1915
.1915
.3413
-.5 .5
1.0
P(-.5<Z<1) = P(-.5<Z<0)+ P(0<Z<1) = = .5328

48. Finding Normal Probabilities
Example - continued
.3413
P(Z<-0.5)=1-P(Z>-0.5)= =0.3085
By Symmetry
P(Z<0.5)

49. Finding Normal Probabilities
Example
The rate of return (X) on an investment is normally distributed with a mean of 10% and standard deviation of (i) 5%, (ii) 10%.
What is the probability of losing money? X
10% 0%
0 - 10 5
(i) P(X< 0 ) = P(Z< ) = P(Z< - 2)
.4772 Z -2 2
=P(Z>2) =
0.5 - P(0<Z<2) = = .0228

50. Finding Normal Probabilities
Example
The rate of return (X) on an investment is normally distributed with mean of 10% and standard deviation of (i) 5%, (ii) 10%.
What is the probability of losing money? X
10% 0% 1
0 - 10 10
(ii) P(X< 0 ) = P(Z< )
.3413 Z -1
= P(Z< - 1) = P(Z>1) =
0.5 - P(0<Z<1) = = .1587

51. AREAS UNDER THE STANDARD NORMAL DENSITY
P(0<Z<1)=.3413 Z 1

52. AREAS UNDER THE STANDARD NORMAL DENSITY
.3413
.4772
P(1<Z<2)= =.1359

53. EXAMPLES P( Z < 0.94 ) = 0.5 + P( 0 < Z < 0.94 )
= =
0.8264
0.94

54. EXAMPLES P( Z > 1.76 ) = 0.5 – P( 0 < Z < 1.76 )
= 0.5 – =
0.0392
1.76

55. EXAMPLES P( -1.56 < Z < 2.13 ) =
= P( < Z < 0 ) + P( 0 < Z < 2.13 )
= =
Because of symmetry
P(0 < Z < 1.56)
0.9240
-1.56
2.13

56. STANDARDIZATION FORMULA
If X~N( , 2), then the standardized value Z of any ‘X-score’ associated with calculating probabilities for the X distribution is:
The standardized value Z of any ‘X-score’ associated with calculating probabilities for the X distribution is:
(Converse Formula)

57. Finding Values of Z
Sometimes we need to find the value of Z for a given probability
We use the notation zA to express a Z value for which P(Z > zA) = A A zA

58. PERCENTILE
The pth percentile of a set of measurements is the value for which at most p% of the measurements are less than that value.
80th percentile means P( Z < a ) = 0.80
If Z ~ N(0,1) and A is any probability, then
P( Z > zA) = A A zA

59. Finding Values of Z Example Solution
Determine z exceeded by 5% of the population
Determine z such that 5% of the population is below
Solution
z.05 is defined as the z value for which the area on its right under the standard normal curve is .05.
0.45
0.05
0.05
-Z0.05
Z0.05
1.645

60. Standardization formula
EXAMPLES
Let X be rate of return on a proposed investment. Mean is 0.30 and standard deviation is 0.1.
a) P(X>.55)=?
b) P(X<.22)=?
c) P(.25<X<.35)=?
d) 80th Percentile of X is?
e) 30th Percentile of X is?
Standardization formula
Converse Formula

61. ANSWERS a) b) c) d) e)
80th Percentile of X is
30th Percentile of X is

62. The Normal Approximation to the Binomial Distribution
The normal distribution provides a close approximation to the Binomial distribution when n (number of trials) is large and p (success probability) is close to 0.5.
The approximation is used only when
np  5 and
n(1-p)  5

63. The Normal Approximation to the Binomial Distribution
If the assumptions are satisfied, the Binomial random variable X can be approximated by normal distribution with mean  = np and 2 = np(1-p).
In probability calculations, the continuity correction improves the results. For example, if X is Binomial random variable, then
P(X  a) ≈ P(X<a+0.5)
P(X  a) ≈ P(X>a-0.5)

64. EXAMPLE Let X ~ Binomial(25,0.6), and want to find P(X ≤ 13).
Exact binomial calculation:
Normal approximation (w/o correction): Y~N(15,2.45²)
25*0.6=15; 25*0.6*0.4=2.45**2
Normal approximation is good, but not great!

65. EXAMPLE, cont. Bars – Bin(25,0.6); line – N(15, 2.45²)
Copied from Casella & Berger (1990)

66. Much better approximation to the exact value: 0.267
EXAMPLE, cont.
Normal approximation (w correction):
Y~N(15,2.45²)
Much better approximation to the exact value: 0.267

67. Exponential Distribution
The exponential distribution can be used to model
the length of time between telephone calls
the length of time between arrivals at a service station
the lifetime of electronic components.
When the number of occurrences of an event follows the Poisson distribution, the time between occurrences follows the exponential distribution.

68. Exponential Distribution
A random variable is exponentially distributed if its probability density function is given by f(x) = e-x, x>=0.
 is the distribution parameter >0).
E(X) =  V(X) = 2
 is a distribution parameter.
The cumulative distribution function is
F(x) =1e-x/, x0

69. Exponential distribution for l-1 = .5, 1, 2
f(x) = 2e-2x
f(x) = 1e-1x
f(x) = .5e-.5x
P(a<X<b) = e-a/ e-b/ a b

70. Exponential Distribution
Finding exponential probabilities is relatively easy:
P(X < a) = P(X ≤ a)=F(a)=1 – e –a/ 
P(X > a) = e–a/ 
P(a< X < b) = e – a/ – e – b/ 

71. Exponential Distribution
Example
The service rate at a supermarket checkout is 6 customers per hour.
If the service time is exponential, find the following probabilities:
A service is completed in 5 minutes,
A customer leaves the counter more than 10 minutes after arriving
A service is completed between 5 and 8 minutes.

72. Exponential Distribution
Solution
A service rate of 6 per hour = A service rate of .1 per minute (l-1 = .1/minute).
P(X < 5) = 1-e-.lx = 1 – e-.1(5) = .3935
P(X >10) = e-.lx = e-.1(10) = .3679
P(5 < X < 8) = e-.1(5) – e-.1(8) =

73. Exponential Distribution
If pdf of lifetime of fluorescent lamp is exponential with mean 0.10, find the life for 95% reliability?
The reliability function = R(t) = 1F(t) = e-t/
R(t) = 0.95  t = ?
R(t)=P(T>t)=exp(-t/0.1)=0.95 solve for t. t=
What is the mean time to failure?

74. GAMMA DISTRIBUTION
X~ Gamma(,)
In graph, k=alpha, theta=beta

75. GAMMA DISTRIBUTION Gamma Function: where  is a positive integer.
Properties:

76. GAMMA DISTRIBUTION
Let X1,X2,…,Xn be independent rvs with Xi~Gamma(i, ). Then,
Let X be an rv with X~Gamma(, ). Then,
Let X1,X2,…,Xn be a random sample with Xi~Gamma(, ). Then,

77. GAMMA DISTRIBUTION Special cases: Suppose X~Gamma(α,β)
If α=1, then X~ Exponential(β)
If α=p/2, β=2, then X~ 2 (p) (will come back in a min.)
If Y=1/X, then Y ~ inverted gamma.

78. Integral tricks Recall: h(t) is an odd function if h(-t)=-h(t);
It is an even function if h(-t)=h(t).
Ex: h(x)=x exp{-x²/2} odd;
h(x)= exp{-x²/2-x} neither odd nor even
odd function =0
even function = 2* even function

79. CHI-SQUARE DISTRIBUTION
Chi-square with  degrees of freedom
X~ 2()= Gamma(/2,2)
In graphs, k=alpha.

80. DEGREES OF FREEDOM
In statistics, the phrase degrees of freedom is used to describe the number of values in the final calculation of a statistic that are free to vary.
The number of independent pieces of information that go into the estimate of a parameter is called the degrees of freedom (df) .
How many components need to be known before the vector is fully determined?

81. CHI-SQUARE DISTRIBUTION
If rv X has Gamma(,) distribution, then Y=2X/ has Gamma(,2) distribution. If 2 is positive integer, then Y has
distribution.
Let X be an rv with X~N(0, 1). Then,
Let X1,X2,…,Xn be a r.s. with Xi~N(0,1). Then,

82. BETA DISTRIBUTION
The Beta family of distributions is a continuous family on (0,1) and often used to model proportions.
where

83. CAUCHY DISTRIBUTION
It is a symmetric and bell-shaped distribution on (,) with pdf
Since
, the mean does not exist.
The mgf does not exist.
 measures the center of the distribution and it is the median.
If X and Y have N(0,1) distribution, then Z=X/Y has a Cauchy distribution with =0 and σ=1.

84. LOG-NORMAL DISTRIBUTION
An rv X is said to have the lognormal distribution, with parameters µ and 2, if Y=ln(X) has the N(µ, 2) distribution.
The lognormal distribution is used to model continuous random quantities when the distribution is believed to be skewed, such as certain income and lifetime variables.

85. STUDENT’S T DISTRIBUTION
This distribution will arise in the study of population mean when the underlying distribution is normal.
Let Z be a standard normal rv and let U be a chi-square distributed rv independent of Z, with  degrees of freedom. Then,
When n, XN(0,1).

86. F DISTRIBUTION
Let U and V be independent rvs with chi-square distributions with 1 and 2 degrees of freedom. Then,

87. MULTIVARIATE DISTRIBUTIONS

88. EXTENDED HYPERGEOMETRIC DISTRIBUTION
Suppose that a collection consists of a finite number of items, N and that there are k+1 different types; M1 of type 1, M2 of type 2, and so on. Select n items at random without replacement, and let Xi be the number of items of type i that are selected. The vector X=(X1, X2,…,Xk) has an extended hypergeometric distribution and the joint pdf is

89. MULTINOMIAL DISTRIBUTION
Let E1,E2,...,Ek,Ek+1 be k+1 mutually exclusive and exhaustive events which can occur on any trial of an experiment with P(Ei)=pi,i=1,2,…,k+1. On n independent trials of the experiment, let Xi be the number of occurrences of the event Ei. Then, the vector X=(X1, X2,…,Xk) has a multinomial distribution with joint pdf

90. BIVARIATE NORMAL DISTRIBUTION
A pair of continuous rvs X and Y is said to have a bivariate normal distribution if it has a joint pdf of the form

91. BIVARIATE NORMAL DISTRIBUTIONIf
, then
and  is the correlation coefficient btw X and Y.
1. Conditional on X=x,
2. Conditional on Y=y,