1. Chapter 5 Joint Probability Distributions and Random Samples  5.1 - Jointly Distributed Random Variables.2 - Expected Values, Covariance, and Correlation.3 - Statistics and Their Distributions.4 - The Distribution of the Sample Mean.5 - The Distribution of a Linear Combination

2. X Density X = Age of women in U.S. at first birth Population Distribution of X Suppose X ~ N( μ, σ ), then… … etc…. μ = 25.4 σ = 1.5  x1x1 x  x  x3x3 x  x4x4 x2x2 x  x5x5 x Each of these individual ages x is a particular value of the random variable X. Most are in the neighborhood of μ, but there are occasional outliers in the tails of the distribution.

3. X = Age of women in U.S. at first birth Sample, n = 400 X Density Sample, n = 400 Sample, n = 400 Sample, n = 400 … etc…. Sample, n = 400 Population Distribution of X σ = 1.5 Suppose X ~ N( μ, σ ), then… μ = 25.4 Each of these sample mean values is a “point estimate” of the population mean μ… How are these values distributed?

4. X Density μ =μ = σ = 1.5 Density μ =μ = Suppose X ~ N( μ, σ ), then… X = Age of women in U.S. at first birth Sampling Distribution of for any sample size n. … etc…. Population Distribution of X μ = 25.4 How are these values distributed? Each of these sample mean values is a “point estimate” of the population mean μ… The vast majority of sample means are extremely close to μ, i.e., extremely small variability. “standard error”

5. Suppose X ~ N( μ, σ ), then… X Density μ =μ = σ = 2.4 Density μ =μ = X = Age of women in U.S. at first birth Sampling Distribution of for any sample size n. … etc…. Population Distribution of X μ = 25.4 “standard error” Each of these sample mean values is a “point estimate” of the population mean μ… The vast majority of sample means are extremely close to μ, i.e., extremely small variability.  for large sample size n. 

6. X ~ Anything with finite μ and σ Suppose X  N( μ, σ ), then… Suppose X ~ N( μ, σ ), then… X Density μ =μ = σ = 2.4 Density μ =μ = X = Age of women in U.S. at first birth Sampling Distribution of for any sample size n. … etc…. Population Distribution of X for large sample size n. μ = 25.4 “standard error” The vast majority of sample means are extremely close to μ, i.e., extremely small variability. Each of these sample mean values is a “point estimate” of the population mean μ… 

7. Density  “standard error”

8. Density    “standard error”  Probability that a single house selected at random costs less than $300K = ? Example: X = Cost of new house ($K) = Cumulative area under density curve for X up to 300. 300 = Z-score

9. Density    “standard error”  Probability that a single house selected at random costs less than $300K = ? Example: X = Cost of new house ($K) 300 = Z-score 0.6554

10. Density    “standard error”  Probability that the sample mean of n = 36 houses selected at random is less than $300K = ? Example: X = Cost of new house ($K) 300 $12.5K = Cumulative area under density curve for up to 300. Probability that a single house selected at random costs less than $300K = ? = Z-score 0.6554

11. Density    “standard error”  Probability that the sample mean of n = 36 houses selected at random is less than $300K = ? Example: X = Cost of new house ($K) 300 $12.5K Probability that a single house selected at random costs less than $300K = ? = Z-score 0.6554 = Z-score 0.9918

12.  Density  “standard error”    approximately  mild skew large 

13. Density  “standard error”    approximately  continuous or discrete, large as n  ,   ~ CENTRAL LIMIT THEOREM ~ 

14. Density  “standard error”    approximately   continuous or discrete, large as n  ,   ~ CENTRAL LIMIT THEOREM ~ Example: X = Cost of new house ($K)

15. Density  “standard error” Density   Example: X = Cost of new house ($K) 300 $12.5K Probability that the sample mean of n = 36 houses selected at random is less than $300K = ? Probability that a single house selected at random costs less than $300K = ? = Z-score 0.9918 = Cumulative area under density curve for X up to 300.

16. 16

17. 17 xf(x)f(x) 00.5 100.3 200.2

18. 18 0.25 5.30 =.15+.15 10.29 =.10+.09+.10 15.12 =.06+.06 20.04

19. 19

20. 20

21. 21 possibly log-normal… each based on 1000 samples but remember Cauchy and 1/x 2, both of which had nonexistent  … CLT may not work! heavily skewed tail More on CLT…

22. More on CLT… X Density  X = Age of women in U.S. at first birth Population Distribution of X Random Variable If this first individual has been randomly chosen, and the value of X measured, then the result is a fixed number x 1, with no random variability… and likewise for x 2, x 3, etc. DATA! X ~ Dist ( μ, σ )

23. More…  X = Age of women in U.S. at first birth Population Distribution of X X Density Random Variable If this first individual has been randomly chosen, and the value of X measured, then the result is a fixed number x 1, with no random variability… and likewise for x 2, x 3, etc. DATA! However, if this is not the case, then this first “value” of X is unknown, thus can be considered as a random variable X 1 itself… and likewise for X 2, X 3, etc. The collection {X 1, X 2, X 3, …, X n } of “independent, identically-distributed” (i.i.d.) random variables is said to be a random sample. X ~ Dist ( μ, σ )

24. More… X = Age of women in U.S. at first birth Population Distribution of X X Density Random Variable X ~ Dist ( μ, σ ) Sample, size n Density etc…… Claim: Proof: Sampling Distribution of for any n

25. More… X = Age of women in U.S. at first birth Population Distribution of X X Density Random Variable X ~ Dist ( μ, σ ) Density etc…… Claim: Proof: Sampling Distribution of for any n

26. M o r e o n C L T … R e c a l l … Normal Approximation to the Binomial Distribution 26 continuousdiscrete P(Success) =  P(Failure) = 1 –  Discrete random variable X = # Successes (0, 1, 2,…, n) in a random sample of size n Suppose a certain outcome exists in a population, with constant probability . We will randomly select a random sample of n individuals, so that the binary “Success vs. Failure” outcome of any individual is independent of the binary outcome of any other individual, i.e., n Bernoulli trials (e.g., coin tosses). Then X is said to follow a Binomial distribution, written X ~ Bin(n,  ), with “probability function” f(x) =, x = 0, 1, 2, …, n.

27. Normal Approximation to the Binomial Distribution 27 continuousdiscrete P(Success) =  P(Failure) = 1 –  Discrete random variable X = # Successes (0, 1, 2,…, n) in a random sample of size n Suppose a certain outcome exists in a population, with constant probability . We will randomly select a random sample of n individuals, so that the binary “Success vs. Failure” outcome of any individual is independent of the binary outcome of any other individual, i.e., n Bernoulli trials (e.g., coin tosses). Then X is said to follow a Binomial distribution, written X ~ Bin(n,  ), with “probability function” f(x) =, x = 0, 1, 2, …, n. CLT See Prob 5.3/7

28. Normal Approximation to the Binomial Distribution 28 continuousdiscrete P(Success) =  P(Failure) = 1 –  Discrete random variable X = # Successes (0, 1, 2,…, n) in a random sample of size n Suppose a certain outcome exists in a population, with constant probability . We will randomly select a random sample of n individuals, so that the binary “Success vs. Failure” outcome of any individual is independent of the binary outcome of any other individual, i.e., n Bernoulli trials (e.g., coin tosses). Then X is said to follow a Binomial distribution, written X ~ Bin(n,  ), with “probability function” f(x) =, x = 0, 1, 2, …, n. CLT ??

29. 29 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation)  = “true” population mean of a numerical random variable X, where  = “true” population probability of Success, where “Success vs. Failure” are the only possible binary outcomes. In general….  In general….  want “nice” properties

30. 30 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation) want “nice” properties In general….  In general….  (see page 253)

31. 31 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation) want “nice” properties In general….  In general….  Recall: Rearrange terms:

32. 32 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation) want “nice” properties In general….  In general….  Recall: Ideally, we would like to minimize MSE, but this is often difficult in practice. However, if Bias = 0, then MSE = Variance, so it is desirable to seek Minimum Variance Unbiased Estimators (MVUE)…