Chapter 5 Joint Probability Distributions and Random Samples  Jointly Distributed Random Variables.2 - Expected Values, Covariance, and Correlation.3 - Statistics and Their Distributions.4 - The Distribution of the Sample Mean.5 - The Distribution of a Linear Combination

X Density X = Age of women in U.S. at first birth Population Distribution of X Suppose X ~ N( μ, σ ), then… … etc…. μ = 25.4 σ = 1.5  x1x1 x  x  x3x3 x  x4x4 x2x2 x  x5x5 x Each of these individual ages x is a particular value of the random variable X. Most are in the neighborhood of μ, but there are occasional outliers in the tails of the distribution.

X = Age of women in U.S. at first birth Sample, n = 400 X Density Sample, n = 400 Sample, n = 400 Sample, n = 400 … etc…. Sample, n = 400 Population Distribution of X σ = 1.5 Suppose X ~ N( μ, σ ), then… μ = 25.4 Each of these sample mean values is a “point estimate” of the population mean μ… How are these values distributed?

X Density μ =μ = σ = 1.5 Density μ =μ = Suppose X ~ N( μ, σ ), then… X = Age of women in U.S. at first birth Sampling Distribution of for any sample size n. … etc…. Population Distribution of X μ = 25.4 How are these values distributed? Each of these sample mean values is a “point estimate” of the population mean μ… The vast majority of sample means are extremely close to μ, i.e., extremely small variability. “standard error”

Suppose X ~ N( μ, σ ), then… X Density μ =μ = σ = 2.4 Density μ =μ = X = Age of women in U.S. at first birth Sampling Distribution of for any sample size n. … etc…. Population Distribution of X μ = 25.4 “standard error” Each of these sample mean values is a “point estimate” of the population mean μ… The vast majority of sample means are extremely close to μ, i.e., extremely small variability.  for large sample size n. 

X ~ Anything with finite μ and σ Suppose X  N( μ, σ ), then… Suppose X ~ N( μ, σ ), then… X Density μ =μ = σ = 2.4 Density μ =μ = X = Age of women in U.S. at first birth Sampling Distribution of for any sample size n. … etc…. Population Distribution of X for large sample size n. μ = 25.4 “standard error” The vast majority of sample means are extremely close to μ, i.e., extremely small variability. Each of these sample mean values is a “point estimate” of the population mean μ… 

Density  “standard error”

Density    “standard error”  Probability that a single house selected at random costs less than $300K = ? Example: X = Cost of new house ($K) = Cumulative area under density curve for X up to = Z-score

Density    “standard error”  Probability that a single house selected at random costs less than $300K = ? Example: X = Cost of new house ($K) 300 = Z-score

Density    “standard error”  Probability that the sample mean of n = 36 houses selected at random is less than $300K = ? Example: X = Cost of new house ($K) 300 $12.5K = Cumulative area under density curve for up to 300. Probability that a single house selected at random costs less than $300K = ? = Z-score

Density    “standard error”  Probability that the sample mean of n = 36 houses selected at random is less than $300K = ? Example: X = Cost of new house ($K) 300 $12.5K Probability that a single house selected at random costs less than $300K = ? = Z-score = Z-score

 Density  “standard error”    approximately  mild skew large 

Density  “standard error”    approximately  continuous or discrete, large as n  ,   ~ CENTRAL LIMIT THEOREM ~ 

Density  “standard error”    approximately   continuous or discrete, large as n  ,   ~ CENTRAL LIMIT THEOREM ~ Example: X = Cost of new house ($K)

Density  “standard error” Density   Example: X = Cost of new house ($K) 300 $12.5K Probability that the sample mean of n = 36 houses selected at random is less than $300K = ? Probability that a single house selected at random costs less than $300K = ? = Z-score = Cumulative area under density curve for X up to 300.

16

17 xf(x)f(x)

= = =

19

20

21 possibly log-normal… each based on 1000 samples but remember Cauchy and 1/x 2, both of which had nonexistent  … CLT may not work! heavily skewed tail More on CLT…

More on CLT… X Density  X = Age of women in U.S. at first birth Population Distribution of X Random Variable If this first individual has been randomly chosen, and the value of X measured, then the result is a fixed number x 1, with no random variability… and likewise for x 2, x 3, etc. DATA! X ~ Dist ( μ, σ )

More…  X = Age of women in U.S. at first birth Population Distribution of X X Density Random Variable If this first individual has been randomly chosen, and the value of X measured, then the result is a fixed number x 1, with no random variability… and likewise for x 2, x 3, etc. DATA! However, if this is not the case, then this first “value” of X is unknown, thus can be considered as a random variable X 1 itself… and likewise for X 2, X 3, etc. The collection {X 1, X 2, X 3, …, X n } of “independent, identically-distributed” (i.i.d.) random variables is said to be a random sample. X ~ Dist ( μ, σ )

More… X = Age of women in U.S. at first birth Population Distribution of X X Density Random Variable X ~ Dist ( μ, σ ) Sample, size n Density etc…… Claim: Proof: Sampling Distribution of for any n

More… X = Age of women in U.S. at first birth Population Distribution of X X Density Random Variable X ~ Dist ( μ, σ ) Density etc…… Claim: Proof: Sampling Distribution of for any n

M o r e o n C L T … R e c a l l … Normal Approximation to the Binomial Distribution 26 continuousdiscrete P(Success) =  P(Failure) = 1 –  Discrete random variable X = # Successes (0, 1, 2,…, n) in a random sample of size n Suppose a certain outcome exists in a population, with constant probability . We will randomly select a random sample of n individuals, so that the binary “Success vs. Failure” outcome of any individual is independent of the binary outcome of any other individual, i.e., n Bernoulli trials (e.g., coin tosses). Then X is said to follow a Binomial distribution, written X ~ Bin(n,  ), with “probability function” f(x) =, x = 0, 1, 2, …, n.

Normal Approximation to the Binomial Distribution 27 continuousdiscrete P(Success) =  P(Failure) = 1 –  Discrete random variable X = # Successes (0, 1, 2,…, n) in a random sample of size n Suppose a certain outcome exists in a population, with constant probability . We will randomly select a random sample of n individuals, so that the binary “Success vs. Failure” outcome of any individual is independent of the binary outcome of any other individual, i.e., n Bernoulli trials (e.g., coin tosses). Then X is said to follow a Binomial distribution, written X ~ Bin(n,  ), with “probability function” f(x) =, x = 0, 1, 2, …, n. CLT See Prob 5.3/7

Normal Approximation to the Binomial Distribution 28 continuousdiscrete P(Success) =  P(Failure) = 1 –  Discrete random variable X = # Successes (0, 1, 2,…, n) in a random sample of size n Suppose a certain outcome exists in a population, with constant probability . We will randomly select a random sample of n individuals, so that the binary “Success vs. Failure” outcome of any individual is independent of the binary outcome of any other individual, i.e., n Bernoulli trials (e.g., coin tosses). Then X is said to follow a Binomial distribution, written X ~ Bin(n,  ), with “probability function” f(x) =, x = 0, 1, 2, …, n. CLT ??

29 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation)  = “true” population mean of a numerical random variable X, where  = “true” population probability of Success, where “Success vs. Failure” are the only possible binary outcomes. In general….  In general….  want “nice” properties

30 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation) want “nice” properties In general….  In general….  (see page 253)

31 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation) want “nice” properties In general….  In general….  Recall: Rearrange terms:

32 PARAMETER ESTIMATOR (Not to be confused with an “estimate”) SAMPLING DISTRIBUTION (or approximation) want “nice” properties In general….  In general….  Recall: Ideally, we would like to minimize MSE, but this is often difficult in practice. However, if Bias = 0, then MSE = Variance, so it is desirable to seek Minimum Variance Unbiased Estimators (MVUE)…