1. Schmidt & Hunter Approach to r
Bare Bones

2. Statistical Artifacts
Extraneous factors that influence observed effect
Sampling error*
Reliability
Range restriction
Computational error
Dichotomization of variables
*addressed in the (bare-bones) analysis

3. Bare Bones r Find weighted mean and variance: Note sample size weight.
Note that for unit weights, the weighted variance estimator is the sample, not population, estimate.

4. Confidence Interval for Mean
There are k studies, with Ni observations.
This is not the only formula they use, but it’s the best one IMHO.

5. Estimated Sampling Error Variance
The variance of r
Estimated variance for a study.
Estimated sampling variance for a meta-analysis. Note mean r is constant. This is the variance of sampling error we expect if all the studies have a common effect estimated by r-bar.

6. Variance of Rho
Classical Test Theory
Sampling Error
A definition

7. Estimated Variance of rho
Note that the variance of rho will be called tau-squared by Hedges -
To find the variance of infinite-sample correlations, find the variance of r in the meta-analysis and subtract expected sampling error variance. Schmidt would be quick to add that part of the estimated variance of infinite-sample correlations is artifactual.

8. Credibility Interval
The credibility interval and the confidence interval are quite different things. The CI is a standard statistical estimate (intended to contain rho, or average of rho). The CR is intended to contain a percentage of the values of a random variable – infinite-sample effect sizes. The S&H value forgets that there is also uncertainty in the mean value; the two should be added. There are Bayesian programs that will do this; there is also an approximation called the prediction interval described in Borenstein et al.

9. Bare-Bones Example (1) Study Ni r 1 200 .20 2 100 3 150 .40 4 80 Mean
132.5
.30
<- Unit weighted mean

10. Bare-Bones Example (2) r Ni rNi .20 200 40 100 20 .40 150 60 80 32 sum
530
152

11. BB Example (3) Recall unwighted or unit weighted mean = .30.
Why are they different?

12. BB Example (4) r Ni
.20
200
1.507
100
.753
.40
150
1.922 80
1.025
Sum
530
5.208

13. BB Example (5)

14. Interpretation
Schmidt says this is a random-effects meta- analysis. It uses a sample of studies to represent a larger population of studies.
People interpret the Credibility Interval, but typically do not recognize that it is poorly estimated.