1. William R. Shadish University of California, Merced
Recent Work
William R. Shadish
University of California, Merced

2. Recent and Current Research
Empirical Studies of Research Methods
Good nonrandomized experiments
Regression discontinuity designs
Analysis and Meta-Analysis of Single-Case Designs
Causal Theory

3. Nonrandomized Experiments
Empirical studies of the conditions under which nonrandomized experiments can approximate results from randomized designs.
Shadish, Clark & Steiner (2008) JASA
Cook, Shadish & Wong (2008) JPPAM

4. Results Really good quasi-experimental design helps a lot
Especially careful attention to a well-matched intact control group.
Reduces the amount of bias to be reduced
Analytic adjustments (PS, ANCOVA) can work, but only if
You have good measures of the selection process
Those measures have high reliability

5. Regression Discontinuity Designs
Assignment to conditions based on a cutoff on a measured pretest variable
We have done similar empirical studies of the conditions under which RDD matches results from RE, with similarly encouraging results

6. Single-Case Designs
On the What Works Clearinghouse Technical Advisory Group, David Rindskopf and I (later joined by Larry Hedges) noted the absence of attention to results from single case designs (SCDs) in deciding what works.
The problem was lack of accepted statistical methods for the analysis and meta-analysis of those designs.
Our research
Developing a d-estimator for SCD
Continued development of multilevel models.

7. Causal Theory
Campbell’s theory of causal inference is over 50 years old (Campbell & Stanley, 1957)
Where does Donald Rubin and Rubin’s Causal Model fit in?
Forthcoming special section of Psychological Methods on this.
Where does Judea Pearl’s graphical approach to causal inference fit in?
Current area of study
In preparation chapter in an APA Handbook of Research Methods probably to come out in 2011 or 2012.

8. Anticipating Some Questions
Are we revising Shadish Cook and Campbell?
Not yet, but we are actively considering it.
It would be nice first to solve the problem of the analysis of SCD (as it relates to the analysis of short time series)
Are we revising Shadish Cook and Leviton (Foundations of Program Evaluation)?
Sadly, probably not.
I’m getting a little out of touch with evaluation theory, as a lot has happened in the last 20 years.

9. Further Readings
Cook, T.D., Shadish, W.R., & Wong, V.C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management, 27,
Shadish, W.R., Rindskopf, D.M. & Hedges, L.V. (2008). The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 3,
Shadish, W.R., Clark, M.H., & Steiner, P.M. (2008). Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random to Nonrandom Assignment. Journal of the American Statistical Association, 103,
Shadish, W.R., & Cook, T.D. (2009). The Renaissance of Field Experimentation in Evaluating Interventions. Annual Review of Psychology, 60,
Shadish, W.R. (in press). Campbell and Rubin: A primer and comparison of their approaches to causal inference in field settings. Psychological Methods.

10. THANKS!
To My Many Coauthors (Tom Cook, MH Clark, Peter Steiner, Larry Hedges, David Rindskopf, Vivian Wong)
and
To You