The Question The first step is deciding on the question to be asked: E.g., Shadish and Baldwin 2005: What are the effects of behavioral marital therapy (BMT) compared to no treatment? The question initially need not be too specific—better to be inclusive at first and then exclude later. You may discover operationalizations of constructs that you did not anticipate Broader inclusion helps explore generalization

The Literature Search: What? Inclusion Exclusion Criteria Which treatments, people, outcomes? For example, in Shadish’s meta-analysis of BMT: Including Cognitive Behavioral? Effects on anything? Just on divorce or the Marital Attitude Survey? Excluding studies of “enrichment”? Which designs: e.g., RCTs only? Language restrictions? Limited to certain years? Both published and unpublished sources?

The Literature Search: Where? The Bibliographies of Past Meta-Analyses and Literature Reviews. The Bibliography of every pertinent study retrieved Standard Electronic Databases E.g., PubMed, Web of Science, PsycInfo, Dissertation Abstracts International What keywords, how far back? Hand Searches of Journals (Recent Years) Funding agencies for final reports. A serious search for unpublished studies? Contacts with Experts Registries of Studies Initiated Archives of convention programs and papers Keep a record of keywords used: Keep a flow chart and record of inclusions-exclusions:

Example Saab et al. 2009 “Oral Antibiotic Prophylaxis Reduces spontaneous Bacterial Peritonities Occurrence and Improves Short-Term Survival in Cirrhosis: A Meta-Analysis” 8 Randomized Trials 647 Patients Search results reported in flow chart

Sample Flow Chart

Most Common Types of Effects Correlations (probably transformed) Standardized mean differences Risk ratios Odds ratios Lots more to say about these later

Fixed- and Random-Effects Meta-Analysis Simple fixed-effects MA Fixed-effects MA with moderators Simple random-effects MA Random-effects MA with moderators (also known as “mixed-effects meta-analysis”)

Most Common Types of Meta-Analysis Fixed-effects MA: fixed effects.pdf Random-effects MA: random effects.pdf Choice of fixed- versus random-effects analysis (traditional and WRONG) Heterogeneity tests What question are you asking? (universe of generalization) Almost always, the correct choice is a prior random-effects model (controversial)

Illustration, and Introduction of metafor Teacher expectancy data (Raudenbush and Bryk) Reading data into R Acquiring the metafor package Simple analyses using metafor What is metafor actually doing?