1. H676 Meta-Analysis Brian Flay WEEK 1 Fall 2016 Thursdays 4-6:50
Waldo 421

2. Week 1 Outline Why do a SR and MA Steps in doing a SR
Eiman
What a Meta-Analysis does
More about protocols
Extracting and coding data
Jafra

3. Types of review designs/methods
Gough et al., 2012

4. Why do a Systematic Review (SR) or a Meta-Analysis (MA)?
Bias in traditional reviews
Traditional reviews were not systematic
Some reviewers had a double standard
Reviewers influenced by pet theories
“Expert” reviewers find studies to suit their purposes/theories
Reliance on p-values
Information overload

5. When is a SR of value? When there is uncertainty
To inform practice and policy
Use “Evidence-based” interventions
Reviews are the unit of progress in science (not individual studies)
As paradigm shifters
Overcome historical assumptions and biases

6. Steps in a SR & MA Clearly define the question (e.g., PICOC)
Determine types of studies - inclusion and exclusion criteria
Conduct comprehensive search, screen results, apply criteria
Critically appraise quality of included studies (Week 2)
Synthesize the studies (estimate and analyze effect sizes) (Weeks 1 & 3)
Consider fixed- and random-effects models (Week 3)
Assess heterogeneity of the studies and conduct subgroup analyses (Week 4)
Consider power, publication and reporting biases, etc. (Weeks 5-7)
Disseminate the findings (Week 8)

7. Be careful - no magic bullet!
Reviews are the unit of progress in science (not individual studies) – but …
Blanket judgments about effectiveness can be meaningless
because they involve extrapolating
from the average study (which may not exist)
to the average citizen (who certainly does not exist).
It’s an iterative process

8. More on finding studies
Eiman
Searching the literature (DeCoster et al., 2009)
Comparison of Web of Science and Google Scholar (Levay et al., 2016)
When does it make sense to perform a MA? (Text, chapter 40)
How far should you go? (Ogilvie et al., 2005a)
How low should you go? (Ogilvie et al., 2005b)

9. Sorts of studies to include
The Hierarchy of Evidence (for effectiveness studies)
Systematic reviews and meta-analyses
Randomized Controlled Trials w definitive results
RCTs with non-definitive results
Prospective cohort studies (& other time-series studies)
Case-control studies (& other quasi-experimental)
Cross-sectional surveys (observational studies)
Case reports
Qualitative studies
It all depends on the question

10. Other inclusion and exclusion criteria
Types of participants
Age range, race/ethnicity, conditions
Types of interventions
Universal vs targeting
Group-based vs one-on-one
Environmental changes vs behavior change
Types of outcome measures
Next slide

11. Inclusion criteria for outcomes
Immediate effects of the interventions
These can be surrogate outcomes
Health behaviors
Primary health outcomes
Long-term consequences
There could be multiple measures of any of these. For most MAs, all should be included and coded.

12. What a MA does Compute an effect size (ES) and variance for each study
Compute a weighted mean ES and its variance
Allows us to make generalizations based on multiple studies, rather than forming conclusions from single studies

13. Example of a forest plot from CMA
This one uses Risk Ratios
Average ES is almost 1.0 (.919)
Plot shows ES in each study with SE and weight
Results consistently show that use of Tamiflu is not significantly associated with risk of hospitalization.

14. Studies with sig. overall ES, even though half of the results were ns & Q is ns

15. Protocol Outline Background Objectives Methods:
Criteria for selecting studies for this review:
Types of studies
Types of participants
Types of interventions
Types of outcome measures
Search methods for identification of studies
Data collection and analysis

16. Summary overview of MA
A statistical technique for combining the findings from independent studies.
Most often used to assess the effectiveness of interventions; by combining data from 2 or more RCTs.
MA of trials provides a precise estimate of intervention effect, weighted by the size of the different studies.
The validity of a meta-analysis depends on the quality of the systematic review on which it is based.
Good meta-analyses look for heterogeneity, and explore the robustness of the main findings using sensitivity analysis.
Adapted from Crombie & Davies, 2009 (Cover)

17. Extracting and Coding data
Jafra
Coding studies - DeCosta, 2009
Data extraction and synthesis - Munn et al., 2014
Preferred reporting items – PRISMA-PP - Moher et al., 2015
Case Studies – Borenstein, 2016