H676 Meta-Analysis Brian Flay WEEK 1 Fall 2016 Thursdays 4-6:50 Waldo 421

Week 1 Outline Why do a SR and MA Steps in doing a SR Eiman What a Meta-Analysis does More about protocols Extracting and coding data Jafra

Types of review designs/methods Gough et al., 2012

Why do a Systematic Review (SR) or a Meta-Analysis (MA)? Bias in traditional reviews Traditional reviews were not systematic Some reviewers had a double standard Reviewers influenced by pet theories “Expert” reviewers find studies to suit their purposes/theories Reliance on p-values Information overload

When is a SR of value? When there is uncertainty To inform practice and policy Use “Evidence-based” interventions Reviews are the unit of progress in science (not individual studies) As paradigm shifters Overcome historical assumptions and biases

Steps in a SR & MA Clearly define the question (e.g., PICOC) Determine types of studies - inclusion and exclusion criteria Conduct comprehensive search, screen results, apply criteria Critically appraise quality of included studies (Week 2) Synthesize the studies (estimate and analyze effect sizes) (Weeks 1 & 3) Consider fixed- and random-effects models (Week 3) Assess heterogeneity of the studies and conduct subgroup analyses (Week 4) Consider power, publication and reporting biases, etc. (Weeks 5-7) Disseminate the findings (Week 8)

Be careful - no magic bullet! Reviews are the unit of progress in science (not individual studies) – but … Blanket judgments about effectiveness can be meaningless because they involve extrapolating from the average study (which may not exist) to the average citizen (who certainly does not exist). It’s an iterative process

More on finding studies Eiman Searching the literature (DeCoster et al., 2009) Comparison of Web of Science and Google Scholar (Levay et al., 2016) When does it make sense to perform a MA? (Text, chapter 40) How far should you go? (Ogilvie et al., 2005a) How low should you go? (Ogilvie et al., 2005b)

Sorts of studies to include The Hierarchy of Evidence (for effectiveness studies) Systematic reviews and meta-analyses Randomized Controlled Trials w definitive results RCTs with non-definitive results Prospective cohort studies (& other time-series studies) Case-control studies (& other quasi-experimental) Cross-sectional surveys (observational studies) Case reports Qualitative studies It all depends on the question

Other inclusion and exclusion criteria Types of participants Age range, race/ethnicity, conditions Types of interventions Universal vs targeting Group-based vs one-on-one Environmental changes vs behavior change Types of outcome measures Next slide

Inclusion criteria for outcomes Immediate effects of the interventions These can be surrogate outcomes Health behaviors Primary health outcomes Long-term consequences There could be multiple measures of any of these. For most MAs, all should be included and coded.

What a MA does Compute an effect size (ES) and variance for each study Compute a weighted mean ES and its variance Allows us to make generalizations based on multiple studies, rather than forming conclusions from single studies

Example of a forest plot from CMA This one uses Risk Ratios Average ES is almost 1.0 (.919) Plot shows ES in each study with SE and weight Results consistently show that use of Tamiflu is not significantly associated with risk of hospitalization.

Studies with sig. overall ES, even though half of the results were ns & Q is ns

Protocol Outline Background Objectives Methods: Criteria for selecting studies for this review: Types of studies Types of participants Types of interventions Types of outcome measures Search methods for identification of studies Data collection and analysis

Summary overview of MA A statistical technique for combining the findings from independent studies. Most often used to assess the effectiveness of interventions; by combining data from 2 or more RCTs. MA of trials provides a precise estimate of intervention effect, weighted by the size of the different studies. The validity of a meta-analysis depends on the quality of the systematic review on which it is based. Good meta-analyses look for heterogeneity, and explore the robustness of the main findings using sensitivity analysis. Adapted from Crombie & Davies, 2009 (Cover)

Extracting and Coding data Jafra Coding studies - DeCosta, 2009 Data extraction and synthesis - Munn et al., 2014 Preferred reporting items – PRISMA-PP - Moher et al., 2015 Case Studies – Borenstein, 2016