Systematic Review & Meta-analysis

Practising EBM – the 4 A’s Step 4 Apply the evidence Step 3 Step 2 Appraise the evidence Step 1 Acquire the best evidence Ask a clinical question

Hierarchy of evidence

What is a systematic review or meta-analysis? A systematic review answers a defined research question by collecting and summarising all empirical evidence that fits pre- specified eligibility criteria. A meta-analysis is the use of statistical methods to summarise the results of these studies.

Why we need systematic reviews Over 20 million citations in PubMed Approx. 75 to 100 RCTs published daily Usually impossible to consider all relevant individual primary research studies in a decision making context

Why we need systematic reviews Minimise the impact of bias/errors Can help to end confusion Highlight where there is not sufficient evidence Combining findings from different studies can highlight new findings Can mitigate the need for further trials Enable practitioners to keep up to date with evidence accumulating in field and to practice evidence-based medicine.

Key Stages in a Systematic Review- the process Define research/review question In consultation/collaboration with the clinical community, commissioners and patient/public representatives Identify relevant studies Develop a comprehensive search strategy and undertake systematic searches of the literature Assess eligibility Select those studies which meet the pre-defined inclusion criteria Data extraction /checking Develop data extraction from into which study information and outcome data can be extracted, checked & verified Synthesis Narratively and/or statistically summarise/describe the data, exploring similarities and differences between studies. Develop review protocol Pre-specify the type of studies to be included, the methods of collating, appraising and analysing data Knowledge translation Review details and results are disseminated to relevant target audiences using appropriate formats Study assessment/appraisal Assess the quality and validity of the included studies using the pre-defined method.

What makes a review “Systematic”? 1. PICO – which you will be familiar with now 2. Using reproducible methods including predefined search strategies 3. A-priori defined inclusion/exclusion criteria and selection, extraction etc. described 4. Often using a mix of objective and subjective assessments 5. This can include meta-analysis and assessment of heterogeneity 6. Not extrapolating beyond the data or populations that were studied

PICO Patient/Person: who does this relate to? Intervention (or cause, prognosis): what is the intervention or cause? Comparison (Is there something to compare the intervention to?) Outcome (What outcome are you interested in?). “To assess the effects of [intervention or comparison] for [health problem] in [types of people, disease or problem, and setting if specified].”

Searching for Information MESH terms and key words/synonyms Medical Subject Heading – controlled vocabulary thesaurus used for indexing articles young; adoles*; teen*; child*................... *end of the ‘stem’ of the word it will automatically search for all the endings for that word stem Word variants : AIDS acquired immunodeficiency syndrome acquired immuno-deficiency syndrome acquired immune deficiency syndrome acquired immune-deficiency syndrome Synonyms e.g. Newborn: infant, toddler, baby, etc. Plurals e.g. child : children OR teenager : teenagers Spelling variants (UK vs US) e.g. randomise/randomize

APHO staff conference 2009 Boolean operators 11

Molecular markers as an indicator in the malignant potential of oral lichen planus: A systematic review “oral lichenoid reaction”, “oral lichenoid lesion”, “oral lichenoid eruption”, “oral lichen planus”, “lichen planus”, and “lichenoid” in combination with “malignant transformation”, “premalignant character”, “cancerization”, “pre-neoplasm”, and “squamous cell carcinoma” all the published papers in English since January 1990 to December 2015. Toward this end, the international electronic databases including PubMed, Google Scholar, Science Direct, Scopus, Cochrane, and ISI Web of Science were searched

Meta-analysis The statistical combination of the results gives a pooled, weighted average of the primary results It weights the effect size (result) of each study in relation to sample size of the study Optional part of SR Systematic reviews Meta-analyses P1 - ….from each of the studies included in the meta-analysis P2 -

Is treatment better than control? Which is the smallest study? Which is the largest study? How many are statistically significant? Smallest P<0.05 P<0.05 Largest Smallest, largest, significant etc…. Can anyone tell the acute illness of our patient? = MI (streptokinase for reducing risk of mortality post MI from thrombotic event) Now, based on this forest plot do we give an antithrombotic drug for reperfusion (in this case streptokinase) = YES Is treatment better than control? How much better?

Heterogeneity (diversity) High heterogeneity = appropriate to pool data? Clinical heterogeneity Variability in the participants, interventions and/or outcomes studied Methodological heterogeneity Variability in study deign and risk of bias Statistical heterogeneity The observed intervention effects being more different from each other than we would expect due to random error (chance) alone

Publication bias Occurs when publication of research results depends on their nature and direction Often happens because smaller studies not submitted/rejected, (of + results) Funnel plots help identify if there is a bias: Largest studies will be near the average (truth), small studies will spread on both sides = symmetric funnel Asymetric funnel indicates publication bias – but not all the time (e.g. heterogeneity) Interpretation difficult if only a few studies in meta-analysis

Funnel plots

Not all systematic reviews are high quality! کیفیت طراحی و متد انجام مرور سیستماتیک بر اعتبار نتایج آن تأثیر بسزایی دارد و مرور سیستماتیکی که به لحاظ متدی ضعیف باشد سبب می‌گردد نتایج مداخلات درمانی به صورت غیرواقعی گزارش گردد و عواقب این گزارش غلط بر درمان بیماران تأثیرات منفی بر جای خواهد گذاشت . پروتکل PRISMA (Preferred Reporting Items For Systematic review and Meta-analysis ) ازجمله دقیق ترین و قابل اعتمادترین ابزارهای ارزیابی مرورهای سیستماتیک محسوب می‌گردد . استفاده از PRISMA استانداری برای افزایش کیفیت گزارشات متاآنالیزها محسوب می‌شود و کاربرد آن باعث می گردد تا گزارش مرورهای سیستمیک و متاآنالیز کاربردی‌تر و قابل اعتمادتر گردد.