Systematic Reviews By Jonathan Tsun & Ilona Blee

What is a systematic review? Define it!  Results from similar randomized trials are brought together (“synthesized”) to help identify which forms of health care work, which don’t work, and which are harmful.  This combination of randomized trials needs to be done in a systematic way – hence it is called a systematic review  The methods used include steps to minimize bias in the process – this is done by including all of the relevant studies  What are the TWO main types of bias? 1. Selection 2. Information

What are the limitations of systematic reviews?  “Synthesis” of results from different studies can oversimplify important distinctions  Different conclusions can be reached depending on the wording of the “ review question ”  For example compare these two questions: to be able to know whether we can predict a problem and intervene early vs. to identify the best ways of diagnosing/identifying problems  Reviews can make it difficult for doctors to apply the results of the review to every situation they come across  over-generalisation, medicine isn’t an exact science!  Findings from systematic reviews aren’t always consistent with other findings of large-scale high quality trials

What is a meta-analysis?  When the results of individual studies are combined to produce an overall statistic.  This can be done without a systematic review by combining results from more than one trial  What is the disadvantage of this?  Without the systematic review, there will be bias present if there is any from that study  This can lead to clinically misleading results!!!

Forest plot – what is it?  A graphical representation of a meta analysis, illustrating the effect sizes of individual studies and the meta-analysis as a whole.  What is “effect size”?  Quantitative measure of the strength of a phenomenon (e.g. correlation between 2 variables, etc)  It will have a list of the studies included in the meta analysis

 What is the line at x=0 called? What does it indicate?  Line of No Effect  If a study’s confidence interval line crosses the Line of No Effect, it demonstrates that at the given level of confidence, the study’s effect size does not differ from no effect. (AKA the study’s findings are not significant)  What does the size of the green square indicate?  It is a graphical representation of the weight of that particular study in the meta-analysis.

 What do the lines sticking out from the green boxes indicate?  They are graphical representations of the size of confidence intervals.  What does the diamond represent?  The effect size of the meta-analysis as a whole  What do the 2 horizontal points of the diamond indicate? What does it mean if it crosses x=0?  The 2 horizontal points indicate the confidence interval for the combined data  If it crosses the line of no effect... Answer is below in the notes.

Funnel Plot – what are they/ what do they do?  They assess whether the results of a systematic review have been influenced by “ publication bias ”  Publication bias occurs when the publication of research results depends on their nature and direction  eg. Positive results are more likely to be published than negative studies and can mean systematic reviews end up biased towards positive results  If the plot is symmetric, this means that there is probably no publication bias  If the plot is asymmetric, this means that publication bias is likely

Symmetrical = no publication bias! Asymmetrical = publication bias likely

If we were to critically appraise a study, what would we ask? (3) 1. Is the study valid? 2. Are the results reliable? 3. Can I generalise from this study to my workplace/own practice? OR will the results help locally?  Why is critical appraisal important?  Ensures that relevant articles contribute towards patient care  Tries to ensure that bias is minimised  Ensures that the results from a study can be used in real life

An example of evidence- based medicine!  CURB-65  This is something you should have covered in CBL and comes up in exams/ on the wards etc.  What does it stand for?  C onfusion  U rea levels of >7mmol/L  R espiratory rate >30 breaths/min  B lood pressure <90mmHg systolic over <60mmHg diastolic  Age > 65 years  You get a point for each of these

What does PRISMA stand for? P referred R eporting I tems for S ystematic reviews & M eta- A nalyses In other words, helps authors report the results of a systematic review More at: statement.org/ statement.org/

Using systematic reviews!

Where do we search for systematic reviews? 1. PICO !  Population/patient problem, intervention, comparator, outcome 2. Cochrane library 3. Medline  What is PICO used for?  Asking a “good” question  What are the steps for using PICO/how do we use PICO?  How to explain this..

Don’t forget to learn your equations and calculations!  Weighted mean difference:  Mean(g1) – Mean(g2)  Standardised mean difference:  Mean(g1) – Mean(g2) pooled sd  Relative risk:   Number needed to treat (NNT):   Risk Ratio:  Be able to interpret risk ratio quite quickly  E.g. RR = 2  Means the intervention outcome is twice as likely compared to the control outcome  E.g. RR = 0.5  Means the intervention outcome is half as likely compared to the control outcome

What is bias?  Any systematic error in an epidemiological study which either underestimates or overestimates the parameter of interest because of a deficiency in the design or execution of the study. SelectionInformation Sampling biasRecall bias Allocation biasObserver/interviewer/reporting bias Responder biasSocial desirability bias Attrition bias Publication bias Healthy worker effect

Hierarchy of evidence! 1. Systematic reviews – always at the top over anything ever because of the way it minimises bias!  Meta-analyses  Non meta-analytic reviews 2. Experimental studies  Randomised controlled trials  Controlled trials 3.Observational studies  Cohort studies  Case-controlled studies 4.Descriptive studies  Cross-sectional designs What order to the study designs go in?

Match the question with the type of evidence needed! QuestionEvidence How effective is this preventative intervention? RCT How effective is this therapy? RCT How effective is this screening/diagnostic tool? RCT Is this a causative factor for this disease?Cohort or case-controlled studies What is the prognosis for this?Cohort or case-controlled studies How do patients feel about their condition?Qualitative experimental studies How common is this condition?Cross-sectional studies How cost-effective is this intervention?RCT

The end!  Any questions?