Moving Cochrane evidence into practice: past, present & future Nicola Lindson-Hawley & Jamie Hartmann-Boyce Managing Editors Cochrane Tobacco Addiction Group Nuffield Department of Primary Care Health Sciences University of Oxford @cochraneTAG 9th June 2016 UKNSCC 2016 London, UK

What we will cover 01 What is Cochrane 02 What is the Cochrane Tobacco Addiction Group (CTAG) 03 The impact of CTAG 04 How to use and interpret Cochrane evidence 05 Where should CTAG focus their efforts in future? 06 How should we disseminate to our users?

Cochrane Organisation named after Archie Cochrane Chalmers, Chalmers and Enkin built on Archie’s work - summarised all the RCT data relating to the care of pregnant women Published as ‘Effective Care in Pregnancy and Childbirth’ (Keirse, Grant et al. 1989) Revolutionised perinatal care and led to the formation of the Cochrane Collaboration in 1993 Archie believed that there wasn’t enough evidence to justify most clinical practice and championed the use of randomised controlled trials

Cochrane WHAT? Gathers and combines the best evidence from research to determine the benefits and risks of treatments/interventions HOW? By systematically reviewing the available evidence, with strong emphasis on quality assessment WHY? To help healthcare providers, patients, carers, researchers, funders, policy makers, guideline developers improve their knowledge and make decisions

Cochrane Protocol => Review => Update 53 topic specific Review Groups, including TAG Worldwide- based in over 100 countries Not-for-profit- largely government funded

Tobacco Addiction Group (CTAG) Founded by GPs working in the Department of Public Health at the University of Oxford (C.Silagy, T.Lancaster, G.Fowler) The group carried out one of the first trials of nicotine patches in primary care They met Iain Chalmers, who was establishing Cochrane, and carried out a prototype review of NRT for smoking cessation Established CTAG in 1996, making 2016 our 20th anniversary

Tobacco Addiction Group (CTAG) @cochraneTAG Tobacco Addiction Group (CTAG) Main focus on interventions for tobacco use & prevention Manage approx. 90 reviews & 350 authors Based at the University of Oxford in the Primary Care Department Funded by the National Institute for Health Research (NIHR) Jamie Hartmann-Boyce Managing Editor Paul Aveyard Editor, University of Oxford Lindsay Stead Information Specialist John Hughes Editor, University of Vermont Tim Lancaster Co-ordinating Editor Nicola Lindson-Hawley Managing Editor Robert West Editor, University College London

CTAG’s aims To inform tobacco control policy internationally to inform research in tobacco control, and help ensure new research is focussed on important unanswered questions to contribute to reducing tobacco use

Example review: NRT Updated 10 times! Influencing healthcare guidance Influencing treatment availability Adapts to usage methods Studies increase, effect size stays stable

Recent evidence: e-cigarettes It is important that TAG reviews move with the times The recent publication of the first version of our e-cigarette review was an example of this (McRobbie et al. 2014) Highlighted the lack of high quality research in the area- only 2 RCTs met the inclusion criteria! Received publicity worldwide, following press release & news briefing at the Science Media Centre in London, UK Due for update this year!

Using and interpreting systematic reviews

In this session we’ll cover…. Sources of evidence How to access systematic reviews via the Cochrane Library How to interpret systematic reviews (key features) Quiz

Where do you look for information? Aims and structure of a Cochrane Review There are three main reasons why systematic reviews are useful: first, they are efficient, bringing together the whole body of research answering a specific question, and saving readers the time and expertise required to locate, appraise and interpret the results. secondly, unlike a single study, a systematic review can help us explore the differences between studies, identifying those factors that influence the effect of an intervention. thirdly, they aim to be reliable, presenting an unbiased, comprehensive picture of the available evidence, that can then be used to support decisions in health care, in policy, or to identify areas where future research may be needed. Traditional narrative reviews have not had this advantage – it’s often not clear how the evidence was selected for inclusion in reviews – perhaps only studies conducted in the last five years, only studies published in English, only studies conducted in large US teaching hospitals, or only studies that agree with the author’s opinion.

Evidence based medicine resource pyramid

Why systematic reviews? efficient way to access the body of research saves time required for searching critical appraisal interpretation of results explore differences between studies reliable basis for decision making unbiased selection of relevant information useful for health care, policy, future research Aims and structure of a Cochrane Review There are three main reasons why systematic reviews are useful: first, they are efficient, bringing together the whole body of research answering a specific question, and saving readers the time and expertise required to locate, appraise and interpret the results. secondly, unlike a single study, a systematic review can help us explore the differences between studies, identifying those factors that influence the effect of an intervention. thirdly, they aim to be reliable, presenting an unbiased, comprehensive picture of the available evidence, that can then be used to support decisions in health care, in policy, or to identify areas where future research may be needed. Traditional narrative reviews have not had this advantage – it’s often not clear how the evidence was selected for inclusion in reviews – perhaps only studies conducted in the last five years, only studies published in English, only studies conducted in large US teaching hospitals, or only studies that agree with the author’s opinion.

Key components of a Cochrane review

Key stages of a systematic review PROTOCOL clearly stated objectives pre-defined eligibility criteria explicit, reproducible methodology Search strategy FULL REVIEW systematic search assessment of validity of included studies systematic synthesis and presentation of findings There are key features that should go into any systematic review, whether Cochrane or not: first, a review should have clearly stated objectives, specifying the question to be answered, and determining what evidence will be sought. the objectives should determine clear eligibility criteria, so that all the relevant evidence is identified, and clear, objective decisions can be made about which studies are included in or excluded from the review. throughout the review, all the methods used should be explicit and transparent, so that in theory, someone could follow along, repeat what you’ve done, and come to the same conclusions, or at least see clearly where they might have made a different decision. once the eligibility and methods are planned, there is a systematic search of the literature, to find all the relevant evidence. the included studies are then carefully assessed, identifying those that might be at risk of bias or inaccurate results. finally, the results of all those included studies are systematically reported, and a synthesis of their findings is presented.

All our reviews are published online in The Cochrane Library. ASK: Has anyone accessed Cochrane reviews in the Library before? The part of the library containing Cochrane reviews is called the Cochrane Database of Systematic Reviews. As of September 2012 it contained 5222 reviews and 2259 protocols for reviews in progress. The Library is published monthly, and currently has an impact factor of 5.912 (2011). On the Library homepage, you’ll also find editorials, special collections of reviews (e.g. around H1N1 flu, disaster relief following major natural disasters, etc.), podcasts and resources for your local journal club. To find out more about what’s in the Library, explore the Learn menu. www.thecochranelibrary.com

Accessing a review on the Library

Participants P Interventions I Comparators C Outcomes O Study type S

What is a forest plot? Graphical display of treatment effects from multiple randomized controlled trials. When these are combined, it’s called a meta-analysis.

Risk ratios (RR) in cessation reviews events in intervention group/participants in intervention group events in control group/participants in control group For cessation, a risk ratio greater than 1 indicates that more people are quitting in the intervention condition. For example, the below data gives us an RR of 1.5. A risk ratio of 1 means as many people quit in the control group as in the intervention group. Intervention quit Intervention total Control quit Control total 15 100 10

Summary of findings tables Summary of key information from review Most important outcomes for someone making a decision

What is GRADE based on? Risk of bias Quality of evidence in GRADE is based on 5 domains Risk of bias The degree to which review authors have judged each included study to be at risk of bias A systematic under or overestimate of an effect due to selective publication of studies. This can happen when the results of a study are not reported. If, for example, studies that detect an effect are more likely to be published than those with negative results, the meta-analysis will be biased towards positive results. Unexplained differences between the results of individual studies (heterogeneity). This can lead to uncertainty about the underlying effect. Results can be downgraded because of indirectness if: Head to head comparisons are not available The population, intervention, comparator or outcome is different from the question the review is trying to address Results are considered imprecise if: The total number of events is small (usually less than 300) The confidence intervals are wide Inconsistency Indirectness Imprecision Publication bias

The blue square represents the point estimate for the effect. In this example, studies are divided into subgroups based on the type of NRT. The size of the square is dependent on the size of the study. Bullen 2010 has 969 participants; Rose 2006 96. Results based on trials - element of chance Each point estimate displayed with a confidence interval (CI) CIs represented by the ends of the lines for individual studies and by the horizontal edges of the diamonds for pooled estimates. 95% CIs provide an estimated range of values within which the true effect is 95% certain to lie. Each line represents an individual study (or intervention versus control comparison) Diamonds show a pooled estimate, or the combined result from all of the studies. Here, there are pooled estimates for studies in each subgroup and for studies overall. This pooled estimate represents the result of the meta-analysis. The vertical line is known as the line of no effect. For our cessation reviews, if the point estimate is on the right side of the line, more people receiving the intervention quit than people in the control group. If the point estimate is on the left side of the line, more people quit in the control group than in the intervention group. If the confidence interval crosses this central line, the effect is not statistically significant, meaning it is possible that the intervention and the control do not have different effects. The blue square represents the point estimate for the effect. Stead LF, Perera R, Bullen C, Mant D, Hartmann-Boyce J, Cahill K, Lancaster T. Nicotine replacement therapy for smoking cessation. Cochrane Database of Systematic Reviews 2012, Issue 11

Quiz Get into small groups You have time to discuss You’ll be asked to share your answers after each question

Q1 How many of these studies have a statistically significant result? none One Two Three or more Q1 All of the studies have confidence intervals that cross the line of no effect (1), so their results aren’t statistically significant.

Q2 In four studies, the point estimate is to the left of the line of no effect In how many studies did more people quit in the control group than in the treatment group? None Two Four Seven

Q3 What risk ratio would you expect to see for this result? a) 0.5 b) 1.0 c) 1.5 Intervention quit Intervention total Control quit Control total 10 100 20 The risk ratio is less than one, because more people quit in the control group than in the intervention group

Q4. What are the PICOs? P I C O S P I C O S Participants Interventions Comparators C Outcomes O Study type S Individuals who smoke P Cytisine I Placebo C Continuous abstinence at longest follow-up O Randomized controlled trials S

Why was the evidence downgraded? (can choose more than one answer) Risk of bias Inconsistency Indirectness Imprecision Publication bias The interventions being tested varied considerably. In addition, there was some evidence from a funnel plot to suggest smaller studies that did not detect significant results had not been published.

Cochrane TAG’s anniversary project AIMS Raise awareness of the group, and what we have achieved so far Identify areas where further research is needed in the areas of tobacco control & smoking cessation by involving our stakeholders Identify specific goals for Cochrane TAG Funded by the NIHR School for Primary Care Research

Cochrane TAG’s anniversary project Stage 1: Identifying uncertainties survey Asked anyone with an interest in tobacco to share the questions they would still like to see answered by tobacco control research

Cochrane TAG’s anniversary project Stage 2: Ranking uncertainties survey Unanswered questions (uncertainties) were separated into 15 research categories, with each category consisting of 3-21 questions. 278 from the 1st survey contacted and asked to rank the 15 research categories in order of their importance. They were then asked to rank the questions in their top 3 175 people completed the whole survey (63% of those invited) 3. Mental health & other substance abuse 1. E-cigarettes 2. Addressing inequalities

Cochrane TAG’s anniversary project Stage 3: Prioritisation Workshop 17th June we are holding a 1-day workshop with 50 of our stakeholders Discussion of where CTAG should focus its future efforts and ways to disseminate our findings The findings of the project will be written up, with our priorities & aims for the future, and published We will begin to work on the priorities before the end of the year and will continue to do so into the future

Any questions?

Workshop Get into small groups (3-5) – choose a scribe who will feed back from each First 10 minutes: identify gaps in the tobacco control evidence base (see survey results if you need a steer) Second 10 minutes: Discuss the best ways Cochrane can get information to you Final 10 minutes: Feedback and discussion

How to find out more or contact us Visit our website: http://tobacco.cochrane.org/ Tweet us: @cochraneTAG Email us: nicola.lindson-hawley@phc.ox.ac.uk Call us: +44 (0)1865 289 320