Cochrane Systematic Reviews & Cochrane Oral Health Group Luisa Fernandez Anne-Marie Glenny Cochrane Oral Health Group, University Dental Hospital of Manchester Emma Tavender Cochrane EPOC Satellite, Melbourne Australia

Aims To discuss the role of Cochrane systematic reviews and meta-analyses Cochrane Oral Health Group – progress, undertaking a systematic review with the group & looking to the future

Cochrane Systematic Reviews & Oral Health Anne-Marie Glenny Cochrane Oral Health Group, University Dental Hospital of Manchester

What are systematic reviews? The process of systematically locating, appraising and synthesising evidence from scientific studies in order to obtain a reliable overview Aim to find all studies addressing the review’s question using an objective and transparent process

Why are they important ? Reduce large quantities of information into manageable portions Formulate policy and develop guidelines Efficient use of resources Increased power/precision Limit bias and improve accuracy

Cumulative Year RCT Pts I I I I I I I I I I I Routine Specific Rare/Never Experimental Not Mentioned p<.01 p<.001 p< Odds Ratio (Log Scale) Favours Treatment Favours Control Textbook/Review Recommendations A. Thrombolytic Therapy

Systematic review Structured process involving several steps: 1.Well formulated question 2.Comprehensive data search 3.Unbiased selection and abstraction process 4.Validity assessment of papers 5.Synthesis of data

1.Well formulated question P articipants I nterventions (Exposure) C omparisons O utcomes

1.Well formulated question Participants Interventions Comparisons Outcomes Pit and fissure sealants versus placebo for the prevention of dental caries in permanent teeth in children and adolescents

Inclusion criteria Participants Interventions Comparisons Outcomes Study design/methodological quality

What type of study design? How effective is paracetamol at reducing pain? Does smoking increase the risk of oral cancer?

STRONGExperimental studies/ clinical trials Randomised controlled trials Non-randomised controlled trials Observational studies Cohorts Case-controls Cross-sectional surveys Case series Case reports WEAK Expert opinion, consensus

2.Search strategy Needs to be as comprehensive as possible Consider –Electronic databases (Cochrane Controlled Trials Register, Medline, Embase) –Reference lists –Handsearching –English language/non-English language –Sources of ongoing and/or unpublished studies

Reporting biases Statistically significant ‘positive’ results are; -more likely to be published -publication bias -more likely to be published rapidly -time lag bias -more likely to be published in English -language bias -more likely to be cited by others -citation bias

3.Unbiased selection and data abstraction process Selection of relevant papers

3.Unbiased selection and data abstraction process Selection of relevant papers Data abstraction/extraction

Data extraction is: Time consuming Often subjective Prone to error Often difficult

3.Unbiased selection and abstraction process Predefined data abstraction form Independently and in duplicate

4. Validity assessment Can be used; –As a threshold for inclusion of studies –As a possible explanation for differences in results between trials –In sensitivity analyses –As weights in statistical analysis of the results

Quality assessment tools Composite scales - assign numerical value to individual items to provide overall estimate of quality – problematic Component approach - assesses relevant methodological aspects individually (e.g randomisation, blinding, drop-outs) - preferred

4. Validity assessment Process should be conducted independently by at least two reviewers Results of the quality assessment should be reflected in the analysis

5.Study synthesis Appropriate pooling –qualitative (narrative) –quantitative (meta-analysis) –inappropriate when data are sparse or heterogeneity exists –Clear presentation of individual studies included in the review

Meta-analysis The process of using statistical methods to combine the results of different studies. The aim is to integrate the findings, pool the data, and identify the overall trend of results (Dictionary of Epidemiology, 1995)

What is a meta-analysis? Optional part of a systematic review Systematic reviews Meta-analyses

When can/should you do a meta-analysis? When more than one study has estimated an effect When there are no differences in the study characteristics that are likely to substantially affect outcome When the outcome has been measured in similar ways When the data are available (beware when only some data are available)

Summary statistic for each study Calculate a single summary statistic to represent the effect found in each study - usually displayed with 95% confidence intervals (CI)

Weighting studies More weight to the studies which give us more information –More participants –More events –Lower variance Weight is proportional to inverse variance

Displaying results graphically Forest plots –Commonly used

there’s a label to tell you what the comparison is and what the outcome of interest is Alderson 2002

At the bottom there’s a horizontal line. This is the scale measuring the treatment effect. Here the outcome is death and towards the left the scale is less than one, meaning the treatment has made death less likely. Take care to read what the labels say – things to the left do not always mean the treatment is better than the control. Alderson 2002

The vertical line in the middle is where the treatment and control have the same effect – there is no difference between the two Alderson 2002

For each study there is an id The data for each trial are here, divided into the experimental and control groups This is the % weight given to this study in the pooled analysis Alderson 2002

Each study is given a blob, placed where the data measure the effect. The size of the blob is proportional to the % weight The horizontal line is called a confidence interval and is a measure of how we think the result of this study might vary with the play of chance. The wider the horizontal line is, the less confident we are of the observed effect. The label above the graph tells you what statistic has been used The data shown in the graph are also given numerically Alderson 2002

The pooled analysis is given a diamond shape where the widest bit in the middle is located at the calculated best guess (point estimate), and the horizontal width is the confidence interval Note on interpretation If the confidence interval crosses the line of no effect, this is equivalent to saying that we have found no statistically significant difference in the effects of the two interventions Alderson 2002

Heterogeneity Clinical heterogeneity – differences in trial characteristics Statistical heterogeneity - the variability in the reported effect sizes between studies –how similar are the results? –are the differences among the results of the trials greater than could be expected by chance alone?

Heterogeneity

Chi-squared test of heterogeneity P<0.1 demonstrates statistically significant heterogeneity –may not be appropriate to pool data

Subgroup analyses Where it is suspected in advance that certain features may alter the effect of an intervention Example –women –a particular age group –those with a specific disease subtype

Subgroup analysis Often misleading- Is there indirect evidence in support of a difference? Did the hypothesis about the difference precede rather than follow the analysis? Is the subgroup analysis one of a small number of hypotheses tested?

Sensitivity analysis Does result change according to small variations in the data and methods? –Choice of treatment effects or method for pooling –Inclusion/exclusion of dubious data –Inclusion/exclusion of trials

A common sensitivity analysis is to repeat the analysis taking out lower quality trials

Useful websites; (comprehensive list of E.B.P. websites and links) –CRD manual on how to conduct a systematic review –DARE - database of abstracts of reviews of effectiveness –reviewer’s handbook –Cochrane Library (abstracts only)

Cochrane Oral Health Group – looking to the future Emma Tavender Cochrane EPOC Satellite, Melbourne Australia

Aims Progress of the group Process of undertaking a systematic review with the group & support available Challenges for the future Questions & answers

Cochrane Oral Health Group Include all RCTs of oral health broadly conceived to include the prevention, treatment and rehabilitation of oral, dental and craniofacial diseases and disorders Achievements 624 members from 40 countries Panel of referees Set up Specialised Register of Trials (21,000) Handsearching programme Annual Evidence Based Dental Practice Course NIDCR funding for oral cancer reviews

Oral Health Group Reviews 68 Protocols 54 Reviews 21 Updates

Growth of Oral Health Group Trials Register January 1998 – January 2006

Editorial Process Register title Prepare protocol Editorial and external review of protocol Protocol entered on Cochrane Library Identify trials Complete systematic review Peer review of systematic review Systematic review entered on Library Update the review regularly (every 2 years)

Editorial support Register title –Check for overlap –Find co-reviewers with similar interests –Assigned Contact Editor

Editorial support Prepare protocol –Reviewers Handbook –RevMan software/support at editorial base –Protocol training workshop (free of charge) –Online learning materials –Help with developing search strategy (TSC)

Editorial Support Identify trials Complete systematic review –Search the OHG Trials Register –Analysis workshop (free of charge) –Statistical & methodological support from editorial base –Help with translations

Editorial support Update the review regularly –Help with identifying new trials –Procedures for changing lead author

Challenges for the future Prioritisation of reviews Diagnostic reviews Reviews of no trials – including other levels of evidence? Umbrella reviews Funding Dissemination

Challenges for the future Prioritisation of reviews –Increasing number of reviews/overlap –Usefulness – answering the right questions for clinicians, consumers, funders, policy makers? –NIDCR funding of oral cancer reviews –OHG Symposium – identify questions/topics –Future – ways of identifying priorities for consumers etc

Challenges for the future Diagnostic reviews –Important area within dentistry –Cochrane Diagnostic Reviews of Test Accuracy Initiative –4 phases: development of materials, piloting, implementation & publication –Piloting phase – 10 test review teams –Training & implementation 2007

Challenges for the future Reviews of no trials – including other levels of evidence? –Scope of OHG includes RCTs and Quasi RCTs –Reviews eg. Endocarditis included other levels of evidence –OHG currently developing guidance for reviewers on how to deal with reviews of no trials and when to include other levels

Challenges for the future Umbrella reviews –Combine the findings of a number of related reviews so information is in one document –Incorporate summary of findings tables –RevMan 5 released in 2007 will include this feature –Cochrane Umbrella Reviews Working Group

Challenges for the future Funding –5 years funding from DoH, UK –Covers salary of staff, administration at editorial base –NIDCR grant to undertake series of oral cancer reviews –NICE, Italian Govn., vCIOH –Alternative methods?

Challenges for the future Dissemination –The Cochrane Library –Journal articles & by products –Courses/conferences –Incorporation in guidelines (HTA, NICE, SIGN) –Improve visibiltiy and utilisation of reviews

Any questions?