Ian Saldanha, MBBS, MPH, PhD Outcomes in clinical trials and systematic reviews – What is the overlap and what should it be? Ian Saldanha, MBBS, MPH, PhD 2017 Society for Clinical Trials Meeting, Liverpool, UK May 8, 2017

Evidence-based healthcare paradigm Generation (Clinical trials) Evidence Synthesis (Systematic reviews) Clinical Policy (Practice guidelines) Evidence-based Healthcare (Clinicians and patients) Slide courtesy Dr. Kay Dickersin

Key points If trials are to impact evidence-based healthcare, their results need to be included in systematic reviews. But, trialists and reviewers studying the same disease do not often report similar outcomes. How much overlap in outcomes is ideal? How do we decide what outcomes are relevant for clinical decision-making?

A recent Cochrane systematic review 48 included trials Primary outcome – “proportion of patients with intraocular inflammation at 1 week of follow-up after surgery” Mean by 1 month 0 trials! 7 trials 11 trials

What is an outcome? In clinical trials, an outcome is a measure or event that are used to assess the effectiveness and/or safety of clinical interventions. Meinert CL. Clinical trials dictionary: Terminology and usage recommendations. 2nd ed. Wiley. Hoboken, NJ: Wiley; 2012.

I V IV II III Five elements of a completely specified outcome Time-Points Domain e.g., 1 month e.g., Intraocular inflammation Method of Aggregation Specific measurement IV II e.g., Slit lamp examination e.g., Proportion Specific Metric e.g., Value at a time-point III Saldanha IJ, et al. PLoS One 2014

(Practice guidelines) So, are clinical trials and systematic reviews measuring examining similar outcomes? Evidence Generation (Clinical trials) Evidence Synthesis (Systematic reviews) Clinical Policy (Practice guidelines) Evidence-based Healthcare (Clinicians and patients)

Our studies in ophthalmology and HIV/AIDS For two areas – Ophthalmology1 and HIV/AIDS2 Examined all outcome domains in all Cochrane reviews and all trials included in the reviews. Ophthalmology – compared outcomes within subgroups by disease (4 diseases) HIV/AIDS - compared outcomes within subgroups by type of intervention (4 types) 1 Saldanha IJ, et al. (Submitted) 2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

What did we find? Topic Area Systematic reviews Clinical trials N Ophthalmology 56 414 HIV/AIDS 84 524 1 Saldanha IJ, et al. (Submitted) 2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

What did we find? Topic Area Systematic reviews Clinical trials N Ophthalmology 56 414 HIV/AIDS 84 524 Outcomes per review or trial 5 7.5 8 1 Saldanha IJ, et al. (Submitted) 2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

What did we find? Topic Area Systematic reviews Clinical trials N Ophthalmology 56 414 HIV/AIDS 84 524 Outcomes per review or trial 5 7.5 8 Unique outcomes 92 248 218 779 1 Saldanha IJ, et al. (Submitted) 2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

One way to look at the overlap HIV/AIDS – Behavioral interventions 85 outcomes in 17 reviews 458 outcomes in 170 trials 9 outcomes (2%) 76 outcomes (16%) 382 outcomes (82%) Outcomes in reviews only Outcomes in reviews and trials 467 total outcomes Outcomes in trials only

Another way to look at the overlap % of trials reporting the outcome 100 50 50 100 % of reviews reporting the outcome

Some scenarios 100 Outcomes popular among trialists % of trials reporting the outcome 50 50 100 % of reviews reporting the outcome

Some scenarios 100 100 Outcomes popular among trialists % of trials reporting the outcome Outcomes popular among reviewers % of trials reporting the outcome 50 50 50 100 50 100 % of reviews reporting the outcome % of reviews reporting the outcome

Some scenarios  100 100 Outcomes popular among trialists % of trials reporting the outcome Outcomes popular among reviewers % of trials reporting the outcome 50 50 50 100 50 100 % of reviews reporting the outcome % of reviews reporting the outcome 100  50 50 100

Some scenarios   100 100 Outcomes popular among trialists % of trials reporting the outcome Outcomes popular among reviewers % of trials reporting the outcome 50 50 50 100 50 100 % of reviews reporting the outcome % of reviews reporting the outcome 100 100  50 50  50 100 50 100

Some scenarios   100 100 Outcomes popular among trialists % of trials reporting the outcome Outcomes popular among reviewers % of trials reporting the outcome 50 50 50 100 50 100 % of reviews reporting the outcome % of reviews reporting the outcome 100 100  50 50  50 100 50 100

What did we find in our data? – Ophthalmology Age-related macular degeneration Cataract Diabetic retinopathy Glaucoma Saldanha IJ, et al. (Submitted)

What did we find in our data? – HIV/AIDS Clinical management Biomedical prevention Behavioral prevention Health services Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

Conclusions Large number of outcomes 3 times more outcomes in trials than reviews Limited overlap Need for consistency in outcome usage Differences in goals and perspectives? Trials: Patient care Reviews: Guidelines and policy

Key points If trials are to impact evidence-based healthcare, their results need to be included in systematic reviews. But, trialists and reviewers studying the same disease do not often report similar outcomes. How much overlap in outcomes is ideal? How do we decide what outcomes are relevant for clinical decision-making?

Thank you! isaldan1@jhmi.edu Dr. Kay Dickersin (Johns Hopkins) Dr. Paula Williamson (University of Liverpool) Dr. Tianjing Li (Johns Hopkins) Dr. Cui Yang (Johns Hopkins) Dr. Jill Owczarzak (Johns Hopkins) Mrs. Kristina Lindsley (Johns Hopkins) Dr. Gianni Virgili (University of Florence) Dr. Leslie Jones (Howard University) Dr. Henry Jampel (Johns Hopkins) Dr. Diana Do (Stanford University) Dr. Roy Chuck (Albert Einstein College of Medicine) Dr. Catherine Meyerle (Johns Hopkins) Dr. Paula Williamson (University of Liverpool) Dr. Tianjing Li (Johns Hopkins) Dr. Anne Coleman (UCLA)