1. Ian Saldanha, MBBS, MPH, PhD
Outcomes in clinical trials and systematic reviews – What is the overlap and what should it be?
Ian Saldanha, MBBS, MPH, PhD
2017 Society for Clinical Trials Meeting, Liverpool, UK
May 8, 2017

2. Evidence-based healthcare paradigm
Generation
(Clinical trials)
Evidence
Synthesis
(Systematic reviews)
Clinical
Policy
(Practice guidelines)
Evidence-based
Healthcare
(Clinicians and
patients)
Slide courtesy Dr. Kay Dickersin

3. Key points
If trials are to impact evidence-based healthcare, their results need to be included in systematic reviews.
But, trialists and reviewers studying the same disease do not often report similar outcomes.
How much overlap in outcomes is ideal?
How do we decide what outcomes are relevant for clinical decision-making?

4. A recent Cochrane systematic review
48 included trials
Primary outcome – “proportion of patients with intraocular inflammation at 1 week of follow-up after surgery”
Mean
by 1 month
0 trials!
7 trials
11 trials

5. What is an outcome?
In clinical trials, an outcome is a measure or event that are used to assess the effectiveness and/or safety of clinical interventions.
Meinert CL. Clinical trials dictionary: Terminology and usage recommendations. 2nd ed. Wiley. Hoboken, NJ: Wiley; 2012.

6. I V IV II III Five elements of a completely specified outcome
Time-Points
Domain
e.g., 1 month
e.g., Intraocular inflammation
Method of Aggregation
Specific measurement IV II
e.g., Slit lamp examination
e.g., Proportion
Specific Metric
e.g., Value at a time-point
III
Saldanha IJ, et al. PLoS One 2014

7. (Practice guidelines)
So, are clinical trials and systematic reviews measuring examining similar outcomes?
Evidence
Generation
(Clinical trials)
Evidence
Synthesis
(Systematic reviews)
Clinical
Policy
(Practice guidelines)
Evidence-based
Healthcare
(Clinicians and
patients)

8. Our studies in ophthalmology and HIV/AIDS
For two areas – Ophthalmology1 and HIV/AIDS2
Examined all outcome domains in all Cochrane reviews and all trials included in the reviews.
Ophthalmology – compared outcomes within subgroups by disease (4 diseases)
HIV/AIDS - compared outcomes within subgroups by type of intervention (4 types)
1 Saldanha IJ, et al. (Submitted)
2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

9. What did we find? Topic Area Systematic reviews Clinical trials N
Ophthalmology 56
414
HIV/AIDS 84
524
1 Saldanha IJ, et al. (Submitted)
2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

10. What did we find? Topic Area Systematic reviews Clinical trials N
Ophthalmology 56
414
HIV/AIDS 84
524
Outcomes per review or trial 5
7.5 8
1 Saldanha IJ, et al. (Submitted)
2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

11. What did we find? Topic Area Systematic reviews Clinical trials N
Ophthalmology 56
414
HIV/AIDS 84
524
Outcomes per review or trial 5
7.5 8
Unique outcomes 92
248
218
779
1 Saldanha IJ, et al. (Submitted)
2 Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

12. One way to look at the overlap HIV/AIDS – Behavioral interventions
85 outcomes in 17 reviews
458 outcomes in 170 trials 9
outcomes
(2%)
76 outcomes
(16%)
382 outcomes
(82%)
Outcomes in reviews only
Outcomes in reviews and trials
467 total outcomes
Outcomes in trials only

13. Another way to look at the overlap
% of trials reporting the outcome
100 50 50
100
% of reviews reporting the outcome

14. Some scenarios 100 Outcomes popular among trialists
% of trials reporting the outcome 50 50
100
% of reviews reporting the outcome

15. Some scenarios 100 100 Outcomes popular among trialists
% of trials reporting the outcome
Outcomes popular among reviewers
% of trials reporting the outcome 50 50 50
100 50
100
% of reviews reporting the outcome
% of reviews reporting the outcome

16. Some scenarios  100 100 Outcomes popular among trialists
% of trials reporting the outcome
Outcomes popular among reviewers
% of trials reporting the outcome 50 50 50
100 50
100
% of reviews reporting the outcome
% of reviews reporting the outcome
100  50 50
100

17. Some scenarios   100 100 Outcomes popular among trialists
% of trials reporting the outcome
Outcomes popular among reviewers
% of trials reporting the outcome 50 50 50
100 50
100
% of reviews reporting the outcome
% of reviews reporting the outcome
100
100  50 50  50
100 50
100

18. Some scenarios   100 100 Outcomes popular among trialists
% of trials reporting the outcome
Outcomes popular among reviewers
% of trials reporting the outcome 50 50 50
100 50
100
% of reviews reporting the outcome
% of reviews reporting the outcome
100
100  50 50  50
100 50
100

19. What did we find in our data? – Ophthalmology
Age-related macular degeneration
Cataract
Diabetic retinopathy
Glaucoma
Saldanha IJ, et al. (Submitted)

20. What did we find in our data? – HIV/AIDS
Clinical management
Biomedical prevention
Behavioral prevention
Health services
Saldanha IJ, et al. Journal of Clinical Epidemiology 2017

21. Conclusions Large number of outcomes
3 times more outcomes in trials than reviews
Limited overlap
Need for consistency in outcome usage
Differences in goals and perspectives?
Trials: Patient care
Reviews: Guidelines and policy

22. Key points
If trials are to impact evidence-based healthcare, their results need to be included in systematic reviews.
But, trialists and reviewers studying the same disease do not often report similar outcomes.
How much overlap in outcomes is ideal?
How do we decide what outcomes are relevant for clinical decision-making?

23. Thank you! isaldan1@jhmi.edu
Dr. Kay Dickersin (Johns Hopkins) Dr. Paula Williamson (University of Liverpool) Dr. Tianjing Li (Johns Hopkins) Dr. Cui Yang (Johns Hopkins) Dr. Jill Owczarzak (Johns Hopkins) Mrs. Kristina Lindsley (Johns Hopkins) Dr. Gianni Virgili (University of Florence) Dr. Leslie Jones (Howard University) Dr. Henry Jampel (Johns Hopkins)
Dr. Diana Do (Stanford University)
Dr. Roy Chuck (Albert Einstein College of Medicine)
Dr. Catherine Meyerle (Johns Hopkins)
Dr. Paula Williamson (University of Liverpool)
Dr. Tianjing Li (Johns Hopkins)
Dr. Anne Coleman (UCLA)