1. Reproducibility, open data and evidence synthesis in clinical research CESS 2016, Budapest
Tim Friede
Department of Medical Statistics
University Medical Center Göttingen

2. Background: Evidence-based medicine
The ultimate aim is to improve patient care
by basing decision-making in clinical practice and in health care provision in general on sound, reliable evidence
gained through well designed, conducted and analysed research studies
Quantitative empirical sciences among the pillars of modern medicine and biomedical research in particular
Hierarchy of evidence …

3. Hierarchy of evidence
From support-your-scientific-indication

4. Systematic Reviews and Meta-analyses
The evidence on a particular research question available at a certain timepoint is summarized in so-called systematic reviews which can include meta-analyses formally integrating data from various studies on a particular research topic.
Steps involved in a systematic review
(Electronic) searches of data bases (literature, clinical trial registries)
Assessment of study quality
Data extraction
Meta-analysis, if applicable

5. Included studies To include or not to include?
PICOS (Population, Intervention, Control, Outcome, Study design)
Quality assessment (e.g. Cochrane risk of bias tool)
Context (e.g. market authorization, reimbursement)

6. Size of database vs. study quality

7. Size of database vs. study quality
No studies, but
those are of very
high quality …
Size of database
Study quality

8. Reduce waste, increase value
Crisis in medical research (including basic sciences as well as clinical research): research perceived as wasteful (or at least inefficient) and results often not reproducible
Series of articles in The Lancet

9. Reduce waste, increase value
Recommendations by Macleod et al (2014) Lancet to avoid waste in biomedical research:

10. Reduce waste, increase value
Recommendations by Macleod et al (2014) Lancet to avoid waste in biomedical research:

11. Reduce waste, increase value
Recommendations by Macleod et al (2014) Lancet to avoid waste in biomedical research:

12. Reproducibility
A number of approaches have been suggested to make research more reproducible including
publication of data
in the context of publishing in biostatistics: a number of journals have implemented reproducible research policies which include the publication of software code and data (see e.g. Hothorn et al (2009) Biom J)

13. Open data: Recent example from nature

14. Open data and clinical trials
In the context of clinical trials (in particular those conducted by the pharmaceutical industry) the pressures have significantly increased over the past years to provide access to trial data which led to the establishment of data repositories (see e.g. Sudlow et al, 2016).
Examples for repositories
clinicalstudydatarequest.com (CSDR.com)
Yale University Open Data Access (YODA)

15. Example: Requesting Open data

16. Key steps in requesting and accessing clinical trial data
Figure from Sudlow et al (2016) BMC Med Res Meth

17. Evidence synthesis Combined analysis of data from more than one study
Meta-analysis: combining data from studies with similar design (e.g. randomized controlled trials)
Generalized evidence synthesis (or cross-design synthesis): combining data from studies with different designs
Types of data
Aggregated on study level (e.g. effect estimates with standard errors)
Individual patient data (open data policies help here!)
Advantages of individual patient data

18. Rare diseases and Evidence synthesis
Definitions of rare diseases (or orphan diseases)
no universal definition of rare diseases
in European Union: a disease affecting 5 in people
in USA: a condition affecting fewer than people
Clinical research in rare diseases difficult
Evidence often scarce
few small studies with varying designs
Evidence synthesis of particular interest (Friede et al, 2016)
e.g. use of external controls, cross-design synthesis

19. Rare diseases and Evidence synthesis
Example: Doxycycline in early Creutzfeldt–Jakob disease (CJD) (Varges et al (2016) JNNP, in press)
n=88
n=12

20. Electronic health records
Routinely collected data more readily available for research purposes, for instance in the form of electronic health records
Great value in particular in rare diseases (Chataway and Friede, 2016)
Developments have the potential of making research more efficient by potentially
facilitating better planning of future studies
reducing the number and size of clinical trials

21. Ethical considerations
Key concepts in clinical trials
data protection: protecting the privacy of individuals
informed consent: patients given appropriate information to make a decision on participation in clinical study
Open data: sharing and reusing data carries some risks for these key concepts in enabling clinical research
Biomaterials: in combination with clinical data
Necessary to develop and implement new concepts to protect the interests of patients (see e.g. Wegscheider and Friede (2016) for a discussion).

22. Conclusions
The advance in data capturing and storage combined with increasing analytical ability carries some promise to make clinical science more efficient and to strengthen the evidence base.
At the same time these developments require a new form a governance in terms of use and access of data to protect patients’ interests.

23. Some References
J. Chataway and T. Friede, The N-MOmentum trial: building momentum to advance trial methodology in a rare disease, Multiple Sclerosis Journal 22 (2016), 852–853.
T. Friede, C. Röver, S. Wandel and B. Neuenschwander, Meta-analysis of few small studies in orphan diseases, Research Synthesis Methods (2016), in press.
T. Hothorn, L. Held and T. Friede, Biometrical Journal and reproducible research. Biometrical Journal 51 (2009), 553–555.
R. Sudlow, J. Branson, T. Friede, D. Morgan and C. Whateley Smith, EFSPI/PSI Working Group on Data Sharing: Accessing and Working with Pharmaceutical Clinical Trial Patient Level Datasets – a Primer for Academic Researchers, BMC Medical Research Methodology (2016), in press.
K. Wegscheider and T. Friede, Do we consent to rules of consent and confidentiality?, Biometrical Journal (2016), submitted.