1. Observational Health Data Sciences and Informatics: A journey to reliable evidence
Patrick Ryan, PhD
Janssen Research and Development
Columbia University Medical Center
7 March 2017

2. Introducing OHDSI
The Observational Health Data Sciences and Informatics (OHDSI) program is a multi-stakeholder, interdisciplinary collaborative to create open-source solutions that bring out the value of observational health data through large-scale analytics
OHDSI has established an international network of researchers and observational health databases with a central coordinating center housed at Columbia University

3. OHDSI’s mission
To improve health, by empowering a community to collaboratively generate the evidence that promotes better health decisions and better care.

4. What is OHDSI’s strategy to deliver reliable evidence?
Methodological research
Develop new approaches to observational data analysis
Evaluate the performance of new and existing methods
Establish empirically-based scientific best practices
Open-source analytics development
Design tools for data transformation and standardization
Implement statistical methods for large-scale analytics
Build interactive visualization for evidence exploration
Clinical evidence generation
Identify clinically-relevant questions that require real-world evidence
Execute research studies by applying scientific best practices through open-source tools across the OHDSI international data network
Promote open-science strategies for transparent study design and evidence dissemination

5. OHDSI community OHDSI Collaborators:
>140 researchers in academia, industry, government, health systems
>20 countries
Multi-disciplinary expertise: epidemiology, statistics, medical informatics, computer science, machine learning, clinical sciences
Databases converted to OMOP CDM within OHDSI Community:
>50 databases
>660 million patients

6. One common data model to support multiple use cases
Drug safety surveillance
Comparative effectiveness
Device safety surveillance
Health economics
Vaccine safety surveillance
Quality of care
Clinical research
Standardized clinical data
Person
Standardized health system data
Standardized meta-data
Observation_period
Location
Care_site
CDM_source
Specimen
Provider
Death
Concept
Payer_plan_period
Standardized health economics
Standardized vocabularies
Vocabulary
Visit_occurrence
Domain
Procedure_occurrence
Concept_class
Cost
Concept_relationship
Drug_exposure
Relationship
Device_exposure
Concept_synonym
Standardized derived elements
Cohort
Concept_ancestor
Condition_occurrence
Cohort_attribute
Source_to_concept_map
Measurement
Condition_era
Drug_strength
Note
Drug_era
Cohort_definition
Observation
Dose_era
Attribute_definition
Fact_relationship

7. Odyssey (noun): \oh-d-si\
A long journey full of adventures
A series of experiences that give knowledge or understanding to someone

8. A caricature of the patient journey
Disease
Treatment
Outcome
Conditions
Drugs
Procedures
Measurements
Person time
Baseline time
Follow-up time

9. Each observational database is just an (incomplete) compilation of patient journeys
Person 1
Conditions
Drugs
Procedures
Measurements
Person time
Disease
Treatment
Outcome
Baseline time
Follow-up time
Person 2
Conditions
Drugs
Procedures
Measurements
Person time
Disease
Treatment
Outcome
Baseline time
Follow-up time
Person 3
Conditions
Drugs
Procedures
Measurements
Person time
Disease
Treatment
Outcome
Baseline time
Follow-up time
Person N
Conditions
Drugs
Procedures
Measurements
Person time
Disease
Treatment
Outcome
Baseline time
Follow-up time

10. Questions asked across the patient journey
Disease
Treatment
Outcome
Which treatment did patients choose after diagnosis?
Conditions
Which patients chose which treatments?
Drugs
Does one treatment cause the outcome more than an alternative?
Procedures
Measurements
How many patients experienced the outcome after treatment?
Does treatment cause outcome?
Person time
Baseline time
Follow-up time
What is the probability I will develop the disease?
What is the probability I will experience the outcome?

11. Classifying questions across the patient journey
Clinical characterization: What happened to them?
What treatment did they choose after diagnosis?
Which patients chose which treatments?
How many patients experienced the outcome after treatment?
Patient-level prediction: What will happen to me?
What is the probability that I will develop the disease?
What is the probability that I will experience the outcome?
Population-level effect estimation: What are the causal effects?
Does treatment cause outcome?
Does one treatment cause the outcome more than an alternative?

12. Complementary evidence to inform the patient journey
Clinical characterization:
What happened to them?
observation
Patient-level prediction:
What will happen to me?
Population-level effect estimation:
What are the causal effects?
inference
causal inference

13. A caricature of the journey of a patient with major depressive disorder
Depression
SSRI
Stroke
Conditions
Drugs
Procedures
Measurements
Person time
Baseline time
Follow-up time

14. In practice, a patient’s journey is a bit more complicated…
Depression
SSRI
Stroke

15. …and every patient’s journey is quite different
Person 1
Depression
SSRI
Stroke
Person 2
Depression
SSRI
Stroke
Person 3
Depression
SSRI
Stroke
, ,

16. Clinical questions that deserve reliable evidence to inform patients with depression
Clinical characterization: What happened to them?
What antidepressant did they choose after their MDD diagnosis?
Which patients chose which antidepressant treatments?
How many patients had ischemic stroke after antidepressant exposure?
Patient-level prediction: What will happen to me?
What is the probability that I will develop major depressive disorder?
What is the probability that I will experience an ischemic stroke?
Population-level effect estimation: What are the causal effects?
Do SSRIs cause ischemic stroke?
Does sertraline cause ischemic stroke more than duloxetine?

17. How should patients with major depressive disorder be treated?

18. How are patients with major depressive disorder ACTUALLY treated?
Hripcsak et al, PNAS, 2016

19. OHDSI participating data partners
Code
Name
Description
Size (M)
AUSOM
Ajou University School of Medicine
South Korea; inpatient hospital EHR 2
CCAE
MarketScan Commercial Claims and Encounters
US private-payer claims
119
CPRD
UK Clinical Practice Research Datalink
UK; EHR from general practice 11
CUMC
Columbia University Medical Center
US; inpatient EHR 4 GE
GE Centricity
US; outpatient EHR 33
INPC
Regenstrief Institute, Indiana Network for Patient Care
US; integrated health exchange 15
JMDC
Japan Medical Data Center
Japan; private-payer claims 3
MDCD
MarketScan Medicaid Multi-State
US; public-payer claims 17
MDCR
MarketScan Medicare Supplemental and Coordination of Benefits
US; private and public-payer claims 9
OPTUM
Optum ClinFormatics
US; private-payer claims 40
STRIDE
Stanford Translational Research Integrated Database Environment
US; inpatient EHR
HKU
Hong Kong University
Hong Kong; EHR 1
Hripcsak et al, PNAS, 2016

20. Treatment pathway study design
>250,000,000 patient records used across OHDSI network
>=4 years continuous observation
>=3 years continuous treatment from first treatment
N=264,841 qualifying patients with depression
Hripcsak et al, PNAS, 2016

21. How are patients with major depressive disorder ACTUALLY treated?
Substantial variation in treatment practice across data sources, health systems, geographies, and over time
Consistent heterogeneity in treatment choice as no source showed one preferred first-line treatment
11% of depressed patients followed a treatment pathway that was shared with no one else in any of the databases
Hripcsak et al, PNAS, 2016

22. Which patients chose which antidepressant treatments?
Conditions
Drugs
Procedures
Measurements
Person time
Depression
sertraline
Baseline time
Follow-up time
Create cohorts for all antidepressant treatments
Summarize all baseline characteristics
Systematically explore differences in populations
Conditions
Drugs
Procedures
Measurements
Person time
Depression
duloxetine
Baseline time
Follow-up time
vs.

23. How many patients experienced the outcome after treatment?
Conditions
Drugs
Procedures
Measurements
Person time
Depression
sertraline
Baseline time
Follow-up time
Stroke
Create cohorts for all outcomes of interest
Summarize incidence of outcomes within each treatment group
Systematically explore risk differences in subpopulations of interest
Conditions
Drugs
Procedures
Measurements
Person time
Depression
duloxetine
Baseline time
Follow-up time
vs.
Stroke

24. Journey toward reliable evidence
Generation
How to produce evidence from the data?
Evaluation
How do we know the evidence is reliable?
Dissemination
How do we share evidence to inform decision making?

25. Following the scientific process
Data Analysis = Evidence
Patient-level
Data
Statistical programming
Evidence repository

26. Following the scientific process in clinical development
Data Analysis = Evidence
Patient-level
Data
Statistical programming
Evidence repository
Internally
Conduct RCTs to collect data for specific question
Pre-specified protocol analyses
Prepare study reports and publications

27. Following the open scientific process in clinical development
Data Analysis = Evidence
Patient-level
Data
Statistical programming
Evidence repository
Internally
Conduct RCTs to collect data for specific question
Pre-specified protocol analyses
Prepare study reports and publications
Externally

28. Following the open scientific process in clinical development
Data Analysis = Evidence
Patient-level
Data
Statistical programming
Evidence repository
Internally
Open Science tenets:
Transparency - all aspects of the evidence generation, evaluation, and dissemination process are fully disclosed to all interested parties
Reproducibility - same data + same analysis = same evidence
Conduct RCTs to collect data for specific question
Pre-specified protocol analyses
Prepare study reports and publications
Externally

29. Following the scientific process in observational research
Data Analysis = Evidence
Patient-level
Data
Statistical programming
Evidence repository
Internally
Data captured during clinical practice
Secondary use based researcher-initiated question
Prepare publications when ‘appropriate’
Externally

30. What is the quality of the current evidence from observational analyses?
August2010: “Among patients in the UK General Practice Research Database, the use of oral bisphosphonates was not significantly associated with incident esophageal or gastric cancer”
Sept2010: “In this large nested case-control study within a UK cohort [General Practice Research Database], we found a significantly increased risk of oesophageal cancer in people with previous prescriptions for oral bisphosphonates”

31. What is the quality of the current evidence from observational analyses?
April2012: “Patients taking oral fluoroquinolones were at a higher risk of developing a retinal detachment”
Dec2013: “Oral fluoroquinolone use was not associated with increased risk of retinal detachment”

32. Proposal: Follow open science in observational research
Data Analysis = Evidence
Patient-level
Data
Statistical programming
Evidence repository
Internally
Data captured during clinical practice
Standardized analytics following community best practice
Evidence used targeted search and exploratory browsing
Externally

33. Proposal: Follow open science in observational research
Data Analysis = Evidence
Patient-level
Data
Statistical programming
Evidence repository
Internally
Open Science tenets:
Transparency
Reproducibility
Replicability - same data + different analysis = similar evidence? different data + same analysis = similar evidence?
Reliability - evidence can be interpreted honestly with known operating characteristics
Efficiency - access to evidence can be minutes instead of months
Data captured during clinical practice
Standardized analytics following community best practice
Evidence used targeted search and exploratory browsing
Externally

34. Join the journey
Discussion / questions / comments