1. Propensity Score Adjustment for Comparative Efficacy Studies of Disease Modifying Agents in Multiple Sclerosis
Carrie M. Hersh, DO,1 Claire Hara-Cleaver, RN,1 Jeffrey A. Cohen, MD,1 Robert A. Bermel, MD,1 Daniel Ontaneda, MD1
1Mellen Center for Multiple Sclerosis Treatment and Research, Neurological Institute, Cleveland Clinic, Cleveland Ohio
AAN 2015
P3.210
Background
Observational studies are valuable to assess comparative efficacy among multiple sclerosis (MS) disease modifying therapies (DMT) but are susceptible to confounding and bias that may produce spurious results even when comparison groups look similar.
Propensity score (PS) analysis is employed to estimate a conditional probability of treatment assignment using baseline covariates. Conditional on the PS, the distribution of the baseline covariates are made similar between the two treatment groups.
PS allows estimation of the causal effect of treatment by limiting bias in observational studies.
Objective
To highlight the utility and proposed methodology of propensity score analysis in observational studies in MS.
Design/Methods
Clinical Procedures and Data Collection
Fingolimod was administered at The Cleveland Clinic Mellen Center in accordance with FDA recommendations.
For comparison purposes, patients directly switched from intramuscular interferon beta-1a (IM IFN beta-1a) or glatiramer acetate (GA) to fingolimod between October 2010 and August 2011 were identified through the electronic medical record.
Baseline and 12 month follow-up data for each patient were collected and stored in a password-protected database.
Propensity Score (PS) Methods/Analysis
Patients switched from IM IFN beta-1a and GA to fingolimod were compared using PS methods based on clinical and objective data, by unadjusted direct comparisons, 1:1 greedy matching without replacement, and Average Treatment effect on the Treated (ATT) weighting.
PS model, defined as propensity to receive IM IFN beta-1a, included the following baseline covariates:
Demographics: age, gender, and race
Disease History: course and duration
MRI Measures: baseline brain MRI lesion burden and number of gadolinium-enhancing lesions
Primary outcome was fingolimod discontinuation rate at 12 months for tolerability. Secondary outcomes included number of clinical relapses, proportion relapse-free, time to first clinical relapse, number of gadolinium-enhancing lesions on brain MRI, and change in Timed 25 Foot Walk (T25FW) at 12 months.
Design/Methods cont.
Unadjusted estimates were calculated using chi-square tests. After matching and weighting, groups were compared with conditional logistic regression to obtain odds ratio estimates for binary outcomes and with linear regression to obtain estimates for continuous outcomes. Survival data were estimated by stratified Cox regression.
Odds ratios refer to IFN beta-1a compared to GA switchers to fingolimod.
Results
317 patients were started on fingolimod.
64 patients switched from IM IFN beta-1a to fingolimod.
78 patients switched from GA to fingolimod.
Baseline information for each patient cohort is presented in Table 1.
Before adjustment, IM IFN beta-1a and GA switchers had similar baseline disease characteristics. There was a borderline statistically significant greater number of female patients in the IM IFN beta-1a cohort compared to GA switchers.
Without adjustments, the range of standardized differences between the GA and IM IFN beta-1a treated patients was wide, to
There was adequate overlap of propensities between the treatment groups (Figure 1).
1:1 PS matching failed to optimally improve covariate balance (Figure 2).
ATT weighting with adjustment on the linear propensity score (“double robust” approach) was superior to 1:1 greedy matching on the linear PS at achieving covariate balance between groups (Figure 3).
Results cont.
Unadjusted and adjusted tolerability and efficacy estimates are presented in Table 2.
Before propensity score weighting, there was a difference in fingolimod discontinuation rates between the IM IFN beta 1a and GA groups that approached significance [OR=0.36; 95% CI (0.12, 1.04)].
Propensity score weighting, which showed excellent covariate balance (Figure 3), decreased the difference in discontinuation rates [OR=0.51; 95% CI (0.16, 1.59)], confirming expected lack of differences between comparison groups.
No significant differences in secondary outcome measures between IM IFN beta-1a and GA switchers to fingolimod were found.
Secondary outcome measures including number of relapses, proportion relapse-free, time to first relapse, number of brain MRI gadolinium-enhancing lesions, and T25FW showed comparable results before and after propensity score weighting.
Similar results were observed following propensity matched analysis.
Conclusions/Discussion
Our results illustrate the utility of propensity techniques to address confounding and bias in observational studies in MS.
Propensity score weighting effectively minimized covariate imbalance between patients who switched from either IM IFN beta-1a or GA to fingolimod.
Results demonstrate feasibility of using ATT weighting to increase comparability in two treatment groups. ATT weighting compares mean outcomes for individuals who received IM IFN beta-1a to the mean outcomes if these same individuals had instead received GA.
MRI covariate measures in our study, although not quantitative, are used by clinicians to make treatment decisions and therefore appropriately balance treatment bias.
As expected, results indicate there are no significant differences in the tolerability and efficacy of fingolimod in patients who directly switched from IM IFN beta-1a compared to GA.
Potentially spurious findings- differences in fingolimod discontinuation rates- were avoided with propensity score analysis.
Dr. Carrie Hersh is supported by National Multiple Sclerosis Society Sylvia Lawry Physician Clinical Fellowship Award FP 1788-A-1. Claire Hara-Cleaver: Consultancy- Biogen Idec, TEVA neuroscience, EMD Serono, Acorda pharmaceuticals, Novartis, and Genzyme. Dr. Jeffrey Cohen has received consulting fees from Biogen Idec, EMD Serono, Genzyme, Novartis, Receptos, Synthon, Teva, and Vaccinex. Dr. Robert Bermel: Consultancy- Biogen Idec, Novartis, Teva, Genzyme, Questcor; Grants/grants pending- Biogen Idec, Novartis. Dr. Ontaneda is supported by KL2 TR000440/TR/NCATS NIH Grant.
Linear PS
Raw PS
Gad
Mod MRI Burden
Mild MRI Burden
Duration
Age
Female
White
RRMS
Figure 3. Standardized difference plot for ATT weighting
Standardized Difference (%)
Figure 1. Density plot of propensity scores (propensity for IFN beta-1a)
Propensity for IFN beta 1a
Table 2: Unadjusted and adjusted outcomes 
Tolerability Outcomes at 12 Months
Causal effect of treatment
Unadjusted
1:1 Matching
ATT Weighting
Discontinuation
0.36
0.40
0.51
OR (95% CI)
(0.12, 1.04)
(0.13, 1.28)
(0.16, 1.59)
Efficacy Outcomes at 12 Months
Relapse Count
0.03
0.02
Difference (95% CI)
(-0.18, 0.12)
(-0.14, 0.09)
Proportion Relapse-Free
1.06
1.00
1.12
(0.37, 3.03)
(0.37, 3.42)
Time to First Relapse
0.96
0.86
0.91
Relative Hazard Rate (95% CI)
(0.36, 2.57)
(0.29, 2.55)
(0.33, 2.53)
Number of Brain MRI Gad-Enhancing Lesions
Timed-25 Foot Walk
0.26
0.28
0.55
(-1.82, 1.29)
(-1.76, 1.20)
(-2.67, 1.57)
Table 1. Baseline demographics and disease characteristics prior to propensity
analysis
Covariate
Treatment
p-value
IFN beta-1a GA
(n=64)
(n=78)
Age at symptom-onset, years
40.3 ± 9.1
p=0.11
Mean ± standard deviation
37.9 ± 9.3
Gender, female sex
38 (59.4)
59 (75.6)
p=0.047
n (%)
Race, white
62 (96.9)
71 (91.0)
p=0.19
Course, RRMS
61 (95.3)
67 (85.9)
p=0.09
Disease Duration, years
6.81 ± 6.6
8.28 ± 6.8
Baseline Brain MRI Lesion Burden
p=0.68
Mild
28 (43.8)
33 (42.3)
Moderate
30 (46.9)
34 (43.6)
Severe
6 (9.4)
11 (14.1)
Number of Brain MRI
0.16 ± 0.4
0.12 ± 0.3
p=0.49
Gad-Enhancing Lesions
Standardized Difference (%)
Figure 2. Standardized difference plot for 1:1 greedy matching
Raw PS
Linear PS
Gad
Mod MRI Burden
Mild MRI Burden
Duration
White
Age
RRMS
Female