Transparency in the Use of Propensity Score Methods John D. Seeger, PharmD, DrPH Chief Scientist, i3 Drug Safety Adjunct Assistant Professor, Harvard School of Public Health September 9, 2008 With thanks to: Alec Walker, Tobias Kurth, Jeanne Loughlin, Mona Eng, and Alex Cole

Propensity Score Analysis – When? Amenable to Propensity Techniques Thanks to S. Schneeweiss

102,400 potential matching groups Motivation Assume matching when comparing 2 treatments: For every drug user with given characteristics Find a comparator with identical characteristics Example: Male, age 45, smoker, with HTN… Matching fails: Age (10 categories) x Sex (2 categories) x Prior diagnoses (5 @ 2 categories each) Prior drug therapy (5 @ 2 categories each) Preceding cost of care (5 categories) 102,400 potential matching groups

Propensity Score Collapses Exposure Predictors Single value Probability: subject will receive therapy vs comparator Removes confounding by components of the score Patient characteristics that favor one therapy over another Permits Restriction Matching Stratification Modeling Weighting

Should Propensity Scores Always be Used? More than 8 events per covariate leads to unbiased estimates So propensity score favored when: Many more persons exposed to drug of interest than study outcomes Common exposure Rare outcome Allows for richer model (more predictors) of exposure than outcome Alternative hypotheses Cepeda S, et al. Am J Epidemiol 2003;158:280-287.

Estimate Propensity Score Predict treatment from baseline covariates within database Inclusion of predictors a-priori (what characteristics are used to prescribe?) Empiric (what differentiates initiators?) Generic (what patterns of healthcare predict initiation?) Coefficients of propensity score Interpretable and Informative

Propensity Score Restriction Sturmer T, et al. J Clin Epidemiol 2006;59:437-47.

Propensity Score Restriction Potential for serious adverse events from error (name confusion) Amaryl (glimepiride an oral hypoglycemic) Reminyl (galantamine for Alzheimer’s disease) 36,816 people with AD diagnosis (14,626 Reminyl dispensings) 236 Amaryl recipients 24 Amaryl recipients in the lowest decile of the propensity score 13 with a single dispensing of Amaryl or no diabetes diagnoses 2 with no diabetes-related claims across entire claim history Medical record review suggested no error Propensity score restriction may be used as a screening method to identify unusual patterns of healthcare for closer scrutiny Possible medication dispensing errors Others Confirmation requires additional data, which could be obtained through medical record review.

Propensity Score Distribution and Strata C-statistic = 0.739

Effect of Temazepam Relative to Zopiclone Transparent analysis: Within-stratum balance Stratum-specific effect estimates as well as pooled estimate Explicit evaluation of potential for effect measure modification

Matching on the Propensity Score Matching can be performed by: Standard automated case-control matching programs where the matching range is specified Nearest available match based on the propensity score Greedy matching techniques (http://www2.sas.com.proceedings/sugi26/p214-26.pdf)

Characteristics Before Matching

Balance Achieved by Matching

Analysis by 2X2 Table

MI Outcome (After Matching) 31% (7%-48%) Risk Reduction HR=0.69 (0.52-0.93) Cumulative Incidence Statin Non-Initiators Statin Initiators Months of Follow-Up

Regression Adjustment with Propensity Scores All study participants are used Still a two-step approach (exposure and outcome) More power compared to including all covariates into the model, since degrees of freedom are gained However, assumes the underlying association between the score and the outcome is modeled appropriately

Weighting IPTW SMR 1/ê(X), in treated 1 in treated 1/(1- ê(X)), in untreated 1 in treated (ê(X)/(1-ê(X)) in untreated IPTW SMR

Baseline Characteristics

Cohort Results

Are Divergent Results Possible? Kurth T, et al. Am J Epidemiol 2006;163:262-70.

What About Unmeasured Confounding? Obesity, Smoking, Exercise

Accounting for Variables had Little Effect

Conclusion Propensity score can be useful for addressing confounding (by indication) Allows for rich model of exposure to be developed Advantageous when number of people with a study outcome is small relative to number of exposed persons and number of potential confounders is large Drug effects (particularly adverse ones) Consider transparency When selecting propensity score When building propensity score When using propensity score

Thank-You John.Seeger@i3DrugSafety.com