1. Propensity Scores October 2014 Alexander M. Walker MD, DrPH Extensive parts of this presentation incorporate the work of John D. Seeger, PharmD, DrPH

2. “In mathematics, you don't understand things. You just get used to them.” - John von Neumann 1903-1957

3. Research Goal  Compare two treatments with respect to a health or economic outcome  “Counterfactual” ideal If the same people had received B instead of A, how would their outcomes have differed?  What is achievable: “similar” not “same” Comparable treatment groups … insofar as you can tell!

4. 4 Pictures for Confounding

5. Comparison of Heterogeneous Groups 5 E1E2

6. 6 E1E2 Internal Composition May Differ

7. 7 Affected individuals E1E2 Risks that Depend on Subgroup Status

8. 8 Affected individuals 50% 15% 50% 15% E1E2 Risks that Depend on Subgroup Status Note that the proportions of affected individuals are different in the yellow and gray groups …

9. 9 50% 15% 50% 15% E1E2 Risks that Depend on Subgroup Status Note that the proportions of affected individuals are different in the yellow and gray groups … … and that the group- specific risks are identical in the two exposed populations E1 and E2.

10. 10 The differences in risk are due to the covariate structure of compared populations, not to the effects of E1 and E2 E1E2 Internal Risk Factor Heterogeneity Creates an Differences in Group Risk

11. 11 Propensity Scores to Create Populations with Similar Covariate Structure

12. 12 E1 E2 Covariate Heterogeneity E1 has more Yellow E2 has more Gray

13. 13 E2 E1 E2 Gray predicts E2 Yellow predicts E1 Covariate Status as a Predictor of Treatment

14. 14 Propensity Scores  PS is the predicted probability of treatment, given all the covariates  Matching on the PS creates study populations that have balance on the covariates  Perfect for a single, dichotomous covariate  Not perfect, but very good for multiple covariates

15. 15 E1 E2 Propensity for Covariate Patterns Think of orange and green as two distinct covariate patterns that have the same predicted Pr(E1). Pr(E1|Orange)=x Pr(E1|Green) =x

16. 16 E1 E2 Gathering subjects with identical propensity puts all individuals with covariate patterns orange and green into the same stratum. Conditioning on Propensity Permits Unconfounded Comparisons At a given propensity level x, there is no association between treatment and covariate patterns. Pr(E1|Orange)=x Pr(E1|Green) =x

17. Formal Expression Propensity(x)  P(T=1|x) = E(T|x) The propensity associated with level x of the covariate X is the probability that treatment is present (equivalently, is “B” as opposed to “A”), given level x, and this is in turn equal to the expected value of treatment, given x. Note that the definition does not specify the parametric form of the Propensity(x). The examples in this talk use a logistic function; others -- including nonparametric functions -- are also used. Notation. A single capital letter denotes a variable, a single lower case letter denotes a particular value for that variable.

18. Probability Calculus Under propensity matching, how do X (covariate status) and T (treatment status) relate to one another? 1.Pr( x, t | p ) = Pr( x | p ) Pr( t | x, p ) Probability Theory 2.Pr( t | x, p ) = Pr( t | p ) p incorporates all information about t that is in x  Pr( x, t | p ) = Pr( x | p ) Pr( t | p )

19. Pr( x, t | p ) = Pr( x | p ) Pr( t | p ) Given a particular value of the propensity score variable, that is at P=p, the covariates X and T are uncorrelated. At particular levels of P individually and therefore collectively (i.e. “conditionally on P”), the lack of correlation guarantees that X cannot confound the association between T and any outcome.

20. 20 Matching on Propensity Scores

21. Propensity Matching: Method  Identify candidate predictors of treatment B v A  Perform a logistic regression of B v A  Obtain from the regression a “predicted” probability of B v A  Sort all members of A and B according to this propensity  Match A patients to B patients on the propensity

22. Duragesic and Long-Acting Opioids DuragesicLA Opioids N5042,201 65+ years29%10% Male35%49% Periph Vasc Disease4%1% Sx of Abd or Pelvis18%10% > 2 hospitalztns 6 mo9%3% 30 days NonRx Costs$1,136$746

23. Straightforward Regression proc logistic data = mother.propensity2 descending; model DuragesicUser = DischCostIndex EncCostIndex RxCostIndex OtherCostIndex RxCostPrior1 OtherCostPrior1 AnyRx OneDisch TwoDisch ThreePlusDisch AnyICD443 AnyICD719 AnyICD724 AnyICD787 AnyICD789 q3_95_new q4_95 q1_96 q2_96 q3_96 q4_96 q1_97 q2_97 q3_97 q4_97 q1_98 q2_98 q3_98 q4_98 hmo men young old /rl; where enrbaseflag = 1 and validindex = 1 and sameday = 0 and medicare = 0 and malignant = 0; output out = mother.propensity3 p = score ; run;

24. Propensity Output Obs PATIENT score 1 2909 0.57475 2 3438 0.32899 3 3841 0.01324 4 5674 0.48411 5 5734 0.06892 6 8573 0.11780 7 10210 0.34692 8 13056 0.09737 9 13376 0.37350 10 16026 0.15635 11 16865 0.06106 12 16949 0.10568

25. 25 E1 Pr(E1)=x E2 (sample) E2 (residual) Choose from E2 a sample that matches E1 in size. Matching on Propensity

26. 26 E1 Pr(E1) = 0.5 E2 At every level of propensity in the constructed cohorts, Pr(E1) = 0.5. Therefore, treatment is uncorrelated with propensity, and you can collapse all the propensity- matched groups together to form a cohort in which all covariate patterns are uncorrelated with treatment, and there will be no confounding bias. Matching on Propensity

27. Stratum I II III IV V

29. Duragesic and Long-Acting Opioids DuragesicLA Opioids N5042,201 65+ years29%10% Male35%49% Periph Vasc Disease4%1% Sx of Abd or Pelvis18%10% > 2 hospitalztns 6 mo9%3% 30 days NonRx Costs$1,136$746

30. Propensity-Matched Cohorts DuragesicLA Opioids N478 65+ years26%25% Male36%33% Periph Vasc Disease4%3% Sx of Abd or Pelvis17%18% > 2 hospitalztns 6 mo8% 30 days NonRx Costs$1,084$1,043

31. Pharmacoepidemiol Drug Saf. 2005 Jul;14(7):465-76.

32. Do Statins Affect Risk of AMI?  The purpose of the study was to assess whether statins affect the risk of risk of acute myocardial infarction (AMI)  Strong predictors for statin use that affect risk of AMI  How to design an observational study?  Note: we would not ordinarily use observational data for efficacy questions, but this serves as a suitable test case because there is a known gold standard

33. +Risk Factors: age (45M, 55F), diabetes, smoking, HTN, low HDL, family history of premature CHD -Risk Factor: high HDL Risk Category LDL to initiate drug Tx LDL Goal of drug Tx No CHD and <2 Risk Factors  190 <160 No CHD and  2 Risk Factors  160 <130 With CHD >130  100 NCEP ATP II guidelines (1993) Good Clinical Practice Creates Confounding

34. Gold Standard for the Effect of Statins CARE Trial Results Sacks FM, et al N Engl J Med. 1996;335:1001-9

35. Data Source Fallon Community Health Plan Central Massachusetts HMO ~200,000 members Claims Data available on: –Enrollment (age, sex, date) –Ambulatory care visits –Hospitalization –Pharmacy dispensings (drug & quantity) –Laboratory tests (tests & results)

36. Patient Entry, Analytic Sequence 1993 199419951996199719981999 1 of 9 Blocks 1)Apply eligibility criteria FCHP member for at least 1 year At least one physician visit in last year LDL, HDL, TG levels in last 6 months At least one physician visit in cohort accrual block No PAD diagnosis before index date Not current statin user 2) Estimate propensity score (statin initiation) 3) Match statin initiators with non-initiators 4) Repeat for all blocks of time 5) Follow matched groups for diagnosis of MI 2nd/9 4 ~35,000 Members All Fallon members with any LDL > 130 mg/dl Require 1 year Enrollment

37. Current Statin Users (1501) Statin Initiators, Eligible (77) Statin Initiators, Not Eligible (34) Non Statin Users, Not Eligible (24,799) Non Statin Users, Eligible (9,639) Month of 1/1/94 Propensity Score Matching Total subjects in cohort (36,050)

38. “Typical” Statin Initiator and Non-Initiator

39. 111% (46%- 204%) Risk Increase Statin Non-Initiators Statin Initiators Months of Follow-Up Cumulative Incidence MI Outcome (Unmatched) HR=2.11 (1.46-3.04)

40. Calculate Propensity Score  Predict treatment Statin initiation vs. not In each 6-month period of cohort accrual  Using baseline covariates  Obtain fitted values from regression  Fitted value for each study subject is the Propensity Score

41. Construct Rich Model  More than 8 events per covariate leads to unbiased estimates  Many more persons exposed to drug of interest than study outcomes  In Drug Safety studies, usually the outcome is rare  Therefore can control for more covariates when exposure is dependent variable than when outcome is Cepeda S, et al. Am J Epidemiol 2003;158:280-287. Propensity modeling permits the introduction of many, many covariates

42. *build model for 9501; proc logistic descending data=new1; model statin = male smok obes age9501 ang9501 usa9501 chf9501 isch9501 ath9501 cva9501 usa9501 mi9501 olmi9501 htn9501 tia9501 afib9501 ascv9501 hth9501 ost9501 cvs9501 htdx9501 circ9501 cond9501 rvsc9501 hhd9501 dysr9501 hrt9501 ns9501 ins9501 diab9501 skca9501 depr9501 adj9501 schz9501 deb9501 rheu9501 days9501 lres9501 tres9501 hres9501 hbac9501 cvhp9501 ekg9501 cvrx9501 cvvs9501 llab9501 lab9501 cvdg9501 hosp9501 rx9501 vist9501 diag9501 ; output out=psmodel pred=PROPSCORE; run;

43. Propensity Regression Parameter Estimates

44. Obs ID STATINPROPSCORE 1 1909 00.57475 2 2438 00.33899 3 3841 00.01324 4 4674 00.48411 5 4734 00.06892 6 5573 00.11780 7 6210 10.34692 8 7056 10.09737 9 7376 10.37350 10 8026 10.15635 11 8865 10.06106 12 9949 10.10568. Output File – Propensity Scores

45. 45

47. Obs ID STATINPROPSCORE 1 1909 00.57475 2 2438 00.33899 3 3841 00.01324 4 4674 00.48411 5 4734 00.06892 6 5573 00.11780 7 6210 10.34692 8 7056 10.09737 9 7376 10.37350 10 8026 10.15635 11 8865 10.06106 12 9949 10.10568. Match exposed to non-exposed according to Propensity Score values

48. Obs ID STATINPROPSCOR 1 1909 00.57475 2 2438 00.33899 3 3841 00.01324 4 4674 00.48411 5 4734 00.06892 6 5573 00.11780 7 6210 10.34692 8 7056 10.09737 9 7376 10.37350 10 8026 10.15635 11 8865 10.06106 12 9949 10.10568. Match exposed to non-exposed according to Propensity Score values

49. Obs ID STATINPROPSCOR 1 1909 00.57475 2 2438 00.33899 3 3841 00.01324 4 4674 00.48411 5 4734 00.06892 6 5573 00.11780 7 6210 10.34692 8 7056 10.09737 9 7376 10.37350 10 8026 10.15635 11 8865 10.06106 12 9949 10.10568. Match exposed to non-exposed according to Propensity Score values

51. Balance Achieved by Matching Only 1 of 52 variables sig. different at P<0.05

52. 31% (7%-48%) Risk Reduction Statin Non-Initiators Statin Initiators Months of Follow-Up Cumulative Incidence MI Outcome (After Matching) HR=0.69 (0.52-0.93)

53. Interpreting Propensity Coefficients 53

57. When Is the Model Sufficient? 57

58. Early Matching Results

59. New Variables Suggested post hoc for the Propensity Score Cardiac Disease  Cardiovascular  Diagnoses  Hospitalizations  Outpatient visits  Medications  EKGs  Number of labs  Number of lipid labs Other Causes of “Medicalization”  Schizophrenia  Adjustment Disorder  Depression  Non-Skin CA  Skin CA  Debility  Rheumatic Disease

60. Imbalance on Non-Included Variables

61. NIVs are Predictors of Statin Initiation

62. New Ranking of Predictors

63. Balance on New Variables

64. Thank You!