1 Hester van Eeren Erasmus Medical Centre, Rotterdam Halsteren, August 23, 2010

2 The propensity score (1) The propensity score is “…the conditional probability of assignment to a particular treatment given a vector of observed covariates.” (Rosenbaum en Rubin, 1983: 41). Used in non-randomized studies to control for selection bias Balance observed pretreatment variables among patient Find an estimate of the average treatment effect But, treatment effect can be different within subgroups

3 The propensity score (2) Univariate propensity scoreMultivariate propensity score Bartak and colleagues (2009)Spreeuwenberg and colleagues (2010) Used for 2 treatment categoriesUsed for > 2 treatment categories Propensity score used in: Matching Stratification Regression Inverse probability weight Combinations of …

Methods in this study To find a treatment effect within subgroups, if the propensity score is applied: Method 1: Regression analysis with propensity score, subgroups and interaction with treatment assignment; Method 2: Weighted regression analysis with inverse of the propensity score (to weight observations), subgroups and interaction with treatment assignment; Method 3: Propensity score applied for groups defined on treatment assignment and subgroups; then, regression analysis with propensity score and dummies for groups Two treatment categories and two subgroups are used in this study 4

Variable selection for propensity score Does the variable for subgroups has to be included? Discussion about variable selection for propensity score; Only variables related to outcome? Only variables related to treatment assignment? Both variables…? In this study; 8 different propensity scores (PS) formulated, based on: Variables related to outcome, with and without subgroup Variables related to treatment assigment, with and without subgroup Both variables…, with and without subgroup Only variables related to both outcome and treatment assignment, with and without subgroup 5

How to test? (1) Real dataset not useful because effects unknown beforehand;  You cannot test whether the effect found is accurate Monte Carlo simulation study to test methods and different propensity scores: Simulate data with known treatment effects Estimate different propensity scores for this data Apply different methods for different propensity scores, for this data  Repeat this process 1000 times 6

How to test? (2) What do you want to know? If the treatment effect estimated is (almost) equal to the treatment effect you used to simulate the data Bias of estimator: difference between estimated treatment effect and the true value of parameter Want to have an unbiased estimate; Less bias indicates a more accurate estimate of the treatment effect Bias is estimated for overall treatment effect and for the treatment effect within subgroups 7

Results 8 N=250;ρ=0N=250;ρ=0.3N=250; ρ=0.7N=500;ρ=0N=500;ρ=0.3N=500; ρ=0.7N=1000;ρ=0N=1000;ρ=0.3N=1000; ρ=0.7 RowMethodBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSE Regression 1PS1, func PS1, func PS1, func PS1, subgr PS2, func PS2, func PS2, func PS2, subgr PS3, func PS3, func PS3, func PS3, subgr PS4, func PS4, func PS4, func PS4, subgr PS5, func PS5, func PS5, func PS5, subgr PS6, func PS6, func PS6, func PS6, subgr PS7, func PS7, func PS7, func PS7, subgr PS8, func PS8, func PS8, func PS8, subgr N=250;ρ=0N=250;ρ=0.3N=250; ρ=0.7N=500;ρ=0N=500;ρ=0.3N=500; ρ=0.7N=1000;ρ=0N=1000;ρ=0.3N=1000; ρ=0.7 RowMethodBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSE Regression 1PS1, func PS1, func PS1, func PS1, subgr PS2, func PS2, func PS2, func PS2, subgr PS3, func PS3, func PS3, func PS3, subgr PS4, func PS4, func PS4, func PS4, subgr PS5, func PS5, func PS5, func PS5, subgr PS6, func PS6, func PS6, func PS6, subgr PS7, func PS7, func PS7, func PS7, subgr PS8, func PS8, func PS8, func PS8, subgr N=250;ρ=0N=250;ρ=0.3N=250; ρ=0.7N=500;ρ=0N=500;ρ=0.3N=500; ρ=0.7N=1000;ρ=0N=1000;ρ=0.3N=1000; ρ=0.7 RowMethodBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSEBiasMSE Inverse PS 1PS1, func PS1, func PS1, func PS1, subgr PS2, func PS2, func PS2, func PS2, subgr PS3, func PS3, func PS3, func PS3, subgr PS4, func PS4, func PS4, func PS4, subgr PS5, func PS5, func PS5, func PS5, subgr PS6, func PS6, func PS6, func PS6, subgr PS7, func PS7, func PS7, func PS7, subgr PS8, func PS8, func PS8, func PS8, subgr

Results (1) Which propensity score is the most accurate within each method tested (tested with ANOVA): But, some values for bias per propensity score where not very different from each other… 9 General treatment effectTreatment effect within subgroups Method 1PS with variables related to outcome Method 2PS with variables related to outcome and variable for subgroups PS with variables only related to outcome and treatment assignment and variables for subgroups Method 3PS with variables related to outcome NA

Results (2) Which method is most accurate when the most accurate propensity scores are compared? Decide on partial effect size of method in ANOVA* For general treatment effect, the partial effect size is 0,028, where method 1 gives the lowest bias (followed by method 3) For treatment effect within subgroups, the partial effect size is 0,051, where method 1 gives the lowest bias too Although the effect sizes for method are not very large, regression analysis with treatment assignment, subgroup, interaction between these and the propensity score, which is estimated with variables related to outcome, seems to be the most accurate method to find treatment effects within subgroups *Effect size – 0,010 = small; 0,059 = medium; 0,138 = large (Cohen, 1988) 10

Discussion (1) Data simulation is done for different settings: Sample size, correlation between covariates and correlation with covariate for subgroups are changed over simulations Results for most accurate propensity score are based on sum of bias over all these settings; comparisons between methods for all propensity scores could give more in depth results The overall bias for different propensity scores was sometimes not very different Model for simulation of data was simple, linear; the relation between variables and outcome in practice can be more complicated …. 11

Discussion (2) Questions? 12