Propensity Score Matching in SPSS: How to turn an Audit into a RCT Outline   What is Propensity score matching? Propensity Score Matching in SPSS Example: Comparing patients with both Gout & diabetes to those with diabetes only Dealing with missing data Mario D Hair Independent Statistics Consultant

What is Propensity score matching? Developed by Rosenbaum & Rubin (1983). Two aspects 1. Generate the propensity score 2. Apply it to balance the data. Search hits using ‘Propensity score matching’ by year. Slide provided by Beng So, ST6 Queen Elizabeth Hospital, Glasgow   Mario D Hair Independent Statistics Consultant

What is Propensity score matching? 1. Generate the propensity score The propensity score is the probability (from 0 to 1) of a case being in a particular group based on a given set of covariates. Generally calculated using logistic regression with group (Treatment /Control) as dependent , covariates as independent variables. Caveats & Limitations Can only be two groups. If more groups need to analyse them pairwise. The propensity score is only as good as the predictors used to generate it. Propensity score not generated for any case with any missing data. Not interested in any aspect of the logistic model other than the probabilities.   The propensity score is a balancing score: The differences between groups on the covariates condensed down into a single score so if two groups balanced on the propensity score then balanced on all the covariates. Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant Slide provided by Beng So, ST6 Queen Elizabeth Hospital, Glasgow Mario D Hair Independent Statistics Consultant

What is Propensity score matching? 2. Apply propensity score to balance the data. Four main applications. Propensity score matching : Match one or more control cases with a propensity score that is (nearly) equal to the propensity score for each treatment case Stratification: Divide sample into strata based on rank-ordered propensity scores. Comparisons between groups are then performed within each stratum. Regression adjustment: Include propensity scores as a covariate in a regression model used to estimate the treatment effect. Weighting: Inverse probability of treatment weighting (IPTW) weights cases by the inverse of propensity score. Similar to use of survey sampling weights used to ensure samples are representative of specific populations. Often used in survival analyses. Austin (2011) reports that propensity score matching is better than stratification or regression adjustment and is at least as good as IPTW. It is increasingly the most widely used method. Mario D Hair Independent Statistics Consultant

Propensity Score Matching in SPSS Available in SPSS V22 but Prior to that only as ‘PS matching’ an extension command that requires both r and the r plug-in. Developed by Felix Thoemmes at Cornell University. PS matching: http://sourceforge.net/projects/psmspss/ contains Latest version of the software, psmatching 3.04 June 2015. (this talk uses 3.03) Installation instructions (in a file called ‘readme.txt’) Thoemmes 2012 paper describing the software (called ‘arxiv preprint.pdf’). Comparison of PS matching & SPSS Propensity score matching   Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant Propensity score matching SPSS V22 PS Matching Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant Example: Comparing 1714 patients with BOTH Gout & diabetes to 15,224 patients with ONLY diabetes Covariates Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant Univariate stats: Comparing BOTH Gout & diabetes to those with ONLY diabetes * p < 0.05 †using t to d conversions d = 2t/sqrt(df) & d = ln(OR)*(√3/π) Mario D Hair Independent Statistics Consultant

PS Matching: Using a file with only the covariates Warning: PS Matching will not work if there are missing values on any variable Mario D Hair Independent Statistics Consultant

Propensity score matching SPSS V22 PS Matching However Propensity Score Matching does work if there are missing values on any variable Mario D Hair Independent Statistics Consultant

PS Matching has more options & diagnostics Mario D Hair Independent Statistics Consultant

PS Matching Outputs : Datasets Matched cases Paired cases wide format Mario D Hair Independent Statistics Consultant

Propensity score Matching SPSS V22 Output Mario D Hair Independent Statistics Consultant

PS Matching Outputs : Diagnostics Samples sizes of matched data Overall balance test (Hansen & Bowers, 2010) chisquare df p.value Overall .883 5.000 .971 Overall balance tests Relative multivariate imbalance L1 (Iacus, King, & Porro, 2010) Before matching After matching Multivariate imbalance measure L1 .290 .167 Covariates Means Treated Means Control SD Control Std. Mean Diff. Before After propensity .136 .097 .135 .058 .073 .524 .005 Age 69.928 65.511 69.938 12.416 11.049 .409 -.001 sex0 .667 .542 .661 .498 .473 .266 .014 sex1 .333 .458 .339 -.266 -.014 curr_smok1 .110 .189 .118 .392 .323 -.252 -.026 Thiazide11 .138 .165 .134 .371 .341 -.077 .012 Diuretic11 .460 .307 .461 .499 .306 -.002 Detailed balance Summary of any unbalanced covariate terms inc interactions Summary of unbalanced covariates (|d| > .25) No covariate exhibits a large imbalance (|d| > .25). Mario D Hair Independent Statistics Consultant

PS Matching Outputs : Diagnostic plots Histogram of propensity scores Jitter plot Mario D Hair Independent Statistics Consultant

PS Matching Outputs : Diagnostic plots Dotplot of standardized mean differences Graphical representation of data from detailed balance stats Covariates Std. Mean Diff. Before After propensity .524 .005 Age .409 -.001 sex0 .266 .014 sex1 -.266 -.014 curr_smok1 -.252 -.026 Thiazide11 -.077 .012 Diuretic11 .306 -.002 Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant Adding the lipid data to matched file using merge where original (non active) is the keyed file Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant Univariate stats: Comparing BOTH Gout & diabetes to those with ONLY diabetes Covariates after matching Age Be f ore Af t er Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant Univariate stats: Comparing BOTH Gout & diabetes to those with ONLY diabetes Lipids after matching Mario D Hair Independent Statistics Consultant

Dealing with missing data 1 Missing data in non covariate data: Use the paired data format Green is paired comparison, red is matched. There are no substantive changes Mario D Hair Independent Statistics Consultant

Dealing with missing data 2 Missing data in covariates: Use multiple imputation Separate creation of propensity scores from the matching Run logistic regression on imputed datasets Aggregate to get mean (median) propensity score Use the aggregate file to do the matching Load in the other variables Use imputation again if missing data in non-covariates Mario D Hair Independent Statistics Consultant

Mario D Hair Independent Statistics Consultant References Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46, 399-424. doi:10.1080/00273171.2011.568786 One of the foremost authors on the subject. Beal S J & Kupzyk K A, An Introduction to Propensity Scores What, When, and How. The Journal of Early Adolescence January 2014 vol. 34 no. 1 66-92 doi:10.1177/0272431613503215. Easy to read introduction   Iacus, S. M., King, G., & Porro, G. (2009). CEM: Coarsened exact matching software. Journal of Statistical Software, 30, 1-27. Reference for ‘relative multivariate imbalance test’ Mitra, R., & Reiter, J. P. (2012). A comparison of two methods of estimating propensity scores after multiple imputation. Statistical methods in medical research, 0962280212445945. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55. doi:10.1093/biomet/70.1.41. Seminal paper. Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of internal medicine, 127(8_Part_2), 757-763. Example of stratification. Thoemmes, F. (2012). Propensity score matching in SPSS. arXiv preprint arXiv:1201.6385. Explains use of ‘ps matching’. Mario D Hair Independent Statistics Consultant

Propensity Score Matching in SPSS: How to turn an Audit into a RCT Outline  What is Propensity score matching? Propensity Score Matching in SPSS Example: Comparing patients with both Gout & diabetes to those with diabetes only Dealing with missing data Thank you: Questions? Mario D Hair Independent Statistics Consultant