1. Propensity Score Matching in SPSS: How to turn an Audit into a RCT
Outline
What is Propensity score matching?
Propensity Score Matching in SPSS
Example: Comparing patients with both Gout & diabetes to those with diabetes only
Dealing with missing data
Mario D Hair Independent Statistics Consultant

2. What is Propensity score matching?
Developed by Rosenbaum & Rubin (1983). Two aspects
1. Generate the propensity score
2. Apply it to balance the data.
Search hits using ‘Propensity score matching’ by year.
Slide provided by Beng So, ST6 Queen Elizabeth Hospital, Glasgow
Mario D Hair Independent Statistics Consultant

3. What is Propensity score matching?
1. Generate the propensity score
The propensity score is the probability (from 0 to 1) of a case being in a particular group based on a given set of covariates.
Generally calculated using logistic regression with group (Treatment /Control) as dependent , covariates as independent variables.
Caveats & Limitations
Can only be two groups. If more groups need to analyse them pairwise.
The propensity score is only as good as the predictors used to generate it.
Propensity score not generated for any case with any missing data.
Not interested in any aspect of the logistic model other than the probabilities.
The propensity score is a balancing score: The differences between groups on the covariates condensed down into a single score so if two groups balanced on the propensity score then balanced on all the covariates.
Mario D Hair Independent Statistics Consultant

4. Mario D Hair Independent Statistics Consultant
Slide provided by Beng So, ST6 Queen Elizabeth Hospital, Glasgow
Mario D Hair Independent Statistics Consultant

5. What is Propensity score matching?
2. Apply propensity score to balance the data. Four main applications.
Propensity score matching : Match one or more control cases with a propensity score that is (nearly) equal to the propensity score for each treatment case
Stratification: Divide sample into strata based on rank-ordered propensity scores. Comparisons between groups are then performed within each stratum.
Regression adjustment: Include propensity scores as a covariate in a regression model used to estimate the treatment effect.
Weighting: Inverse probability of treatment weighting (IPTW) weights cases by the inverse of propensity score. Similar to use of survey sampling weights used to ensure samples are representative of specific populations. Often used in survival analyses.
Austin (2011) reports that propensity score matching is better than stratification or regression adjustment and is at least as good as IPTW. It is increasingly the most widely used method.
Mario D Hair Independent Statistics Consultant

6. Propensity Score Matching in SPSS
Available in SPSS V22 but Prior to that only as ‘PS matching’ an extension command that requires both r and the r plug-in. Developed by Felix Thoemmes at Cornell University.
PS matching:
contains
Latest version of the software, psmatching 3.04 June (this talk uses 3.03)
Installation instructions (in a file called ‘readme.txt’)
Thoemmes 2012 paper describing the software (called ‘arxiv preprint.pdf’).
Comparison of PS matching & SPSS Propensity score matching
Mario D Hair Independent Statistics Consultant

7. Mario D Hair Independent Statistics Consultant
Propensity score matching SPSS V22 PS Matching
Mario D Hair Independent Statistics Consultant

8. Mario D Hair Independent Statistics Consultant
Example: Comparing 1714 patients with BOTH Gout & diabetes to 15,224 patients with ONLY diabetes
Covariates
Mario D Hair Independent Statistics Consultant

9. Mario D Hair Independent Statistics Consultant
Univariate stats: Comparing BOTH Gout & diabetes to those with ONLY diabetes
* p < †using t to d conversions d = 2t/sqrt(df) & d = ln(OR)*(√3/π)
Mario D Hair Independent Statistics Consultant

10. PS Matching: Using a file with only the covariates
Warning:
PS Matching will not work if there are missing values on any variable
Mario D Hair Independent Statistics Consultant

11. Propensity score matching SPSS V22 PS Matching
However Propensity Score Matching does work if there are missing values on any variable
Mario D Hair Independent Statistics Consultant

12. PS Matching has more options & diagnostics
Mario D Hair Independent Statistics Consultant

13. PS Matching Outputs : Datasets
Matched cases
Paired cases wide format
Mario D Hair Independent Statistics Consultant

14. Propensity score Matching SPSS V22 Output
Mario D Hair Independent Statistics Consultant

15. PS Matching Outputs : Diagnostics
Samples sizes of matched data
Overall balance test (Hansen & Bowers, 2010)
chisquare df
p.value
Overall
.883
5.000
.971
Overall balance tests
Relative multivariate imbalance L1 (Iacus, King, & Porro, 2010)
Before matching
After matching
Multivariate imbalance measure L1
.290
.167
Covariates
Means Treated
Means Control
SD Control
Std. Mean Diff.
Before
After
propensity
.136
.097
.135
.058
.073
.524
.005
Age
69.928
65.511
69.938
12.416
11.049
.409
-.001
sex0
.667
.542
.661
.498
.473
.266
.014
sex1
.333
.458
.339
-.266
-.014
curr_smok1
.110
.189
.118
.392
.323
-.252
-.026
Thiazide11
.138
.165
.134
.371
.341
-.077
.012
Diuretic11
.460
.307
.461
.499
.306
-.002
Detailed balance
Summary of any unbalanced covariate terms inc interactions
Summary of unbalanced covariates (|d| > .25)
No covariate exhibits a large imbalance (|d| > .25).
Mario D Hair Independent Statistics Consultant

16. PS Matching Outputs : Diagnostic plots
Histogram of propensity scores Jitter plot
Mario D Hair Independent Statistics Consultant

17. PS Matching Outputs : Diagnostic plots
Dotplot of standardized mean differences
Graphical representation of data from detailed balance stats
Covariates
Std. Mean Diff.
Before
After
propensity
.524
.005
Age
.409
-.001
sex0
.266
.014
sex1
-.266
-.014
curr_smok1
-.252
-.026
Thiazide11
-.077
.012
Diuretic11
.306
-.002
Mario D Hair Independent Statistics Consultant

18. Mario D Hair Independent Statistics Consultant
Adding the lipid data to matched file using merge where original (non active) is the keyed file
Mario D Hair Independent Statistics Consultant

19. Mario D Hair Independent Statistics Consultant
Univariate stats: Comparing BOTH Gout & diabetes to those with ONLY diabetes Covariates after matching
Age Be f
ore Af t er
Mario D Hair Independent Statistics Consultant

20. Mario D Hair Independent Statistics Consultant
Univariate stats: Comparing BOTH Gout & diabetes to those with ONLY diabetes Lipids after matching
Mario D Hair Independent Statistics Consultant

21. Dealing with missing data 1
Missing data in non covariate data: Use the paired data format
Green is paired comparison, red is matched. There are no substantive changes
Mario D Hair Independent Statistics Consultant

22. Dealing with missing data 2
Missing data in covariates: Use multiple imputation
Separate creation of propensity scores from the matching
Run logistic regression on imputed datasets
Aggregate to get mean (median) propensity score
Use the aggregate file to do the matching
Load in the other variables
Use imputation again if missing data in non-covariates
Mario D Hair Independent Statistics Consultant

23. Mario D Hair Independent Statistics Consultant
References
Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46, doi: / One of the foremost authors on the subject.
Beal S J & Kupzyk K A, An Introduction to Propensity Scores What, When, and How. The Journal of Early Adolescence January 2014 vol. 34 no. 1 66-92 doi: / Easy to read introduction
Iacus, S. M., King, G., & Porro, G. (2009). CEM: Coarsened exact matching software. Journal of Statistical Software, 30, Reference for ‘relative multivariate imbalance test’
Mitra, R., & Reiter, J. P. (2012). A comparison of two methods of estimating propensity scores after multiple imputation. Statistical methods in medical research,
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, doi: /biomet/ Seminal paper.
Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of internal medicine, 127(8_Part_2), Example of stratification.
Thoemmes, F. (2012). Propensity score matching in SPSS. arXiv preprint arXiv: Explains use of ‘ps matching’.
Mario D Hair Independent Statistics Consultant

24. Propensity Score Matching in SPSS: How to turn an Audit into a RCT
Outline 
What is Propensity score matching?
Propensity Score Matching in SPSS
Example: Comparing patients with both Gout & diabetes to those with diabetes only
Dealing with missing data
Thank you: Questions?
Mario D Hair Independent Statistics Consultant