Propensity Score Matching A Primer in R 1 David Zepeda Assistant Professor Supply Chain & Information Management Center for Health Policy and Healthcare Research Brown Bag Series April 1, 2015

Outline 1. Problem description 2. Theory 3. Two-Step Approach 4. Implementation in R 5. Example 1 – Hospitals 6. Example 2 – Primary Care Clinics 7. Example 3 – Farm Land 8. References 2

Problem 3

4

5 An observational unit is generally assigned only one of the two treatments. The treatment is not randomly assigned. Results in a number of potential problems regarding bias and model dependence.

Problem 6 Source: Ho, D. E., Imai, K., King, G. & Stuart, E.A Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15:

Theory 7

8

9

10

Two-Step Approach 11

What is R? A language and environment for statistical computing and graphics Provides a wide variety of statistical and graphical techniques Is highly extensible Provides an Open Source route to participation Great care has been taken over the defaults for the minor design choices in graphics User retains full control Available as Free Software! Allows users to add additional functionality Can be extended (easily) via packages. The R Project for Statistical Computing Implementation in R 12

M ATCH I T Package Dichotomous treatment variable Experimental and observational data Improving parametric statistical models Reduces model dependence Semi-parametric and non-parametric preprocessing Assess covariate distributions in the two groups (i.e., balance) Large range of matching methods Exact Subclassification Nearest neighbor Optimal Genetic Implementation in R 13

Exact matching Simplest version of matching Match each treated unit to all possible control units Exactly the same values on all the covariates Sufficient matches often cannot be found Subclassification Forms subclasses with “close” distributions of covariates Various subclassification schemes Can be used in conjunction with other matching methods Nearest neighbor matching Selects “best” control matches for each treated unit Chooses the control unit not yet matched closest to treated unit Implementation in R 14

Optimal matching Finds matched samples with smallest average absolute distance Helpful when there are not many appropriate control matches Genetic matching Uses a genetic search algorithm Optimal balance achieved after matching Performs statistical tests for determining balance Variety of options for matching methods Number of matched control units Matching with or without replacement Kernel matching Discard treated units, control units, or both Number of subclasses Distance measurement (i.e., logit) Implementation in R 15

Association between hospital system affiliation and hospital inventory in California hospitals (Zepeda, Nyaga, & Young, WP 2015) California hospital data from 2007 – observations (126 affiliated with smaller hospital systems) Preprocessing of data on affiliation with smaller hospital systems Genetic matching method 2 control observations with replacement for every treated observation 126 observations in treatment group 156 observations in control group Propensity score balancing improved by 95% Example 1 16

Example 1 17

Association between IT-leveraging capability and high quality diabetes care in Minnesota primary care clinics (Zepeda & Sinha, WP 2015) Minnesota primary care clinics in observations (135 with high IT-leveraging capability) Preprocessing of data on high IT-leveraging capability Optimal matching method 1 control observations without replacement for every treated observation 135 observations in treatment group 135 observations in control group Propensity score balancing improved by 76% Example 2 18

Example 2 19

Effect of easements on the selling price of farms in Minnesota (Taff & Weisberg, 2007) Federal Conservation Reserve Program (CRP) Temporary conservation easement by USDA (10-15 years) Annual payment by USDA for enrolled land Land valuation theory predicts that temporary easements should have no effect on value of properties Data Oct 1, 2002 – Sep 30, 2004 Farm properties with short-term conservation easements Farm properties with no conservation easements Covariates 2,937 property sales (271 were restricted by CRP contracts) Example 3 20

The primary objective Compare 271 sales with CRP restrictions to sales without Standard observational study approach Use all sales with no CRP as a comparison group Potential problem Properties sold without a random assignment Differences between observable sample and target population may be a cause for bias Using propensity score matching Mimic a randomized experiment Sample of non-CRP and CRP sales Closely agree on salient property characteristics (i.e., balance) Example 3 21

Example 3 22 Medians Upper 75% Lower 25% Dotted lines = 95%

Six models developed and tested Models 1 – 3: use all data, CRP and portion of land RESTRICTED Model 4: restricts data to sales with PRODUCTIVITY measure Model 5: matched sample on CRP restriction Model 6: matched sample with PRODUCTIVITY measure Consistency in results CRP contracts negatively associated with sales prices Most of CRP effect is captured by RESTRICTED amount Counter to land valuation theory Example 3 23

Example 3 24

The R Project for Statistical Computing M ATCH I T R Package Ho, D. E., Imai, K., King, G. & Stuart, E.A Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15: Examples Zepeda, D., Nyaga, G., & Young, G Supply Chain Risk Management and Hospital Inventory: Effects of System Affiliation. Working Paper. Zepeda, D. & Sinha, K. IT-Leveraging Capability for Reducing Health Care Disparities: An Empirical Analysis of Primary Care Operations. Working Paper. Taff, S.J. & Weisberg, S Compensated short-term conservation restrictions may reduce sales prices. The Appraisal Journal, Winter. References 25

Thank You! 26 David Zepeda Assistant Professor Supply Chain & Information Management