Methods of Economic Investigation Lecture 12 Matching Methods - 2 Methods of Economic Investigation Lecture 12

Last Time Ways to define a ‘control group’ if you don’t have an experiment Difference-in-Differences Assume: Fixed Differences over time Attribute any change in trend to treatment Propensity Score Matching Assume: Treatment, conditional on observables, is as if randomly assigned Attribute any difference in outcomes to treatment

Choices when doing p-score matching Sample with or without replacement One-to-one or one-to-many matching How many observations to use for a match What criteria to just how close is “close enough”

How close is close enough? No “right” answer in these choices—will depend heavily on sample issues How deep is the common support (i.e. are there lots of people in both control and treatment group at all the p-score values Should all be the same asymptotically but in finite samples (which is everything) may differ

Tradeoffs in different methods Source: Caliendo and Kopeinig, 2005

How to estimate a p-score Typically use a logit Specific, useful functional form for estimating “discrete choice” models You haven’t learned these yet but you will For now, think of running a regular OLS regression where the outcome is 1 if you got the treatment and zero if you didn’t Take the E[T | X] and that’s your propensity score

The Treatment Effect CIA holds and sufficient region of of common support Difference in outcome between treated individual i and weighted comparison group J, with weight generated by the p-score distribution in the common support region J is comparison group with |J| is the number of comparison group units matched to i N is the treatment group and |N| is the size of the treatment group

General Procedure Run Regression: Dependent variable: T=1, if participate; T = 0, otherwise. Choose appropriate conditioning variables, X Obtain propensity score: predicted probability (p) 1-to-1 match estimate difference in outcomes for each pair Take average difference as treatment effect 1-to-n Match Nearest neighbor matching Caliper matching Nonparametric/kernel matching Multivariate analysis based on new sample

Standard Errors Problem: Estimated variance of treatment effect should include additional variance from estimating p Typically people “bootstrap” which is a non-parametric form of estimating your coefficients over and over until you get a distribution of those coefficients—use the variance from that Will do this in a few weeks

Some concerns about Matching Data intensive in propensity score estimation May reduce dimensionality of treatment effect estimation but still need enough of a sample to estimate propensity score over common support Need LOTS of X’s for this to be believable Inflexible in how p-score is related to treatment Worry about heterogeneity Bias terms much more difficult to sign (non-linear p-score bias)

Matching + Diff-in-Diff Worry that unobservables causing selection because matching on X not sufficient Can combine this with difference and difference estimates Take control group J for each individual i Estimate difference before treatment If the groups are truly ‘as if’ random should be zero If it’s not zero: can assume fixed differences over time and take before after difference in treatment and control groups

Bottom Line… Matching Methods used to replicate experimental methods Need to believe independence, conditional on X’s If matching assumption is right, can estimate the TOT without worrying about selection bias