1. Methods of Economic Investigation Lecture 12
Matching Methods - 2
Methods of Economic Investigation
Lecture 12

2. Last Time
Ways to define a ‘control group’ if you don’t have an experiment
Difference-in-Differences
Assume: Fixed Differences over time
Attribute any change in trend to treatment
Propensity Score Matching
Assume: Treatment, conditional on observables, is as if randomly assigned
Attribute any difference in outcomes to treatment

3. Choices when doing p-score matching
Sample with or without replacement
One-to-one or one-to-many matching
How many observations to use for a match
What criteria to just how close is “close enough”

4. How close is close enough?
No “right” answer in these choices—will depend heavily on sample issues
How deep is the common support (i.e. are there lots of people in both control and treatment group at all the p-score values
Should all be the same asymptotically but in finite samples (which is everything) may differ

5. Tradeoffs in different methods
Source: Caliendo and Kopeinig, 2005

6. How to estimate a p-score
Typically use a logit
Specific, useful functional form for estimating “discrete choice” models
You haven’t learned these yet but you will
For now, think of running a regular OLS regression where the outcome is 1 if you got the treatment and zero if you didn’t
Take the E[T | X] and that’s your propensity score

7. The Treatment Effect
CIA holds and sufficient region of of common support
Difference in outcome between treated individual i and weighted comparison group J, with weight generated by the p-score distribution in the common support region
J is comparison group with |J| is the number of comparison group units matched to i
N is the treatment group and |N| is the size of the treatment group

8. General Procedure Run Regression:
Dependent variable: T=1, if participate; T = 0, otherwise.
Choose appropriate conditioning variables, X
Obtain propensity score: predicted probability (p)
1-to-1 match
estimate difference in outcomes for each pair
Take average difference as treatment effect
1-to-n Match
Nearest neighbor matching
Caliper matching
Nonparametric/kernel matching
Multivariate analysis based on new sample

9. Standard Errors
Problem: Estimated variance of treatment effect should include additional variance from estimating p
Typically people “bootstrap” which is a non-parametric form of estimating your coefficients over and over until you get a distribution of those coefficients—use the variance from that
Will do this in a few weeks

10. Some concerns about Matching
Data intensive in propensity score estimation
May reduce dimensionality of treatment effect estimation but still need enough of a sample to estimate propensity score over common support
Need LOTS of X’s for this to be believable
Inflexible in how p-score is related to treatment
Worry about heterogeneity
Bias terms much more difficult to sign (non-linear p-score bias)

11. Matching + Diff-in-Diff
Worry that unobservables causing selection because matching on X not sufficient
Can combine this with difference and difference estimates
Take control group J for each individual i
Estimate difference before treatment
If the groups are truly ‘as if’ random should be zero
If it’s not zero: can assume fixed differences over time and take before after difference in treatment and control groups

12. Bottom Line… Matching Methods used to replicate experimental methods
Need to believe independence, conditional on X’s
If matching assumption is right, can estimate the TOT without worrying about selection bias