Copyright © 2015 Inter-American Development Bank. This work is licensed under a Creative Commons IGO 3.0 Attribution-Non Commercial-No Derivatives (CC-IGO BY-NC-ND 3.0 IGO) license ( and may be reproduced with attribution to the IDB and for any non-commercial purpose. No derivative work is allowed. Any dispute related to the use of the works of the IDB that cannot be settled amicably shall be submitted to arbitration pursuant to the UNCITRAL rules. The use of the IDB’s name for any purpose other than for attribution, and the use of IDB’s logo shall be subject to a separate written license agreement between the IDB and the user and is not authorized as part of this CC-IGO license. Note that link provided above includes additional terms and conditions of the license. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Inter-American Development Bank, its Board of Directors, or the countries they represent.

Matching Rosangela Bando Office of Strategic Planning and Development Effectiveness January 2012

Source: Gertler et al. (2011)

Motivation Many times when evaluating a program, we have no influence over program assignment. The evaluation problem How do we know what would have happened without the program? In other words,, is not observable. Matching proposes a possible solution: Identify a group of no beneficiaries with similar observable characteristics and assume that this is how the beneficiary group would have behaved in the absence of the program. 5

Conceptual Framework Notation: i individual variable of interest 1 if it receives the treatment, 0 otherwise x vector of characteristics Let: Variable of interest without treatment Variable of interest with treatment Main question of interest What is the impact of a hypothetical change in D over y keeping x constant? 6

Conceptual Framework Frequently used estimates of causal effects: Average Treatment Effect (ATE) Average Treatment Effect on the Treated (ATET) ATE=ATET+Selection Bias 7

Necessary Assumptions to Calculate Treatment Effects 8 Density Propensity Score High probability of participating given X Density of participating Density of not participating Matched area

Assumptions 1. Independence conditional on the mean: 2. Common support: Intuitively: 1. Selection for treatment is based only on observable variables 2. For each value of x there are units of treated and un-treated observations Assume that there are no general equilibrium effects (i.e. Treatment doesn’t affect those in the comparison group) 9

Assumptions to calculate treatment effects Matching is especially useful when: We can control for a wide number of x variables The parameter of interest is the ATET Definition Matching estimator 10

Propensity Scores We can generate a comparison group by selecting individuals with the same observable characteristics. Exact matching: x has discrete values and the sample contains many observations for each value of x Propensity Score Matching: if the data generating process justifies matching as a function of x, then the use of propensity score matching is also justified (Rosenbaum & Rubin (1983)) 11

General Formula to Calculate ATET Method Simple Closest neighbor Where the definition of the kernel can vary. For example, Epanechnikov (or quadratic): 12

Stratifying and Intervals The effect on the block b is defined as Where I (b) is the set of units in block b, are the treated individuals in block b and are the individuals not treated in block b. The effect based on stratification is defined as: 13

Radius Matching Given that Assuming the superposition principle, we have: 14

How to calculate treatment effects using propensity score matching 1. Use a model to determine the probability to participate. The most frequently used model is logit, followed by probit. The linear model is preferred by some as it tends to cluster less around 0 and 1 2. Use estimated values to generate the scoring p(x) for all members of the treatment and comparison groups 3. Match pairs Restrict the sample to the common support Determine a tolerance limit (closest neighbor, non-linear matching, multiple matching) Once the pairs are formed, estimate the impact by calculating the mean of the results of interest between participants and their non- participating pairs 15

Some Binomial Response Models 16

Errors The most common way to calculate errors is with bootstrap (ex. psmatch2 on STATA) The intervals diverge from the real values even in the most simple case (exact matching). The behavior of the variance using propensity score matching is still under investigation. Recommended reading: Abadie and Imbens, 2008 “On the Failure of the Bootstrap for Matching Estimators" (Econometrica) 17

Conclusions Matching only helps to control for observable differences, but not for unobservables Do not do a matching over variables that may change as a result of participating in the program Matching is best when combined with other techniques like Diff-in- Diff The impact evaluation may not be possible if the treatment and comparison groups are too different (no common support) 18

References Cameron, Colin A. and Pravin K. Trivedi, Microeconometrics : Methods and Applications, Cambridge University Press, May Abadie A. and Imbens G 2008 On the failure of the boostrap for matching estimators. Econometrica 76,