1. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 1 Overview of non- experimental approaches: Matching and Difference in Difference Estimators

2. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 2 Vgl. Hagen, Tobias und Bernd Fitzenberger (2004), Mikroökonometrische Methoden zur Ex-post-Evaluation, in: Hagen, Tobias und Alexander Spermann, Hartz-Gesetze – Methodische Ansätze zu einer Evaluierung, ZEW-Wirtschaftsanalysen, 74, S.45-72 1.Matching: Concept & Assumptions 2.Matching vs. Regression 3.Propensity Score Matching 4.Matching approaches 5.Matching and DiD

3. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 3 Cross Section DataPanel Data „Regression Discontinuity Design“ „Propensity Score Matching“ Instrumental Variables Approach (IV) Before After Estimators Difference in Difference Estimators (DiD) „ Propensity Score Matching“ + DiD Multivariate Separation Rate Models („Timing of Events“) Evaluation with non-experimental approaches

4. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 4 „selection on observables“ „selection on unobservables“ Methods of Regression „Propensity- Score- Matching“ Difference in Difference Estimators (DiD) Selection Models Instrumental Variable Approaches (IV) Evaluation with non-experimental approaches

5. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 5 Matching Concept: A participant‘s outcome variable is compared to the outcome variable of one or several non participants as similar as possible to the participant.

6. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 6 Assumptions: A1: Only observable factors influence participation and outcome variable simultaneously (Counter-example: motivation)  CMI / CIA A2: „common support“ is given, i.e.

7. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 7 Conditional Independence Assumption (CIA): Conditional on x, C and (y 0,y 1 ) are independent. Where x is the vector of all observed variables

8. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 8 Conditional Mean Independence Assumption (CMI): and CIA is more restrictive than CMI. CMI is sufficient to identify ATT.

9. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 9 Equation can also be stated as ATT can be estimated consistently  Exact matching: for each participant find (at least) one non participant equal in x

10. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 10 Note: A linear regression model following the equation is also conditioned on observable variables which do have an influence on participation and the outcome variable.

11. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 11 Disadvantage compared to Matching: Assumption of a linear functional relation Projection into areas with no observations More restrictive assumptions with regard to disturbances:

12. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 12 Problem: In case vector x is large (many variables), it is unlikely to find a similar non participant for every participant with regard to all characteristics. Solution: Matching of participants and non participants based on their estimated propensity scores: p(x)≡P(C=1|x)

13. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 13 Rosenbaum und Rubin (1983)* show that if CIA holds then: Hence, matching of participants and non participants based on propensity scores is sufficient. * Rosenbaum, P.R. und D.B. Rubin (1983), The Central Role of the Propensity Score in Observational Studies for Causal Effects, Biometrika 70 (1), S.1-55

14. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 14 Advantages: Vector x is reduced to a one- dimensional probability p(x) Easier to find „good matches“ „Common Support“ 0<p(C=1|x)<1  No comparison of „incomparable“ individuals Necessary Assumption for Matching:

15. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 15 Individuals with equal characteristics of variable x should show a positive probability to be participants as well as a positive probability to be non participants. Status of participation cannot be predicted using x.

16. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 16 Procedure Estimation of a model for participation C conditioned on explanatory vector x (e.g.: Probit) Prediction of individual propensity scores based on the estimation Estimation of ATT as:

17. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 17 A weighted average of all non participants‘ outcome variables is subtracted from every participants outcome variable. Where: N 1 Number of participants N 0 Number of non participants iIndex of participants jIndex of non participants w ij Weights

18. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 18 The smaller the difference in estimated propensity scores |p i -p j | or equivalently the more similar participant i and non participant j are, the higher the weights Weights of different Matching approaches differ:  „Nearest-Neighbour-Matching“  „Caliper Matching“  „Kernel-based Matching“

19. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 19 „Nearest-Neighbour-Matching“ There is only one member of the control group per participant: The non participant minimizing |p i -p j | This „next neighbour‘s“ weight is set to 1

20. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 20 „Caliper Matching“ Participant i is only included in the ATT calculation in case a next neighbour within a priori specified range |p i -p j | can be found  Modification of „Nearest-Neighbour- Matching“  Reduces the probability of „bad matches“

21. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 21 „Kernel based Matching“ Use of several or all non participants as control group for every participant Weight for non participant j is a negative function of |p i -p j |.

22. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 22  All available information is included  Reduction of the estimation‘s variance  Danger of higher selectivity bias as a participant can be assigned to unsimilar non participants

23. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 23 Matching Problem: Only observed factors are permitted to simultaneously influence participation of the treatment and outcome variable. Solution: Combination of Matching and Difference in Difference approach Condition: Availability of panel data

24. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 24 Reminder: We now estimate:

25. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 25 Consideration of the change of participants‘ outcome variable over time that exceeds the change of control groups‘ outcome variable change Not a comparison of sample means but a comparison of each participant with one or more non participants very similar to him

26. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 26 Example: Productivity effects of personnel policy measures based on enterprise data  Calculation of the measure‘s mean productivity effect on enterprises executing the measure (ATT)

27. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 27 1999: Period of time before the measure (t) 2000: Implementation 2001: Period of time after the measure (t‘)

28. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 28 Procedure 1.Estimation of the „propensity score“ for personnel policy measures taken in 2000 2.Determination of enterprises for comparison with a kernel based matching approach 3.Calculation of ATT

29. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 29 Results:  ATT equals (95.401-94.066)-(102.614-97.802)=- 3.837€. i.e. the personnel policy measure leads to a negative productivity effect ParticipantsControl GroupDifference y 1999 94.06697.802-3.736 Y 2001 95.041102.614-7.573 Difference9754.812-3.837

30. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 30 There is a difference of 3.736€ between the outcome variable of program and control group already before the measure‘s implementation  Interpretation as selectivity bias due to unobserved variables  Selectivity bias can be eliminated by calculating Before After Differences in case the bias is time constant  Another possible reason for negative ATT estimates: Anticipation effects

31. Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 31  Example illustrates simplicity and potential weakness of this approach