1. Yannick LHORTY TEPP, FR CNRS 3435 Evaluation of Public Policies in France : At this point, where are we ? ASIRPA Conference June 13th 2012

2. Very practical views on impact evaluations in France, based on three case studies of main recent reforms i) the reduced Social Security Contributions ii) the RSA reform, in its experimental phase and its generalization phase, iii) the hundreds of social innovations financed by the Fund of experimentation for youth since 2009. Where are we now ? Motivations 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions

3. The parabole of perfect world 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions Large spread of new experimental technologies in France, since 2007 Moving towards a complete set of evaluations tools, from ex ante modeling and simulations to expost experimental protocoles Strong supports from the demand side, with many calls for tenders Appel à projet dexpérimentation sociale (2007 – 6 M) Fond national dexpérimentations sociales pour la jeunesse (2009 – 150 M ; 2011 – 240 M) Many new suppliers : J-PAL, PSE, CREST, TEPP, LIEPP-Science Po Very good fit between these new technologies and new public local needs Fully realistic ?

4. Case Study 1 : Reductions of Social Security Contributions 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions The device

5. Case Study 1 : Reductions of Social Security Contributions 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions Reforms have been always mixed with something else… Working time reductions in 1993, and 1998-2002 Minimum wage increases in 1995, 1998-2002 and 2003. => An identification problem Reforms are general, all French firms have been concerned => No natural counterfactual, no experimentation Targeted on low wages => Heterogeneity among fims and between workers Unstable over time Not enough time between two shocks Differences between exante and expost evaluations

6. Case Study 1 : Reductions of Social Security Contributions 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions Conclusion 1 The norm among Public policies Is that they are unevaluable policies according to the academic standards Other examples : Principaux Investissements davenir, Pôles de compétitivité, Loi TEPA, Golden rule for fiscal policies, etc.

7. Case Study 2 : RSA Reform 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions The device Gross income Net income RMI 62 %

8. Case Study 2 : RSA Reform 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions Conclusion 2 Never forget that policymaker s preferences are not fully compatible with a transparent, honest, independant and external evaluation Experimentation of RSA (before June 2009) 51 territories among 34 regions (French departments) Presidents of départments refused the randomization principle : They prefered to choose themselfves test territories Generalization of RSA(after 2009) Government decided to generalise the RSA before the end of experimentation => lack of observations to get precise conclusions

9. Case Study 3 : New programs for Youth 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions Since 2009 : several calls for tenders from the Experimentation for Youth Funds More than 450 projects have been financed with a couple experimenter/evaluator each time Two interesting facts : 1. Reports from the Scientific council of the Funds shows that more than 95 % of all evaluations are not randomized experiments, even statistical evaluation, but qualitative evaluations 2. They also show that academics dominate quantitative assesment but private firms dominate qualitative evaluations Conclusion 3 There is a huge gap between demand for impact evaluation studies and capacity of suppliers

10. Conclusions 1. Les métiers de demain 2. Le canal à mobiliser 3. Comment sont triés les CV? 4. Quest ce que la discrimination? 5. La discrimination existe mais… 1. Introduction 2. Géographie des problèmes 3. Géographie des solutions 4. Conclusions 1. Introduction 2. The perfect world 3. The real world : Three case studies 4. Conclusions The perfect world doesnt exist The spread of new experimental technologies in France doesnt mean these technologies are omnipotent and that all public policy are evaluable with existing technics Demand for evaluations, reflected by many calls for tenders, doesnt mean demand for external and independant evaluation Many new suppliers, but still very small. Impact Evaluation is an emerging field of research in France