1. Supporting material for CIE
Video tutorials
standard introductory e-learning tools
to learn about CIE: outsourcing, application, use of results, choice of data
to add subtitles
to share them with the consultants

2. Tips and tricks on how to plan intervention design
in view of future counterfactual impact evaluations
European Commission Centre for Research on Impact Evaluation

3. A What’s inside the magic box? Tricks
Some practical precautions that can make CIE easer to be carried on.
Tips
Hints on the most suitable CIE method to be applied in each situation.
Typical situation
For each trick, the situation in which it best applies.

4. Main message of the videoB
Always keep in mind the importance of creating a proper comparison group!

5. Example 1

6. Example 1
Launch of “National Supported Work Demonstration” by Manpower Demonstration Research Corporation.
Aim of the intervention: to offer job training programs to particular worker categories in need.
To participate, people need to apply.

7. Example 1 Treated group: random selection among applicants.
Control group: not selected applicants.
Intervention: 9 to 18 months of payed supported work.
To disentangle the average treatment effect (ATE) : compare average employment rate of the treated and non-treated group 6 months after the end of the training.

8. Example 1
Trick
Randomise treatment: randomisation creates two similar groups, except for one of them being treated.
Tip
The average treatment effect is given by the difference between the average employment rate of the treated and non treated.
Typical situation
In case you have more applications than available slots, allocate them trough randomisation.

9. Example 3

10. Example 3
As in example 1, the intervention consists in offering job training programs.
Priority is given to poorer applicants.
Participation (treatment) will be decided upon a threshold calculated with respect to a yearly income indicator.

11. Example 3
Treated group: all individuals with income indicator below the threshold level of 250 €.
Control group: all individuals with income indicator above the threshold level (excluded from the intervention).
The effect of the intervention will be computed by comparing the average employment rate of treated people just below the income threshold level with that of untreated workers just above the same threshold.

12. Example 3
Trick
Allocate treatment on the basis of a score (forcing variable) with a threshold.
The threshold must fall in the middle of the distribution of the forcing variable: ensure having both treated units and controls!
Need to collect data both for the treated and the controls.
The RDD works as a local randomisation around the threshold.

13. Example 3 Tips Regression Discontinuity Design (RDD).
Typical situation
Treatment assignment is a function of some variable (income, age).

14. Example 2
Example 4
Example 5
Example 6
Example 7

15. Summing-up
Take-home message Use foreseen (and unforeseen) characteristics of the policy intervention to create an appropriate comparison group!

16. Summing-up
Several typical situations imply specific class of CIE methods:

17. Summing-up Not all methods are applicable in a single intervention.
Different methods estimate different kinds of impact:
RDD estimates the impact for individuals at the threshold
PSM estimates ATT
DiD estimates ATE
IV estimates LATE
 Choose the most appropriate impact for your intervention

18. Summing-up
When designing the intervention, pave the way for the application of 2 or 3 methods
Save money
Have reliable results
CIE methods can be combined and refined to fit special situations and purposes  ask to a CIE expert ( market)!

19. Stay in touch
CRIE website: crie.jrc.ec.europa.eu YouTube: JRC CRIE Yammer: