1. ESF programme targets in Social Inclusion
European Commission
14 March 2013

2. 1. Aspects of target setting
Indicators: output, result -> SMART
Type of target
Absolute numbers, percentages (for result only)
Baseline setting for result targets (not output
Adjusting targets
(Des)aggregation

3. 1. Aspects of target setting
Data
Challenge: Different definitions (e.g. disabled)
Data availability
Characteristics of target group are sensitive data, not easy to link to administrative data
Surveys: participants difficult to trace, non-response
EU-SILC (successor to ECHP)
Risk of poverty (household income below 60% of national median income)
Material deprivation
Living in a workless household
No participation ALMP, only training (not necessary ESF funded)

4. 2. Methods (general = for output and result)
Not many examples, also not Worldbank UN
In practice: combination of (simple) methods
For each method:
Pros and cons
Type of data needed
Relevance for Social Inclusion
Qualitative and Quantitative methods

5. 2. Methods (general, qualitative)
Expert opinions, focus group, Delphi method (different rounds, collective intelligence) +
No data required
Quick and relatively low costs
Support for acceptance, experts/public
Important for checking other methods -
Intransparant
Undesirable influenced during process (dominant, hidden agenda)
Less exact, reliable

6. 2. Methods (general, qualitative)
Reference values
Other studies, literature search +
Quick and simple
Can be sound from other (good quality) studies -
Comparability
Characteristics of target group
Economic and labour market situation
Legal and institutional setting

7. 2. Methods (general, quantitative)
Trend analysis
Different functions
Straight trend: C(t) = a + b*t
Other forms: exponential, parabolic, logistic +
Easy
Only indicator data needed -
Mechanic, no explanation for changes

8. 2. Methods (general, quantitative)
Economic optimum
Needed:
- output / result indicator by subsets
- Restrictions for allocation +
Makes optimal use of differences -
Data on differences in costs/result needed

9. 2. Methods (general, quantitative)
Shift-share analysis
Needed: historical values of indicator, shares of subsets (e.g. area)
Changes in shares and indicator values +
Shows analytical path
No microdata needed -
No causality included
Underlying target setting required

10. 2. Methods (general, quantitative)
Shift-share analysis example
Total change: -9, of which:
Indicator change: -17
Changing shares: +4
Cross product: +4

11. 3. Methods for result targets
target group
programme
result
selection effects
non-programme factors

12. 3. Methods for result targets
Selection effects
Creaming, cherry picking, however ‘deadweight loss’ for disadvantaged groups smaller
But not always undesired
Solutions
Adapt definition of indicator (not possible for common result indicators, for subsets (incorporate)
Do nothing if selection desirable
Adjust target levels

13. 3. Methods for result targets
Non-programme factors
Business cycle
Incorporate forecast in targets
Adjusting target method in cases of unforeseen changes (economic crisis)

14. 3. Methods for result targets
Macroeconomic time series
Trends, non-programme factors +
Changes in target values explained
Target adjusting easy -
Medium complex
Enough time observations

15. 3. Methods for result targets
Panel data
Time series and cross section (Member States, regions) +
Changes in target values explained
Data available -
Pooling too restrictive (not the same causal relationships in different MS regions)

16. 3. Methods for result targets
Example Ecorys
Panel data model for adjusting result targets
EMPL (MS,t) = b0 + b1 X + b2 GDP (MS,t)
1% change GDP – 1% change EMPL, not very robust
However social inclusion: disadvantaged groups
Method can be used (not done by Ecorys)
Alternative: microdata on EUSILC

17. 3. Methods for result targets
Microeconometric approach
All aspects are covered, results are modeled at the individual level
Usually quasi-experimental approach as other approaches fail
Social experiment
Discontinuity approach
Matching +
All relevant effects, including effectiveness of intervention -
Data availability
Complex and black box

18. 3. Methods for result targets
Microeconometric for adjusting targets
Examples Bartik et al (2009) and Ecorys (2011)
Bartik (t=quarterly data)
Y (i,area,t)=B0 + B1 * (i,area,t) + B2 * D (area,t)
Y=indicator value (employment, earnings, attainment of degree, literacy)
D=local unemployment rate
B2=elasticity
New target = old target + B2 * change D(area,t)

19. 4. Statistical software
Excel:
Trend analysis
Economic optimum
Shift-share analysis
Unit-cost
Simple regression
Econometric modelling:
SAS, Stata, E-views
Not SPSS

20. 5. Methods applied to SI
Available methods <-> available data for SI
Many methods but not many applied for target setting, in particular SI
Data availability for SI targets challenging
No complete data overview for Member States
Output indicators usually less complicated
Easiest for result indicators: employment of disadvantaged groups (disabled, migrants, low educated, double definitions)
Less easy for ‘other’ disadvantaged groups, other areas (poverty, homelessness)

21. 5. Methods applied to SI
Output indicators
Unit cost from historical data -> yes
Correction for inflation (base year)
Differences in characteristics population? -> correction: shift-share -> needed number of participants per characteristic and costs
Additional: trend (other than inflation)
Benchmarking methods to raise target levels at lower level
Unit cost from historical data -> no
Other reference values indicator -> yes
Correction if possible for differences
Other reference value indicator -> no
Qualitative methods

22. 5. Methods applied to SI
Result indicators
Historical data available-> yes
Creaming -> Targets for subgroups
Microdata available -> micro-econometric approach, if not
Data available from other areas, Member States -> macroeconomic panel data approach, if not
Data on labourmarket /business cycle: timeseries modeling, if not
Trend analysis

23. 5. Methods applied to SI
Result indicators
Historical data available-> no
Other reference values indicator -> yes
Correction if possible for differences
Other reference value indicator -> no
Qualitative methods
Adjusting targets
Macro- or microeconometric model with business cycle -> apply coefficients from model
Otherwise use elasticities from other studies (if transferable)

24. Use several methods, including qualitative (check at end of process)
Target is not forecast, can be challenging
Try to be as transparent as possible
Missing data can create data agenda for future
Use target levels within your own organisation