ESF programme targets in Social Inclusion European Commission 14 March 2013 wim.zwinkels@epsilon-research.nl

1. Aspects of target setting Indicators: output, result -> SMART Type of target Absolute numbers, percentages (for result only) Baseline setting for result targets (not output Adjusting targets (Des)aggregation

1. Aspects of target setting Data Challenge: Different definitions (e.g. disabled) Data availability Characteristics of target group are sensitive data, not easy to link to administrative data Surveys: participants difficult to trace, non-response EU-SILC (successor to ECHP) Risk of poverty (household income below 60% of national median income) Material deprivation Living in a workless household No participation ALMP, only training (not necessary ESF funded)

2. Methods (general = for output and result) Not many examples, also not Worldbank UN In practice: combination of (simple) methods For each method: Pros and cons Type of data needed Relevance for Social Inclusion Qualitative and Quantitative methods

2. Methods (general, qualitative) Expert opinions, focus group, Delphi method (different rounds, collective intelligence) + No data required Quick and relatively low costs Support for acceptance, experts/public Important for checking other methods - Intransparant Undesirable influenced during process (dominant, hidden agenda) Less exact, reliable

2. Methods (general, qualitative) Reference values Other studies, literature search + Quick and simple Can be sound from other (good quality) studies - Comparability Characteristics of target group Economic and labour market situation Legal and institutional setting

2. Methods (general, quantitative) Trend analysis Different functions Straight trend: C(t) = a + b*t Other forms: exponential, parabolic, logistic + Easy Only indicator data needed - Mechanic, no explanation for changes

2. Methods (general, quantitative) Economic optimum Needed: - output / result indicator by subsets - Restrictions for allocation + Makes optimal use of differences - Data on differences in costs/result needed

2. Methods (general, quantitative) Shift-share analysis Needed: historical values of indicator, shares of subsets (e.g. area) Changes in shares and indicator values + Shows analytical path No microdata needed - No causality included Underlying target setting required

2. Methods (general, quantitative) Shift-share analysis example Total change: -9, of which: Indicator change: -17 Changing shares: +4 Cross product: +4

3. Methods for result targets target group programme result selection effects non-programme factors

3. Methods for result targets Selection effects Creaming, cherry picking, however ‘deadweight loss’ for disadvantaged groups smaller But not always undesired Solutions Adapt definition of indicator (not possible for common result indicators, for subsets (incorporate) Do nothing if selection desirable Adjust target levels

3. Methods for result targets Non-programme factors Business cycle Incorporate forecast in targets Adjusting target method in cases of unforeseen changes (economic crisis)

3. Methods for result targets Macroeconomic time series Trends, non-programme factors + Changes in target values explained Target adjusting easy - Medium complex Enough time observations

3. Methods for result targets Panel data Time series and cross section (Member States, regions) + Changes in target values explained Data available - Pooling too restrictive (not the same causal relationships in different MS regions)

3. Methods for result targets Example Ecorys Panel data model for adjusting result targets EMPL (MS,t) = b0 + b1 X + b2 GDP (MS,t) 1% change GDP – 1% change EMPL, not very robust However social inclusion: disadvantaged groups Method can be used (not done by Ecorys) Alternative: microdata on EUSILC

3. Methods for result targets Microeconometric approach All aspects are covered, results are modeled at the individual level Usually quasi-experimental approach as other approaches fail Social experiment Discontinuity approach Matching + All relevant effects, including effectiveness of intervention - Data availability Complex and black box

3. Methods for result targets Microeconometric for adjusting targets Examples Bartik et al (2009) and Ecorys (2011) Bartik (t=quarterly data) Y (i,area,t)=B0 + B1 * (i,area,t) + B2 * D (area,t) Y=indicator value (employment, earnings, attainment of degree, literacy) D=local unemployment rate B2=elasticity New target = old target + B2 * change D(area,t)

4. Statistical software Excel: Trend analysis Economic optimum Shift-share analysis Unit-cost Simple regression Econometric modelling: SAS, Stata, E-views Not SPSS

5. Methods applied to SI Available methods <-> available data for SI Many methods but not many applied for target setting, in particular SI Data availability for SI targets challenging No complete data overview for Member States Output indicators usually less complicated Easiest for result indicators: employment of disadvantaged groups (disabled, migrants, low educated, double definitions) Less easy for ‘other’ disadvantaged groups, other areas (poverty, homelessness)

5. Methods applied to SI Output indicators Unit cost from historical data -> yes Correction for inflation (base year) Differences in characteristics population? -> correction: shift-share -> needed number of participants per characteristic and costs Additional: trend (other than inflation) Benchmarking methods to raise target levels at lower level Unit cost from historical data -> no Other reference values indicator -> yes Correction if possible for differences Other reference value indicator -> no Qualitative methods

5. Methods applied to SI Result indicators Historical data available-> yes Creaming -> Targets for subgroups Microdata available -> micro-econometric approach, if not Data available from other areas, Member States -> macroeconomic panel data approach, if not Data on labourmarket /business cycle: timeseries modeling, if not Trend analysis

5. Methods applied to SI Result indicators Historical data available-> no Other reference values indicator -> yes Correction if possible for differences Other reference value indicator -> no Qualitative methods Adjusting targets Macro- or microeconometric model with business cycle -> apply coefficients from model Otherwise use elasticities from other studies (if transferable)

Use several methods, including qualitative (check at end of process) Target is not forecast, can be challenging Try to be as transparent as possible Missing data can create data agenda for future Use target levels within your own organisation