Comment on EU precision requirements EFS Evaluation Partnership Meeting Brussels 12.03.2015.

indicator values “are to be reported with a relative standard error (i indicator values “are to be reported with a relative standard error (i.e. the standard error divided by the estimate) not exceeding 2%" Reasons to use the „relative” precision requirement none (?) when all indicators are measured on the same scale Reasons not to use the „relative” precision requirement Required sample size unknown before the survey Huge samples may be required Unrealistically high response rate may be required Logical inconsistencies

Practical consequences – simulation What will be the required sample size? Scope Investment Priority „Active Inclusion…” Reporting for AIR 2019 All relevant common longer-term result indicators Assumptions Took real data on participant population and indicator values from a Polish 2007-2013 programme Stratified sample – proportional allocation + booster

Precision requirements Three versions tested EC proposal from Rome – relative std error ≤ 2% (margin of error ≤ 4% of the estimated ind. value margin of error < 4% of the population size) Standard error ≤ 2% of the population size (margin of error ≤ 4% of the population size) Standard error ≤ 5% of the population size (margin of error ≤ 10% of the population size)

Practical consequences Rel std err ≤ 2% Std err ≤ 2% Std err ≤ 5% Required sample size 142 485 7 822 1 386 Requirement met? probably yes Required response rate 90% 15% 3% Estimated cost €661 883 €55 899 €26 937

Thank you for your attention!