1. Catherine Renne Insee 15 11 2012 Measuring sampling error in business surveys The case of the French monthly industry survey

2. 15 11 2012 Measuring sampling error in business surveys 2 Introduction Definitions Census : Exhaustive observation of all the elements of a population Sampling : Selection methods of the elements of a population Two types of sampling Probability sampling : each element of the population has a known non-zero probability of being selected. Stratified sampling, cluster sampling,… Non-probability sampling : elements are selected from the population in some non random manner Ex : quota sampling 17 MS 9 MS

3. 15 11 2012 Measuring sampling error in business surveys 3 Key issues of probability sampling 1. Selection of the sample Sampling method, selection process, sample size,… 2. How to aggregate the answers Calculation of the estimator In other words : how to draw conclusions from sampling data? Calculation of the random error In other words : how accurate is the aggregation procedure? Good sampling methods involve To have an exhaustive list of the units of the target population Sample frame issues To know exactly what we want to measure via the survey Variable of interest, notion of « true value »

4. 15 11 2012 Measuring sampling error in business surveys 4 Basic vocabulary of sampling Parameter It is a fonction of the individual unknown values Y i in the population of size N. This is the « true value » Example : {1,2,3,4) are the elements of the population. Y1=6000 ; Y2=12000 Y3=8000 ; Y4=6000 The estimator If n is the size of the sample, the formula that aggregates the n value y i is called the estimator of :

5. 15 11 2012 Measuring sampling error in business surveys 5 Sampling error The value of the estimator depends on the selected sample Bias If we could select all the possible samples, we would know the average value of the estimator The bias is the difference between this average and the true value Example : n=2. s1={1,2}, s2={1,3}, s3={1,4}, s4={2,3}, s5={2,4}, s6={3,4} The average of these six values is 8000. This is the « true value » The estimator is unbiased Variance 8 000 6 0007 0009 00010 000

6. 15 11 2012 Measuring sampling error in business surveys 6 Sampling error in short Low bias, low variance Low bias, high variance High bias, low variance High bias, high variance

7. 15 11 2012 Measuring sampling error in business surveys 7 The statistical law of the estimator

8. 15 11 2012 Measuring sampling error in business surveys 8 Sampling plan The estimator Principle of calculation Unweighted questions Weighted questions The French business industry survey

9. 15 11 2012 Measuring sampling error in business surveys 9 Sampling plan Sampling frame: the statistical business register Stratification criteria : activity (3 digits of the NACE) enterprise size 20-99 employees ; 100-499 employees ; 500 or more Number of units per strata proportional to the shares of total turnover Exhaustive strata : >500 employees or >150 million turnover

10. 15 11 2012 Measuring sampling error in business surveys 10 The estimator Two kind of questions At the enterprise level : Ex : Your outlook for total French Industry ? Likely changes in next 3 months At the activity level : Ex : How has your production developed over the past 3 months? It has… + increased = remained unchanged - decreased Answers are not weighted Answers are weighted

11. 15 11 2012 Measuring sampling error in business surveys 11 Principle of calculation The answers of enterprises are associated with an auxiliary variable Y : yi=1 if + increased yi=0 if = remained unchanged yi=-1 if - decreased Balance of opinion is the average of Y. Yi=1Yi=0Yi=-1

12. 15 11 2012 Measuring sampling error in business surveys 12 Unweighted questions

13. 15 11 2012 Measuring sampling error in business surveys 13 Weighted questions

14. Thank you for your attention ! Contact Ms. Catherine Renne Tél. : +33 1 41 17 60 56 Courriel : catherine.renne@insee.fr Insee 18 bd Adolphe-Pinard 75675 Paris Cedex 14 www.insee.fr Informations statistiques : www.insee.fr / Contacter lInsee 09 72 72 4000 (coût dun appel local) du lundi au vendredi de 9h00 à 17h00