1. The 6th Conference on Survey Sampling in Economic and Social Research September 21-22, 2009 Katowice, Poland Criticalities in Applying the Neyman’s Optimality in Business Surveys: a Comparison of Selected Allocation Methods Paola M. Chiodini a,d, Rita Lima c, Giancarlo Manzi b,d, Bianca Maria Martelli c, *, Flavio Verrecchia d b.martelli@isae.it a. Department of Statistics, Università di Milano-Bicocca, Milan, Italy b. Department of Economics, Business and Statistics, Università degli Studi di Milano, Milan, Italy c. ISAE, Rome, Italy d. ESeC, Assago (MI), Italy

2. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 2 DISCUSS POSSIBLE MORE EFFICIENT SAMPLE DESIGNS FOR THE ISAE BUSINESS TENDENCY (BTS) SURVEY –BTS Economic features –BTS Statistical features –Operational bounds TO MEET EVERYBODY’S NEEDS WHILE STRENGHTENING OUTCOMES RELIABILITY (INDUSTRIAL CONFIDENCE) AIM OF THE PAPER

3. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 3 BTSECONOMIC FEATURES BTS ECONOMIC FEATURES Business Tendency Surveys investigate CONFIDENCE of economic agents CONFIDENCE can be defined as the (positive) attitude of economic agents toward both firms’ (internal) and country’s (external) variables –Corresponding Universe real value unknown To this purpose BTS collect information about a wide range of variables selected for their capability, when analysed together, to give an overall picture of industrial sector of the economy (OECD 2003)

4. assessments expectationsThe survey ask entrepreneurs and managers assessments on current trends and expectations for the near future regarding both their own business and the general situation of the economy qualitativeBusiness Tendency Survey thus collect qualitative information, mainly with a three options ordinal scale September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 4 BTSECONOMIC FEATURES BTS ECONOMIC FEATURES

5. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 5CONFIDENCE “balances”Answers obtained from the survey are quantified in form of “balances”, i.e. differences between positive and negative answers’ percentages The statistical series derived from business tendency surveys are particularly suitable for monitoring and forecasting business cycles confidenceThe aggregation of selected series (order book level, production expectations and stock) gives the confidence indicator leading capabilitiesConfidence indicators (and some single series too) often have leading capabilities and are widely used in the analysis of the economic cycle (recessions/expansions)

6. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 6 SHORT SURVEY HISTORY The manufacturing survey began 1959 on a quarterly basis and became monthly 1962 on a limited number of questions (purposive panel) During the years the survey was broadly modified to meet upcoming occurrences: –1986 the sample was updated in order to provide information also a regional level adopting a stratified (sector/region/size) partially random sample –1998 the Neyman’s optimal allocation of the reporting units to sample strata based on workforce variance was introduced (Cochran 1977) –2003 data processing was upgraded introducing a two-stage weighting system (sample weights and size weights) according to OECD (2003) able to assure a fully fledged comparability between local and national data

7. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 7 GDP and CONFIDENCE Confidence well fit the GDP shifts In recent times (since April 2009) positive signals from the survey (last available GDP figures Q II 2009: very negative)

8. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 8 EUROPEAN REFERENCE FRAME The Survey is part of the Joint Harmonised Business and Consumer Survey (BCS) program of the European Commission The project began 1962 and ISAE (formerly ISCO) was one of the founder member The principle of harmonisation underlying the project aims to produce a set of comparable data for all European countries (EC 2007) To achieve this goal institutes have to: –Use the same harmonised questionnaire –To strictly respect the Commission timetable in carrying on the survey and transmitting the results Institutes are relatively free to define any other aspects of the entire process (apart from a minimum sample size)

9. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 9 FRAME : ASIA archive of Italian active firms (last update 2006): + complete universe of firms – relatively late update BTS Statistical features: SAMPLE DESIGN QUESTIONNAIRE: fixed by Commission. Can only be integrated DATA COLLECTING MODE: CATI (Computer Aided Telephonic Interviewing), partly integrated with fax (foreseen some CAWI): Keep ASIA as FRAME MIXED MODE

10. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 10 OPERATIONAL CONSTRAINS –EC: SAMPLE SIZE4000recommended SAMPLE SIZE about 4000 units (firms/kind of activity units), bound to the country population size Very strict TIMING CONSTRAINTS : –MONTHLY FREQUENCY, –12 DAYS DATA COLLECTION –1 WEEK PROCESSING RESULTS LOCAL INFORMATION –NATIONAL: LOCAL INFORMATION Governmental priority Possible revenues PRESERVING “LOYAL” FIRMS –ISAE: PRESERVING “LOYAL” FIRMS: Research purposes of longitudinal analyses Conflicting with sampling theory (Panel rotation)

11. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 11 BTS STATISTICAL FEATURES As the total sample size is predetermined (about 4000 units), to increase precision is then mainly possible to work on: –Strata definition (partially predetermined and bound to economic and administrative settings) –Units’ allocation to Strata –Panel maintenance –Non response handling –Weighting

12. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 12 STRATA DEFINITION STRATA defined according to: ECONOMIC SECTORS –19, nearly EC requests, adapted to Italian economy AREAS (NUTS1) –4, administrative classification, widely different in size FIRMS’ SIZE (by workforce) –Small (10-49 ), Medium (50-249), Large (>=250). Distribution is right (positively) skewed because of the presence of few “large” establishments and many “small” units Minimum threshold of 10 employees –About 80% of total workforce

13. FIRMS BY STATA Nord OvestNord EstCentroSud e IsoleTotal 10-4950-249250 &+10-4950-249250 &+10-4950-249250 &+10-4950-249250 &+ 10-12. Manufacture of food, beverages and tobacco products149623356171527741108290121856175127045 13. Manufacture of textiles158434255575881010478242723234094 14. Manufacture of wearing apparel1472140231856151231230102912989666406 15. Manufacture of leather and related products31846189312512205314196255234278 16-17. Manufacture of wood and paper products12391541714041681586089107504434753 18. Printing and reproduction of recorded media966751074060550533131721.2733 19. Manufacture of coke and refined petroleum products3297165.19747874188 20-21. Manufacture of chemical and pharmaceutical products680304903281111422767322212712102 22. Manufacture of rubber and plastic products151129242939190155469354496154148 23. Manufacture of other non-metallic mineral products9381261812652264985711115120410414914 24. Manufacture of basic metals65220841275119121603561473441693 25. Manufacture of fabricated metal products, except machinery and equipment61996224145284382819491751119412191116162 26. Manufacture of computer, electronic and optical products71714528439971826057151132451918 27. Manufacture of electrical equipment1050206358081612936254151942742945 28. Manufacture of machinery and equipment n.e.c.324369282283059510473912185236039000 29-30. Manufacture of transport vehicles699183843969530313701323785202225 31. Manufacture of furniture9279541730242199319675276364647 32. Other manufacturing61982127191118506364185612289 33. Repair and installation of machinery and equipment14508169794626573148496234170 Total2579240356522243533054341430314901841178611999585710

14. TOTAL WORKFORCE BY STATA Nord OvestNord EstCentroSud e IsoleTotal 10-4950-249250 &+10-4950-249250 &+10-4950-249250 &+10-4950-249250 &+ 10-12. Manufacture of food, beverages and tobacco products278932343240923315252819132670189358486762132959160705503274208 13. Manufacture of textiles3273933020261121071079556052187566527155150622917905152306 14. Manufacture of wearing apparel25793131411692032975147871117022033859437342410077853805184838 15. Manufacture of leather and related products5503412438617289124604817368401099363411108946211071115532 16-17. Manufacture of wood and paper products2261415000130082584216333700215504768351241291243161876147213 18. Printing and reproduction of recorded media1748773784488133775689279486742984238352791549.72081 19. Manufacture of coke and refined petroleum products70713344508372510.46574622591360473341916152 20-21. Manufacture of chemical and pharmaceutical products1495532746587627149112428071434376673237841062811261184492 22. Manufacture of rubber and plastic products302412760128723190221809454431069183812428904963943247169315 23. Manufacture of other non-metallic mineral products178151226114026244142352528662157791067266122163791132218186734 24. Manufacture of basic metals1357721680467495770133078195326236375906298633521818130240 25. Manufacture of fabricated metal products, except machinery and equipment11244055834174548465639819120843446315080460835425193663622434851 26. Manufacture of computer, electronic and optical products1425214888300188887979391024951553913192211027444238119713 27. Manufacture of electrical equipment205422054431418167871603722000683758341559037592206 163760 28. Manufacture of machinery and equipment n.e.c.640916854245279572445817062515144391128477471002250961063405491 29-30. Manufacture of transport vehicles1432219757113600837698713084860877599108364722872133591268329 31. Manufacture of furniture1619691891547324352194773791691083483290985350704874137038 32. Other manufacturing1112081043685137141030713520899131371794319961232678509 33. Repair and installation of machinery and equipment24766735340531693238207231111825795974154745411143699639 Total487054395926501659427476321858273047259079135768139368215102108625754793340442

15. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 15 10 - 49 FIRMS POPULATION BY SIZE 50 - 249 250 - Total

16. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 16 SIMULATION

17. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 17 UNIT ALLOCATION TO STRATA: SIMULATIONS SETTINGS REFERENCE POPULATION: ASIA INDUSTRIAL SECTOR –85710 ENTERPRISES –3040422 PERSONS EMPLOYED 3 DIMENSIONS –AREAS (NUTS1) –ECONOMIC SECTORS –FIRMS’ SIZE 226 STRATA 500 REPLICATES SIMULATION TECHNIQUE: SEQUENTIAL UNIT SELECTION

18. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 18 UNITS ALLOCATION TO STRATA: ALTERNATIVE ALLOCATION METHODS UNIFORM (21 units per stratum) PROPORTIONAL (f h 4,4%) NEYMAN (x-optimal) ISAE (NEYMAN x-optimal on areas; winsorised 5%) AOSU(n 1h ): UNIFORM(n 1h ) + NEYMAN(n 2h ) –n 1h = 1, 2, …, 21 –n 2h = n h -n 1h –so that: n 1h = 0 then AOSU0 = NEYMAN n 1h = 21 then AOSU21 = UNIFORM APSU(n 1h ): UNIFORM(n 1h ) + PROPORTIONAL(n 2h )

19. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 19 UNIT ALLOCATION TO STRATA: SIMULATION METHOD START RANDOM UNIT SELECTION (SEQUENTIALY RANKED) REPLICATION Simulation DW If replicate < 500 If replicate = 500 Allocation Methods Neyman samples ISAE samples AOSU(n 1 ) samples … OUTPUT END OVERALL STATS DOMAIN STATS INFERENCE

20. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 20 Total workforce) Distribution of Replication (Total workforce) NeymanISAE AOSU3 AOSU9 UNIFORM PROP.

21. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 21 OVERALL POPULATION

22. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 22 Total workforce) REPLICATION BOX PLOT (Total workforce)

23. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 23 STATISTICS Bias = N  – N r  Total Error (TE) = |Bias| + N r  Relative Total Error (RTE) = TE / N r  Range = max(N r  ) - min(N r  ) Where: –  : Population mean – r  : Replication mean – r  : Replication STD –N : # Enterprises

24. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 24 UNIT ALLOCATION: Statistics |BIAS|STDTERANGE isae2832215822441124361 neyman1352133721472114837 aosu15202164822168128626 aosu39222225323176126329 aosu93452356823914129781 uniform1416095661096326455 apsu36370123260129630648787 proportional110931770731881661017149

25. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 25 Total workforce) REPLICATION BOUNDED BOX PLOT (Total workforce)

26. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 26 UNIT ALLOCATION: Statistics BOUNDED UNIT ALLOCATION: Statistics Bound: Max 50% allocation per strata Minimum 3 unit per strata |BIAS|STDTERANGE aosu315134644447957256410 aosu927244636249086269084 aosu24 (i.e uniform)8035968860491362685 apsu36462123244129706644813

27. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 27 DOMAIN ANALYSIS

28. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 28 STRATA COVERAGE AOSU, UNIF, PROP: 0 strata with 0% allocation NEYMAN: 12 strata with 0% allocation ISAE: 7 strata with 0% allocation

29. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 29 STRATA STATISTICS CV s = r  s / r  s Bias s =  s – r  s Total Error s (TE s ) = |Bias s | + r  s Relative Total Error s (RTE s ) = TE s / r  s = (|Bias s | / r  s ) + r CV s Where: –  s : Strata population mean – r  S : Strata replication mean – r  S : Strata replication STD

30. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 30 STRATA BOX PLOT: |Bias| by strata (|Bias s |)

31. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 31 STRATA BOX PLOT: CV of replication means by strata ( r CV s )

32. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 32 STRATA BOX PLOT: Relative Total Error by strata (RTE s )

33. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 33 UNIT ALLOCATION TO STRATA: Statistics RTE Max (|BIAS s | / r  s ) Max ( r CV s ) Max (RTE s ) isae0,00670,03150,56640,5979 neyman0,00640,03150,56640,5979 aosu10,00660,02440,42500,4491 aosu30,00690,02020,27750,2778 aosu90,00720,01410,15490,1624 uniform0,01830,02260,40420,4152 apsu30,03880,05821,00521,0094 proportional0,05630,10331,66451,6713

34. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 34 CONCLUDING REMARKS AND OPEN QUESTIONS Strata allocation: best proposal seem to be: Overall population: Neyman Domain analysis: Approach based on Neyman and strata representativeness constraints The AOSU(n 1 ) family ISAE They allow to strike a balance between theory and practical constraints

35. September, 21-22 2009, 6th Conference “Survey Sampling in Economic and Social Research “, Katowice, Poland 35 THANK YOU FOR YOUR ATTENTION!