1 Sharing best practices for the redesign of three business surveys Charles Tardif, Business Survey Methods Division,Statistics Canada presented at the ICES-III, session 75 June 21 st, 2007

2 Alternative title Can we design a monthly Unified Enterprise Survey?

3 Outline What are monthly business surveys? – a simplified view Common objectives of monthly business surveys Statistics Canada (STC) monthly business surveys: Monthly Survey of Manufacturers (MSM) Monthly Survey of the Food Services and Drinking Places (MFS) Monthly Wholesale and Retail Trade Survey (MWRTS)

4 Outline (contd) Commonalities between the surveys Comparing survey elements Summary – A monthly unified enterprise survey? Conclusion

5 What are monthly business surveys? – a simplified view Production of monthly estimates mainly on financial information (sales, revenues, expenses, etc): By geographic level By industrial level, classified by the North American Industrial Classification System (NAICS) With a targetted quality, expressed in terms of Coefficients of Variation (CV).

6 Common objectives of monthly business surveys Measure trends and levels for key financial variables Establish best possible survey design to meet the surveys objectives Meet the 6 elements of the STC Quality Framework: Relevance, accuracy, timeliness, accessibility, interpretability, coherence Maximise use of administrative data, mainly tax data, to reduce response burden

7 STC monthly business surveys MSM Manufacturers MFS Restaurants MWRTS Retail MWRTS Wholesale Variables of interest Shipments, inventories and orders Sales, Number of locations Sales, Number of locations Sales, Inventories Geo levelProvinces, territories Industrial level (NAICS) 311 to 339, at the 3rd to 6th digit 722, at the 4th digit 44 and 45, for 19 trade groups (TG) 41, for 15 trade groups (TG)

8 STC monthly business surveys (contd) MSM Manufacturers MFS Restaurants MWRTS Retail MWRTS Wholesale Population size (establishments or clusters of est.) 100,00090,000180,000100,000 Collection units11,0002,10012,0008,000 Number of domains More than 1, possible domains (19 TGs) 195 possible domains (15 TGs) Yearly revenues (in billions)

9 Commonalities between the surveys All surveys: share common objectives undertook or completed a redesign or a restratification in the last 2 years are facing pressure to make a more extensive use of Tax data have skewed populations have an annual counterpart, all integrated in the Unified Enterprise Survey (UES) are managed by different subject matter divisions, although centralized methodological support.

10 Commonalities between the surveys One monthly unified survey? Well see what has been done so far to harmonize the different surveys.

11 Comparing survey elements MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail Frame Before BR, no exclusions Business Register (BR), with the exclusions of the non-employing establishments Now BR, no exclusions, using a Survey specific Universe Frame Sampling Unit Before Establishment level Company level Now Establishment level or cluster of establishments

12 Comparing survey elements MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail Stratification variables BeforeProvince/territory, NAICS and a size measure: Gross Business Income (GBI) from the BR NowProvince/territory, NAICS and a size measure: annualized monthly data, data from annual survey, tax data, GBI Take-none stratum BeforeBottom 2% by province No take-none stratum 5% by geo and TGs NowBottom 10% (MSM, MFS) or 5% (MWRTS) by province and stratification NAICS, subject to a cap.

13 Comparing survey elements Stratification – Now: Differences between surveys, but all: Differences between surveys, but all: are using the Lavallée-Hidiroglou algorithm (efficient with skewed population) to stratify their population with the following characteristics: 1 Take-all / must-take stratum by prov/NAICS 1 or a few take-some strata by prov/NAICS Minimum sample size per stratum Capped design weight Oversampling for out-of-scopes, deaths and non- response

14 Comparing survey elements Use of GST - Now: All surveys are making use of GST data, although in a different way: MSM and MWRTS: Micro approach, with selected units replaced by GST data through modeling. Units have a design weight > 1. Use of GST data varies by NAICS for MSM Standard stratification design explained in the previous slide is used

15 Comparing survey elements GST micro approach, selected units Take- somes 1 Take- somes 2 Take-nones Take-alls S1S2

16 Comparing survey elements Use of GST - Now: All surveys are making use of GST data, although in a different way: MFS: Micro approach, where a sample of units known as simples are selected to model the value of all the other simple units. All units have a weight of 1. Independent stratification design for these simple units. Applied for selected combinations of provinces and NAICS.

17 Comparing survey elements GST micro approach, all units modelled Take-alls Take-nones Simples Complex units

18 Comparing survey elements MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail Sample selection STC Generalized Sampling (GSAM) system for sample selection Random sampling within each stratum Random sampling within each stratum and for the GST modelling Systematic sampling within each stratum Same sample, sampling births every month

19 Comparing survey elements MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail Edit and imputation Before MSM in- house E&I program MFS in- house E&I program MWRTS in- house E&I program Now BANFF, a STC generalized system for E&I MWRTS in- house E&I program All are doing outlier detection, historical edits, trend and mean imputation

20 Comparing survey elements MSM Manufacturers MFS Restaurants MWRTS Wholesale, Retail Estimation STC Generalized Estimation System (GES) used to produce the estimates. Variance -Sampling variance computed using GES -Computing variability from other sources (imputation, use of admin data) being considered. Publication -Estimates published in The Daily, a STC publication. -Estimates available in CANSIM.

21 Summary - A monthly unified enterprise survey? Advantages Taking advantage of the best practices of each survey to integrate all monthly surveys Easier to ensure coherence between the monthly surveys – and their annual counterpart, for comparison purposes Annual surveys are already integrated

22 Summary - A monthly unified enterprise survey? Advantages (contd) Would facilitate the implementation of changes to the surveys (no need for a guinea pig): For example, introduction of the GST was done at different times for the 3 surveys Implementation of better measures of variability could be done more efficiently: Integrating more components of variability Variance of the differences between the estimates

23 Summary - A monthly unified enterprise survey? Issues - Methodological As seen, there are conceptual differences between the surveys: Different data elements collected Level at which the information is collected is different Use of administrative data is different Although these differences could be factored in the sample design

24 Summary - A monthly unified enterprise survey? Issues - Operational Currently managed by three different subject matter divisions, but centralized methodological support Impact on the field work

25 Conclusion Past: Used to be important differences between the monthly surveys Present: Important steps have been made to share best practices between the monthly surveys to harmonize them Future: Should we go one step further and integrate them in a Monthly Unified Enterprise Survey?

26 For more information, please contact Pour plus dinformation, veuillez svp contacter