1. Istat - Structural Business Statistics
Purpose: to measure the impact of frame errors on SBS sampled-based estimation
Contents
Sample-based estimates - quality
Frame errors
The SME survey and the Business register - quality aspects
The SME survey sampling and estimation method
Simulation study
Measures of errors
Results and future plans
19th Roundtable Meeting

2. Istat - Structural Business Statistics
Quality: accuracy
It denotes the closeness of estimates to the true value
Errors:
sampling errors
non sampling errors
coverage (frame) much very much
non response much very much
processing
model
measurement
variability bias
much not much
Overall error: MSE = [bias]2 + Variance
19th Roundtable Meeting

3. Istat - Structural Business Statistics
Frame errors (1)
under-coverage: BR does not reflect units in scope for the survey
omission (lags and leakage)
falsely not active units
mistakes in stratification variables
impact on estimations increasing bias
19th Roundtable Meeting

4. Istat - Structural Business Statistics
Frame errors (2)
over-coverage: BR considers in scope businesses that are out
duplications
falsely active units
mistakes in stratification variables
impact on estimations increasing bias and sampling error
19th Roundtable Meeting

5. Istat - Structural Business Statistics
Frame errors (3)
Incorrect information: units correctly recorded hold incorrect variables
stratification v.
identification v. (location)
For postal survey errors in location cause a non response
impact on bias and sampling variance (reduction of the sample size).
Frame errors and non-responses are correlated
19th Roundtable Meeting

6. Istat - Structural Business Statistics
SME survey and BR (1)
yearly
sample survey
BR is the frame for sampling and grossing up
produces estimates of aggregated economic variables (turnover, value added at factor cost, total purchases of goods and services, personnel costs,..)
In the estimation process frame errors and non-responses are treated similarly
sampling errors are calculated (CVs)
Errors in the BR Errors in estimations
19th Roundtable Meeting

7. Istat - Structural Business Statistics
SME survey and BR (2)
Tool: Statistical information system
To manage links between each surveys (SBS and STS) and the BR (by the unique ID code)
to monitor surveyed information for samples coordination ( to reduce statistical burden)
to record response (non response)
19th Roundtable Meeting

8. Istat - Structural Business Statistics
SME survey and BR (3)
Quality information:
a system of DATE
reference date of the survey
date questionnaires have been sent
response date
a system of CODES
1) Respondents
2) Total non response
3) not useful data
4) Rejected, unknown, moved
5) Units are ceased, not active, in bankruptcy
19th Roundtable Meeting

9. Istat - Structural Business Statistics
Quality information- Dates
19th Roundtable Meeting

10. Istat - Structural Business Statistics
SME survey and BR (4)
CODES
Type 4 (rejected, unknown, moved ) and type 5 (units ceased, not active, in bankruptcy)
FRAME ERRORS
type 4 counts for errors in location variables (i.e. ADDRESS)
type 5 depends on delay in BR updating (i.e. state of activity)
19th Roundtable Meeting

11. Istat - Structural Business Statistics
19th Roundtable Meeting

12. Istat - Structural Business Statistics
The SME survey sampling and estimation method
target population: enterprises <100 persons employed
Stratified simple random sample
26,000 strata (NACE Rev.1.1, Size, Region)
sample size:120,000
Survey technique: postal questionnaire; 2 call-backs
Calibration estimators methodology (Deville and Särndal,1992):
- estimates are constrained to two auxiliary variables, known totals in the population (n.of enterprises and persons employed)
19th Roundtable Meeting

13. Istat - Structural Business Statistics
Simulation study (1)
Variable of interest:
Business Turnover (SME estimate) (BT)
Variable used in the estimation process:
Business fiscal turnover (BFT)
High correlation between BT and BFT
BFT (Fiscal register)
Advantage
Availability for
the whole BR population
Disadvantage
Lack of quality
(missing and outliers)
19th Roundtable Meeting

14. Istat - Structural Business Statistics
Simulation study (2)
Data set 1 =respondent units to SME
Data set 2 =respondent units to SME+ non respondent units to SME for frame errors
Estimate 1 ( ) and Estimate 2 ( ) are calculated at NACE 3-digits.
Measures of errors:
Bias
Sampling error (CVs)
MSE
19th Roundtable Meeting

15. Istat - Structural Business Statistics
Effect on estimates’ bias (1)
19th Roundtable Meeting

16. Istat - Structural Business Statistics
Effect on estimates’ bias (2)
19th Roundtable Meeting

17. Istat - Structural Business Statistics
Effect on estimates’ sampling error
19th Roundtable Meeting

18. Istat - Structural Business Statistics
Effect on estimates’ sampling error
19th Roundtable Meeting

19. Istat - Structural Business Statistics
Effect on estimates’ sampling error
19th Roundtable Meeting

20. Istat - Structural Business Statistics
MSE=(bias)2 + sampling variance
19th Roundtable Meeting

21. Istat - Structural Business Statistics
19th Roundtable Meeting

22. Istat - Structural Business Statistics
Results
Overestimate of the BFT in the population (both estimate 1 and estimate 2)
The inclusion of Frame errors :
reduces the bias (estimate 2 is lower that estimate 1)
reduce the sampling variance (sample size increase)
improvement of MSE
19th Roundtable Meeting

23. Istat - Structural Business Statistics
Interpreting results and future plan
the estimation method
the peculiarity of the variable of interest (very high figures)
the structural difference between the respondents and the frame errors units
Next step: to add a further constrain to the estimates procedure of economic variables that is the BFT in order to reduce bias
19th Roundtable Meeting