1 Assessing inconsistencies in reported job characteristics of employed stayers: An analysis on two-wave panels from the Italian Labour Force Survey, Francesca Bassi*, Alessandra Padoan** and Ugo Trivellato*** *Statistics Department, University of Padova **Statistics Office, Regione Veneto ***Statistics Department, University of Padova, and CESifo European Conference on Quality in Official Statistics, Rome, 8-11 July 2008

2 FOCUS OF THE PAPER Measurement error in information on industry and occupation. Yearly transition matrices for workers who are continuously employed over the year and did not change job (263,884 units). Italian Quarterly Labour Force Survey

3 OUTLINE OF THE PAPER 1)The context of the analyses 2)Descriptive indicators of (dis)agreement 3)Testing whether the consistency of information increases when the number of categories is collapsed 4)Examination of the patterns of inconsistencies among response categories 5)Comparison of alternative classifications jointly by occupation and industry

4 INDUSTRY Collected by an open-ended question 12 categories (ATECO2002): Agriculture; Mining and raw material extraction; Manufacturing; Construction; Wholesale and retail trade; Accommodation and food services; Transportation and communication Financial and real estate activities; Professional and support service activities; Public Administration, defence and compulsory social services; Education, health and other social services; Other public, social and personal service activities Istat suggests to use the 12-category classification 1. THE CONTEXT OF THE ANALYSES

5

6 OCCUPATION Collected by a closed form question 11 categories: Manager, Executive, Clerk, Workman, Apprentice, Outworker Entrepreneur, Professional, Own-account worker, Member of a producers’ cooperative, Contributing family worker Istat suggests to use the binary classification Employee,Self-employed

7

8 2. DESCRIPTIVE INDICATORS OF (DIS)AGREEMENT P = percentage of frequencies outside the main diagonal e i = net difference rate I i = index of inconsistency K = Cohen’s Kappa

9 MAIN RESULTS Industry is reported with less inconsistency than occupation. There is no significant trend in the indices K coefficients are high and statistically significant, but we expect them equal to 1 P assumes non negligible values

10 3. COLLAPSING CATEGORIES The hierarchical Kappa coefficient allows to verify if aggregating categories improves agreement K 2 implies a less disaggregated classification than K 1 W ii’ are chosen so that they imply aggregation among categories identifying similar employment

11 MAIN RESULTS Industry: Switching from 12 to 6 categories significantly improves agreement in all panels Reducing further to 5 categories significantly improves agreement in 7 out of 10 panels No significant increase is obtained when reducing to 3 categories Istat uses the 12-category classification

12

13 MAIN RESULTS Occupation: Switching from 11 to 6 categories significantly improves agreement in all panels Reducing further to 2 categories significantly improves agreement in all panels Istat uses the 2-category classification

14

15 4. PATTERNS OF INCONSISTENCIES Goodman quasi-independence model is used to evaluate if, when we leave the main diagonal cells aside, the remaining cells show particular patterns of disagreement Accepting the model, implies that errors in reporting employment occur randomly Rejecting the model implies that there are systematic patterns of associations in errors

16 MAIN RESULTS Industry: The quasi-independence model is always rejected (12, 6, 5 and 3 categories) The BIC index is lower with 6 categories Estimated residuals suggest non-random measurement error affecting responses in each wave of the survey Occupation: The quasi-independence model is always rejected (11, 6 and 2 categories) The BIC index is lower with 2 categories Estimated residuals suggest that the binary classification has become too rigid for the Italian labour market

17 5. ALTERNATIVE CLASSIFICATIONS JOINTLY BY OCCUPATION AND INDUSTRY 4-category joint classification: Self-employed Employee in agriculture Employee in industrial sector Employee in services 4- category alternative classification: Self-employed Employee in agriculture Employee in industrial sector and private services Employee in Public Administration and social services

19 MAIN RESULTS The ‘alternative classification’ has a (significantly) higher level of agreement in (8) 9 out of 10 panels. The ‘alternative classification’ has a significantly higher level of agreement in the overall sample.

20 CONCLUSIONS - 1 1)Aggregating categories improves agreement 2)The best levels of aggregation are for industry: Agriculture, Manufacturing and mining, Construction, Wholesale and trade, Other activities for occupation: Self-employed, Employee for occupation and industry jointly: Self-employed, Employee in agriculture, Employee in industrial sector and private services, Employee in Public Administration and social services

21 CONCLUSIONS - 2 3)Estimated residuals from the model of quasi-independence suggest that even cross-section information is affected by non-random measurement error 4)May dependent interviewing help in reducing inconsistencies?

22 SELECTED REFERENCES Bound J., C. Brown and N. Mathiowetz (20002). Measurement error in survey data. In J.J. Heckman and E. Leamer (Eds.), Handbook of Econometrics. Volume 5, Amsterdam, Elsevier Science, Goodman L.A. (1968). The analysis of cross-classified data: independence, quasi-independence, and interaction in contingency tables. Journal of the American Statistical Association, 63, Landis J.R. and G.G. Koch (1977). The measurement of observer agreement for categorical data. Biometrics, 33, Mathiowetz n. (1992). Errors in reports of occupation. Public Opinion Quarterly, 56, Sala E. and P. Lynn (2006). Measuring change in employment characteristics: the effects of dependent interviewing. International Journal of Public Opinion Research, 18,