1. Validation of ESeC: The Effect of Coding Procedures and Occupational Aggregation Level Cornelia Hausen, Jean-Marie Jungblut, Walter Müller, Reinhard Pollak, Heike Wirth Mannheim, MZES and ZUMA ESeC Validation Conference 18 – 20 January 2006 Portuguese Statistical Office, Lisbon, Portugal

2. German Team: Validation of ESeCLisbon, 19 – 20 January 20062 Overview 1 Criterion Validation of ESeC matrix (4-digit) 2 Aggregation of ESeC: 2-/3digit matrices 3 A Comparison between ESeC and EGP 4 Class Effects on Risk of Unemployment

3. German Team: Validation of ESeCLisbon, 19 – 20 January 20063 1 Criterion Validation: Operational Issues Database: BIBB/IAB (1998/1999) and GSOEP (2001) Employment Status –Problem: Concept of Managers/Supervisors not well established –Different distinction: „Position with Employer“ (PwE) –Self-Employed –Managers: ISCO 11-, 12, and 13 2-digit codes –Supervisors: standard (direct measurement) vs. proxy (PwE) Employment Relation Indicators –Monitoring problems –Asset Specificity –Long-Term Employment –Career Prospects

4. German Team: Validation of ESeCLisbon, 19 – 20 January 20064 1 Criterion Validation: Approach 1.Calculation of mean ER-score for each ESeC class 2.Calculation of mean ER-score for each combination of ISCO code and employment status (OUG) 3.Comparison of the mean ER-score of ESeC classes with the mean ER- score for each OUG 4. Reallocation of class codes if evidently suggested by the ER-indicators e.g. Religious Professionals  changes for 36 ISCO-categories, 10.3% respondents reallocated 5. Test whether the revised matrix performs better: increase of within- class homogeneity and between-class heterogeneity ISCO(4)SE 10+ SE <10 SE NO MAN >10 MAN <10 SUPEMP 2460 122Illicit 22

5. German Team: Validation of ESeCLisbon, 19 – 20 January 20065 1 Criterion Validation: Approach 1.Calculation of mean ER-score for each ESeC class 2.Calculation of mean ER-score for each combination of ISCO code and employment status (OUG) 3.Comparison of the mean ER-score of ESeC classes with the mean ER score for each OUG 4. Reallocation of class codes if evidently suggested by the ER-indicators e.g. Religious Professionals  changes for 36 ISCO-categories, 10.3% respondents reallocated 5. Test whether the revised matrix performs better: increase of within- class homogeneity and between-class heterogeneity ISCO(4)SE 10+ SE <10 SE NO MAN >10 MAN <10 SUPEMP 2460 111Illicit 11

6. German Team: Validation of ESeCLisbon, 19 – 20 January 20066 Overview 1 Criterion Validation of ESeC V3 matrix (4-digit) 2 Aggregation of ESeC: 2-/3digit matrices 3 A Comparison between ESeC and EGP 4 Class Effects on Risk of Unemployment

7. German Team: Validation of ESeCLisbon, 19 – 20 January 20067 2. Aggregation of ESeC 2-/3-digit: Approach Starting base for generating the 3-digit and 2-digit matrices: –revised German 4-digit matrix –German modal values based on BIBB/IAB Stock/Production/Transport Clerks ISCO(4)SE 10+ SE <10 SE NO MAN >10 MAN <10 SUPEMP 4131144Illicit 6 (68)7 (132) 4132144Illicit 2 (75)3 (179) 4133144Illicit 2 (62)3 (55) ISCO(3)SE 10+ SE <10 SE NO MAN >10 MAN< 10 SUPEMP 413144Illicit 23

8. German Team: Validation of ESeCLisbon, 19 – 20 January 20068 2. Aggregation of ESeC 2-/3-digit: Findings Table 2.4: Proportion of Correspondence Between Different Aggregation Levels of ISCO in %. Effects of the aggregation for the performance of ESeC with respect to the ER-indicators? 4-digit vs. 3-digit93.8 4-digit vs. 2-digit84.6

9. German Team: Validation of ESeCLisbon, 19 – 20 January 20069 2 Aggregation of ESeC 2-/3-digit: ER-Indicators (Figure 2.1)

11. German Team: Validation of ESeCLisbon, 19 – 20 January 200611 2 Aggregation of ESeC: Implications of Modal Rule I Problem: UK modal value does not correspond to German modal value Stock/Production/Transport Clerks ISCO(4)SE 10+ SE <10 SE NO MAN >10 MAN <10 SUPEMP 4131144Illicit 67 4132144Illicit 23 4133144Illicit 23 ISCO(3)SUPEMP Germany41323 UK41367

12. German Team: Validation of ESeCLisbon, 19 – 20 January 200612 2 Aggregation of ESeC: Implications of Modal Rule II  for a harmonized 3-digit matrix, country specific modal values should be taken into account Table 2.5: Proportion of Discrepant Class Codes Depending on the Country-Specific Modal Value. BIBB/IAB, 1999. 3-digit7.5% 2-digit10.4%

13. German Team: Validation of ESeCLisbon, 19 – 20 January 200613 Overview 1 Criterion Validation of ESeC V3 matrix (4-digit) 2 Aggregation of ESeC: 2-/3digit matrices 3 A Comparison between ESeC and EGP 4 Class Effects on Risk of Unemployment

14. German Team: Validation of ESeCLisbon, 19 – 20 January 200614 3 A comparison of ESeC and EGP Primary focus: –Quality of the crosswalk from the German National Occupational Classification (KldB) to ISCO88 –KldB (Klassifikation der Berufe) standard used by official statistics in Germany some official data (e.g. LFS) also include ISCO (generated by a mapping procedure KldB => ISCO) –Researchers either use ISCO or KldB, seldom both no empirical knowledge which kind of occupations might pose crosswalk problems –ESeC in the LFS will be based on the crosswalk KldB=>ISCO => Implications for ESeC ?

15. German Team: Validation of ESeCLisbon, 19 – 20 January 200615 3 A comparison of ESeC and EGP EGP: intermediary tool to examine crosswalk problems –well examined coding routine to generate EGP directly from the KldB Step 1: –EGP_ISCO: identical coding routine, but KldB codes were replaced by ISCO_3d codes –Comparison of EGP_KldB and EGP_ISCO => where are movements between classes and why ? Step 2: –Comparison of ESeC and EGP_ISCO => differences in the allocation of occupations to EGP and ESeC classes: crosswalk problems, conceptual and other reasons ? Step 3: –Employment relation indicators: ESeC and EGP compared

16. German Team: Validation of ESeCLisbon, 19 – 20 January 200616 Step 1: Correspondence between EGP_KldB and EGP_ISCO Main findings: –marginal distributions of EGP_KldB and EGP_ISCO nearly identical –agreement rate diagonal cells: 92% –mismatches mostly concentrated on neighbour classes (e.g. I => II; IIIa=> V) –more ‚irritating‘ mismatches (e.g. II => V) are restricted to a few cases Further checking on the data shows that mismatches are –mainly due to the loss of information because of different aggregational levels KldB_3d (369 categories) to ISCO_3d (116 categories) –only to a very small extent due to (german specific) crosswalk problems e.g. upper secondary teachers/Gymnasiallehrer can not be distingished from other secondary teachers => high correspondence rate between EGP_KldB and EGP_ISCO –crosswalk is not a problem it is a problem of the aggregational level –a better job could be done, if ISCO_4d were available in the data =>

17. German Team: Validation of ESeCLisbon, 19 – 20 January 200617 Step 2: Correspondence between ESeC_3d and EGP_ISCO =>

18. German Team: Validation of ESeCLisbon, 19 – 20 January 200618 Step 2: Correspondence of ESeC_3d between EGP_ISCO Further analysis show that the deviations between ESeC and EGP are –mainly due to effects of the employment status variable used: EGP class coding is based on the ‚status within employment‘ variable –differentiates between blue- and white-collar workers, civil servants and management positions, and –within these groups there is a further distinction between different hierachical levels ESeC is based on the ‚supervisory‘ concept –manager, supervisors, employees => ‚status within employment‘ variable thus enables a finer assignment of occupations to classes =>

19. German Team: Validation of ESeCLisbon, 19 – 20 January 200619 Step 3: Employment Relation Indicators: ESeC – EGP => Model fit for EGP for nearly all ER-indicators higher than for ESeC versions ‚Status within employment‘ variable leads to greater homogeneity within the classes and a larger variance between the classes than the supervisory variable =>

20. German Team: Validation of ESeCLisbon, 19 – 20 January 200620 Overview 1. Criterion Validation of ESeC V3 matrix (4-digit) 2. Aggregation of ESeC: 2-/3digit matrices 3. A Comparison between ESeC and EGP 4. Class Effects on Risk of Unemployment

21. German Team: Validation of ESeCLisbon, 19 – 20 January 200621 4 Class effects on risks of unemployment Construct validation analysis: class effects on risks of unemployment 1.comparison of the prototype and the revised German matrix 2.effects of different levels of aggregation (ISCO 4,3,2 digit) 3.ESeC compared to EGP –Dependent variable have been unemployed in the past at least once (dummy coded) –Control variables age, gender, Geman citizenship, part-time, East-West Germany; educational level (CASMIN) =>

22. German Team: Validation of ESeCLisbon, 19 – 20 January 200622 Risk of unemployment : Main Findings I Model fit –Pseudo-R 2 nearly identical for the different versions EGP (Kldb; ISCO) slightly better than all ESeC versions Class effects –clear hierarchy regarding the risk of unemployment –ranking of classes nearly constant for all versions =>

23. German Team: Validation of ESeCLisbon, 19 – 20 January 200623 Risk of unemployment : Main Findings II Effect size: –larger for EGP than for ESeC, and Unemployment risk of EGP farmers much lower than for ESeC farmer EGP : unemployment risk for class IIIb (ESeC 7) higher than for class VI (ESeC 8) –a similiar pattern is found only for the revised ESeC matrix (4_d) => further analysis needed to examine whether these findings –an effect of the ‚status within employment‘ variable, or –an effect of differences in the allocation of specific occupations to classes =>

24. German Team: Validation of ESeCLisbon, 19 – 20 January 200624 Conclusions ESeC will be an important improvement of international comparative research, but there are a few issues left where a further discussion is needed: 1.Supervisory concept  concept should be clarified and an international agreed procedure should be established 2.Aggregation Level of ISCO –the higher the aggregation level of ISCO used for ESeC the lower the within-class homogeneity  A future ESeC (at least in LFS) should be based on ISCO-4digit –model fit of the aggregated matrices are slightly better when country specific modal values ared, but also comes along with a higher between-countries heterogeneity => what is the lesser evil? 3.Procedural or Interpretative equivalence ? –In sum the revised German matrix reveals only a slightly better model fit than the prototype but there are some occupations where we would prefer a change in the ESeC prototype version  equivalence of procedure: identical matrix for all countries? => equivalence of meanings: allowing a certain amount of variation in the matrix between countries?

25. German Team: Validation of ESeCLisbon, 19 – 20 January 200625 Correspondence between EGP_KldB and EGP_ISCO <=

26. German Team: Validation of ESeCLisbon, 19 – 20 January 200626 Correspondence between ESeC_3d and EGP_ISCO (col %) <=

27. German Team: Validation of ESeCLisbon, 19 – 20 January 200627 Position within employment – Employment Status <= =>

28. German Team: Validation of ESeCLisbon, 19 – 20 January 200628 Position within employment – Employment Status <=

29. German Team: Validation of ESeCLisbon, 19 – 20 January 200629 ESeC class 2 => EGP: Movers and Stayers <=

30. German Team: Validation of ESeCLisbon, 19 – 20 January 200630 Step 3: Employment relation indicators: ESeC – EGP => <=

31. German Team: Validation of ESeCLisbon, 19 – 20 January 200631 Employment relation indicators : ESeC – EGP compared <=

32. German Team: Validation of ESeCLisbon, 19 – 20 January 200632 4 Class effects on risks of unemployment: Findings <= =>

33. German Team: Validation of ESeCLisbon, 19 – 20 January 200633 Odds of having been unemployed – class effects <=

34. German Team: Validation of ESeCLisbon, 19 – 20 January 200634 Odds of having been unemployed – class effects

35. German Team: Validation of ESeCLisbon, 19 – 20 January 200635 Odds of having been unemployed – class effects

36. German Team: Validation of ESeCLisbon, 19 – 20 January 200636 Odds of having been unemployed – class effects

37. German Team: Validation of ESeCLisbon, 19 – 20 January 200637 Appendix: Criterion Validation: Main Results

38. German Team: Validation of ESeCLisbon, 19 – 20 January 200638 Appendix: Aggregation of ESeC: 2-/3-digit Matrices I Starting base: revised German 4-digit matrix and German modal values

39. German Team: Validation of ESeCLisbon, 19 – 20 January 200639 Appendix: Aggregation of ESeC: 2-/3-digit Matrices II

40. German Team: Validation of ESeCLisbon, 19 – 20 January 200640 Appendix: Aggregation of ESeC

41. German Team: Validation of ESeCLisbon, 19 – 20 January 200641 Appendix: Operationalization en detail I Supervisor function: –Standard version: “Do you have co-workers for whom you are their direct supervisor?” –Proxy version: based on “Position with Employer” (14 categories): foreman or master, employee with high level managerial tasks, high-level and executive level civil servants Work Autonomy: Factor Scores, 3 items (almost always – hardly ever) –Work tasks are prescribed in all details –An identical work operation recurs in all details –A precise number of product units, a minimum work performance or the time to carry out a specific work task is prescribed

42. German Team: Validation of ESeCLisbon, 19 – 20 January 200642 Appendix: Operationalization en detail II Asset Specificity: information on the qualification job holders in reality have combined with their assessment whether the job could be done by a worker with lower qualifications. Starting from a distinction between the following three levels of the highest (vocational) qualification declared by the respondent –no vocational qualification –vocational school, vocational training etc. –foreman degree (corresponding to post-secondary non-tertiary education or (tertiary) college degree the score for the measure is reduced by one level whenever the respondent indicates that his job could be performed with a lower education than he himself has

43. German Team: Validation of ESeCLisbon, 19 – 20 January 200643 Appendix: Operationalization en detail III Long-Term Employment: –Residuals of number of years with the current employers controlled for variation in length of labour force experience (number of years with the current employer) and gender Career Prospective: –extent of further education in the last five years (with the present employer)