1. ISTAT - Italian National Institute of Statistics Labour Force Survey Division Unit “Methods for LFS data treatment” European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki ( Finland ) Longitudinal data from Italian Labour Force Survey Barbara Boschetto boschett@istat.itboschett@istat.it Antonio R. Discenza discenza@istat.itdiscenza@istat.it Francesca Fiori frfiori@istat.itfrfiori@istat.it Carlo Lucarelli calucare@istat.itcalucare@istat.it Simona Rosati sirosati@istat.itsirosati@istat.it

2. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Outline of the presentation Issues related to the production of longitudinal microdata and gross flows estimates consistent with the official quarterly estimates a specific focus is devoted to the weighting procedure, which account both for a suitable reference population and compensate for the total non-response at subsequent waves the most relevant methodological problems addressed are:  definition of a suitable reference population for the longitudinal sample  longitudinal non-responses and eligibility  coherence between cross-sectional and longitudinal estimates

3. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Household rotation scheme 2-2-2 in the Italian LFS 50% of the sample overlaps after 1 quarter and 1 year

4. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Net changes in quarterly levels are the final result of a high number of gross flows of different nature and different size  Demographic flows: –Children aged 15 entering working age –Deaths –Internal and International migration  Labour status transitions: –Flows between the three main activity states (employment, unemployment and inactivity)

5. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki GROSS LABOUR MARKET FLOWS EMPLOYMENT UNEMPLOYMENT INACTIVE DEATHS AND PEOPLE LEAVING MUNICIPALITIES CHILDREN AGED 15 AND PEOPLE ENTERING MUNICIPALITIES

6. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Choice 1: the reference population is equal to the population of the initial quarter Ideally, longitudinal data from LFS should represent the whole initial population. However, the initial population actually change during the period of observation because of deaths and internal and international migrations. Thus longitudinal data could represent the whole initial population only if the LFS was designed like a “proper” panel, in which all the individuals in the initial sample were “followed” for a new interview at a later stage.  This means that the information must be collected also on people moving to another municipality or to another country.  Actually, information for those persons who left the country during a given period is usually never available.

7. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Figure 1: Scheme for a “desirable” complete matrix with stocks and gross flows from LFS.

8. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Choice 2: the reference population is a specific longitudinal population However, even the population resident in a country at the beginning of the period, which is still resident in the country at the end of the reference period, can experience movements to and from the different municipalities (internal migration). Usually, in the LFS, people moving out of the household, across the country, are not “followed” for re-interview. Is it still correct to use the initial population as the reference population ? In fact, in the longitudinal sample we have information only about those individuals still resident in the same municipality at the end of the period. The longitudinal component (sub-sample) of the Italian LFS requires thus the specification of a suitable reference population.

9. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Choice 2: the reference population is a specific longitudinal population If we weight the longitudinal sample to the initial population we make a very strong assumption: the behaviour of individuals which moves out of the municipality from one wave to another is similar to those who do not move. We have, thus, at least two problems:  Actually, at least in Italy, these two groups are very different  Moreover, if we use the longitudinal microdata to produce flow estimates, there are no records of individuals moving to other regions/country and/or dying (whereas they do exist in the population).

10. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Definition of longitudinal population The Longitudinal Population is defined in Italy as the population which is resident in the same municipality for the entire 12 months period  excluding –deaths –those who have moved to other Italian municipalities (change of residence) –Migrants to other countries It is computed from population register data on resident population; it is classified by broad age groups, geographical area (NUTS III) and nationality (Italian, EU, non-EU) It is fully consistent with the reference population of EU-LFS quarterly data was also ensured.

11. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Figure 1: Scheme for the two transition matrices referred to the longitudinal population and to internal migrants..

12. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Figure 3: Scheme for a “actual” complete matrix with stocks and gross flows from Italia LFS. The longitudinal estimates must be consistent with the “official” estimates provided by the cross-sectional samples (the full sample) at the beginning and at the end of the observed period. Using specific constraints in the calibration procedure used to weight longitudinal sample it is possible to reduce the risk of obtaining inconsistent results.

13. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Longitudinal non-responses and eligibility A very important aspect of the longitudinal component of the LFS is usually affected by unit non-response in subsequent waves, such as:  Municipality non-response: some (very small) municipalities are substituted in July at the beginning of a new annual survey cycle and some others may, for different reasons, fail to provide the interviews in subsequent waves;  Household non-response: all the members of the household do not fill in the questionnaire because they refuse to respond;  Individual non-response: some members of the household do not fill in the questionnaire because they refuse to respond, or they cannot be contacted or left the household to create a new household in the same municipality. Unit non-response may produce bias if non-respondents have significantly different labour features with respect to respondents.

14. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Figure 4: Classification of individuals from the initial sample and eligibility in the Italian LFS (in presence of longitudinal non-response). we don’t have enough information to distinguish not- respondents eligible from thise not-eligible.

15. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Eligibility All the individuals can be classified into two groups:  Eligible: –they represent part of the longitudinal population (because still living in the same municipality), –they should be re-interviewed at the subsequent wave. –some of them are non-respondents in the final quarter, so that they must be considered in a model for treatment of non- response (they must be represented by individuals with similar characteristics);  Not-eligible: –they left the initial population during the observed period (deaths and migrations) –they do not represent part of the longitudinal population –they must be excluded from a model for treatment of non- response.

16. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Step 1 : All the individuals which are linkable/matchable at the beginning of the period are selected.  are all the individuals of the two rotation groups which overlap at 12 month and resident in those municipalities which provided interviews for both waves; they can be considered like a random sub-sample of the whole cross-sectional sample their base longitudinal weights are obtained from cross- sectional weights applying the following correction Weighting longitudinal data in three steps 1/3

17. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Step 2 : Accounts for bias due to municipality non-response Accounts for the differences between the rotation groups which overlaps and those who don’t; Help to ensure consistency between longitudinal and cross- sectional “official” estimates the first calibration procedure makes matchable individuals at the beginning of the period  represent exactly the same cross-sectional population of the whole cross-sectional sample.  provide exactly the same cross sectional “official” estimates for a number of relevant figures (cross-classification of sex, region, age group, labour activity status, education, etc.). Weighting longitudinal data in three steps 2/3

18. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Thus, from the base longitudinal weights and for all the linkable individuals the intermediate longitudinal weights are obtained as result of a minimum constrained problem as follows: Weighting longitudinal data in three steps 2/3

19. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Step 3 : Adjusts for bias due to individual non-response Make weighted longitudinal-sample totals conform to the longitudinal population. The hypothesis underlying is that the non-response is random inside the cells resulting from nesting population by gender, by age groups and NUTS1, NUTS2 and NUTS3 domains Weighting longitudinal data in three steps 3/3

20. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki only for linked individuals the final longitudinal weights are computed applying a new calibration stage to make weighted longitudinal component totals conform to the longitudinal population under the following constraints Weighting longitudinal data in three steps 3/3

21. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Flow chart of weighting procedure Step 2 FINAL CROSS-SECTIONAL WEIGHTS INTERMEDIATE LONGITUDINAL WEIGHTS

22. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Flow chart of weighting procedure Step 3 INTERMEDIATE WEIGHTS FINAL WEIGHTS

23. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki 49 2 495 547 Deaths Employed 20.346 353 1.281 21.980 Unemployed 489 449 514 1.452 Inactive 1.260 757 23.131 25.149 Total 22.095 1.559 24.926 48.581 Total Labour Status at 2008Q1 Inactive Longitudinal PopulationEmployedUnemployed Labour Status at 2007Q1 Net change due to Longitudinal Population flows + 115 817 102 377 1.296 People Leavingthe Municipalities 22.846 1.556 26.021 50.424 Population aged 15+ 2007Q1 00 584 Children aged 15 1075 202 359 1.636 People Entering the Municipalities 23.170 1.761 25.870 50.801 Population aged 15+ 20087Q1 Net change in cross-sectional employment +324 Net change due to Migratory flows + 258 Net change due to Demographic flows - 49 Complete Matrix with net and gross flows. Quarter 1 2007 – Quarter 1 2008. (Thousands)

24. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Transition Matrix for longitudinal population. Quarter 1 2007 – Quarter 1 2008. (Thousands) Employed 20.346 353 1.281 21.980 Unemployed 489 449 514 1.452 Inactive 1.260 757 23.131 25.149 Total 22.095 1.559 24.926 48.581 Total Labour Status at 2008Q1 Inactive Longitudinal PopulationEmployedUnemployed Labour Status at 2007Q1 Net change +105 Leaving employment 1.634 Entering employment 1.749 Persistence in employment almost 3.400 movements

25. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Main Findings Potentials  Longitudinal data provide extremely useful insights on labour market dynamics  Are obtained without important additional costs, but with high investment in methodology  Can be produced regularly on quarterly bases Constraints  EU-LFS is not a panel survey, thus longitudinal estimates can refer only to a specific longitudinal reference population  Known totals for this longitudinal reference population must be available for weighting  Methods for non-response treatment must be used to reduce bias  Methods to ensure consistency with cross-sectional estimates must be used

26. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Some Examples of Analysis of Labour Market from Quarter 1 2004 – Quarter 1 2008 using 12 months longitudinal data

27. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Women have lower persistence probability and higher transition probability to inactivity Employment: persistence and transition probabilities by gender and region. 2007Q1 – 2008Q1 South has lower persistence probability and much higher transition probability

28. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki High segmentation in persistence and transition Employment: persistence and transition probabilities by job characteristics. 2007Q1 – 2008Q1

29. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Unemployment: transition probability to employment by duration of search at the starting point Transition probability is inversely correlated to the duration of search for employment Opportunities to get an Employment for long term Unemployed are stable in the period

30. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki Huge differences in the persistence and transition probabilities between North and South Unemployment: persistence and transition probabilities by sex and NUTS1 region. 2007Q1 – 2008Q1 Higher probability to get an employment for men Higher probability to leave labour force for women

31. European Conference on Quality in Official Statistics – Q2010 3 - 6 May 2010 - Helsinki THANK YOU FOR YOUR ATTENTION