2. The obsession with weight in the modelling world And it’s ancillary affects on Analysis

3. The basic The basic idea of sampling The basic idea of sampling The reason behind complicating a good idea The reason behind complicating a good idea The implication when modelling data The implication when modelling data

4. How Sampling Works. 1% Random (systematic) 3% Random 5% Random 10% Random 2.5% Stratified Now let’s assume that we had some idea about the picture we wanted to see. And we decide to stratify the sample. In this case we decide to sample different areas of the picture at different rates, the backgroud, the dress, the face, the hands, etc... Imagine a well known picture Since a picture is made up of points of colour (pixels), we will sample the points of colour at different rates.

5. How Sampling Works. 3% 5%10% 2.5% Stratified 1%

6. How does this affect modeling or analysis The sample is no longer simply random The sample is no longer simply random We purposefully biaised the sample to gain efficiencies to meet other goals We purposefully biaised the sample to gain efficiencies to meet other goals This bias is corrected when we apply the design weights. This bias is corrected when we apply the design weights.

7. If you were to analyse each stratum separately Each part can actually be treated as surveys each with a simpler design The sampling frame or design allows you to keep all these part together in a cohesive way for analysis.Framework Still there would be some difficulty associated with the correction for non-response and final callibration (post)

8. The way we sample is reflected and corrected by how we weight the data in the end. How to interpret sampling If you looked only at the parts we sampled If you looked only at the parts we sampled –You wouldn’t get an accurate picture. –All the parts would be there but not in the right proportions. The design weights compensate for the known distortions. The final weights include estimated distortions. The design weights compensate for the known distortions. The final weights include estimated distortions.

9. What would you use to base the fundamental multivariate relationships in your model or analysis ?

10. Steps to calculate the weights – Basic overview At the survey design stage, some factors are used to determine the sample size required At the survey design stage, some factors are used to determine the sample size required Probability of selection calculated Probability of selection calculated First series of adjustments for non- response First series of adjustments for non- response Post-stratification Post-stratification

11. Factors to determine the sample size Characteristics to be estimated (small proportions) Characteristics to be estimated (small proportions) Required precision of the estimates (targetted CV) Required precision of the estimates (targetted CV) Variability of the data Variability of the data Expected non-response rate Expected non-response rate Size of the population Size of the population

12. Original design weight Once the sample is selected in each stratum, calculate the original weight: Once the sample is selected in each stratum, calculate the original weight: –N h /n h, where « h » is the stratum Since the sample is selected from LFS, get original weight from LFS. Since the sample is selected from LFS, get original weight from LFS. –Adjustments for the number of available children.

13. Non-response adjustment Adjustments must be made to take into account the total non-response Adjustments must be made to take into account the total non-response Characteristics of respondents vs non- respondents are analyzed: Characteristics of respondents vs non- respondents are analyzed: –Province, income, level of education of parents, depression scale of PMK, urban/rural, etc.

14. Post-stratification Adjustment factor calculated in order to post-stratify the sample to known population counts, by: Adjustment factor calculated in order to post-stratify the sample to known population counts, by: –Province, age, gender

15. Final weight W f = W i X Adj 1 X Adj 2 W f = W i X Adj 1 X Adj 2 Where Where –W f : Final weight –W i : initial weight –Adj 1 : Non-response adjustment –Adj 2 : Post stratification

16. Link between analysis and the sample design (weight) Child’s Ability Child’s Ability IntelligenceIntelligence Social environment SchoolSchool TeachersTeachers MaterialsMaterials CurriculumCurriculum Grade level SubjectSubject ProvinceProvince Province is a stratum The proportion of kids in the sample being taught the PEI curriculum is much larger than what’s found in the population

17. Link between analysis and the sample design There are very few things in a child’s life that is not related to where they live. In the city versus in a small village In a small province versus a large one what social/educational programs are offered what social support and services are offered regional cultural differences to name a few…

18. Weights for cycle 4 –Cross-sectional weights –Longitudinal weights, including the converted respondents. –Longitudinal weights, children introduced in C1 and respondent to all cycles. NEW –Not to mention the bootstrap weights, which are used for an entirely different purpose.

19. Cross-sectional Weights Available for all cycles, up to Cycle 4. Available for all cycles, up to Cycle 4. When are they used? When are they used? Cycle 4 cross-sectional weights: Cycle 4 cross-sectional weights: –to represent the population aged 0-17 in 2000-01. –…–…–…–… Cycle 1 weights: Cycle 1 weights: –to represent the population aged 0-11 in 1994-95.

20. Cross-sectional Weights - Cycle 4 - Warning In Cycle 4, children with a cross- sectional weight come from 4 different cohorts (introduced in 1994, 1996, 1998 and 2000). In Cycle 4, children with a cross- sectional weight come from 4 different cohorts (introduced in 1994, 1996, 1998 and 2000). By 2000, the 1994 cohort has been around for 6 years: By 2000, the 1994 cohort has been around for 6 years: –cross-sectional representativity decreases over time because of sample erosion and population change (immigration).

21. Cross-sectional Weights - Cycle 5 For Cycle 5 (2002-2003), no children aged 6 and 7. For Cycle 5 (2002-2003), no children aged 6 and 7. In addition, the 1994 cohort’s cross- sectional representativity has declined even further (erosion and immigration). In addition, the 1994 cohort’s cross- sectional representativity has declined even further (erosion and immigration). As a result, cross-sectional weights will be calculated only for children aged 0-5. As a result, cross-sectional weights will be calculated only for children aged 0-5.

22. Cross-sectional weights in a nutshell Cross-sectional weights must be used when the analysis concerns a specific year, when you want a snapshot of the situation at a specific point in time. Cross-sectional weights must be used when the analysis concerns a specific year, when you want a snapshot of the situation at a specific point in time.

23. Longitudinal Weights Longitudinal weights represent the population of children at the time they were brought in to the survey. Longitudinal weights represent the population of children at the time they were brought in to the survey. –Children introduced in Cycle 1: longitudinal weights represent the population of children aged 0-11 in 1994-95.

24. Longitudinal Weights (continued) –Children introduced in Cycle 2: longitudinal weights represent the population of children aged 0-1 in 1996-97. –Children introduced in Cycle 3: longitudinal weights represent the population of children aged 0-1 in 1998-99. –Children introduced in Cycle 4: longitudinal weights represent the population of children aged 0-1 in 2000-01.

25. When are longitudinal weights used? When you want to track a cohort of children introduced in a particular cycle and see how they’ve developed over time. When you want to track a cohort of children introduced in a particular cycle and see how they’ve developed over time.

26. Longitudinal Weights - Cycle 4 Something new in Cycle 4: Something new in Cycle 4: 2 sets of longitudinal weights: 2 sets of longitudinal weights: –Set 1: Weights for children who responded in their first cycle and in Cycle 4 (possible non-response in Cycle 2 or 3) –Set 2: Weights for those introduced in cycle 1 who responded in every cycle. NEW.

27. Longitudinal Weights - Cycle 4 Difference between the 2 sets of longitudinal weights Difference between the 2 sets of longitudinal weights –To avoid total non-response in Cycle 2 or 3, the set of weights for those who responded throughout can be used. –If you’re only interested in the changes between Cycle 1 and Cycle 4 directly, the longitudinal weights including converted respondents can be used.

28. Examples Following are real examples taken from the NLSCY data Following are real examples taken from the NLSCY data

29. Weighting - Examples Average weights in Cycle 4. 5-year-old 5-year-old 7 1-year-olds Prince Edward Island

30. 15-year-old Weighting - Examples Average weights in Cycle 4 (continued) Ontario Ontario 712 712 15-year-olds

31. Example: Proportion of children aged 0-17, by province, Cycle 4, UNWEIGHTED 24% of Canada’s children live in the Maritime provinces … whereas in reality... 24% of Canada’s children live in the Maritime provinces … whereas in reality...

32. Example: Proportion of children aged 0-17, by province, Cycle 4, WEIGHTED Whereas in reality…7.3% of children live in the Maritime provinces. Whereas in reality…7.3% of children live in the Maritime provinces.

33. Number of children aged 0-15 by year of age, Quebec, Cycle 3, unweighted The conclusion is obvious… The conclusion is obvious… Huge increase in births in 1993 and 1997!!!!! Huge increase in births in 1993 and 1997!!!!! AgeBirth Year Sample size Percentage 019983064.9% 119971,05516.8% 219963265.2% 319954497.1% 419944056.4% 519931,62725.8% 619923135.0% 719912013.2% 819902654.2% 919891702.7% 1019882213.5% 1119871542.4% 1219862243.6% 1319851672.7% 1419842413.8% 1519831712.7% Total6,295

34. Number of children aged 0-15 by year of age, Quebec, Cycle 3, WEIGHTED So much for the pseudo baby boom... So much for the pseudo baby boom... AgeBirth Year Population Percentage 0199873,2545.2% 1199778,7695.5% 2199684,7136.0% 3199588,6626.2% 4199487,8956.2% 5199391,4666.4% 6199295,1016.7% 7199178,8825.6% 81990116,7528.2% 9198973,4515.2% 101988107,8197.6% 11198775,1305.3% 12198698,2026.9% 13198579,4005.6% 141984100,3857.1% 15198391,2056.4% Total1,421,086

35. Conclusion To be obsessed with weights is a good thing…where statistical analysis is concerned To be obsessed with weights is a good thing…where statistical analysis is concerned