1. 13.Analysis Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data

2. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 1 Prerequisites Recommended modules to complete before viewing this module  1. Introduction to the NLTS2 Training Modules  2. NLTS2 Study Overview  3. NLTS2 Study Design and Sampling  NLTS2 Data Sources, either 4. Parent and Youth Surveys or 5. School Surveys, Student Assessments, and Transcripts  9. Weighting and Weighted Standard Errors

3. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 2 Prerequisites Recommended modules to complete before viewing this module (cont’d)  NLTS2 Documentation 10. Overview 11. Data Dictionaries 12. Quick References Helpful  Implications for analysis 6. Data Content 7. Parent/Youth Survey Data

4. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 3 Overview  Posing questions to the NLTS2 database Types of questions; getting to the answers  Comparative variables  Analysis demonstration using longitudinal data  Analysis plan for this demonstration  Results output for this demonstration  Interpretation of results for this demonstration  Review steps required for this demonstration  Closing  Important information

5. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 4 Posing Questions to the NLTS2 database Types of questions  Descriptive questions—e.g., means, frequency distributions for an identified group  Comparative questions—e.g., differences between subgroups of interest, tests of significance  Longitudinal questions—i.e., relationships between variables measured at different time points  Explanatory questions—i.e., questions regarding relationships between variables, usually requiring controls for covariates

6. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 5 Examples of types of questions Descriptive/comparative  How proficient are high school students with disabilities in reading?  How does proficiency differ by disability category? By race/ethnicity? Longitudinal/comparative  If youth has been employed, how does income change over time for youth with disabilities as a group? How does it change at the individual level?  How do changes in wages over time differ by disability category? By race/ethnicity? Explanatory  What factors relate to variations in the employment of youth with disabilities?

7. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 6 Getting to the answer 1.Develop an analysis plan. 2.Identify/create variables.  Identify data source.  Specify variable names.  Be aware of variable forms (e.g., continuous, categorical,) and respondent subset.  Restructure variables if applicable (e.g., regroup into different categories).  Create variables as needed (e.g., scales from multiple items). 3.Select  Analysis approach from options appropriate to the question.  Software procedure(s) to execute approach. 4.Execute plan.

8. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 7 Comparative variables A set of demographic variables are commonly used for comparative analyses in NLTS2 reports and publications.  NLTS2 demographic variables are by wave and included in every school and parent/youth sourced files. The demographic variables are  Primary disability category  Age category  Race/ethnicity  Gender  Household income category

9. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 8 Comparative variables Grade level and urbanicity are used with school-based data.  Demographic variables are based on location of the school district where youth attended secondary school.  School related demographic variables are included with school data files but not applicable to parent/youth survey files.

10. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 9 Analysis demonstration using longitudinal data Research questions for this demonstration  How do wages change over time for youth with disabilities as a group?  How do they change at the individual level?  How do changes in youth wages over time differ by parent’s household income?

11. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 10 Analysis plan for this demonstration Select procedures that will describe these changes.  In this demonstration, we will use procedures that provide weighted frequencies, percentages, means, and standard errors. Select procedures to make comparisons.  In this demonstration, we will produce crosstabulations and means by a comparison variable and determine whether differences are statistically significant.

12. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 11 Analysis plan for this demonstration Select variables that measure change.  Compare items in Parent/Youth Wave 2 with Parent/Youth Wave 5.  Wage youth earned at current or most recent job (Wave 2 np2HourlyWage compared with Wave 5 np5T4h_L4h). Select comparison variables.  Comparison variables are often demographics such as primary disability category, gender, and race/ethnicity.  Parent/guardian’s household income will be the comparison variable in this demonstration (W5_IncomeHdr2009).

13. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 12 Analysis plan for this demonstration Compare time 1 and time 2.  One approach to observe a difference between time 1 and time 2 is to compare a time 1 mean with a time 2 mean calculated across the entire sample.  To measure individual difference, an approach is to calculate the difference between that youth’s time 1 measure and his or her time 2 measure.  To calculate the difference between time 1 and time 2, there must be a value for both time 1 and time 2 to be included in this analysis.

14. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 13 Analysis plan for this demonstration Missing values  Expect to have fewer cases to analyze in a longitudinal analysis than in a separate analysis of either time 1 or time 2. Youth must have data in both waves to be included. Create new variables  Difference = time 2 – time1  The order is important; subtracting time 1 from time 2 will result in a positive number if the change has been in a positive direction.

15. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 14 Results output for this demonstration Means: hourly wages in Wave 2 and Wave 5 Mean: difference in wages between Wave 2 and Wave 5 Percentage who had a decrease or increase in wages Crosstabs by parent/guardian household income

16. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data NLTS2 restricted-use data NLTS2 data are restricted. Data used in these presentations are from a randomly selected subset of the restricted-use NLTS2 data. Results in these presentations cannot be replicated with the NLTS2 data licensed by NCES. 15

17. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 16 Means of Hourly Wages in Wave 2 and Wave 5 Estimate Standard Error Unweighted Count MeanHourly pay youth earned at job (in or out of sch) in Wave 2 (np2HourlyWage) 7.34.3161229 MeanHourly wage young adult earns at current or most recent job in Wave 5 (np5T4h_L4h) 10.17.2552189 Results output for this demonstration These results cannot be replicated with full dataset; all output in modules generated with a random subset of the full data.

18. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 17 Mean Difference in Wages between Wave 2 and Wave 5 EstimateStandard Error Unweighted Count MeanChange in wages between Waves 2 and 5 (W25_Wage_chg) 3.40.404967 Results output for this demonstration These results cannot be replicated with full dataset; all output in modules generated with a random subset of the full data.

19. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 18 Percentage Who Had a Decrease or Increase in Wages between Waves 2 and 5 W25_Wage_chgCat Estimate Standard Error Unweighted Count % of Total(1) Decrease in wages (2) No change (3) Increase in wages Total 7.7% 4.6% 87.7% 100.0% 1.6% 2.9% 3.2%.0% 125 30 812 967 Results output for this demonstration These results cannot be replicated with full dataset; all output in modules generated with a random subset of the full data.

20. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 19 Mean Difference in Wages, by Household Income between Wave 2 and Wave 5 Household Income (W5_IncomeHdr2009)Estimate Standard Error Unweighted Count (1)$25,000 or lessMeanChange in wages between Waves 2 and 5 (W25_Wage_chg) 2.70.503194 (1)$25,001 - $50,000Mean(Change in wages between Waves 2 and 5 W25_Wage_chg) 3.62.539224 (1)More than $50,000 Mean(Change in wages between Waves 2 and 5 W25_Wage_chg) 3.63.762540 Results output for this demonstration These results cannot be replicated with full dataset; all output in modules generated with a random subset of the full data.

21. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 20 Percentage Decrease or Increase in Wages by Household Income between Waves 2 and 5 (W25_Wage_chgCat) (W5_IncomeHdr2009 Income) (1) $25,000 or less (2) $25,001– $50,000 (3) More than $50,000 Total (1) Decrease in wages% within (W5_IncomeHdr2009) Income category for tables Estimate4.6%6.8%10.5%7.7% Standard Error1.9%2.0%3.2%1.6% Unweighted Count273661124 (2) No change% within (W5_IncomeHdr2009) Income category for tables Estimate11.1%3.4%1.1%4.6% Standard Error9.3%1.9%.5%2.9% Unweighted Count861630 (3) Increase in wages% within (W5_IncomeHdr2009) Income category for tables Estimate84.3%89.8%88.4%87.6% Standard Error9.0%2.9%3.3%3.2% Unweighted Count159182463804 Total% within (W5_IncomeHdr2009) Income category for tables Estimate100.0% Standard Error.0% Unweighted Count194224540958 Results output for this demonstration These results cannot be replicated with full dataset; all output in modules generated with a random subset of the full data.

22. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 21 Interpretation of results for this demonstration Is it significant?  We use an Excel worksheet.  Enter (1) the percentage and standard error for a category and (2) the percentage and standard error for the comparison category.  The significance is calculated by a formula stored in Excel. These results cannot be replicated with full dataset; all output in modules generated with a random subset of the full data.

23. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 22 Interpretation of results for this demonstration Interpreting longitudinal results.  A positive number indicates a positive change between time 1 and time 2 for difference in wage.  A negative number indicates a negative change between time 1 and time 2.  Numbers close to 0 indicate little or no change. What does it mean?  A positive or negative change may be simple to interpret, but what does little or no change mean? What explains the change?  For example, postsecondary attendance might mean a change in employment.

24. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 23 Interpretation of results for this demonstration Was anything significant?  Not with the randomly selected subset of data.  The full data set might have a different result if the standard errors were smaller.  Or perhaps household income was not the best comparison variable to use. Things to think about  What other comparison variables might show significant differences between comparison groups?  Are there other difference in employment measures that might be worth exploring?  Were these the best waves of data to compare?  Should any types of youth be included or excluded from this analysis?

25. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 24 Interpretation of results for this demonstration More things to think about  Did we have a good comparison group? Would it have been better to select Waves 3 and 5 instead of 2 and 5 as more youth are out of secondary school in Wave 3 than in 2? Would it have been better to run the analysis on a subgroup of only those who were out of secondary school at each wave?  The answer is…it is important to select the appropriate subsample as well as to select the appropriate variables.

26. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 25 Review steps required for this demonstration Plan for analysis  Identify the research question.  Find the variables needed.  Determine if any variables need to recoded. Collapse categories. Create a longitudinal measure.  Identify the appropriate subsample if applicable.

27. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 26 Plan for analysis (cont’d)  Identify the appropriate procedures. Descriptive Crosstabs Means Means by comparison variables if calculating means Test significance of comparisons Review steps required for this demonstration

28. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 27 Closing Topics discussed in this module  Posing questions to the NLTS2 database  Comparative variables  Analysis demonstration using longitudinal data  Analysis plan for this demonstration  Results output for this demonstration  Interpretation of results for this demonstration  Review steps required for this demonstration

29. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 28 Closing Next module  14a. Accessing Data Files in SPSS or  14b. Accessing Data Files in SAS

30. 13. Demonstration: Descriptive/Comparative Analysis Using Longitudinal Data 29 Important information  NLTS2 website contains reports, data tables, and other project-related information http://nlts2.org/http://nlts2.org/  Information about obtaining the NLTS2 database and documentation can be found on the NCES website http://nces.ed.gov/statprog/rudman/http://nces.ed.gov/statprog/rudman/  General information about restricted data licenses can be found on the NCES website http://nces.ed.gov/statprog/instruct.asphttp://nces.ed.gov/statprog/instruct.asp  E-mail address: nlts2@sri.com