1. 6. Implications for Analysis: Data Content

2. 1 Prerequisites Recommended modules to complete before viewing this module  1. Introduction to the NLTS2 Training Modules  2. NLTS2 Study Overview  3. NLTS2 Study Design and Sampling  For parent/youth data sources: 4. NLTS2 Data Sources: Parent and Youth Surveys  For school or student data sources: 5. NLTS2 Data Sources: School Surveys, Student Assessments, and Transcripts

3. 6. Implications for Analysis: Data Content 2 Overview  NLTS2 sample  Multiple data sources  NLTS2 data collection  Availability of data  Data discrepancies  Missing data  Response rates  Restricted-use NLTS2 data  Closing  Important information

4. 6. Implications for Analysis: Data Content NLTS2 restricted-use data NLTS2 data are restricted. Data used in these presentations are from a randomly selected subset of the restricted-use NLTS2 data. Results in these presentations cannot be replicated with the NLTS2 data licensed by NCES. 3

5. 6. Implications for Analysis: Data Content 4 NLTS2 sample Sampled in two stages  First stage: Local education agencies (LEAs) and state- operated special schools.  Second stage: Students sampled to represent those in the 12 federally defined disability categories. Students were 13 to 16 in grade 7 or above, and in middle or high school. Complex sampling has implications for weighting.

6. 6. Implications for Analysis: Data Content 5 Multiple data sources Parent/youth survey  A parent interview or mail survey questionnaire completed by youth’s parent or guardian  A youth interview or mail survey questionnaire completed by youth School characteristics survey  School-level information about the school that the student attended Teacher survey  A general education academic subject teacher about student’s experiences in general education academic classroom Student’s school program survey  School staff member who knew about student’s program, including transition and vocational education Student assessments  Direct assessment of student’s scholastic abilities  Student interview on attitudes about school, self-determination  Alternate Assessment completed by student’s teacher if student was unable to participate in the Direct Assessment and interview Secondary school transcripts

7. 6. Implications for Analysis: Data Content 6 Data sources Although data were collected from many sources (parents, school administrators, teachers, youth), the unit of analysis is the individual youth.  In a given wave, a single teacher may have responded to multiple surveys for students within the teacher’s school or classroom, but each survey is in reference to a single student. Generalizations cannot be made about families, communities, LEAs, schools, teachers, or classrooms.  Although many youth may have attended any single school, data from the School Characteristics Survey are stored in a youth-level record not a school-level record.

8. 6. Implications for Analysis: Data Content 7 NLTS2 data collection Parent/guardian interviews/surveys  Collected in 5 waves Youth interviews/surveys  Collected in 4 waves School data  Collected in 2 waves Student assessments  Assessed once when youth was 16 years or older Multiwave collection resulting in one file included with Wave 2 data Secondary school transcripts  Collected for the school years when youth in secondary school included with Wave 5 data

9. 6. Implications for Analysis: Data Content 8 NLTS2 data collection: Timeline Wave 1Wave 2Wave 3Wave 4Wave 5 Year 1 2000- 2001 Year 2 2001- 2002 Year 3 2002- 2003 Year 4 2003- 2004 Year 5 2004- 2005 Year 6 2005- 2006 Year 7 2006- 2007 Year 8 2007- 2008 Year 9 2008- 2009 Year 10 2009- 2010 Parent telephone interviews or surveys Youth telephone interviews or surveys Direct assessment and in-person interviews Teacher Survey Student’s School Program Survey School Characteristics Survey Transcripts

10. 6. Implications for Analysis: Data Content 9 Availability of data Not all sources of data are available for a given youth.  Within a wave, a youth may have one or more sources of data, but not necessarily all sources. For example, there may be a Wave 1 Parent Interview for a youth but no Wave 1 school-level data or vice versa.  Across waves, a youth may have data for a given source in one wave and not another. For example, there may be a Wave 1 Parent Interview and a Wave 5 Parent/Youth Interviews but no Wave 2 through 4 Parent/Youth data.

11. 6. Implications for Analysis: Data Content 10 Data discrepancies Similar items in multisource, longitudinal data do not always agree.  For example, all youth in the sample were in special education in 2000. However Some parents indicated that their son or daughter was never in special education. Parents reported disabilities other than those that the district and/or school has reported.  A parent may have reported the youth dropped out in one wave as well has having reported he or she graduated from high school in another wave.

12. 6. Implications for Analysis: Data Content 11 Data discrepancies A secondary school transcript may report a different high school leaving date than reported in a parent/youth interview. A parent and a youth may provide conflicting responses on such things as high school leaving status, postsecondary attendance, or employment status.  Example: Parent/guardian and youth responses for why youth left secondary school may differ.  Example: Parent/guardian and youth responses for whether youth attended a 2-year/junior college may differ.

13. Implications for Analysis: Data Content Data discrepancies: Examples 12 These results cannot be replicated with full dataset; all output in modules generated with a random subset of the full data.

14. 6. Implications for Analysis: Data Content 13 Missing values There are many reasons for data to have missing values, which are an unavoidable part of survey work. Data can be completely missing for a given source.  The appropriate respondent to complete the survey could not be located. Example: School had no record of student attending that school. Example: Parent had moved and we no longer had contact information to reach the family. Example: Foster parent no longer had youth in his or her care.  Respondent refused to complete a questionnaire or participate in an interview.  Youth attended a known school but school did not compile a transcript for that youth.

15. 6. Implications for Analysis: Data Content Missing values Respondent may have had data for a given data source, but data can be missing within a file for that source.  Respondent did not know or refused to answer a question.  The question was not applicable.  The question was skipped by design.  Respondent missed a page in a questionnaire.  Respondent terminated the interview before it was completed.  Respondent completed a different version of an instrument that does not have that particular item. For example, respondent completed the mailed Family Survey in lieu of the Parent/Guardian interview. 14

16. 6. Implications for Analysis: Data Content 15 Missing values Pay attention to the number of respondents when conducting any analysis.  Across waves, the number of respondents for a given data source will vary.  Within a wave, the number of respondents for each data source will be different from file to file.  Within a data collection instrument, n’s will vary item by item.

17. 6. Implications for Analysis: Data Content 16 Missing values Longitudinal and/or multisource analyses are affected by missing values.  The more sources of data that are being combined, the larger the likelihood that data will be missing for at least one of those sources.  The number of respondents is typically smaller for cross- instrument or cross-wave analysis than for analysis of a single instrument. For example, an analysis that requires a respondent to have all 5 waves of Parent/Youth Survey data will have a smaller number of respondents than any single wave of Parent/Youth Survey data.

18. 6. Implications for Analysis: Data Content 17 Response rates Data collected from multiple sources and multiple points in time generated different response rates. Generally there were higher response rates in earlier waves than in later waves.  Lower response rate between Wave 1 Parent and Wave 2 Parent/Youth largely due to attrition. Combining data from multiple sources can result in a smaller n if requiring that all respondents have all sources.

19. 6. Implications for Analysis: Data Content 18 Response rates Parent/Youth Surveys W1 Parent Survey82% W2 Parent/Youth Survey61% W3 Parent/Youth Survey50% W4 Parent/Youth Survey50% W5 Parent/Youth Survey48% School Surveys W1 (Teacher, Program, School)36%,53%, 57% W2 (Teacher and Program)41%, 52% Student Assessments W1 Administration63% W2 Administration72% Transcripts [multiple waves]81%

20. 6. Implications for Analysis: Data Content 19 Restricted-use NLTS2 data NLTS2 Web Seminar data  NLTS2 data and statistical output are restricted and require licensure for use.  Data used in these presentations are a random subset of the full NLTS2 database.  Results cannot be replicated with fully licensed data as demonstrations and examples are generated using a random subset of the data. If you plan to use NLTS2 data, you will need to obtain a license through NCES.  http://nces.ed.gov/statprog/rudman/

21. 6. Implications for Analysis: Data Content 20 Closing Topics discussed in this module  NLTS2 sample  Multiple data sources  NLTS2 data collection  Availability of data  Data discrepancies  Missing data  Response rates  Restricted-use NLTS2 data

22. 6. Implications for Analysis: Data Content 21 Closing Next module for Parent/Youth data sources:  7. Implications for Analysis: Parent/Youth Survey Data Next module for school or student data sources:  8. Implications for Analysis: School Survey, Student Assessment, and Transcript Data

23. 6. Implications for Analysis: Data Content 22 Important information  NLTS2 website contains reports, data tables, and other project-related information http://nlts2.org/http://nlts2.org/  Information about obtaining the NLTS2 database and documentation can be found on the NCES website http://nces.ed.gov/statprog/rudman/http://nces.ed.gov/statprog/rudman/  General information about restricted data licenses can be found on the NCES website http://nces.ed.gov/statprog/instruct.asphttp://nces.ed.gov/statprog/instruct.asp  E-mail address: nlts2@sri.com