1. Developing a Severity-of-Disability Scale and Modeling Early Reading and Math Performance in a Longitudinal Study of Preschoolers with Disabilities Elaine Carlson Tamara Daley Frank Jenkins Westat The Pre-Elementary Education Longitudinal Study is funded by the U.S. Department of Education.

2. PEELS design 6-year national longitudinal study of preschoolers with disabilities, currently in its fourth year. Designed to describe  the children,  the services they receive,  their transitions from early intervention to preschool and preschool to elementary school, and  their early academic and functional skills. Sample includes 3,104 children who were ages 3-5 and were receiving special education services in 2003-04. Produces statistical estimates that generalize to the national population of children with disabilities ages 3-5.

3. PEELS design, cont’d Data collection  a one-on-one assessment of early academic skills,  a parent telephone interview,  mail questionnaires to children’s teachers and school-, district-, and state-level administrators, and  indirect assessments of students’ social skills, problem behaviors, academic skills, and motor skills

4. Sample design Nationally representative sample of 223 LEAs (208 in Wave 1, plus 15 added in Wave 2) LEAs stratified by  enrollment size  geographic region  wealth

5. LEA participation Main sample  709 released for recruiting  245 agreed in 2001  189 participated in 2003-04 (Wave 1) Nonresponse sample  32 of 464 non-recruited LEAs selected  19 participated in 2003-04 (Wave 1) Supplemental sample to add 1 previously nonparticipating SEA  15 of 24 participated in 2004-05 (Wave 2) Total Sample = 223 LEAs

6. Family participation Wave 1 main and nonresponse samples (ages 3-4 in 2003-04) 52,871 on sampling frame 5,330 selected 4,072 eligibility determined 2,906 agreed (81% of those eligible)

7. Family participation Wave 2 supplemental sample (ages 4-6 in 2004-05) 7,727 on sampling frame 542 selected 433 eligibility determined 198 agreed (63% of those eligible) Total sample = 3,104 families

8. PEELS schedule

9. Unweighted response rates, by wave Wave 123 SEA questionnaire 100%-- LEA questionnaire 84%-- Principal/program director questionnaire* 73%77%56% Teacher questionnaire 79%84% Parent interview 96%93%88% Child assessment 96%94%93% English direct assessment 84%87%88% Alternate assessment 11%7%5% - Not applicable * QED data were used to supplement these data, bringing percentage of children with some school data in waves 1, 2, & 3 to 94%, 95%, & 94%, respectively

10. Demographic characteristics Gender Male 70.7 Female29.3 Race/ethnicity American Indian/Alaska Native3.3 Asian3.2 Black or African American14.2 White84.6 Pacific Islander0.8 Hispanic, any race21.0 NOTE Demographic and household characteristics are for the unweighted sample. SOURCE: U.S. Department of Education, National Center for Special Education Research, Pre-Elementary Education Longitudinal Study (PEELS), “Parent Interview (Wave 1 for main and nonresponse samples, Wave 2 for supplemental sample),” previously unpublished tabulation.

11. Household characteristics Mother’s education <H.S diploma18.9 H.S. diploma or GED31.0 Some college29.4 4-year degree or higher21.0 Household Income $20,000 or less26.3 $20,000-$40,00028.3 >$40,00045.3 SOURCE: U.S. Department of Education, National Center for Special Education Research, Pre-Elementary Education Longitudinal Study (PEELS), “Parent Interview (Wave 1 for main and nonresponse samples, Wave 2 for supplemental sample),” previously unpublished tabulation.

12. Primary disabilities Speech or language impairment49.8 Developmental delay26.4 Autism6.7 Mental retardation4.0 Learning disability2.5 Other health impairment2.4 Orthopedic impairment1.6 Emotional disturbance1.1 Low incidence disability5.5 NOTE Low incidence includes visual impairment, hearing impairment, deaf-blindness, traumatic brain injury, multiple disabilities. SOURCE: U.S. Department of Education, National Center for Special Education Research, Pre-Elementary Education Longitudinal Study (PEELS), ‘Early Childhood and Kindergarten Teacher Questionnaires (Wave 1 for main and nonresponse samples, Wave 2 for supplemental sample),” previously unpublished tabulation.

13. Developing a Severity-of- Disability Scale

14. Severity Under IDEA, children classified according to one of 13 disability categories Children in each category have a range of abilities Heterogeneity within categories often overlooked Categorical labels especially problematic for preschoolers

15. Approaches to measurement Pediatric health conditions (PEDI, WeeFim)  Self-care, mobility, and social cognition  Require a certain threshold of impairment Single disability conditions in children  Hearing, vision, language, depression ICIDH (WHO, 1980)  Published as a research tool  Not widely adopted for use

16. Approaches to measurement ABILITIES Index (Simeonsson & Bailey, 1991): Severity in 9 domains  Audition  Behavior and social skills  Intellectual function  Limbs  Intentional communication  Tonicity  Integrity of physical health  Eyes  Structural status

17. Approaches to measurement (cont’d) Since its original publication, 6 new domains added (Simeonsson, 2006):  Regulation of attention,  Regulation of activity level, and  Regulation feeling/emotions,  Academic skills,  Motivation, and  Impulse control

18. ABILITIES index In Special Education Expenditures Project (SEEP) (Chambers et al., 2004) :  Federal disability category able to explain only 10% of the variation in total expenditures  In contrast, the ABILITIES Index alone was able to account for 40% of the variation in total educational expenditures for special education students  Including the federal disability categories increased the variance accounted for by 2%

19. Question of current study To what extent do functional markers of severity of childhood disability predict measurement of child outcomes?

20. Overall method Identify items from parent interview similar to ABILITIES domains Create composite domains where necessary Examine distribution of severity within the 15 domains Examine regressions of the 15 domains on multiple outcomes Create short version of index using 6 domains Compare 6 domain and 15 domain versions of index to outcomes to select better version Compare 6 domain version of index with other indicators of severity

21. Outcome measures Cognitive  PPVT-III  Woodcock-Johnson III: Letter-Word Identification  Woodcock-Johnson III: Applied Problems Social/Behavioral  PKBS Social Competence  PKBS Problem Behaviors Alternate Assessment  ABAS Conceptual Domain  ABAS Practical Domain  ABAS Social Domain Wave 1 data

22. Creating the severity measure Used parent report from Wave 1 CATI Created 4-point scale for each domain (normal/typical, mild, moderate, severe) 11 domains already with 4 point scales: 4 levels of severity 4 domains with 3 point scales: 1  1, 2  2, 3  4 to preserve range

23. Creating the severity measure Some domains matched to a single question  Use of arms (within limbs domain): “How well does {child} use {his/her} arms and hands for things like throwing, lifting, or carrying?”  Motivation: “Some children try to finish things, even if it takes a long time. How much does this sound like {child}… ” For domains with multiple questions, worked with ABILITIES Index author to best reflect original intention of domain

24. Example: Combining Items to Create ‘Communication with Others’ When {CHILD} talks to people {he/she} doesn’t know well, is {he/she} 1: Very easy to understand 2: Fairly easy to understand 3: Somewhat hard to understand 4: Very hard to understand 5: DOES NOT OR WILL NOT TALK AT ALL Compared with other children about the same age, how well does {CHILD} make {his/her} needs known to you and others? Communication can be any form, for example crying, pointing or talking. Would you say {he/she} 1: Communicates just as well as other children 2: Has a little trouble communicating 3: Has a lot of trouble communicating 4: Does not communicate at all? 1: Communicates just as well as other children and very easy to understand 2: Some difficulties communicating or being understood 3: Moderate difficulties communicating or being understood 4: Does not communicate at all or very hard to understand

25. Results Population weighted percentages (n = 2,986) Audition, vision, use of arms, legs, regulation of emotions  80-95% in normal/typical category Inappropriate or unusual behavior, overall health, social skills, use of hands, understanding  50-60% in normal/typical, 20-30% mild

26. Results Population weighted percentages (n = 2,986) Motivation, regulation of attention, regulation of activity level,  20-30% in normal/typical, 33-42% mild, 16- 30% severe Communicating with others  29% in normal/typical, 9% mild, 45% moderate, 17% severe Intellectual function  10% in normal/typical, 43% mild, 32% moderate, 14% severe

27. Regressions on outcome variables All predictors entered simultaneously; examined beta weights and bivariate correlations between predictors and each outcome Significant predictors of at least three outcomes: cognition, communicating with others, understanding, overall health, and regulation of activity level Significant predictor of two outcomes: regulation of attention The remaining severity domains generated a mixture of significant and nonsignificant associations No significant loading on any outcome variables: use of hands and arms and use of legs

28. Creation of indices Index A, 15 items: Sum of all domains Index B, 6 items: cognition, communicating with others, understanding, overall health, regulation of activity, regulation of attention No significant differences in correlations between Index A and outcomes and Index B and outcomes

29. Index A and B correlations with outcomes Index A 15 var * Index B 6 var* PPVT -.32-.36 Letter-Word Identification -.22-.26 Applied Problems -.40-.45 PKBS Social Skills Composite -.47-.43 PKBS Problem Behavior Composite.35-.35 ABAS Conceptual Domain -.53-.46 ABAS Practical Domain -.53-.43 ABAS Social Domain -.40-.35 *The associated p-values are less than.0001 for all coefficients.

30. Final severity measure items Cognition Communicating with others Understanding Overall health Regulation of activity Regulation of attention

31. Distribution of final severity measure

32. Validation: correlations between Index B and other indicators Wave 1 Parent report: Age at which children began receiving special education or therapy service r (2,802) = -.22, p <.0001 Wave 1 Teacher report: Amount of modification needed to curriculum materials r (248) =.42, p <.0001 Wave 1 Teacher report: Number of services the child receives in school r (2,014)=.37, p <.0001

33. Validation: comparison of mean scores on severity measure From teacher/parent declassification measure, Wave 1: Children remaining in special education M = 13.2 Children no longer receiving special education M = 10.7, p <.0001 From assessment measures, Wave 1: Children taking the alternate assessment M = 16.3 Children completing the direct assessment M = 12.4, p <.0001

34. Summary To what extent do functional markers of severity of childhood disability predict measurement of child outcomes?  Of the 15 domains examined, most were significant predictors of at least two outcomes  An index of only six variables was as effective as the longer version  Severity was significantly correlated with intervention variables  Severity differentiated children in two groups

35. Modeling Early Reading and Math Performance

36. Hierarchical analysis PEELS has children in naturally-occurring hierarchies:  time points within children  children within districts  districts

37. What we want to know  What factors relate to children’s cognitive growth over 3 years? What we have  observations of PEELS children over 3 years.  Yearly information about child’s SES, health, severity, and services received  Yearly measures of 3 academic outcomes  Adapted Peabody Picture Vocabulary Test (PPVT)  Woodcock-Johnson III: Letter-Word Identification  Woodcock-Johnson III: Applied Problems

38. Problem: How to make sense out of longitudinal data Hierarchical data is clustered: i.e., repeated measures are not independent observations. Standard regression assumes independent observations. Ordinary repeated measures analyses do not allow for missing time points or clustering. Repeated cross-sectional analyses ignore the growth of individual children. Mean growth is not the same as growth of individuals.

39. Solution: Hierarchical linear modeling Data is modeled at 3 levels of hierarchy at the same time. Most of the clustering in the sample is accounted for, leading to correct statistical tests. Focus is on individual growth profiles. Modeling seeks to explain differences in growth between children.

40. Hierarchical structure of data District 1 District n Districts S1S1 Si S1S1 Students W/in District T1 T2 T3 T1 T2 T3.. 3 time points per student.. T1 T2 T3 T1 T2 T3 Level 3 Level 2 Level 1

41. HLM model Level 1: Repeated observations within child is the outcome for individual i measured at wave t. is the age of the child at time at time t. is the growth intercept: average achievement for the individual is the growth curve slope: How much the outcome changes over years. is a random error term.

42. HLM model Level 2: Children nested within districts. is an intercept for child ji, is the individual growth curve for child ji to are child factors that predict growth. is the deviation from average achievement for each child. is the deviation from average growth for each child.

43. HLM model Level 3: Districts Where, is the achievement status of district j, is the grand mean of achievement, and is the district effect on achievement.

44. Hypothetical growth curve HLM Individual slopes estimated for high & low severity children <- Low severity <- High severity

45. Individual slopes estimated for high & low severity children <- Low severity <- High severity

46. Four Sets of Predictors Predictors are added in four sets of similar variables. These sets include  SES: e.g., mother’s education, SES scale  Severity: e.g., severity scale, age services started  Health: e.g., child’s general health, health scale  Services: e.g., % time in a regular classroom, parent involvement scale

47. Predictors of PPVT growth Accounted for 22 % of PPVT growth Note: Controlled for age and cohort PredictorFactorEffectProb Intercept229.55.00 Slope54.24.00 # Years in special educationSeverity-1.37.01 Severity of disability scaleSeverity-.78.00 Ease of transitionService-1.85.01 % Time in a regular classroomService.03

48. Predictors of Letter-Word growth Accounted for 57 % of Letter-Word growth Note: Controlled for age and cohort PredictorFactorEffectProb Intercept318.18.00 Slope37.76.00 Years in high poverty schoolSES2.10.00 Mother had some education after HSSES2.92.00 # Years in special educationSeverity-1.62.00 Age services startedSeverity.09.02 Severity of disability scaleSeverity-.69.00 Ease of transitionService-2.61.00 % Time in regular classroomService.08.00 Parent involvement scaleService1.49.03

49. Predictors of Applied Problems growth PredictorFactorEffectProb Intercept393.26.00 Slope24.00.00 Household IncomeSES-1.26.03 Child is HispanicSES1.53.03 Mother had some education after HSSES1.42.02 Parent SES scaleSES1.38.01 Problems with healthHealth2.50.00 Health scaleHealth1.18.05 Child’s general healthHealth2.49.00

50. Predictors of Applied Problems growth (continued) Accounted for 12 % of Applied Problems growth Note: Controlled for age and cohort PredictorFactorEffectProb # Years in special education Severity -1.02.00 Age services started Severity.09.00 Severity of disability scale Severity -.24.02 % Time in regular classroom Service.03.00 Parent involvement scale Service.75.06

51. Summary Scores increased from 1½ to 2 standard deviations over 3 years SES, severity, health, & service predictors accounted for 12% to 57% of growth Percent of growth accounted for PPVT22% Letter-Word57% Applied Problems12%

52. Summary (cont.) Service-related predictors of growth  % Time in a regular classroom Positively related to growth for all outcomes  Parent involvement Positively related for 2 of 3 outcomes (Letter-Word & Applied Problems)  Ease of transition Positively related for 2 of 3 outcomes (PPVT & Letter-Word)

53. Summary (cont.) Other predictors of growth  # Years child was in special education Predicted lower growth for all outcomes  Parent’s rating of severity Predicted lower growth for all outcomes  Mother’s education Positively related to growth for 2 of 3 outcomes (Letter-Word & Applied Problems)

54. WEBSITE: WWW.PEELS.ORGWWW.PEELS.ORG