Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State & Chris Schatschneider, Carol Connor & Stephanie Al Otaiba

Expanding our search for moderators of intervention A little about me – Behavioral genetics background – Interested to move these findings to schools Even with modest effect sizes, individual differences in intervention response Bioecological model (Bronfenbrenner & Ceci, 1994) – Provides framework for differentiating students based on non-intervention related traits Lets follow individualized medicine

Integrative Data Analysis (IDA) Item-level pooled data (Curran & Hussong, 2009) Capitalizes on cumulative knowledge – Longer developmental time span – Increased statistical power – Increased absolute numbers in tails Controls for heterogeneity – Sampling, age/grade, cohort, geographical, design, measurement

Project KIDS Expanded definition of moderators of response to intervention – Cognitive, psychosocial, environmental, familial/genetic risk IDA across 9 completed intervention projects – Approximately 5600 kids Data entry of item level data common across at least 2 projects – ~30 different assessments Questionnaire data collection

Proof of Concept Behavior problems and achievement are associated More behavior problems are typically seen in LD populations Is adequate vs inadequate response status differentiated by behavior problems?

Method Participants – ISI intervention through FL LDRC (Al Otaiba et al., 2011) RCT: 23 treatment, 21 contrast teachers 556 kindergarteners A2i recommendations vs enhanced standard practice – RTI Intervention through FL LDRC (Al Otaiba et al., 2014) RCT: 34 classrooms, kids randomized st graders regular RTI vs dynamic RTI – ISI intervention project (Connor et al., 2007) RCTish : 22 treatment, 25 contrast teachers, 3 pilot 821 first graders A2i recommendations vs standard practice

Method Measures – WJ Tests of Achievement Letter-Word Identification (LWID) Pre- and post-intervention testing periods – Social Skills Rating Scale: Behavioral Problems subscale Teacher completed during intervention year 07/08 K ISI LDRC Mean (SD) 09/10 1 st RTI LDRC Mean (SD) 05/06 1 ISI Mean (SD) WJ LWID Fall12.03 (5.50)26.66 (9.03)24.28 (7.97) WJ LWID Spring21.75 (7.09)38.73 (8.07)36.54 (7.39) SSRS.50 (.44).34 (.39).53 (.44)

r=.89 r=-.23 r=-.26

Results: Calibration LWID IDA Randomly selected 1 time point/child/project to form “calibration sample” for LWID Decision to include only items > 5% endorsement rate Reduced item sample from 75  38 – Items 11 to 49

Results: Calibration LWID IDA Hahahaahaaa.

Results: Calibration LWID IDA Non-linear factor analysis – Multiple group analysis to arrive at single factor for LWID – Using measurement (in)variance principles across the 3 projects, using modindices in Mplus to adjust for project variant differences Key decision: We wanted to constrain based on meaningful differences – Chi-square difference value set to equal Cohen’s d =.20

Metric (in)variance -localized misfit for items 32, 35, 36, 37, 39, & 46-49

Scaler (in)variance -localized misfit for thresholds of items 46, 47 & 49

Residual variance (in)variance -all could be constrained to equal

Factor variance (in)variance -All could be constrained to be equal

Final Model -using all data, set parameters based on final factor model, exported factor scores

Results: SSRS

Results r =.97r =.98

Results r =.96

Now I have equivalent data!!! It’s a lot of steps to get a regular old data set Now I can answer content questions, but with more kids in a more generalizable sample So, are behavior problems bad for treatment response?

Results: Response Proc mixed: covariance adjusted LWID score – 1712 children

Results: Response 818 treatment children

Results: Response 818 treatment children Unresponsive Cutoff < 20% N=164!

Results Logistic regression – SSRS behavior problems significant predictor of response status (OR = 1.58, CI = ) average behavior problems = 41% probability of being “unresponsive” greater than average behavior problems(+ 1SD) = 50% probability of being “unresponsive” Less than average behavior problems (-1SD) = 34% probability of being “unresponsive”

Conclusions Response status is differentiated by behavior problems – Mo’ behavior problems, mo’ (reading) problems!

Overall IDA conclusions IDA is a “cheap” way to get more power, more n at tails, and show more generalizable effects Given how similar many of our projects are, consider doing item-level data entry – Easy potential to combine data – Can you do factor analysis and IRT? You can do IDA*! These data are more useful together than apart – IRT within and between samples? – Treatment effectiveness across samples? – Characteristics of lowest responders?

Project KIDS goals Intervention response status is known Can we predict responders/non-responders with questionnaire data? – Family history 1 st degree vs 2 nd degree, dosage – Cognitive correlates Executive functioning, ADHD – Behavior Comorbid behavior issues – Environments Home literacy environment vs neighborhood vs school Individualize instruction based on child traits – No cheek swab needed

Acknowledgements Stephanie Al Otaiba Carol Connor Chris Schatschneider Great staff & grad students, and many wonderful data enterers NICHD grant HD072286

r =.95

r =.78 r =.88

What about without DIF? r =.97 r =.99