Christopher Schatschneider Supporting Effective Instruction: Technology and Professional Development Carol McDonald Connor Frederick J. Morrison Christopher Schatschneider Barry Fishman

Coming to PD through the back door Individualizing Student Instruction Study RFT to test the impact of child characteristics-by-instruction interactions on student outcomes Aptitude by treatment interactions

Designing the PD Protocol Used the research base to develop a “state of the art” professional development protocol, delivered to all treatment group teachers We assumed that, with our coaching, the teachers would be able to implement ISI in the first experimental year with enough fidelity so that we could test our hypotheses most PD research suggests that it takes 2 to 3 years for teachers to fully implement a new practice

What was the Intervention? Created algorithms based on the HLM results in our Beyond the Reading Wars paper Assessment to Instruction (A2i) software created to compute recommended amounts and types of instruction for each child Embedded in planning software design to scaffold research-based reading practices

Multiple Dimensions of Instruction Teacher Managed Child Managed Code Focused Alphabet activities Letter Sight-Sound Phonological Awareness Onset-rime, blending and segmenting Word Segmentation Spelling phonics worksheets, handwriting activities decoding activities Meaning Focused Vocabulary Teacher Read Aloud Student Read Aloud, Choral Group Writing, Writing Instruction, Model Writing Listening Comprehension Discussion Student Buddy Reading Sustained Silent Reading Reading Comprehension worksheets Student Individual Writing Pair writing HLM to computer

Algorithm Results TM-CF TMCFa = ((End of Year Target - (.2* LW GE))/(.05 + (.05 * LW)))+13. TMCF_Recommended = (TMCFa - (.82 * Month )).

Algorithm Results for CM-MF CMMFa = ((3.76 – End of year target + (1.4 * Vocabulary AE))/(.30)) - 14. CMMFsl = 10-(.24*CMMFa). CMMF_Recommended= CMMFa + .5*(CMMFsl * Month).

PD Protocol Focused on using A2i to plan instruction and then on implementing the recommended amounts for each child in the classroom Grouping small group rather than whole class using A2i recommended groupings (homogeneous) Classroom organization Effective child-managed activities Individualizing time, content, and delivery no one way to individualize instruction Research-based practice

Mentor or Coaching Model “Research Partners” classroom based – 2 hours bi-weekly School level meetings Individual meetings Individualized Same amount of time Content and focus varied

Did the PD work? Evidence from child outcomes Evidence from A2i software tracking Evidence from changes in teacher knowledge Evidence from classroom observations

Evidence from Child Outcomes HLM - Treatment versus Control Student Reading Comprehension Outcomes Mean scores controlling for fall vocabulary, passage comprehension, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0, n = 616 students

Evidence from Software Tracking A2i Use and Reading Comprehension AE = 8.2 years AE = 6.0 years HLM fitted growth curves controlling for fall vocabulary, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0,

Evidence from changes in Teacher Knowledge Teacher Knowledge Test Descriptives and Reliability Assesses teachers’ understanding of English phonology, orthography, and morphology, and concepts of literacy acquisition and instruction 34 multiple choice and 11 short answer items Administered fall and spring Cronbach’s alpha = .87. Teachers in the treatment group had significantly higher spring TKS scores compared to control teachers Controlling for school SES status, other PD opportunities, teacher credentials and fall TKS scores standardize beta = .40 A2i and TKS correlations Fall TKS and A2i total use did not correlate Spring TKS and A2i correlation = .58

Importance of Teacher Knowledge Scores on the TKS ranged from 9 to 36 (M = 23.45, SD = 7.27). First Graders – end of year grade equivalent score of 1.9 = 428

Evidence from Classroom Observations Child-managed Pair 4.1. Literacy Codes: 4.1.2. Phoneme Awareness 4.1.3. Syllable Awareness 4.1.4. Morpheme Awareness 4.1.5. Onset/Rime Awareness 4.1.6. Word ID/Decoding 4.1.7. Word ID/Encoding 4.1.8. Fluency 4.1.9. Print Concepts 4.1.10. Oral Language 4.1.11. Print Vocabulary 4.1.12. Reading Comprehension 4.1.13. Text Reading 4.1.14. Writing 4.1.15. Library 4.1.16. Assessment 4.1.2. Phoneme Awareness 4.1.2. Phoneme Awareness 4.1.2.2. Blending 4.1.2.3. Elision/Initial 4.1.2.4. Elision/Final 4.1.2.5. Elision/Vowel 4.1.2.6. Elision/Medial 4.1.2.7. Substitution/Initial 4.1.2.8. Substitution/Final 4.1.2.9. Substitution/Vowel 4.1.2.10 Substitution/Medial 4.1.2.11 Segmenting/Counting Here is a screenshot of our coding software, and, to give you a sense of the detail with which we are able to code, here is a list of our instructional categories. Each category is further broken down into specific activities. For example,…

TCM Small-group Code-focused

Teacher-Managed Instruction Small Group Descriptive Statistics treatment Mean Std. Deviation N TM-Whole Class-Code Focused .00 .0000 .00000 227 1.00 .0000 .00012 186 Total .0000 .00008 413 TM-Whole Class-Meaning Focused .00 .2739 1.14832 227 1.00 .5086 1.29146 186 Total .3796 1.21898 413 TCM-Whole Class-Code Focused .00 9.4729 8.54112 227 1.00 14.1571 13.77630 186 Total 11.5825 11.43157 413 TCM-Whole Class-Meaning Focused .00 20.3966 19.69005 227 1.00 21.7216 12.53186 186 Total 20.9933 16.84113 413 Whole Class

Child Managed Instruction

Winter Observed – A2i recommended amounts Distance From Recommendation Absolute Values * Simple Differences

Effect of Distance from Recommendations Winter TM-CF DFR Winter CM-MF DFR 464 = 1.8, 470 = 2.1 about a 3 month difference in GE 450 = 1.9, 458 = 2.5 .6 GE or about a 5 and a half month difference in GE 464 = 1.8, 470 = 2.1 or about a 3 month difference in GE

HLM - DFR predicting student outcomes Cumulative fall, winter and spring DFR for TM-CF and CM-MF DFR Treatment teachers < DFR Control Teachers Total Amounts of instruction did not predict student outcome growth (residualized change) Cumulative TM-CF and CM-MF DFR negatively predicted both Passage Comprehension and Letter-Word Identification residualized change TM-CF DFR amount and change fall to spring CM-MF DFR amount

Discussion Finding effective ways to change teacher practice quickly are critical For Random Field Trials Improving student learning Why did our PD work? Practice-based PD versus General Knowledge Focus on A2i, which was designed to scaffold the kinds of instruction research suggests are more effective in improving students’ reading skills Did the technology really make a difference? We don’t know – and will need to conduct a RFT to find out