1. Independent versus Computer-Guided Oral Reading:
Equal-time Comparison of Sustained Silent Reading (SSR) to an Automated Reading Tutor that Listens
Jack Mostow, Greg Aist, Juliet Bey, Paul Burkhead, Andrew Cuneo, Brian Junker, Susan Rossbach, Brian Tobin, Joe Valeri, and Sara Wilson
Project LISTEN, Carnegie Mellon University
Thanks to: LISTENers, NSF, schools

2. Outline of talk Project LISTEN’s Reading Tutor Study design Results
Conclusions

3. Project LISTEN’s Reading Tutor (video)
John Rubin (2002). The Sounds of Speech (Show 3). On Reading Rockets (Public Television series commissioned by U.S. Department of Education). Washington, DC: WETA.
To be broadcast in fall 2002.

4. Project LISTEN’s Reading Tutor (video)
Rubin, John (2002). The Sounds of Speech (Show 3). On Reading Rockets (Public Television series commissioned by U.S. Department of Education). Washington, DC: WETA.
To be broadcast in fall 2002.

5. Can computer-guided oral reading beat independent practice
Can computer-guided oral reading beat independent practice? Study design
Treatments (6 months)
Sustained Silent Reading 20 minutes/day in classroom
Current practice
Reading Tutor 20 minutes/day in lab
During SSR – equal-time comparison!
Students (N = 193, grades 1-4)
Two suburban Blue Ribbon Schools of Excellence
Selection: anyone in grade 1, low readers in grades 2-4
Excluded students seeing Reading Specialist
Random assignment: matched within class by pretest

6. Sustained Silent Reading in a classroom

7. Using the Reading Tutor in a lab

8. Study design: measures
Pre-test in September 2000, post-test in March-April 2001
Woodcock Reading Mastery Test: word identification, word attack, word comprehension, passage comprehension
Comprehensive Test of Phonological Processing: blending, elision, rapid letter naming (Wagner, Torgesen, & Rashotte)
Fluency (also Jan., May): words per minute on grade-level text
Test of Written Spelling (Larsen, Hammill, & Moats)
Elementary Reading Attitude Survey (McKenna, Kear, & Ellsworth)
Teacher surveys: class activities, individual students
Site visit reports: attrition, treatment fidelity
Reading Tutor recorded: logs, oral reading, opinion polls

9. Fidelity to study design: diluted but still equal-time
Attrition
Sustained Silent Reading: 10 of 98 left study
Reading Tutor: 7 of 95 left study
Sustained Silent Reading implementation (from site visits)
Occasional deviations inevitable in practice
Reading Tutor usage (from November–March logs)
Design: 20-minute sessions; actual: (average 16.5)
Count only time actually logged in to Reading Tutor
Design: daily sessions: actual: % (average 68%)
E.g. when lab monitor absent, stay in class for SSR instead
Omit 335 sessions (2%) not matched to known students
Design: 40 hours; actual: hours (average 18)

10. Statistical analysis Outcome variable = pre- to post-test gain
Method = ANOVA
Treatment and gender as fixed effects
Class as a random effect
Significant pretests as covariates
Results
Significance (p) for estimated effects of treatment
Effect size (ES) = difference in estimated mean gains / average standard deviation

11. Whom did the Reading Tutor help most
Whom did the Reading Tutor help most? Between-treatment differences in gains
Everyone
Word identification: p=.03, ES=.19
Boys
Word comprehension: p=.04, ES=.42
Spelling: p=.02, ES=.57
1st graders (40% of sample)
Word comprehension: p=.02, ES=.51
Passage comprehension: p=.05, ES=.38
Phonological awareness (blending + elision): p=.03, ES=.55
Fluency: p=.02, ES=.64
Spelling: p=.01, ES=.72

12. Conclusions NRP: (human-)guided oral reading helps children
This study: so can machine-guided oral reading
Reading Tutor beat Sustained Silent Reading on multiple skills
In barely 16 minutes per day
Despite excellent classroom instruction
Next:
What helps which students and skills?
Analyze data from studies (N=654)
See publications at
Questions?

13. What the Reading Tutor does
Reader can:
log in/out
pick story
read/write
get help
go on/back
Reading Tutor:
listens
helps
tries again
goes on

14. Reading Tutor session Log in by picking (or adding) name on menu.
Read or write stories
Student and Reading Tutor take turns picking stories.
Reading Tutor may insert preview and/or review.
Story step types = read, listen, spell, edit, pick.
until Goodbye or timeout (and maybe log back in).

15. Which Reading Tutor actions help which students and skills?
Relate word ID, fluency gains to time allocation
Partial correlation controls for pretest scores
R=+.44 with % time reading (p<.05)
R=-.45 with % time picking stories (p<.05)
Embed experiments in the Reading Tutor itself
Compare ways to introduce a new word
Ask students to rate stories they finish
Poll students about reading and the Reading Tutor