Independent versus Computer-Guided Oral Reading: Equal-time Comparison of Sustained Silent Reading (SSR) to an Automated Reading Tutor that Listens Jack Mostow, Greg Aist, Juliet Bey, Paul Burkhead, Andrew Cuneo, Brian Junker, Susan Rossbach, Brian Tobin, Joe Valeri, and Sara Wilson Project LISTEN, Carnegie Mellon University www.cs.cmu.edu/~listen Thanks to: LISTENers, NSF, schools

Outline of talk Project LISTEN’s Reading Tutor Study design Results Conclusions

Project LISTEN’s Reading Tutor (video) John Rubin (2002). The Sounds of Speech (Show 3). On Reading Rockets (Public Television series commissioned by U.S. Department of Education). Washington, DC: WETA. To be broadcast in fall 2002.

Project LISTEN’s Reading Tutor (video) Rubin, John (2002). The Sounds of Speech (Show 3). On Reading Rockets (Public Television series commissioned by U.S. Department of Education). Washington, DC: WETA. To be broadcast in fall 2002.

Can computer-guided oral reading beat independent practice Can computer-guided oral reading beat independent practice? Study design Treatments (6 months) Sustained Silent Reading 20 minutes/day in classroom Current practice Reading Tutor 20 minutes/day in lab During SSR – equal-time comparison! Students (N = 193, grades 1-4) Two suburban Blue Ribbon Schools of Excellence Selection: anyone in grade 1, low readers in grades 2-4 Excluded students seeing Reading Specialist Random assignment: matched within class by pretest

Sustained Silent Reading in a classroom

Using the Reading Tutor in a lab

Study design: measures Pre-test in September 2000, post-test in March-April 2001 Woodcock Reading Mastery Test: word identification, word attack, word comprehension, passage comprehension Comprehensive Test of Phonological Processing: blending, elision, rapid letter naming (Wagner, Torgesen, & Rashotte) Fluency (also Jan., May): words per minute on grade-level text Test of Written Spelling (Larsen, Hammill, & Moats) Elementary Reading Attitude Survey (McKenna, Kear, & Ellsworth) Teacher surveys: class activities, individual students Site visit reports: attrition, treatment fidelity Reading Tutor recorded: logs, oral reading, opinion polls

Fidelity to study design: diluted but still equal-time Attrition Sustained Silent Reading: 10 of 98 left study Reading Tutor: 7 of 95 left study Sustained Silent Reading implementation (from site visits) Occasional deviations inevitable in practice Reading Tutor usage (from November–March logs) Design: 20-minute sessions; actual: 12-24 (average 16.5) Count only time actually logged in to Reading Tutor Design: daily sessions: actual: 34-82% (average 68%) E.g. when lab monitor absent, stay in class for SSR instead Omit 335 sessions (2%) not matched to known students Design: 40 hours; actual: 10-30 hours (average 18)

Statistical analysis Outcome variable = pre- to post-test gain Method = ANOVA Treatment and gender as fixed effects Class as a random effect Significant pretests as covariates Results Significance (p) for estimated effects of treatment Effect size (ES) = difference in estimated mean gains / average standard deviation

Whom did the Reading Tutor help most Whom did the Reading Tutor help most? Between-treatment differences in gains Everyone Word identification: p=.03, ES=.19 Boys Word comprehension: p=.04, ES=.42 Spelling: p=.02, ES=.57 1st graders (40% of sample) Word comprehension: p=.02, ES=.51 Passage comprehension: p=.05, ES=.38 Phonological awareness (blending + elision): p=.03, ES=.55 Fluency: p=.02, ES=.64 Spelling: p=.01, ES=.72

Conclusions NRP: (human-)guided oral reading helps children This study: so can machine-guided oral reading Reading Tutor beat Sustained Silent Reading on multiple skills In barely 16 minutes per day Despite excellent classroom instruction Next: What helps which students and skills? Analyze data from 2001-2002 studies (N=654) See publications at www.cs.cmu.edu/~listen Questions?

What the Reading Tutor does Reader can: log in/out pick story read/write get help go on/back Reading Tutor: listens helps tries again goes on

Reading Tutor session Log in by picking (or adding) name on menu. Read or write stories Student and Reading Tutor take turns picking stories. Reading Tutor may insert preview and/or review. Story step types = read, listen, spell, edit, pick. until Goodbye or timeout (and maybe log back in).

Which Reading Tutor actions help which students and skills? Relate word ID, fluency gains to time allocation Partial correlation controls for pretest scores R=+.44 with % time reading (p<.05) R=-.45 with % time picking stories (p<.05) Embed experiments in the Reading Tutor itself Compare ways to introduce a new word Ask students to rate stories they finish Poll students about reading and the Reading Tutor