1. Micro-analysis of Fluency Gains in a Reading Tutor that Listens:
Jack Mostow and Joseph Beck
Project LISTEN (
Carnegie Mellon University
Society for the Scientific Study of Reading
12th Annual Meeting, June, 2005
Funding: National Science Foundation, Heinz Endowments 1

2. Wide vs. repeated guided oral reading
Guided oral reading builds fluency [NRP 00].
Typically repeated oral reading
Is repeated reading better than wide reading?
Unclear! [Kuhn & Stahl JEP 03]
Past work analyzed reading rates on passages and word lists.
E.g. scramble word order to study context effects [Levy, …]
This talk: use finer-grained Reading Tutor data. 2

3. Project LISTEN’s Reading Tutor: Rich source of guided oral reading data
database:
8 diverse schools
600 students (K-6)
26,000 sessions
600,000 sentences read
4 million words heard
Get hours from Joe 3

4. Reading speeds up with practice: example
Initial encounter of muttered:
I’ll have to mop up all this (5630 ms) muttered Dennis to himself but how
5 weeks later (different word pair in different sentence):
Dennis (110 ms) muttered oh I forgot to ask him for the money
Word reading time = latency + production time  1/fluency
Beck et al. [TICL 04] used latency to assess proficiency (R2>.8).
What predicts word reading time? 4

5. Proficient readers are faster.5

6. Long words are slower. 6

7. Reading time speeds up over successive encounters, but by less and less.7

8. Linear model of word reading speedup
Predicted variable is speedup on the same word
Reduction in word reading time
From one (“practice”) encounter of a word
To the next (“test”) encounter
N = 243,172 opportunities for speedup
By 352 students (gr 1-6) with WRMT pretest scores
Include rereading as practice but not as test
Exclude encounters after the first 8
Exclude 36 stop words (the, a, …) 8

9. Predictors of speedup What is the student’s reading level?
How many letters long is the word?
How often has the student seen the word before? 9

10. Contextual predictors
Has the student seen the practice sentence before?
(Students have not seen test sentences before.)
Has the student seen the practice word pair before?
Has the student seen the test word pair before? 10

11. Results Higher readers speed up less: 3 ms less per grade level.
Longer words speed up more: 2.4 ms more per letter.
Speedup averages 18 ms per encounter (for the first seven).
Speedup averages 30 ms after the first encounter, then 3 ms less after each subsequent encounter. 11

12. Context effects
A new practice sentence helps 27 ms more than an old one.
Wide reading beats rereading!
A new practice word pair helps 21 ms more.
A test word pair seen before is 13 ms faster. 12

13. Are these effects statistically reliable?
Standard errors are 0~3 ms.
But: Linear model ignores dependencies
Within student (other than proficiency)
Within word (other than length)
So: Compute p by counting N students, not encounters.
Restrict to students with 50+ encounters in each condition
Compare how many students do better in one condition vs. the other.
Use sign test to compute significance.
New practice sentences beat old ones for 100 of 135 students (p = .000).
No marked differences between groups
New practice word pairs beat old ones for 151 of 222 students (p = .000). 13

14. Does word speedup relate to test score gains?
In contrast, # encounters doesn’t predict gains (R2<.001). 14

15. What affects fluency growth?
Practice on specific words
Especially in new sentences
Practice on specific word pairs
Helps if tested, but new practice pairs are better
Practice in decoding new words
Not analyzed in this talk 15

16. Conclusions Wide guided oral reading beats repeated reading!
Correlational, not experimental; what uncontrolled sample bias?
Excludes rereading but not recency effects
Relevant to human-guided oral reading?
Is rereading more motivating for poor readers?
Wide reading requires more text and more guidance!
Micro-analysis of tutor data finds subtle (ms) effects
Large, fine-grained, longitudinal sample
Speech recognition is imperfect
But unlikely to be biased wrt our variables
Thank you! Questions?
See papers & videos at 16

17. Thanks to fellow LISTENers & friends
Tutoring:
Dr. Joseph Beck, mining tutorial data
Prof. Albert Corbett, cognitive tutors
Becky Kennedy, linguist
Joe Valeri, activities and interventions
Listening:
Dr. Evandro Gouvea, acoustic training
John Helman, transcriber
Dr. Mosur Ravishankar, speech recognizer
Programmers:
Andrew Cuneo, application
Field staff:
Kristin Bagwell
Julie Sleasman
Dr. Roy Taylor
Grad students:
Kai-min Chang, LTI
Cecily Heiner, MCALL
Ayorkor Mills-Tettey, RI
Interns:
Alisa Grishman
Brooke Hensler
James Leszczenski
Rachel Minkoff
Research partners:
DePaul
U. Toronto
U. British Columbia
Ghana
Advisory board
Kathryn Ayres, children’s stories
Rollanda O’Connor, reading
Charles Perfetti, reading …
Schools 17