1. I C A R E: Independent Comprehensive Adaptive Reading Evaluation: WHY & HOW
Barbara Wise, Ron Cole, Bryan Pellom, Sarel Van Vuuren
Center for Spoken Language Research
University of Colorado
& with Lynn Snyder, Tim Weston, Richard Olson, Jan Keenan, Jariya Tuantranont, Nattawut Ngampatipatpong, Andi Hagen, & Sharon Vaughn
Institute for Educational Science
November 12, 2004

2. Colorado Literacy Project
A comprehensive reading program:
Teach any child to read and learn from text
Optimize instruction for different kids
Scale up nationally and globally
A test bed for research on
Reading and comprehension instruction
Individual differences in instructional needs
Interactions with Virtual Teachers

3. Components of Colorado Literacy Project
ICARE: rd -7th (K--3rd??)
Independent Comprehensive Adaptive Reading Evaluation: (also Lu Sessions, Tammy Tomczyk)
Foundations to Literacy: KG--3rd (--> 5??)
Foundational reading exercises
Books for fluent reading and understanding
(also Scott Schwartz, Taylor Struemph, Gordon Golding, Pam Cole)
Summary Street: 5-->; (K--5??)
Comprehension and learning
Interactive Books and LSA
(Walter & Eileen Kintsch, Tom Landauer, Donna Caccamise, Dave Wade-Stein)

4. Sponsors & Grants
IES: ICARE: Independent Comprehensive Adaptive Reading Evaluation $1,600,000 Wise, B., Cole, R., Pellom, B, & Van Vuuren, S. 10/1/04--9/30/08
NICHD/IERI : 1R01HD Cole, R., Barker, L., Schwartz S., Snyder, L., Wise, B. "IERI: Scaling up Reading Tutors," $1,000,000.00, NIH. 9/27/02 - 9/30/04.
NSF/ITR: REC Kintsch, W., Landauer T., Caccamise, D., Cole, R., "ITR/PE: Latent Semantic Analysis Theory and Technology," $2,400,000, NSF, 09/01/ /31/06.
NSF/IERI: EIA Kintsch, W., Caccamise, D., Cole, R., Olson, R., Snyder, L., "IERI: Scalable and Sustainable Technologies for Reading Instruction and Assessment," $5,997,404, NSF, 07/01/ /26/06
NSF/ITR: IIS Cole, R., van Santen, J., Movellan, J., "ITR: Creating the Next Generation of Intelligent Animated Conversational Agents," $4,000,000, NSF, 09/01/ /31/05.
and past NIH grants HD11683 and HD22223 Talking Computers Help Children with Reading Disabilities, and for the bases of many of the word-level tests. Olson, R. & Wise, B

5. Outline Why build I C A R E ? Goal: Independent Comprehensive Adaptive
How?
Simple Model for Comprehensive & Adaptive
Technology for Independence
4 stages for building and validating
Where are knowledge and measures strong, where gaps
Discussion and Plans

6. ICARE Independent Comprehensive Adaptive Reading Evaluation
Why build It?
Need:
Instructional Profiling and RtI measures
Problem:
Not enough trained evaluators nor time
Goal:
Positive, Engaging, Efficient
Good readers out in < 15 minutes
Kids in Need Out in minutes
Work in English, ESL, Spanish

7. ICARE Independent Comprehensive Adaptive Reading Evaluation
Two Kinds of Assessment Needed
(Fuchs et al 2004)
Evaluation Level for
Identification and Evaluation of Programs
WJ-III, CTOPP, & ICARE are this kind
Response to Intervention
RtI, CBM
for stability, ongoing evaluation
DIBELS and FtL are this kind

8. ICARE Independent Comprehensive Adaptive Reading Evaluation
Efficient, Effective, Engaging
Find kids in need or at risk for
Difficulties in Reading Comprehension
For different reasons
Funded Grades rd th
Proposed KG rd

9. ICARE Independent Comprehensive Adaptive Reading Evaluation
Goals:
Instructional Needs Profiles to
Guide areas of needed Instruction
For classroom Instruction
For Study Plan for FtL
Evaluate effectiveness of Programs
2 forms for repeated measure

10. All ICARE measures (eventually)
Adaptive
Only take tests if you need them
Only take items you need
Independent
Version with Speech Recognition
Version without Speech Recognition (Forced Choice, mouse-clicked)

11. Can Speech Rec do it? Speech recognition for
Reading out loud, for fluency, accuracy: SONIC: Bryan Pellom
Accuracy 94% with known text, & improving
Spoken responses to focused questions
Transcription of spoken summaries for automatic grading
And if not-- Mouse Clicks!

12. How to Make it Adaptive ? How will we build it and validate it?
Use Simple Model for basic profiles
What underlies reading comprehension ?
Get Profile of Instructional Needs for all with needs or at risk
How will we build it and validate it?
Where are knowledge and measures strong, where gaps

13. Theoretical Base Simple Model of Reading (Gough et al., 1996)
Skilled Reading = product of Processes underlying
Word Reading:
Alphabet, Phonological Awareness, Decoding, Sight, Automaticity
times
Listening Comprehension:
Construct Gist , Vocabulary , Working Memory....
Text Cohesion (cross-text connections, bridging inferences)
Rapid Naming affects both

14. Basic Profiles from Simple Model

15. ICARE: Efficient and Adaptive.. Time Goals for Good Readers
Exit Kids (<15 min) if at grade level on “screening measures”
Fluent and Accurate Word Reading
Fluent Reading with Comprehension
Spelling ( risk for later problems ? )
Vocabulary (risk for later problems ?)
Congratulations! Keep reading and writing well!

16. ICARE Time Goals for Kids At Risk for or with Difficulties in Reading Comprehension
Problem at Word Level? (20-25 min)
Investigate processes under Word Reading
Problem w/ Listening Comp (20-25 min)
Problem with both (40-45 min)

17. Possible Word Level Tests if difficulty with Time limited Word reading
Untimed Word Reading
Rapid Naming Letters & Numbers
Spelling
Time-limited or Untimed Decoding
Orthographic Coding
Phonological Awareness
Phoneme Deletion (if Decode < = 3rd or)
Phoneme Reversal (if Decode > 3rd

18. Comprehension Measures if Reading Comprehension < Grade
Listening Comp and Vocab: OK?
Just do Word Level Measures, otherwise:
Paragraph Listening
Main idea , summaries
Concrete and abstract inferencing
Cross-text cohesion, pronoun referents
Reading Comp at TWR level
Prosody and fluency at TWR level
Reflects and develops comprehension
Phonological or Working Memory

19. Concurrent Validity Measures For ICARE measures & Adaptive System
1. Woodcock-Johnson III
Untimed Word Read, Spell, Reading Comp, Fluency, Vocabulary, Abstract Words, Analogies
2. CELF: Listening Comprehension
3. GRADE: Reading Comp, Analogies, Inferencing
4. WORD: Vocabulary, Multiple Meanings
5. TOWRE
Time-limited word reading TWR & Time-limited decoding
6. CTOPP
Phoneme Deletion, Reversal, Rapid Naming, Phonological Memory
7. PIAT Spelling: Orthographic Coding

20. ICARE will Build on FtL Interface and Functions

21. What is a Virtual Evaluator?
A Lifelike 3D Computer Character
with personality, attitude, and built-in knowledge about language
that engages users in natural face-to-face conversation
to produce engaging and effective learning experiences

22. Marge

23. Foundational Reading Skills Exercises
More than 30 exercises:
Alphabet,
Phonological awareness,
Decoding and encoding (spelling) sound-out words
Reading and Spelling Sight words
Automaticity
Vocabulary
Contextual feedback, reinforcement and individualized instruction by the Virtual Teacher

24. Interactive Books Student can listen as teacher reads
Student can read aloud, get help, & computer listens and tracks reading
Student can read silently & get help
Multimodally assess comprehension

25. Interactive Books Engage kids in comprehension activities
Background Check
Main Idea and Inference questions
Summarize
Word- spotting
Base for assessments of
Reading and Listening Comprehension
Summarizing
Inferencing (Abstract and Concrete)
Vocabulary (Single and Multiple meanings)

26. Click on image ? interaction

27. ICARE measures In various stages of development
1. Develop Items.....
Have for many word level, from Twin & ROSS
Develop for new measures, under LC especially
2. Test in Long Form
To develop ordering, speech recognition
3. Test in Adaptive Form
Skip up to error, down to basal, up to ceiling 5 / 7 X
Concurrent validity
4. Adaptive System
... ESL and Bilingual Spanish Versions

28. Comprehension, Fluency and Prosody at TWR level

29. Main Idea Multiple choice

30. Summarization using Summary StreetTM

31. Untimed Letter Sound ID
Instruction: Click on the letter that makes the beginning sound of the word I say.
pig

32. Rapid Naming of Letters with speech recogntion
Instruction: Read as many letters as you can in 45 seconds

33. Untimed Word reading without speech recogntion
Instruction: Click on the word I say
fed

34. Time-limited Word Reading without speech recognition
Words stay on screen for 2 sec. then bubbles pop Score final response
girl

35. Time-limited Word Reading with speech recognition
Word stays on screen for 1 sec. then bubble pops Child reads. Score final response
Keep up
the good work!

36. Time-limited Word Reading with speech recognition
Word stays on screen for 1 sec. then bubble pops Child reads. Score final response
Keep up
the good work!

37. Phoneme Elision without speech recogntion
Say “nest.”
What sounds
like
“nest”
without sss?

38. Articles and Websites for Foundations to Literacy, COLit, ICARE
Learning to Read with a Virtual Tutor: Foundations to Literacy
Wise, Cole, Van Vuuren, Schwartz, Snyder, Ngampatipatpong, Tuantranont, & Pellom. In Kinzer, C. & Verhoeven, L. (Eds). Interactive Literacy Education. Mahwah, NJ: Lawrence Erlbaum and on website above