1. Automating Early Assessment of Academic Standards for Very Young Native and Non-Native Speakers of American English better known as The TBALL Project (Technology Based Assessment of Language and Literacy)
NSF-IERI award REC
Abeer Alwan (UCLA),Shrikanth Narayanan (USC), and David Pearson (UCB)
IV. Reporting: What teachers see
Speech Recognition Approach
Recorded 300 children from 6 elementary schools; more than 80% of them are bilingual. Data used to train the ASR system. System has a 93% verification accuracy and correlates well with teacher scoring.
ASR Approaches:
Pronunciation modeling
Speaker adaptation techniques
Noise robust (front end and/or back end)
Background
Report of National Reading Panel (2000) advocates use of classroom-based assessments
Many classroom assessments are not psychometrically sound
Investment of time limits widespread use
Distinguishing Features
- Study the speech of native English and of Bilingual (Hispanic origin) K-2 students
- Longitudinal and cross-sectional studies
- Balance pedagogy and technology: strong interdisciplinary interactions between EE, CS, Education, Psychology and Linguistics
- Correlation of measures with later performance
- Validation of system, children’s performance, and teachers’ practices
Spanish-accented English
Field Testing
Four schools in Los Angeles and one in Oakland with ELL populations; Two schools in Los Angeles, two in Oakland with native English speakers (total of 240 K-2 students)
Very positive feedback from teachers in terms of students’ enthusiasm, ease of use of, and efficiency in conducting the tests
I. Assessment Framework
Many different aspects of reading skills
Phonemic Awareness; Alphabet Identification; Letter-sound knowledge, Blending, Spelling, Segmenting. Word Recognition (Rate and Accuracy). Picture naming. Syntax, Comprehension
Framework is hierarchical rather than a uniform approach to assessment
All students take benchmark assessments
Some students take 'drill down' assessments
Teachers have guidance on what to assess
TBALL Specific Aims
Report of National Reading Panel (2000) advocates use of classroom-based assessments
Many classroom assessments are not psychometrically sound
Investment of time limits widespread use
Acoustic phonetic knowledge transfer
/v/ /f/ (very)
/dh/ /d/
English Phoneme
Acoustically similar Spanish Phonemes
Map
Think
Listen
Produce
Jose /h ow z ey/
Joint /jh oy n t/
/h oy n t/
Orthographical knowledge transfer
/j/ /h/ (word initially)
III. HCI and Database Design
Reading in Context
Several children found it easier to read words in context than to identify them in a word list.
Typical mispronunciations: Him/heem, as/us will/weell, read/reet, with/whit, by/be
However, when the words were embedded in a sentence, the children had correct pronunciations. For example, the child that said ‘reet’ for ‘read’ had the proper pronunciation when he read : ‘I can read my book’
Syntax also plays a role. One child didn’t distinguish between ‘Came’ and ‘Come’. However, when he was presented the sentence: ‘I came to play but my friend said “ I will not play with you”, he read ‘came’ correctly.
TBALL Team
Vocab and Topic Knowledge
Oral Lang. Comp.
(Syntax)
I.I.
The Dam (decodable word list)
Irregular Word list
I.I.
Displaying data for different groups and needs (including age-appropriate HCI design)
Narrative Listening Comp.
Schools:
5 LAUSD schools
UES
2 Bay Area schools
Narrative Oral Reading
I.I.
The Dam (decodable word list)
K/1 High Frequency Word List
NarrativeReading Comp
San Diego High Frequency Word List
Vocab and Topic Knowledge
Written Lang.
Comp.
(Syntax)
I.I. EE
GSEIS CS EE
Linguistics
Psychology
Education
BPST
Phonemic Awareness
I.I.
Rapid Naming
Letter
Sound
Comp.
11 graduates & 7 undergraduate students, 6 teachers
Query-based Datamining
Database design allocates a place to put the collected data and its context, e.g.,
Demographic info from parent, date, TIME, test
Content material from test
Later the data can used for computations, e.g.,
Words in isolation correct: 21/51 = 41%
Words in connected text: 20/36 = 55%
Teacher or administer can easily access the data through queries, e.g.,
How did non-native speakers do relative to native speakers on task x?
Did a child’s performance change from K to G1?
Components
II. Automatic Speech Recognition
Challenges:
Children have shorter vocal tract lengths (hence, higher resonances) and higher pitch
Significant intra- and inter-speaker variability
Significant variability in pronunciations due to different linguistic backgrounds, misarticulations, and signal to noise ratio of the recording environment
How to distinguish reading errors from pronunciation differences
Looking to the Future
Beta version of the system, fall 2006; Alpha version, fall 2007
Assess other skills (mathematical and scientific reasoning)
Extend the current system to other grade levels and language pairs
Refine assessment tasks, materials, and automated techniques based on feedback from teachers and children
Address validity, utility, and impact for native and non-native speakers
Train teachers to use the system, deploy in more classrooms
Please visit us at:
Present auditory, text, graphical stimuli
Measure decoding, comprehension skills
Score, analyze, and adapt to responses
Query-based datamining: monitor progress, compare, experiment
Displays and summary screens for teachers to combine data to help make decisions
Instructional guidance for teacher development