Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University

Outline Background Research Discussion and future research

User-centered learning Approaches in educational research –Authentic –User initiated –Motivating –Individual needs

MASLA project Models of Adaptive Second Language Acquisition Combination of Computer Science and Second Language Acquisition Goal: building a model for personalized digital language learning web based applications How can learning materials automatically be adapted to fit the characteristics and preferences of the language learner? Criterion is learning effect.

Requirements for adaptivity Annotated learning material –domain model Knowledge about learner characteristics –user model User model + domain model -> adaptation model (rules) (Dexter model, 1990; AHAM model (De Bra, 2000))

MASLA Framework Graphical User Interface Curriculum L2 - proficiencies Learning contents Learning styles Learner backgrounds

Task: Vocabulary learning through reading Incidental vocabulary learning (side effect of reading for comprehension) ZOPD (Vygotsky, 1962); Comprehensible input (Krashen, 1987) –Assessing learner proficiency –Assessing text difficulty based on frequency information from corpora => combined in text coverage

Text Coverage more difficult easier

Interpreting text coverage Hazenberg, 1994; Laufer, 1989; Vermeer, 1998 Lemma Coverage: –85%: Global understanding –90%: Good understanding –95%: Almost complete understanding

Effective Instruction Comprehensible but challenging Lemma coverage 85% - 92% Support from input modification –Dictionary/glossary (see Hulstijn et al., 1996; Plass et al., 1998; Watanabe, 1997) –User initiated “focus on form”

Text Coverage Top criterion Bottom criterion

Summary of research background Web based tool for automatic adaptive selection of the appropriate text for a specific user. Automated analysis of text difficulty. User proficiency calculation from score on vocabulary test. User gets text that is comprehensible but challenging and has input modification for unknown words to support for understanding the text.

Research questions A. Adaptive selection of texts leads to: A learning effect for all users No difference between learners with different proficiency levels B.Using input modification: There is a relation between noticing and retention (There is no difference in this relation for different proficiency levels)

Method (1) Subjects (N=32) Reading Texts (16) –4 clusters Input modification

Text coverage for selected texts Almost complete comprehension Global comprehension

Mean text coverage per cluster Almost complete comprehension Global comprehension

Method (1) Subjects (N=32) Reading Texts (16) –4 clusters Input modification

Method (2) Data collection: –User logging and tracking –Testing material Vocabulary proficiency test Text specific vocabulary tests Comprehension questions Procedure

Learning gains Procedure

Results (1) A mean learning effect occurred for all clusters –5% learning gains No significant difference between groups –both pre and posttest scores –learning gains

Results (2) Correlation between noticing and retention –Mean Φ correlation for subjects:.28 –Mean Φ correlation for items:.50 in general, the use of the dictionary was limited –No significant difference between proficiency groups In lookup behavior In correlation

Conclusion Automated assessment of texts based on corpora information is a useful indication of text (task?) difficulty. Adaptive selection of texts based on vocabulary proficiency works. Open, web based learning environment provides flexibility in the curriculum and opportunities for individualized tasks.

Discussion and future work Increase learning gains –More adaptivity in text selection Increase exposure to target words Based on observed behavior Increase usability of input modification –Individualize annotation Based on observed behavior More focus on form Use different corpus for text coverage –Now children’s corpus, future Celex/CGN –Unknown lemmas –Multiword expressions