0 Automated Formative Assessment: Providing Linguistic Support through Online Modules Presented by: Ken Petersen 11/18/2011 ASEES 42nd Annual Convention Washington, DC

1 What is Formative Assessment? Formative assessment allows a teacher to capture learning as it occurs, and to make appropriate instructional adjustments. It is distinct from summative assessment in several aspects: o Frequency o Goals  Stakes  Stakeholders o Outcomes  What effect does assessment have on the learner’s linguistic development and/or language career path? o Shelf-life of the data

2 Online Formative Assessment The state of the art for the critical languages is less than optimal. o First generation Web-based CALL applications provide little data on learner performance and live in strict isolation from one another.  Russnet, Langnet, GLOSS, Arabic Without Walls, CenAsiaNet, etc. o Development of good materials is time and labor intensive. Russnet has been providing learners and teachers the opportunity to use online pedagogical materials for formative assessment for over a decade. o Reading, listening and writing activities that capture:  Scores  Student mistakes  Student essays  Teacher feedback o Data collected to date:  Activities completed: 321,368  Mistakes: 566,077  Student Essays: 26,302  Writing Comments: 2,735

3 Moving Forward Opportunities for online formative assessment are growing at a staggering rate. o Access to authentic language content (text, audio and video) o Online communication provides immediacy in feedback o Natural Language Processing (NLP) tools and resources are moving online  NLP Tools  Part of speech taggers  Morphological analyzers  Syntactic parsers  NLP resources  Electronic dictionaries  Bi-text corpora o Web-services are allowing data and functionality to be shared between applications.

4 Generating Dynamic Activities Together with the Center for Research in Computational Linguistics, American Councils is developing a suite of online tools for generating “on the fly” support for Russian lexis and morphology. The technology can embed instructional support into Russian texts from a broad array of contexts. The learner performance data derived from these activities could be used to provide pedagogically meaningful feedback to learners and teachers. An initial prototype is the Russian News Reader.

5 Russian News Reader (RNR): Search

6 RNR: Search Result

7 RNR: Lexical Support

8 RNR: Passive Morphological Support Highlight: Verbs

9 RNR: Passive Morphological Support Fill-in the Blanks: Adjectives

10 RNR: Passive Morphological Support Multiple-choice: Adjectives

11 Dynamic Activities & Formative Assessment The challenge at hand is to effectively model the data that we might capture from learner-text interactions. We have robust knowledge of lexis and morphology: o Part of speech o Gender o Number o Case We need to create a model that will provide informative and actionable evidence for formative assessment. There need to be intuitive graphical representations of the data. Khan Academy provides a nice framework for this.

12 Contact Ken Petersen American Councils for International Education 1828 L Street NW, Suite 1200 Washington, DC You can download this presentation at: