StoryStation: Agent-based scaffolding of metacognitive processes for writing Peter Wiemer-Hastings DePaul University Kath Glasswell University of Illinois Chicago

Overview Metacognitive processes in writing StoryStation gives support and feedback for story writing Current directions How we’ll evaluate Comparisons

Writing is a complex act Writing involves the conscious manipulation and orchestration of a number of processes and sub-processes. Skilled writers work like busy switchboard operators (Flower & Hayes, 1980) switching their attention between transcription, audience awareness, language use, content generation, text organization and topic knowledge. Skilled writers manage this almost invisibly. They self-regulate their own composing processes.

Self-regulation in composing models Flower & Hayes (1980) –Proposed that a great part of skilled writing was the writer’s ability to monitor and direct composing processes Bereiter & Scardamalia (1986) –Proposed an ‘executive control’ facility that took account of managing and orchestrating composing constraints

Self regulation and writing Self regulation influences writing in two main ways Writers develop a metacognitive competence through the use of self-regulatory processes. They exert more control over their writing, setting goals for writing, monitoring the text produced and comparing it to mental representations of intended meaning. These self-regulatory mechanisms act as ‘change- inducing agents’ (Graham & Harris, 2000) which help develop both the piece under construction and the writer’s self-regulating behaviors.

Revision as primary process for self-regulation Revision is a key focus for the integration of these two aspects of self regulation in writing –Much research has focused on students’ revising processes suggesting that the ability to reflect on the written product, judge its effectiveness given the context and make changes to improve it are central to being a skilled writer.

Three main assumptions of StoryStation Individual differences in self-regulatory strategy and knowledge are linked to differences in writing performance. Helping students develop self-regulatory strategies will improve writing competence. AI agents can provide structured feedback on student writing which will scaffold the development of students’ self-regulatory strategies for writing.

Teaching for self-regulation can improve writing competence Evidence from intervention research studies –Training in specific self-regulatory strategy use leads to improvements in writing competence (Graham & Harris, 2000) Evidence from classroom researchers –Self regulation is fostered through the meaningful feedback on writing (Atwell, 1998, Calkins, 1994, Graves, 2003) –Effective teaching interactions for writing promote reflection, self-regulation and responsibility in writing (Anderson, 2000, Glasswell, 2003).

Developing SRL in classroom environments Providing for meaningful dialogue about writing processes Teacher / Buddy conferencing Self-conferencing

Sensitive feedback on writing can promote self-regulation McNaughton (1995) –argues that teaching is the only profession which has as its goal the desire to put oneself out of business (1995) For self regulation less is NOT more. –Theories of teaching as ‘assisted performance’ (Tharpe & Gallimore, 1988, Rogoff, 1990) focus on the gradual transfer from teacher to learner of responsibility for task completion. –The goal of teaching is the independent, self regulated performance of the learner.

Sensitive agent-based feedback can also promote self-regulation At present SS is designed to act as an aid to revising student writing Story Station uses agents to ask questions and give feedback aimed at developing self-regulatory strategies and independence in learner writers Like models of good teaching SS can be set to vary the level of assistance given to individual writers The goal is similar to that of teacher conferencing. Students will, through interacting with the agents, come to internalize and apply independently the questioning that promotes reflection on writing and self regulation of writing processes

StoryStation first draft Designed and implemented by Judy Robertson Students hear stories, write their own versions StoryStation provides resources (dictionary, thesaurus, wordbanks) Also provides feedback (spelling, characterization, word use) Different types of feedback provided by different agents

Current goals Engage students in dialog about their stories. The goal is similar to that of teacher conferencing. Students will, through interacting with the agents, come to internalize and apply independently the questioning that promotes reflection on writing and self regulation of writing processes Like models of good teaching, vary the level of assistance given to individual writers

Agent assistance for writing: Awareness of Audience and Purpose (reflecting on writing demands) Content/Ideas (characters/setting/theme/main events) Structure/Sequence (beginning/middle/end) Word choice (vocabulary use) Sentences (style and correctness) Word-solving (Spelling agent) Punctuation (checking for sentence, simple and complex punctuation)

Sample: Audience awareness Did you decide you were writing this for? (tick these boxes) Did you take that into account when you were writing? –Yes. Good. Thinking about who about who you are writing for helps you write a more effective story. What kinds of things did you do when you were writing to take account of your reader? –I decided not to put the scary part in, in case they were too frightened. –I knew that she would want to see a lot of detail so I added in information. – I was writing for my brother so I didn’t put in stuff he already knew. –No. Maybe next time you write if you take into account who you are writing for it will help you write a more effective story. For example, if you were writing this for your friend you might try to include lots of scary words and phrases to make the story more exciting to read.

Sample: Content Did you include ALL the important characters in the story? –Yes I don’t see Momma Bear in here, would you like to include her now? Where would you like your readers to meet her? Good job, you have included them. How did you help your readers get a feel for these characters? How did you do that? –I put in a lot of description about the bears – I made poppa bear really big and I wanted him to be bad-tempered and I wanted baby bear to be really cute so I described him as fluffy and made him cry a lot.

Evaluation We hope for improvements in product, but they will be hard to measure Examine differences in student’s talk about how they approached their writing in peer conferences (before and after StoryStation use)

Summary Writing revision depends on metacognitive processes –Planning –Self-monitoring –Reflection StoryStation fosters by modeling metacognitive dialogs with the students about their writing and writing processes

Comparison NORMIT: supporting self-explanation –Query students about what they’ve done –Student’s justifications are textual –Differences due to domains Differences –Domain: Database design and Similarities

StoryStation genesis Writing is difficult Natural language processing techniques can support reasonable feedback within restricted domains Linda Flower’s voices

Negotiated Construction of Meaning Goal of writing is to create a meaningful text The creation process involves negotiating with voices which constrain what the writer can do Constraints come from social, linguistic, and internal considerations

Some voices –Reader: What do I want R to know? What do I know about what R knows? What do I want from R? –Situation: What am I supposed to write about? What do I know about my topic? What do I want to say about it? –Socio-linguistic context: How can I say this in a way that my teacher will accept?