1. Assessing L2 writing performance: Reconsidering writing prompts as genre-based tasks Heidi Byrnes German Department Georgetown University byrnesh@georgetown.edu TBLT 2007 University of Hawai’i at Manoa Sept. 21, 2007 GUGD curriculum web page: http://www3.georgetown.edu/departments/german/programs/curriculum/index.html byrnesh@georgetown.edu

2. Underlying Assumptions  L2 writing prompts can/should - be reconsidered as “tasks”, particularly as genre-based tasks - be linked to quality of performance - facilitate the assessment of L2 writing performance  L2 writing in genre-based tasks can - Provide a favorable environment for integrating “language” and “content”  Assessment of L2 writing performance should - Consider language knowledge and content knowledge in an integrated fashion

3. “Tasks” in L2 education (1) An educational “activity ”  With a particular objective, appropriate content, specified work procedure, range of outcomes, necessitating the use of language (Breen 1997, Van den Branden 2006)  In the target language, focused on meaning rather than form ( Nunan 1989)  Meaning-oriented language use, resembling language use in the real world, often occasioned by a problem, involving a workplan toward achieving a certain outcome ( Ellis 2003, Skehan 1998)

4. “Tasks” in L2 education (2)  Users of tasks Teachers and learners, curriculum developers, researchers, assessment specialists (Bygate et al. 2001)  Uses of tasks, as a way of –Specifying learning goals –Designing and organizing educational activities in support of learning –Ascertaining ability to use language (assessment) –Exploring implications of findings for curriculum and instruction

5. “Tasks”: Relating “language” and “content/meaning ”  Language AND content  Language knowledge AND content/background knowledge  Focus on form AND focus on meaning  Ability to use/perform AND ability to develop  Additive conceptualization of task  No conceptual reach into content/meaning in social context

6. Needed: New conception of “language” “Our approach to the teaching of any phenomenon depends critically on our conception of this phenomenon. Unless we can base language teaching and learning on a richly revealing comprehensive account of what kind of phenomenon language is, we are not in a position to answer the many questions that arise in educational contexts. Similarly, the value and success of any approach designed to support second or foreign language teaching … will depend critically on the conception of language that informs it.” ( Matthiessen 2006: 31)

7. Probing two constructs of “task” (1) Language analyzed as Formal system of rules Fundamentally functional social semiotic system Meaning making Expressing things; Focus on personal meanings; “Negotiation of meaning”. Construing/creating that which is being expressed; Developing a voice within socially situated, socially preferred forms of language use. Semantics and lexicogrammar Inventories Strategic resources

8. Probing two constructs of “task” (2) Context and content Given “out there” Socially constructed; Language as primary semiotic system; “Context of culture” + “context of situation” participating in the construal of meaning Language learning as Gains in accuracy, fluency, complexity of processing in all modalities Learning how to mean; Building up ‘meaning potential’ in a range of registers and text types/genres Focus of attention Sentence-level processing Oral/written texts as environment arising from and construing social structures and meanings

9. Probing two constructs of “task” (3) Processing orientation Syntagmatic syntactic rules orientation Paradigmatic meaning and choice orientation “Knowledge” imagined and valued as Cognition; “Cognitively demanding;” “Higher-order thinking” Language-based shaping of world; Expressed in texts from 2 semiotic perspectives  congruent semiosis  non-congruent (“metaphorical”) semiosis Integration through Psycholinguistic mechanisms focused on individual language user Semiotic processes focused on construing and shaping the physical and social world = meaning making

10. Needed: New understanding and descriptions of language  The challenge is to relate sociocultural context and the “content” that is likely to be enacted to its particular wording in language, that is, we need rich accounts of semantics and lexicogrammar.  “On the one hand, we need semantics as an ‘interface’ between context and lexicogrammar … a fully fledged semantics of text. … Language learners need to learn semantics as a strategic resource … a resource for transforming what is not meaning into meaning, construing their experience of the world as meaning and enacting social roles and relations as meaning; and this will provide them with the ‘bridge’ to lexicogrammar.  On the other hand, both semantics and lexicogrammar need to be learned as resources rather than as inventories.

11. Needed … ... To meet the challenge of relating language to context, we also need to have a systematic and comprehensive account of context … Like language, context is a semiotic system, but it is a different kind of semiotic system … language is a denotative semiotic system (that is, a semiotic system that has its own expression plane), whereas context is a connotative semiotic system (that is, a semiotic system that has other semiotic systems as its expression plane). Context is realized by language, and by other denotative semiotic systems such as gesture and facial expressions; and also by non-semiotic, social systems.” (Matthiessen 2006, 37-38)

12. SFL approach to relating meaning/content and language  The process of meaning making is multifunctional in all languages  Three metafunctional modes of meaning –Construing our experience in the world around us and inside us as meaning = ideational metafunction (Meaning as “representation”: participants, processes, circumstances) –Enacting our social roles and relations as meaning = interpersonal metafunction (Meaning as “exchange”) –Presenting the meaning construed and enacted as a flow of information = textual metafunction (meaning as “message/text”: “coherence, cohesion”)

13. SFL approach (ctd.)  These resonate with three contextual variables - field: “what” is being talked about (social activity) (participants, processes, circumstances) - tenor: “how” the participants relate to each other in relation to the activity (mood, modality, appraisal) relation to the activity (mood, modality, appraisal) - mode: “how” the oral/written text is structured (theme and information structure; logical relationships in clause complexes; grammatical metaphor)

14. A trinocular conception of meaning/context Language comprises  Ideational resources naturalize reality  “Field knowledge” enables participation in domestic, recreational, academic, public, and professional activities  Interpersonal resources negotiate social relations  ”Interpersonal knowledge” enables a way of valuing and assessing these activities and enacting power and solidarity in relation to shared values  Textual resources manage information flow  “Textual knowledge” enables ways of phasing the ideational and interpersonal meanings into textures that are responsive/sensitive to the communicative demands of oral and written discourse.

15. Metafunction in relation to language, register, and genre

16. Register and genre in SFL  Register – a “constellation” A constellation of lexicogrammatical features that realizes a particular situational context, characterized in terms of ranges of field, tenor, and mode values in the contexts within which they are located  Genre – a “text type”  Recurrent configuration of meanings  Culture as a system of genres  Staged, goal-oriented social process, realized in text  Staged: more than one phase of meaning to work through the genre  Goal-oriented: unfolding phases toward meeting expectations  Social: genres occur in social settings and are interactively undertaken  The “teachability” of genres

17. Big Challenges  What constitutes good L2 writing?  How can we link content knowledge and language knowledge while setting some limits to the assessment of “content”?  How does instructed L2 writing develop?  Does it/should it figure into the assessment of L2 writing performance?

18. Writing development in the GUGD: a “case study” Writing tasks as genre-based writing tasks  Embedded in a four year integrated content oriented and task-based curriculum  Learning goal of advanced multiple literacies in several languages  Explicit pedagogy for writing  Writing development integrated into L2 development in all modalities  Assessment of writing development in curricular context Example: Level III writing task, within 5-level curriculum. Course focus “German stories, German histories” (Post WWII – present). Unit: fall of the wall.

19. Foci for genre-based writing tasks  Task Appropriateness: Understood in terms of –Generic moves –Positioning of writer toward an imagined reader audience  Content: Understood in terms of –Information provided within each of the generic moves  Language Focus : at 3 strata of language system –Discourse, sentence, lexicogrammatical system

20. Move to handout for  End of curricular level expected writing performance profile: Levels I – IV of the five-level curriculum  Specification of language foci at 3 strata of language system: Level III  Genre “(Political) appeal” = Aufruf. Text used in class to exemplify the genre.  Analysis of the text’s generic moves  Lexicogrammatical analysis of the moves “problem statement” and “appeal for action” in terms of field, tenor, and mode.  General guidelines for the creation of genre-based tasks  Aufruf task sheet, in terms of task appropriateness (genre), content, and language focus  Feedback sheet for the writing task  Guidelines for the assessment of writing performances

21. Linking content/context and language in genre-based writing tasks: Pedagogical considerations  Purposeful texts approximating real-life writing in terms of topics, possible relationships between writer and audience, purposes, textual demands in terms of level of L2 ability  Focus on creating meaning (content/knowledge) in a textual world through and in and with language “forms” at all strata of the system that should be chosen for their meaning-making capacity in relation to a genre.  Culturally authentic language use within the genre as culturally validated forms of texts and registers with their preferred bundling of lexicogrammatical features  Development of a competent non-native voice within stable, yet flexible generic structures

22. Pedagogical considerations (ctd.)  “Conceptual space” through metalanguage for a focus on content: manifestations of “field” in the various genre moves  Motivated development from more narrative genres, which privilege congruent semiosis, to more abstract, essayistic, public genres, which privilege noncongruent (metaphorical) semiosis  long-term trajectory toward the secondary discourses of the academy (see e.g., Byrnes & Sprang, Christie, Macken-Horarik, Martin, Rothery, Schleppegrell)  “ Plausibility” for preferred lexicogrammatical choices – Pawley and Syder’s “native-like selection”, particularly at the phrasal and clausal level, increasingly more important for more advanced learners.  Link between L1 and L2 literacies; between learning a language and learning in other disciplinary areas.

23. Implications for assessment The previous categories and pedagogical approaches to L2 writing provide ways of considering writing development as well as writing quality Assessment can be imagined at the intersection of a number of the constructs SFL makes available:  What metafunctions are handled and how well?  What major subaspects of these metafunctions are handled and how well?

24. Implications for assessment (ctd.)  What language strata are competently used? from word to phrase, to clause, to clause complex, to text, - -- linking into cultural context, and from cultural context back to language  What is the breadth and depth of registerial and generic resources available for use?  What is the user’s location on the cline between the resources of the system and instantiation of those resources in a text?

25. Assessment (ctd.)  What kind of knowledge is being created: congruent – noncongruent forms of semiosis?  What kinds of logical relationships are established and how? In simplex clauses? In complex clauses? What forms of grammatical are deployed? For what functions?  How is the phrase level expanded? Expansion of clausal and phrasal capacity  How is information structure textured? themes, hyperthemes; different forms of coherence

26. Assessment (ctd.) … and many more!  In each case, language use is linked to meaning, content, context. BUT …  There are also necessary limits set to what is being assessed

27. Benefits of assessment for teachers and learners  Predicts systematically the likely linguistic features that must be in focus  Spells out the criteria for successful writing  Helps teachers formalize and be more objective about their intuitive judgments about writing quality  Is formative and diagnostic with high potential for washback into educational processes

28. Benefits (ctd.)  Clarifies the demands associated with a particular writing task  Therefore has a high likelihood of enhancing L2 writing development  Provides a language for reflecting on writing  Links the goals of L2 writing with those in other subject matter areas

29. The challenges for this approach  Requires reconceptualizing language as a social semiotic and language development in terms of users’ ability to deploy generically appropriate resources for meaning making  Requires much higher levels of awareness (and responsibility) by practitioners for knowing how genres/texts work, that is, how meanings are realized in language  Requires similar awareness on the part of the testing community and a willingness to reconsider a number of constructs that have been central to assessment/testing

30. How to take up the challenge … Until we know more about instructed L2 writing development, perhaps inquiry is best located within an educational context -With an explicit curricular trajectory -That integrates all aspects of educational work: curriculum, materials development, pedagogy, assessment -Uses to advantage the specificity that assessment demands regarding what constitutes competent L2 writing while also informing assessment on the nature of L2 development in writing, -And makes appropriate adjustments to improve educational outcomes.

31. … and gain large benefits It constitutes  Socially responsive work –Outcomes of our programs: what is the value added on the basis of our educative work  Socially responsible work –Development of sophisticated, multiple literacies in a multilingual and multicultural world  Enhances L2 learning –Links content and language learning, thereby providing perhaps the most deep-seated form of motivation for learning languages.