Assessing Standards-Based Language Performance in Context Assessment Assessing Standards-Based Language Performance in Context March 29, 2011 Shrum/Glisan Chapter 11

Terminology Assess Test Evaluate Grade Assessment Evaluation Assess: gather info about and measure a learner’s level of knowledge or skills Test: vehicle for determining a learner’s level of knowledge or skills Evaluate: interpret/assign a value to info about a learner Grade: convert assessment info into a form that is understandable to the learner (letter grade, points on a rubric, numerical score, written feedback) Assessment: describing/reporting a learner’s performance Evaluation: assigning a value judgment to that performance HISTORICALLY PURPOSE OF TESTING WAS TO EVALUATE ACHIEVEMENT AND ASSIGN GRADES; NOW ASSESSMENT BECOMING MORE IMPORTANT AS VEHICLE FOR PROVIDING FEEDBACK TO LEARNERS, IMPROVING LEARNER PERFORMANCE, AND ASSESSING/INFORMING INSTRUCTION

Paradigm Shift Begin with the end in mind! (Backward Design) Assessing from standpoint of multiple perspectives Need for formative and summative assessment Must be contextualized Expanded role of teacher and student Consider what learners should be able to do by the end of a period of instruction; what assessments would best serve to assess and track progress Provide ongoing opportunities for students to show what they know and can do with the language (portfolios, informal observation, student feedback, etc.) Formative: occurs along with learning, helps students repair and improve. Summative: occurs at the end of a course of study, designed to measure what learner can do at that point, usually “final exam” No place for decontextualized testing of discreet grammar points; should be series of interrelated tasks that reflect 3 modes of communication; task should be performance-based, communicative activity that reflects use of language in real world Teacher tells how students will be assessed, shows sample of meets/exceeds performance, gives rich feedback; student has multiple opportunities for growth and improvement, learn through assessment; participate in assessment process (portfolios)

Standards-Based Authentic Assessment Contextualized Meaningful communication Elicit a performance Encourage creativity Adapted to serve as formative or summative assessment Address at least one mode of communication Used or adapted to address standards These are goals…difficult to implement! Must be a process, district-wide New Shrum Gleason page 53 for National Standards: Come up with example of Standards-based authentic assessment

Continuum of Test Item Types Natural-situational Unnatural-contrived Direct Indirect Integrative/Global Discrete point Natural-situational: learners may encounter in outside world (response to letter or email); unnatural-contrived: traditional test items that often focus on isolated grammatical structures/vocab (fill-in-the-blank verb formation) Direct: incorporate contexts, problems, solution strategies used in real life (give a talk to peers); indirect: extracts knowledge and skills out of real-life contexts (multiple-choice grammar test) Integrative/global: learners use various components of language at same time, requiring multiple modes/skills (listen to CD segment, identify main ideas, use information in discussion or composition); discrete point: tests one point at a time (multiple choice, true/false, matching, completion) MOST LANGUAGE TESTS LIE SOMEWHERE ON THIS CONTINUUM

Implications for Teaching Selection of assessment should depend on teacher’s objectives Direct test types may be more valid measures than indirect Must understand limitations of discreet-point testing What teacher intends to be assessed (for reading comprehension, multiple choice may be ok; for interpersonal speaking, integrative assessment with real-life communication task) Students may at first prefer “one right answer” tests, however recent research says students acquire a more positive attitude toward direct tests because they have face validity and show them what they can do with the language in real-life contexts; especially enthusiastic if it reflects classroom instruction Can be used to assess grammar and vocab, but not production or performance. Teacher cannot assume that learner will be able to use the item appropriately!

Assessment Formats Prochievement (proficiency + achievement) Monologue: Describe your routine for a typical day during the school week. Tell what you do in the morning, during the day at school, and at home in the evening, and at what times. Then describe how this routine may be different on the weekend. Be sure to include interesting details. Prochievement: born out of desire to blend proficiency-based instruction performance with grammar and vocab being taught in the classroom (role plays, paired interviews, picture descriptions, task-based discussions, writing activities). Example is designed to assess the use of reflexive verbs AND intermediate-level speaking proficiency. Performance-based: requires learner to use repertoire of knowledge and skills to create a product or response, either alone or in groups; use of prompts, more than one right answer. Prochievement focuses on narrow area of knowledge and skills; performance based require greater integration of accumulated knowledge and skills (call teacher and leave a message with certain info; write letter to exchange student about your school and community) Interactive model: assesses interpretive reading, listening, and viewing. Parallels Interactive Model for Integrating the Three Modes of Communication Authentic: tasks that mirror those faced by individuals in the real world Standards-based: tasks that address one or more goal areas in the SFLL

Assessment Formats Prochievement (proficiency + achievement) Performance-Based Interactive Model Authentic Standards-based Prochievement: born out of desire to blend proficiency-based instruction performance with grammar and vocab being taught in the classroom (role plays, paired interviews, picture descriptions, task-based discussions, writing activities). Example is designed to assess the use of reflexive verbs AND intermediate-level speaking proficiency. Performance-based: requires learner to use repertoire of knowledge and skills to create a product or response, either alone or in groups; use of prompts, more than one right answer. Prochievement focuses on narrow area of knowledge and skills; performance based require greater integration of accumulated knowledge and skills (call teacher and leave a message with certain info; write letter to exchange student about your school and community) Interactive model: assesses interpretive reading, listening, and viewing. Parallels Interactive Model for Integrating the Three Modes of Communication. In example (Appendix 11.2 ) 1. students read/listen to authentic text 2. focus on situational context (identify main ideas) 3. focus on information (identify details) 4. focus on grammatical competence (use grammatical structures to further explore ideas 5. focus on intent of text (students develop their points of view) Authentic: tasks that mirror those faced by individuals in the real world Standards-based: tasks that address one or more goal areas in the SFLL.

Once again: Authentic and Standards-Based Assessment “Authentic tasks require learners to address an actual audience and mirror challenges faced by real individuals in real-world settings” Shrum/Glisan p. 371 The teacher is not the only audience IPA (Integrated Performance Assessment) May be used for formative or summative assessments; allow teachers to “assess what we value so that we value what we assess” Speech to classmates; email to penpal; brochure for someone really traveling Used to be called “PAU” (Performance Assessment Unit); recently designed by ACTFL to address need for measuring student progress in attaining competencies described in national standards and proficiency guidelines within authentic contexts; cyclical approach seen in Interactive Model (SG p. 379)

Classroom Assessment Techniques (CATs) Informal, formative assessment strategies Aimed at improving instruction vs. assigning grades/points Simple, non-graded, anonymous, in-class activities Gives useful feedback to teacher and student The function of CATs is to improve the quality of student learning, not provide evidence for evaluating or grading students Usually done at the end of a class period Example: Exit cards, the Minute Paper (purpose: to provide rapid feedback on whether the teacher’s objective for the lesson matched what learners actually learned. Increase in student satisfaction as a result of having a voice in their learning