Ch. 11 Assessing Standards-Based Language Performance in Context Song Xiaonan and Will Newman

Concepts Assess: to gather information about and measure a leanrer’s level of knowledge or skills. Test: a vehicle for determining a learner’s level of knowledge or skills. Evaluate: to interpret and/or assign a value to information about a learner. Grade: to convert assessment information about a learner into a form that is understandable to the learner, such as a letter grade, points on a rubric, numerical score, or written feedback.

That’s why we need assessment.

The purpose of the Assessment Administrative: General assessment Placement Exemption Certification Promotion

Instructional: Diagnosis Evidence of progress Feedback to the respondent Evaluation of teaching or curriculum Research: Evaluation Experimentation Knowledge about language learning and language use

Current understanding of assessment Administration of tests and assignment of grades; In today’s foreign language classes: Understanding the language learning process; Determining the difficulties students may experience and misconceptions that they may have, and documenting students’ language development over time.

Planning for assessment in a new paradigm “Planning begins with a consideration of what learners should be able to do by the end of a period of instruction and what assessments would best serve to assess achievement and track progress; you explored this type of backward design planning process in Chapter 3, in which assessment plays a pivotal role. Within backward design, you anticipate and even plan your assessments as part of designing a thematic unit, before instruction begins” (p. 395-396).

http://www1.chapman.edu/cll/faculty/piper/2042/assess.htm

Stages of Backward Design

“The new vision for assessment highlights the need for both formative and summative measures, assessment within meaningful and authentic (i.e., real-world) contexts, and opportunities for students to exhibit creativity and divergent responses. … ” (p. 397-398)

What is Standardized tests? “Standardized tests, also referred to as norm-referenced tests, measure learners’ progress against that of other learner in a large population; examples are the SAT, the TOEFL, Advanced Placement Tests, and PRAXIS exams. … Standardized tests typically follow a uniform procedure for administration and scoring” (p. 400).

Summarive vs. Formative Assessments “Summative assessment often occurs at the end of a course and is designed to determine what the learner can do with the language at that point”. “Formative assessments are designed to help for or shape learners’ ongoing understanding or skills while the teacher and learners still have opportunities to interact for the purposes of repair and improvement within the instructional setting.” (p. 401).

Similarities between Summative and Formative Assessments Systematic, planned, and connected to the curriculum; Many assessment tasks of both are similar (i.e., role-play situation, p. 401-402);

Types of Foreign Language Assessments (Figure 11.4) Proficiency-based/ Prochievement Performance-based Interactive Authentic Standards-based

Prochievement Format “Learner perform tasks designed for a particular level of proficiency in order to determine their ability to performance specific language functions within contexts and content areas, using a particular text type and level of accuracy” (p. 404, Figure 11.4). To assess the degree to which students have achieved the objectives of a particular lesson or thematic unit. To assess their ability to function along a proficiency- based continuum (Gonzalez Pino, 1989). “Formats include role-plays, paired interviews, picture descriptions, task-base discussions, and writing activities that assesses grammar/vocabulary as well as intermediate-level writing proficiency” (p. 404).

Performance-Based Format: Assessing Global Communication “Learners use their repertoire of knowledge and skills to create a product or a response, either individually or collaboratively ” (p. 404, Figure 11.4). PALS Project is an effective example of performance-based assessment. http://www.fcps.edu/is/worldlanguages/pals/ PALS tasks template (Figure 11.8, p. 407): Theme and topic (as determined by the school curriculum) A statement of the task objective The task description The minimal descriptions for completing the task Suggestions Directions

Interactive Model “Learners interact or are engaged in listening, reading, or viewing an authentic text, and they use that knowledge to communicate their opinions or to perform a related task” (p. 404, Figure 11.4). Swaffar, Arens, and Byrnes (1991) proposed a strategy for assessing interpretive reading, listening, and viewing, which integrated the Three Modes of Communication.

Key processing factors of Interactive model: Informational background: Informational background: the reader’s context Metacognition: How does the reader structure comprehension? Intent: Why is the text being read? The learner’s language ability As is illustrated in this design on page 409, that “the interactive format can test learners on their interpretive listening/reading/viewing abilities; grammatical, lexical, and cultural knowledge; ability to interact with the text; and presentational writing, all with the framework of a real context” (p. 409).

Authentic Assessment (p. 410) Is realistic (tests abilities in real-world situations) Requires judgment and innovation Use language in a meaningful way Contexts from workplace, civic life, or personal life Assesses ability to negotiate a complex task Opportunities for practice, consultation of resources, and feedback

Evaluating Authentic and Performance Based Tasks: Scoring Rubrics (p All rubrics contain three common features: They focus on measuring a stated objective They use a range to rate performance They contain specific performance characteristics, arranged in levels indicating the degree to which a standard of performance has been met

Elements of a typical rubric (p. 412-413) A scale of possible points to be assigned in scoring work, on a continuum of quality (highest being excellent) Descriptors for each level of performance Can be generic or task-specific (assesses overall impression) Can be holistic or analytic (assesses individual elements of language use)

Chelsea F.C. ESL Coordinator You have been hired to coordinate the English Language Program for Chelsea Football Club in London. Your job is to help international players improve their skills in spoken English. Today you will assess the English interpersonal and presentational speaking skills of several international players that the club is thinking of hiring.

General Information All perspective players are non-native speakers of English. You are looking for players that can handle post- match interviews and product indorsements in English. Design an analytic rubric to assess the players. Assess interview skills in authentic situations.

Design a Rubric for Assessing Potential Players (p.415) Step 1: Look at performance task and decide what the dimensions or criteria for performance should be. Task specific – Interviews Analytic or Holistic? Select 3 to 5 Criteria for evaluation

Design a Rubric for Assessing Potential Players (p. 415) Step 1: Look at performance task and decide what the dimensions or criteria for performance should be. Task specific – Interviews Analytic or Holistic? Select 3 to 5 Criteria for evaluation

Criteria for Evaluation Comprehensibility Confidence Impact of Performance Overall Polish and Organization Sophistication of knowledge or argument Focus on overall effectiveness of performance, more than specific content or grammatical points.

Step 2: Decide how many levels of performance to include (p. 415) 4. Exceeds Expectations 3. Meets Expectations 2. Almost Meets Expectations 1. Does Not Meet Expectations 4 should be reserved for performances that are truly exemplary Rubric should inform students of their actual level in each criteria. These scores do not have to directly translate into letter grades.

Step 3: Write the performance descriptions for each level of the rubric (p. 415) What would an excellent performance look like? Performance should resemble what will be required or desired in the real world. Start with 4. Exceeds Expectations and 1. Does Not Meet Expectations Fill in the other two levels after looking at the best and worst descripcions.

In tables, come up with descriptions for your assigned criteria Comprehensibility Confidence Impact of Performance Overall Polish and Organization Sophistication of knowledge or argument Be sure to include descriptions for each of the four levels 4. Exceeds expectation, 3. Meets expectation, 2. Almost meets expectation, 1. Does not meet expectation

In the FL classroom Step 4. Give each student a copy of the rubric (p. 415) Step 5. Constantly revise rubrics (p. 416) Keep in mind that scores on the rubric do not necessarily translate directly to letter grades. It is possible for nobody to receive a 4 or a 1.

Assessing Potential Players Watch each interview Pay special attention to how the player performs with respect to your teams criteria Score the player based on your teams levels of performance Discuss with your team

Zlatan Ibrahimovic - Sweeden http://www.youtube.com/watch?v=x2R28Avtbe g 1:25-2:30

Cristiano Ronaldo - Portugal http://www.youtube.com/watch?v=IaBziWYcw K4

Sergio Aguero - Argentina :35

Back to the classroom (p. 417) Raw scores cannot be converted directly to percentages Align rubric to schools grading philosophy B = “Meets expectations” F = 60% = “Does not meet expectations” There are many websites and computer programs that can help you do this

Standards-Based Integrated Performance Assessments (p. 418-424) Performance is effectively assessed in tasks that test learners’ knowledge and skills in authentic contexts Students need to know what a good performance looks like Students should receive high quality feedback Is closely linked to instruction and should lead to improvement in student performance

Integrated Performance Assessments (p. 418-424) Consists of a series of tasks at three levels: Novice Learner, Intermediate Learner, and Pre- Advanced Learner Should involve all three modes of communication Should be integrated with other goal areas of the standards, such as culture Cyclical approach including modeling, practice, performance, and feedback

Portfolio Assessment (p. 424-429) Can be used to assess language skills at a specific time or progress in language skills over time Helps students to assess and revise their own work Can be used to manage many different elements of language learning and use Often include self-reflections justifying the inclusion of each artifact Allows students to reflect on what they learned

Learner Self-Assessment (p. 429-430) Students assess what they are able to do and what they need to work on Can be used with another type of assessment, allowing students to see both what they think they can do and what they can actually do

Interactive Homework (p.430-433) Can help to involve parents in the language learning process Parents help student with homework assignment Since parents do not know the target language, they are also involved in the learning process

Oral Proficiency Interview (p.435 – 438) Standardized oral assessment (ch. 8) A trained interviewer converses with the student, gradually increasing the proficiency level needed, in order to assess the student’s proficiency in accordance with the ACTFL guidelines Teachers can adopt some aspects of the OPI in class assessments