Review course concepts Module 8: Review

Course Review This PPT will review concepts that we have covered this session. We will review: Course objectives Course modules Course concepts Future directions

Course Objectives

Course Objectives This course had 8 course objectives. Review the objectives on the next slide and try to remember a few things that you learned for each objective. Consider whether you feel better prepared as a teacher in each of these assessment areas.

8 Course Objectives Understand and apply key concepts of assessment Understand bottom-up and top-down subskills involved in reading, listening, speaking, and writing Identify text features of reading and listening passages Identify key aspects of rubrics for speaking and writing tasks Understand how questions relate to test design Understand features of good questions and prompts Identify online assessment-related resources Understand how to review English language tests

Course Modules

Course Modules There were 8 modules in this course. Review the modules on the next slide and try to remember a few things that you learned within each module. Consider whether you feel better prepared as a teacher in each of these assessment areas.

Course Modules Introduction Reading Listening Speaking Writing Test Design and Statistics Task-based Assessment and Professional Development Review of Key Concepts from the Class

Course Concepts

Assessment Design The first three steps in creating an assessment are: Defining your test construct Defining the test purpose Defining the target language use

Assessment Design The cycle of creating an assessment consists of the following steps: Think about the purpose of the test Define the construct you want to measure Decide what content/student learning outcomes to assess Decide what items to use to assess the content Create a table of specifications Write the items Revise items

Reading In bottom-up processing, readers start with the smallest units of language. They understand each unit and then fit them together into a larger text. Readers use bottom-up processing with: Word recognition Phonological processing Orthographic processing Semantic and syntactic processing

Reading In top-down processing, readers understand the big picture of what they are reading. Top-down processing involves activation of schemata. (Schemata is a reader’s prior knowledge.) Readers use top-down processes when they Understand the main idea of a text, Understand the purpose of a text, Relate the text to their own knowledge, and Make inferences that are probable.

Reading Item Formats Common item types to use with reading: Multiple choice Sentence Completion Fill-in-the-blank True/false/not stated Short answer Matching

Reading passages When choosing reading passages, it is important to consider the following things: Students’ purpose Type of text Similarity of the texts Number of texts

Reading passages Features that contribute to the difficulty of a reading passage: Background knowledge vs. text knowledge Presence of text while answering questions Text length Item dependence

Reading passages To understand the complexity of a reading passage, it may be useful to use one of these three readability tools: Flesch-Kincaid (Embedded in Microsoft Word) Lexile Analyzer Compleat Lexical Tutor

Vocabulary There are many aspects of vocabulary that can be assessed. Here are a few: Receptive vs. Productive vocabulary knowledge Selective vs. Comprehensive vocabulary knowledge Discrete vs. Embedded Context independent vs. Context dependent Size vs. depth High frequency vs. low frequency Specialized / technical Concrete vs. abstract Grammatical class Collocations

Vocabulary A few common item formats to test vocabulary: Multiple choice Matching Sentence completion Fill-in-the-blank Short answer

Listening Below are tasks that illustrate bottom-up processing: 1. Recognize word divisions 2. Recognize key words 3. Recognize key transitions in a discourse 4. Use knowledge of word-order patterns to identify discrete parts of spoken language 5. Recognize grammatical relations between elements in sentences 6. Recognize the function of word stress in a sentence 7. Recognize the function of intonation in a sentence

Listening Below are tasks that illustrate top-down processing: 1. Ability to identify the gist or main idea of a discourse 2. Ability to identify the purpose of a discourse 3. Ability to deduce meaning of words from context 4. Ability to make textually-based inferences 5. Ability to distinguish between literal and figurative meanings 6. Ability to distinguish between facts and opinions

Listening passages Features that impact the difficulty of listening are: Content Form Grammar Phonology Discourse Type Register Context

Listening item formats Gap-fill Note-taking Multiple choice Completing diagrams or tables Multiple choice with pictures Sorting information Short answer questions Spoken response True / false / not stated Written response Matching

Speaking Let’s quickly review a few common functions of speech: Asking for information Giving information Expressing a reaction Negotiating Problem solving Small talk Politeness Making excuses

Speaking Here are a few tasks you can use to assess speaking: Description task Narrative task Instruction task Decision task Comparing and Contrasting task Explaining and Predicting task Structured Speaking task Role-play Interview

Speaking task A speaking task should include: -Purpose of the task -Time limit -Approximate length of response -Directions for responding -How responses will be scored/graded

Scoring speaking Steps to follow when scoring speaking: Choose the type of rubric Find rubric or design rubric Share rubric with students Choose how many raters (single rater or double rater) Train the rater(s) Rate live speaking performance or rate delayed speaking performance (a recording)

Writing Knowing how to write involves many subskills, including: Knowledge of writing (orthography, spelling, punctuation) Knowledge of phonology (sound/letter relationships) Vocabulary knowledge (multiple definitions for the same word) Syntactic knowledge (grammar, structure) Knowledge of written organizational patterns Knowledge of register (formal/informal)

Writing Three ways to assess writing are: Indirect Direct Portfolio

Writing We can identify some characteristics of writing tasks that will vary from task to task. These variations can make tasks easier or more difficult. These characteristics include: Topic Stimulus Genre Audience Time allowed Length

Writing A writing task should include: -Purpose of the task -Time limit -Approximate length of response -Directions for responding -How responses will be scored/graded

Rating scales Remember, rating scales (or rubrics) consist of the following three elements: Sub-Construct Score Scale definition / Scale descriptor

Holistic rubrics Strengths Weaknesses Emphasizes what student does well One score does not provide diagnostic information Reduces risk of the rater focusing on one aspect of writing (e.g., grammar) It is difficult to interpret a single score Students may have uneven abilities across subskills Reduced reliability

Analytic rubrics Strengths Weaknesses Training raters is easier when there are different categories Writing is more than the sum of its parts Analytic rubrics keep raters from collapsing categories Information from analytic rubrics may not be easy for students to interpret There is no assurance that analytic rubrics will be used the way that they are intended.

Grammar Grammar knowledge consists of several components, including: Phonological form (e.g., segmental forms, minimal pairs) Lexical form (e.g., spelling, word formation) Morphosyntactic form (e.g., affixes and suffixes, word order)

Grammar item formats Grammar can be assessed using the following item types: Multiple choice Multiple choice error identification Discrimination task Noticing task Fill-in-the-blank Matching Short answer Role play tasks

Reliability Inter-rater reliability: the amount of agreement between two raters Intra-rater reliability: how consistent a rater is with him or herself Test reliability: how consistent the measure is. This refers to the results we get from a test or measurement instrument, not to the measurement instrument itself.

Reading Reading is an active process. It is the result of a combination of bottom-up and top-down processes. Bottom-up and top-down processing refers to the way that people internalize and understand information. This knowledge has implications for assessment. Reading comprehension tests often include questions to assess both bottom-up and top-down processes.

Future Directions

Future directions We hope that you will use the information you have learned from this course to: Use your checklist to evaluate tests. Share what you have learned with other teachers through articles and presentations. Be an assessment resource for teachers and students at your school. Continue to use language testing resources on the internet to refresh and refine your knowledge of language assessment.

Next Steps Now, you should do assignment #1. In this assignment, you will create a personalized language test checklist for your context.