Common Formative Assessments for Science Monica Burgio Daigler, Erie 1 BOCES

Review  Look at Deconstruction Template and Trend Map  Summarize work/progress thus far

Overview of CFAs Read handout from Larry Ainsworth, the CFA guru Tossed Terms Discuss

Summative -- Assessment OF Learning Summative assessment for unit, quarter, semester, grade level, or course of study Provides “status report” on degree of student proficiency or mastery relative to targeted standard(s) Stephanie L. Bravmann, Assessment’s “Fab Four.”Education Week, March 17, 2004, p. 56

Summative -- Assessment OF Learning Answers question: “Have students achieved the goals defined by a given standard or group of standards?” Helps teachers judge effectiveness of their teaching practices Intended to support the assignment of grades

Formative -- Assessment FOR Learning Formative: given before and during the teaching process Diagnostic: intended to be used as a guide to improve teaching and learning Answers key questions: Do students possess critical pre-requisite skills and knowledge? Do students already know some of the material that is to be taught?

Formative -- Assessment FOR Learning Provides teachers with information they need to create appropriate work for groups of learners or individual students Not typically used to assign grades

Assessment Inventory LIST all the important assessments you administer to students during the year. RANK each assessment as a 1, 2, or 3 in terms of having a real impact on instruction and student learning, with a “1” having the greatest impact and a “3” having the least impact.

Discussion Which assessments did you rank a “1”? Which assessments did you rank a “1”? Are you administering any assessments that ranked a “3”? Are you administering any assessments that ranked a “3”? Are there assessments that have no purpose? Are there assessments that have no purpose? Which of the assessments are used formatively? Which of the assessments are used formatively? What type of remediation is given? What type of remediation is given?

Step by Step…. How to create a quality CFA… How to create a quality CFA…

Steps to Creating a CFA Identify power standards from longitudinal data Determine topics to address with CFA Deconstruct test items to determine skills and big ideas Write essential questions matched to skills and big ideas Design CFA

General Item Writing Tips… Quality items should Quality items should –Reflect higher order thinking skills (HOTS) –Students should not be able to answer by recall, the should have to apply their knowledge –Be brief and clear—goal is to “test mastery of material, not students’ ability to figure out what you’re asking.” –Richard Stiggins

Things to Consider… Design fair and bias free items (no bias toward gender, ethnicity, or language) Design fair and bias free items (no bias toward gender, ethnicity, or language) Format items to match district benchmark assessments, final exams, and state tests Format items to match district benchmark assessments, final exams, and state tests

Top 5 Roadblocks to Effective Item Writing 1. Unclear Directions or Ambiguous Statements 2. Unintentional Clues 3. Complex Phrasing 4. Difficult Vocabulary W. James Popham, Test Better, Teach Better, 2003, p. 64

Two Major Types of Assessment Items Selected Response Constructed Response

Selected Response Requires students to select one response from a provided list or provide very brief answer Requires students to select one response from a provided list or provide very brief answer Types include: multiple choice, true-false, matching, short answer/fill-in Types include: multiple choice, true-false, matching, short answer/fill-in Can be used to effectively assess students knowledge of factual information, basic concepts, and basic skills Can be used to effectively assess students knowledge of factual information, basic concepts, and basic skills

Selected Response Benefit: Student answers can be quickly scored and objectively scored as correct or incorrect Benefit: Student answers can be quickly scored and objectively scored as correct or incorrect Drawback: Tends to promote memorization of factual information, rather than evidence of higher-level understanding—unless the items are deliberately designed to do so. Drawback: Tends to promote memorization of factual information, rather than evidence of higher-level understanding—unless the items are deliberately designed to do so.

Reasons for Selected Response Better content domain sampling Better content domain sampling Higher reliability Higher reliability Greater efficiency Greater efficiency Objectivity Objectivity Measurability for HOTS Measurability for HOTS Mechanical Scoring Mechanical Scoring Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, pp

Reasons Against Selected Response Emphasis on learning of isolated facts (teaching to the test) Emphasis on learning of isolated facts (teaching to the test) Inappropriate for some writing/creative purposes Inappropriate for some writing/creative purposes Lack of student writing Lack of student writing Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, pp

Multiple Choice to Assess Higher Level Thinking? Common myth is that multiple choice items ONLY assess lower-level thinking skills and therefore will not be appropriate for evaluating students higher-level thinking skills.

Dispelling the Myth “Research has not yet been done to prove this point. If you want to measure understanding and some types of mental skills and abilities, multiple choice formats can be written to measure these behaviors effectively.” Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, p. 98

Criteria for Writing Selected Response Items 1. Write clearly in a focused manner 2. Ask a question with only one best answer 3. Write items consistent with grade level reading expectation 4. Eliminate clues leading to correct answer 5. Make response options brief Adapted from Richard J. Stiggins, Student-Centered Assessment, 2001.

Constructed Response Includes short-and extended response (i.e. short answer prompts, essays, and problem solving requiring writing Includes short-and extended response (i.e. short answer prompts, essays, and problem solving requiring writing Requires students to organize and use knowledge and skills to answer a question or complete a task Requires students to organize and use knowledge and skills to answer a question or complete a task Requires scoring guide (rubric) to evaluate degree of student proficiency Requires scoring guide (rubric) to evaluate degree of student proficiency

Constructed Response Items Benefit: Provide teachers with more valid inferences about student understanding than those derived from selected response items Benefit: Provide teachers with more valid inferences about student understanding than those derived from selected response items Drawbacks: Take longer to score; can have errors in design; dependent on student writing proficiency; challenge to score accurately Drawbacks: Take longer to score; can have errors in design; dependent on student writing proficiency; challenge to score accurately

More Valid Inferences from Constructed Response “Because a student really needs to understand something in order to construct a response based on that understanding students’ responses to these sort of items will better contribute to valid inferences than will students’ answers to selected-response items” W. James Popham, Test Better, Teach Better, 2003, p. 87

Resources for Common Formative Assessment Items Textbook Questions (that meet criteria for well-written items) Textbook Questions (that meet criteria for well-written items) Assessment or evaluation components of text series Assessment or evaluation components of text series State Exams State Exams Check for permission to duplicate copyrighted material! Check for permission to duplicate copyrighted material!

Creating a Scoring Guide Scoring Guide (Rubric) Scoring Guide (Rubric) –A set of general and/or specific criteria used to evaluate student performance on a given task –Descriptions of competence or proficiency –Identifies degree of proficiency student has reached in relation to particular standard/content area Proficient Proficient –The level of performance students must meet to demonstrate attainment of particular standards

Avoid Subjective Language Such as… Such as… –Complete –Partial –Adequate –General –Successful –Good –Nice –Some –Few –Many –Most –Little –Creative

Strive for Objective Language Language that is Language that is –Specific –Measurable –Observable –Understandable –Matched to task directions

Next Steps…