Formative Assessments New Teacher Series Day 3 December 1, 2010 Stephanie Lemmer slemmer@kresa.org Sharon Dodson sdodson@kresa.org

Today’s schedule Survey The “I Cans” The 5 Keys of Assessments Purposes - Formative/Summative Learning Targets Assessment Maps

What you need to make today successful Handout Packet Content Expectations and/or learning standards for a course you teach

Current Beliefs and Practice Survey Current Beliefs and Practice Use of the responder system. Reinforce that there are no absolute correct answers but that over the course of the two days, we will be presenting information and soliciting their views regarding each of these.

Formative Assessment Critical Learning Objectives At the completion of today, it is our goal that you will be able to state the following: I can tell another person the difference between summative and formative assessments. I can articulate critical learning targets to my students in student-friendly language. I can match the appropriate type of assessment to a learning target for my students. I can determine if feedback is descriptive or evaluative. I can explain the importance of actively involving students in the assessment process. Introduction of the “I can” statement. Reasons for. Slightly different from the essential questions used in the UBD model. Use what works for you.

5 Keys of Quality Assessments Key 1 – Purpose Key 2 – Clear Learning Targets Key 3 – Effective Design Key 4 – Effective Communication Key 5 – Student Involvement

Personal Reflection Think of a time when you were assessed and it was a negative experience. What made it negative? Now think of a time when you were assessed and it was a positive experience. What made it positive?

Key 1 Purpose Seinfeldclip. What went wrong here? 8 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 8

What are purposes for assessment? To begin thinking about assessment quality, think about these questions: What is the purpose of assessment? How do we use the results? Take a few minutes to write down as many uses as you can think of. Handout page 1 Additional Presentation Activity #1 “Balance in the Classroom” fits here. © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 9

Classroom Assessments Think of the assessments you give. Why do you give them? List all of the reasons that come to mind. Use chart paper and markers. Develop as a grade level or table group. © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 10

Two Purposes for Assessment SUMMATIVE Assessments OF Learning How much have students learned as of a particular point in time? FORMATIVE Assessments FOR Learning How can we use assessments to help students learn more? We can divide the purposes of assessment into two categories: assessment of learning and assessment for learning. Summative assessment, or assessment of learning, measures the level of achievement at a point in time. Standardized tests and common assessments fall into this category. Any work that is evaluated that counts toward the report card grade we can consider an assessment of learning. So, if you think about all the assessments given over a trimester or quarter and how many of them are figured into the grade, you’ll discover that a lot of them, if not most of them, are assessments of learning. Formative assessment, or assessment for learning, on the other hand, is not an accountability tool, but a teaching tool. We can conduct assessments to make decisions about instruction before the learning process or during the learning process. For example, we conduct pretests to help us decide where to begin with certain groups of students, and we give students quizzes to help them decide what their strengths are and what they need to focus on. Handout page 2 Additional Presentation Activity #2 “Differences Between Assessment for and of Learning” fits here. © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 11

Assessments FOR and OF Learning Watch the video clip of Rick Stiggins discussing the differences between assessments FOR and OF learning. Track the differences on the chart. In your own words, briefly summarize the difference between formative and summative assessments? Time to discuss. Share the distinction. © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 12

Assessment for learning Assessment for learning Balanced Assessment Formative Formal and informal processes teachers and students use to gather evidence to directly improve the learning of students assessed Summative Provides evidence of achievement to certify student competence or program effectiveness (Paraphrase slide) You can see all three categories of decision-makers represented here. Remember, “formative” refers to the manner in which an assessment is used, not to any particular assessment itself. Formative assessments serve to support learning, not merely to verify it. Handout page 5 Assessment for learning Use assessments to help students assess and adjust their own learning Assessment for learning Use classroom assessments to inform teacher’s decisions Formative uses of summative data Use of summative evidence to inform what comes next for individuals or groups of students © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 13

Balanced Assessment “If we wish to maximize student achievement in the U.S., we must pay greater attention to the improvement of classroom assessment. Both assessment of learning and assessment for learning are essential. But one is currently in place, and the other is not.” Rick Stiggins, 2002

Assessment Research Study S.D. gains Bloom (1984) 1.0 – 2.0 Black and Wiliam (1998) 0.4 – 0.7 Miesels, et.al. (2003) 0.7 – 1.5 Rodriguez (2004) 0.5 – 1.8 FORMATIVE ASSESSMENT There is significant research showing that assessment for learning practices increase student motivation and achievement. Most famous--Two British researchers, Paul Black and Dylan Wiliam, reviewed 250 studies on the effects of formative assessment on achievement. Three Qs: Is there evidence that use of FA increase student ach? Improvement needed? What? 0.7 Standard Deviation Score Gain = 25 Percentile Points on ITBS (middle of score range) 70 SAT Score Points; 4 ACT Score Points Largest Gain for Low Achievers © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 15

Needed Improvements Increased commitment to high-quality formative assessments Increased descriptive feedback, reduced evaluative feedback Increased student involvement in the assessment process Black and Wiliam noted three fundamental changes that are necessary for schools to attain those gains: a commitment to developing high-quality formative assessments, increasing descriptive feedback and reducing evaluative feedback, and increased student involvement in the assessment process. These three findings are the foundation of the ETS ATI classroom assessment professional development program. We urge you to commit them to memory: commit to improving quality, provide feedback that is descriptive, and involve students every step of the way. Handout page 2 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 16

Key 2 Clear and Appropriate Learning Targets © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 17

Clear Learning Targets Know what kinds of targets are represented in curriculum Know which targets each assessment measures Communicate the learning targets in advance in language students can understand Key 2, Clear Targets, asks us to be clear about the targets we intend to teach and assess before we plan our assessments. We need to know what kind of target the content standard represents, so we teach to the intended level of cognitive challenge, and so we can select the appropriate assessment method. We need to know what targets each assessment measures so we can ensure that our tests match our teaching and so that we can keep track of learning standard by standard. And we need to make our targets clear to students in advance of, or during, the learning. Handout page 6 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 18

Clear Learning Targets: Research on Student Benefits Students who could identify their learning scored 27 percentile points higher than those who could not (Marzano, 2005) Making targets clear to students increases their achievement. (Read slide.) Students can hit any target they can see that holds still for them. This is common sense and many teachers do it already. The goal here is for all to fit this practice into their teaching and to understand why it increases student learning. Handout page 7 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 19

We Need Clear Learning Targets to. . . Ensure that there is a common understanding of what needs to be learned. Know if the assessment adequately covers what we taught. Correctly identify what students know and don’t know. Have students self-assess or set goals likely to help them learn more. Without clear targets we can’t do any of the following. (Read slide.) In short, we can’t assess well or communicate effectively. Handout page 7 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 20

Learning Targets Any achievement expectations we hold for students Statements of what we want students to learn We call our statements of expected learning by many names. Whatever term you use, just know that we refer to it as a learning target. © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 21

Which of these are actually Learning Targets? Senior project Model of a fort Present a persuasive argument State report Diorama If I think the state report is a learning target, how am I going to evaluate it? The diorama? © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 22

A Social Studies Example Chapter 10 Korean War World History Create a timeline Subject Topic Resource Are we doing the subject, topic, resource, and activity on our lesson plans without clearly specifying the learning target? Activity Understand recurring conflicts that lead to war Learning target © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 23

Kinds of Learning Targets Master content knowledge Use knowledge to reason and solve problems Demonstrate performance skills Create quality products We can think of achievement targets as falling into one of four categories: Knowledge targets involve things like math facts, important dates in history, and grammar rules—anything we want student to know outright. Reasoning targets are learning expectations that call for students to use the knowledge in some way—to create an hypothesis, or to analyze a political argument, for example. Performance skill targets call for students to do things like give an oral presentation, read aloud with fluency, or use equipment correctly. A performance skill target is something we have to watch or listen to in order to evaluate. Product targets specify that students will create a product—a fitness plan in health, a physical model in science, or a research report in English. We evaluate the characteristics of the product to give evidence of achievement of these kinds of learning targets. Handout page 6 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 24

Kinds of Learning Targets with Associated Verbs Knowledge Reason Skill Product List Predict Measure Construct Define Infer Demonstrate Develop Understand Classify Use Create Recognize Evaluate Operate Produce Explain Summarize Calculate

Learning Targets by Content Area What kinds of learning targets are most common in your content area? Elbow partner share © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 26

Converting Learning Targets to Student-Friendly Language Identify important or difficult learning goal. Identify word(s) needing clarification. Define the word(s). Rewrite the definition as an “I can” statement, in terms that your students will understand. Try it out and refine as needed. Have students try this process.

Student-Friendly Language Word to be defined: SUMMARIZE to give a brief statement of the main points, main events, or important ideas Student-friendly language: I can summarize text. This means I can make a short statement of the main points or the big ideas of what I read.

Student-Friendly Language Word to be defined: PREDICTION A statement saying something will happen in the future Student-friendly language:

Student-Friendly Language Word to be defined: PREDICTION A statement saying something will happen in the future Student-friendly language: I can make predictions. This means I can use information from what I read to guess at what will happen next.

Student-Friendly Language Learning Target: “Deeply examine policy issues…” Word to be defined: EXAMINE A process by which problems, alternate views and reasons for differing views for a given situation are understood. I Can Statement: I can “examine.” This means I can state the problems, describe alternative views, and understand the reasons for these different views. This is an example of a common learning target: make inferences. Do students know what an inference is? Without this understanding, they are likely to respond as one fifth-grader did: “I hate those questions where they don’t tell you the answer.” If we provide students with a definition that helps them understand what kind of thinking is involved, they will have a much greater chance of being able to do it when asked. Handout page 7 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 31

Your Turn… Choose either “analyze” or “describe” and convert it into student-friendly terms Definition: Student-friendly language: Finally have the tables do this for the standard they deconstructed previously.

Clear and Appropriate Learning Targets - Summary Things to remember Different types of targets Clarify targets by using student-friendly language Post targets or have students keep them (refer to targets) Connect learning targets to learning activities and assessments

From Curriculum Documents to Learning Targets The Assessment Map

Use the Assessment Map to Define Learning Targets Work time Use the Assessment Map to Define Learning Targets Need to determine the process

Key 3 Effective Design

Sound Assessment Design Select a proper assessment method Select or create quality items, tasks, and rubrics Sample—gather enough evidence Control for bias Design assessments so students can self-assess and set goals So, good assessment begins with a clear purpose and clear targets. Key 3 is Sound Assessment Design. High-quality assessments are not built first and then retro-fitted into a context. After we have established the purpose and identified the learning targets to be assessed, then we are ready to (read slide). Handout page 8 Additional Presentation Activity #5 “Why Accuracy Is Important” fits here. © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 37

Possible Assessment Methods Selected Response Extended Written Response Performance Assessment Personal Communication The assessment methods available to us fall into one of four categories: (read slide). The methods are not interchangeable; some fit some contexts but not others. No method is inherently superior to the others. All are viable choices, depending on two variables: purpose and target. Handout page 8 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 38

Sound Design Assessment methods match learning targets. Sample is representative of what was taught. Items, tasks and scoring guides are well-written. Bias is avoided.

Target X Method Match SR EWR PA PC KNOW REASON SKILLS PRODUCT

Target-Method Match Activity

+ + ? + + + + + Target - Method Match SR EWR PA PC KNOW REASON SKILLS This chart is one of the icons of the ETS ATI program. It shows the strong matches between kind of learning target and type of assessment. With practice, teachers learn which method to select and which methods to avoid for any given learning target, a requirement of high-quality assessment. Handout page 9 (In the text [CASL], each cell is filled in with a rationale for the strong and weak match, and several pages are devoted to an explanation of why each is strong or weak. Read that section [pages 95 – 106] carefully before using this slide.) SKILLS + + PRODUCT + © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 43

Okay but not Good Match efficient Good Match Reasoning Think-aloud Good match for Some patterns Of reasoning Reasoning Inferred by observation Think-aloud w/follow-up questions Possibly okay Good Match for Oral comm. only Good Match Good Match for Writ. comm. only Good Match

The Assessment Map Identify your “I cans” as Knowledge, Skill, Reasoning, or Performance items Next, select a method of assessment that would sample that ability effectively and efficiently.

Key 4 Effective Communication

Effective Communication “The most powerful single modification that enhances achievement is feedback. The simplest prescription for improving education must be ‘dollops of feedback’.” John Hattie (1992) “…all forms of feedback are not equally effective.”

Research Quotes on Effects of Feedback Read the quotes provided on the handout. Choose 1 quote that is most meaningful to you at this time.

Feedback On your own think about what you know to be the characteristics of effective feedback. Worksheet on feedback page 73 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 49

What Makes Feedback Effective? Describes features of work or performance Relates directly to the learning targets and/or standards of quality Points out strengths and gives specific information about how to improve We are all used to steady streams of evaluative feedback: B+, 91%, meets standards, and so forth. Mostly, we give students symbols and numbers to tell them how they’re doing, with nothing at all in those symbols and numbers to tell them what that could do differently or better. Descriptive feedback points out successes and gives specific information about how to improve the performance or product. Comments directed to the quality of the work--what was done well and what needs improving--increase student interest in the task and level of achievement. Frequently feedback is used to push students to “do more” or to “do better,” without being specific enough to help students know what to do. This type of feedback is generally ineffective. Descriptive feedback lacks praise or blame, and focuses on giving the student a recipe for improvement, during the learning, while there is still time to do something about it. And, it models the kind of thinking students will engage in when they self-assess. Handout page 10 (Note that on handout page 10, there is also a description of evaluative feedback. The information is not represented on the PowerPoint slide, but it is referred to in the talking points.) © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 50

Source  Characteristics of Feedback from Classroom Assessment  Number of Studies*  Effect Size  Percentile Gain or Loss in Student Achievement  Bangert-Drowns, Kulik, Kulik, & Morgan (1991)  Right/wrong  6  -.08  -3    Provide correct answer  39  .22  8.5  Criteria understood by students vs. not understood  30  .41  16  Explain  9  .53  20  Repeat until correct  4  Fuchs & Fuchs (1986)  Displaying results graphically  89  .70  26  Evaluation (interpretation) by rule  49  .91  32  *Indicates the number of studies that were examined by the researchers to compute an effect size. See Technical Note 1.2 for discussion of an effect size. 

Summary of the Research Formative classroom assessments should be frequent and provide many opportunities for feedback. Feedback should give students a clear picture of their progress on learning goals and how they might improve. Feedback on classroom assessments should encourage students to improve Marzano, 2006

Evaluative vs. Descriptive Feedback Evaluative feedback sums up achievement and assigns a label. It expresses a judgment. Descriptive feedback offers information that can be used by students to take action to improve.

Descriptive or Evaluative? Table Activity Mark each example of descriptive feedback with a D and each example of evaluative feedback with an E. If you believe it is neither, mark it with an X.

Effective Communication Provide students with descriptive feedback Involve students in tracking and communicating about their learning Use grading practices that accurately communicate about student learning The most accurate assessment is wasted if its results are miscommunicated, or if they are communicated to students in ways that shut learning down. In Key 4, we focus on what needs to be in place so that everyone who receives assessment information can use it effectively. This includes descriptive feedback, grades, portfolios, student-involved conferences, and standardized test scores. Each of these aspects of effective communication is an area of study in the CASL program. Example of the MEAP and being used by various groups incorrectly (e.g. in the paper, by real estate agents, even by us) Handout page 10 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 56

Effective Communication Continue Assessment Map

Key 5: Student Involvement © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 58

Student SELF-assessment is crucial for feedback to be used effectively Student SELF-assessment is crucial for feedback to be used effectively. Students are the ones who must ultimately take action to bridge the gap between where they are and where they are heading. The transition from feedback to self-monitoring can occur only when the student comes to know what constitutes quality. --Sadler, 1989 © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 59

Student as User of Assessment Information (Sadler) Where am I going? Where am I now? How can I close the gap? Relate a dibels example Any other examples © 2007 Educational Testing Service Assessment Training Institute www.ets.org/ati 503.228.3060 60

Student Motivation and Involvement Where am I going? Provide a clear statement of the learning target Use examples and models Where am I now? 3. Offer regular descriptive feedback 4. Teach students to self-assess and set goals How can I close the gap? 5. Design focused lessons 6. Teach students focused revision 7. Engage students in self-reflection; let them keep track of and share their learning

Student Involvement “The most important instructional decisions are made, not by the adults working in the system, but by students themselves.” CASL 2006

Emily’s Story: Assessment for Learning Read Emily’s story. Note what Emily’s teacher did to enhance student involvement, motivation, and achievement. Now read her writing samples. What does Emily have to say about this? (video) What does this look like in social studies?

Involving the Student Clear learning targets in student friendly language- made known at the outset to the student Instruction that models what success looks like Assessments that are fair – no surprises, no excuses

Wrap up “I Can” Review

Exit Card What is your next step? Wrap up Exit Card What is your next step?