1. Going Deeper with TBTs and the Ohio 5-Step Process Getting Results: Impacting Each Student within All Students Going Deeper with TBTs and the Ohio 5-Step Process 2011-2012 SPDG Professional Development

2. Our Targets For Today COMMON Summative & Formative Questions Embedded Learning Targets Categories of Learning Aligning Assessments Formative Classroom Assessments

3. Data: Digging Deeper in Steps 1 and 2 of Ohio 5-Step Protocol This document/product/software was supported in whole or in part by the U.S. Department of Education, Office of Special Education Programs, (Award #Q27A090111A, CFDA 84.027A, awarded to the Ohio Department of Education). The opinions expressed herein do not necessarily reflect the policy or position of the U.S. Department of Education, Office of Special Education Programs, and no official endorsement by the Department should be inferred. Teacher Based Teams

4. The Ohio 5-Step Process: A Cycle of Inquiry

5. Teacher Based Teams: Steps 1 & 2 Training Outcomes Analyze “common” assessment questions to determine: – Summative or Formative in Nature – Distracters Deconstruct embedded learning targets and identify category of learning for each Apply knowledge of how to determine cut scores. Apply and analyze the use of classroom formative assessments

6. 6 Roles and Responsibilities Working Agreements Facilities Getting Started Parking Lot Materials

7. Working Agreements Stay Focused Manage Electronic Devices Practice Timely Attendance Pass Notes instead of Sidebar Talking Respectfully Challenge One Another Listen Actively Participate to the Fullest of Your Ability

8. Quick Check for Learning Anticipation Guide Statements that can be reacted to without having read the text. Statements that challenge beliefs. General rather than specific statements. Statements that relate to the upcoming content. Formative classroom assessments for learning

9. Inclusion Activity Anticipation Guide 1.Answer each question 2.Talk at your table about your responses

10. Do you agree? 1.Yes 2.No Think of literacy as a spine; it holds everything together. The branches of learning connect to it, meaning that all core content teachers have a responsibility to teach literacy. 1234567891011121314151617181920 2122232425262728293031323334353637383940 4142434445464748

12. Do you agree? 1.Yes 2.No Offered a list of standards, we should scrutinize each one but also ask who came up with them and for what purpose. 1234567891011121314151617181920 2122232425262728293031323334353637383940 4142434445464748

13. Do you agree? 1.Yes 2.No We test too much. 1234567891011121314151617181920 2122232425262728293031323334353637383940 4142434445464748

14. Summative district and state assessments (aggregated, disaggregated; srand, item, and student work) Data about people, practices, perceptions (e.g., demographic, enrollment, survey, interview, observation data, curriculum maps) Benchmark common assessments (e.g., end-of-unit, common grade-level tests reported at item level; aggregated, disaggregated; strand, item, and student work) Formative common assessments (e.g., math problem of the week, writing samples, science journals, other student work) Formative classroom assessments for learning (e.g., student self-assessments, descriptive feedback, selected response, written response, personal communications, performance assessments) Annual 2-4 times a year Quarterly or end of unit 1-4 times a month Daily - Weekly The Data Coach’s Guide: Love, Stiles, Mundry & DiRanna, c. 2008 Embedded throughout our Session

15. Summative district and state assessments (aggregated, disaggregated; srand, item, and student work) Data about people, practices, perceptions (e.g., demographic, enrollment, survey, interview, observation data, curriculum maps) Benchmark common assessments (e.g., end-of-unit, common grade-level tests reported at item level; aggregated, disaggregated; strand, item, and student work) Formative common assessments (e.g., math problem of the week, writing samples, science journals, other student work) Formative classroom assessments for learning (e.g., student self-assessments, descriptive feedback, selected response, written response, personal communications, performance assessments) Annual 2-4 times a year Quarterly or end of unit 1-4 times a month Daily - Weekly The Data Coach’s Guide: Love, Stiles, Mundry & DiRanna, c. 2008 More summative in the types of questions Questions are based on specific deconstructed learning targets

16. We must create/use assessments that will gather data that are the right “grain size.” That is, the information teachers need from formative assessments must be specific enough to be able to tell them what to do next in their classrooms. Kim Marshall (2008).The Teacher as Assessment Leader, Edited by Guskey, pg. 43

17. How We Assess Learning Target Students will be able to solve problems requiring them to add fractions with like and unlike denominators.

18. SummativeFormative Solve: ½ + ¾ = (a)4/6 (b) 1 (c) 1 1/4 (d) 4/8 What is the least common denominator for the following fractions? 1/2 2/3 3/4 (a) 24 (b) 12 (c) 6 (d) 9 What Is the Difference Between These Two Items?

19. Formative assessment provides information about how to differentiate classroom instruction for the next lessons. Assessment that Makes Sense, Chris Jakicic, pg. 44 & 42 These (formative assessments) are not improvised or spontaneous; rather, they are administered at a particular time in the teaching-learning process for the teacher to get feedback about how to proceed.

20. Summative district and state assessments (aggregated, disaggregated; srand, item, and student work) Data about people, practices, perceptions (e.g., demographic, enrollment, survey, interview, observation data, curriculum maps) Benchmark common assessments (e.g., end-of-unit, common grade-level tests reported at item level; aggregated, disaggregated; strand, item, and student work) Formative common assessments (e.g., math problem of the week, writing samples, science journals, other student work) Formative classroom assessments for learning (e.g., student self-assessments, descriptive feedback, selected response, written response, personal communications, performance assessments) Quarterly or end of unit 1-4 times a month Daily - Weekly The Data Coach’s Guide: Love, Stiles, Mundry & DiRanna, c. 2008 ½ + ¾ = (a) 4/6 (b) 1 (c) 1 1/4 (d) 4/8 What is the least common denominator for the following fractions? 1/2 2/3 3/4 (a) 24 (b) 12 (c) 6 (d) 9

21. Can we mix summative and formative type questions in the same assessment?

23. I Don’t Like “Tricking” Students Multiple choice items MUST be created with “diagnostic” distracters that can pinpoint what is understood and not understood. They allow for teaching with precision. Distracters are more difficult to write than the correct answers!

24. Best Distracters A plant is able to grow larger because A. it gets its food from the soil. Misconception A student who chooses this answer does not understand that nutrients are manufactured internally by the plant. B. it turns water and air into sugar. Oversimplification The student understands that food is manufactured internally but does not understand that water & carbon dioxide (from the air) are used to make sugar and oxygen. C. it has chlorophyll to produce food. Overgeneralization The student does not understand that some parasitic plants do not contain chlorophyll. D. it adds biomass through photo- synthesis. Correct answer

25. Activity: Assessment Type In small groups, analyze the assessment to determine whether each question is summative or formative. How do you know? Choose 1 summative question and rewrite it as formative. Choose 1 question and determine what kind of distracters are used.

26. “We have trusted those we believe to be more knowledgeable in test item development and statistical validity and reliability to develop our assessments.” “The problem isn’t that test makers have failed in their task; the assessments that textbook and testing companies have designed have accomplished exactly what they are capable of accomplishing.“ So What May Be Missing in Company-Generated Test Items? Popham 2001, 2008a

27. Summative district and state assessments (aggregated, disaggregated; srand, item, and student work) Data about people, practices, perceptions (e.g., demographic, enrollment, survey, interview, observation data, curriculum maps) Benchmark common assessments (e.g., end-of-unit, common grade-level tests reported at item level; aggregated, disaggregated; strand, item, and student work) Formative common assessments (e.g., math problem of the week, writing samples, science journals, other student work) Formative classroom assessments for learning (e.g., student self-assessments, descriptive feedback, selected response, written response, personal communications, performance assessments) Annual 2-4 times a year Quarterly or end of unit 1-4 times a month Daily - Weekly The Data Coach’s Guide: Love, Stiles, Mundry & DiRanna, c. 2008 HOW MUCH IS TOO MUCH!?

28. Frequent Testing/Effects of Testing.34 Hattie 2009

29. Feedback.73 Hattie 2009

30. What Does Feedback Mean? “The mistake I was making was seeing feedback as something teachers provided to students – they typically did not, although they made claims that they did at the time, and most of the feedback they did provide was social and behavioral. It was only then when I discovered that feedback was most powerful when it is from the student to the teacher that I understand it better. When teachers seek, or at least open to, feedback from students as to what students know, what they understand, where they make errors, when they have misconceptions, when they are not engaged – then teaching and learning can be synchronized and powerful. Feedback to teachers helps make learning visible. “ (Hattie 2009 pg. 173)

31. Formative Evaluation! 0.90 Hattie 2009

32. Pay attention to the formative effects of your teaching, as it is these attributes of seeking formative evaluation of the effects (intended and unintended) of the programs that makes for excellence in teaching. Hattie pg. 181 2009 Formative Evaluation! 0.90

33. TRY USING THIS SCENERIO… T1 We assess too much. I don’t have time for teaching. T2 Do you watch basketball? T1 Yeah T2 During a recent high school basketball playoff game, the scoreboard completely malfunctioned midway through the game. So the refs kept the score and time on the sidelines. As it came close to the end of the game, the visiting team was down by 2 points, but they did not realize it nor how much time was left. The clock ran out before they took their final shot. T1 That’s not right!

34. T2 Of course not. If the scoreboard had been working, the entire end of the game could have been different. So you’d agree that a working scoreboard is critical for sporting events, correct? T1 Of course. T2 At the end of the day, assessments where students give teachers feedback is like fixing the scoreboard. Relying on state tests is like covering up the scoreboard at the beginning of the game and then uncovering it at the end of the game to see if you won. At that point, there’s nothing you can do to change the outcome! We use benchmark common assessments to keep the scoreboard uncovered so we can make the necessary adjustments to be able to win the game.

35. 3-2-1 Write 3 key terms related to what you know or have learned. Write 2 questions you have. Write 1 way you can apply your learning. Formative classroom assessments for learning Quick Check for Learning

36. So… IF: We understand … “the way assessment questions are written determines the kind of data we receive,” THEN: We need to ensure that all of the learning targets embedded in our standards are identified. Do I need summative or formative data? I need to make sure I assess the embedded targets.

37. What Is “Deconstructing”? “Taking a broad and/or unclear standard, goal, or benchmark and breaking it into smaller, more explicit learning targets that can be incorporated into daily classroom teaching” Stiggins et al. 2006 aka Unwrapping & Unpacking

38. Deconstructing Standards into a Progression of Learning Targets A systematic process to identify embedded learning targets in standard indicators so that nothing essential is missed during instruction Learning targets: What students should know, understand and be able to do to master the indicators Sequence the targets into a developmentally appropriate learning progression

39. Deconstructing the Golf Game Will be able to perform basic golf skills, model acceptable social behavior and apply fundamental knowledge of the rules of golf to enable participation in a regulation or modified game of golf.

40. 1.Perform Basic Golf Skills Proper grip Stance Alignment of feet to target Distance of body from ball Bend at the waist Swing fundamentals Wrist position at top of swing 2. Acceptable Social Behavior All swings count towards your score Clubs are not thrown Mark your ball when it is on the putting green Keep the pace of play Do not talk or make noise while others are swinging

41. Tips for Deconstructing Analyze the wording of the standard/objective to determine key concepts and key skills  Read through indicators  Circle verbs to identify key skills  Underline nouns and noun phrases to identify key concepts Example: Create grade-appropriate real-world problems involving any of the four operations using multiple strategies, explain the reasoning used, and justify the procedures selected when presenting solutions.

42. Deconstructing Practice Activity: Describe what happens when two forces act on an object in the same or opposing directions. 1. I can give the definition of force. 2. I can tell what 3 changes might happen to an object when a force is applied. 3. I can write in words and also create a diagram to explain what happens when two forces act on an object in the same direction. 4. I can write in words and create a diagram to explain what happens when two forces act on an object in opposite directions. Example

43. 12th Grade Research Content: Short Research Projects Sustained Research Projects Self-Generated Questions Multiple Sources of Information Relevant Information Authoritative Print Sources Digital Sources Advanced Searches Evaluation of Sources Flow of Ideas Audience & Purpose Plagiarism Citation Format Writing Standard: Research to Build and Present Knowledge 7. Conduct short as well as more sustained research projects to answer a question (including a self-generated question) or solve a problem; narrow or broaden the inquiry when appropriate; synthesize multiple sources on the subject, demonstrating understanding of the subject under investigation! 8. Gather relevant information from multiple authoritative print and digital sources, using advanced searches effectively; assess the strengths and limitations of each source in terms of the specific task, purpose, and audience; integrate information into the text selectively to maintain the flow of ideas, avoiding plagiarism and overreliance on any one source and following a standard format for citation. Writing Standard: Research to Build and Present Knowledge Example

44. New Content Standards Choose a content area from the samples on your table. In Pairs or Triads, “Deconstruct, Unwrap, Unpack” one of them and record the Embedded Learning Targets in the first column of the template. At your tables share your work. Standard Embedded Learning Targets Type of Target: Knowledge, Reasoning, Skill, or Product Best Assessment Method

45. Quick Check for Learning Linking 1st person makes a statement about something s/he has learned. Person to the right “links” to the 1 st person’s statement. Continue the “links” until everyone at the table has participated. Formative classroom assessments for learning

46. Four Categories of Learning 1. Knowledge: Fact-based learning easily assessed 2. Reasoning: Require application of knowledge to make comparisons, predictions, or evaluative judgments 3. Skills: Require an observable demonstration of the proper application of knowledge. 4. Products: Require application of knowledge to generate specific, predefined pieces of work common to a particular domain of study Stiggins et al. 2006

47. Knowledge Examples Identify metaphors and similes Read and write quadratic equations Describe the function of a cell membrane Know the multiplication tables Explain the effects of an acid on a base

48. Reasoning Examples Use statistical methods to describe, analyze, evaluate, and make decisions. Make a prediction based on evidence. Examine data/results and propose a meaningful interpretation. Distinguish between historical fact and opinion.

49. Performance/Skill Examples Measure mass in metric and SI units Use simple equipment and tools to gather data Read aloud with fluency and expression Participates in civic discussions with the aim of solving current problems Dribbles to keep the ball away from an opponent

50. Product Examples Construct a bar graph Develop a personal health-related fitness plan Construct a physical model of an object Write a term paper to support a thesis

51. Standard/Benchmark: Produce writing to communicate with different audiences for a variety of purposes. Learning Targets: What are the knowledge, reasoning, skill, or product targets underpinning the standard? Knowledge Targets Skill Targets Reasoning Targets Product Targets Know what a sentence is Understand concept of word choice Distinguish the uses or meanings of a variety of words (word choice) Hold a pencil correctly Print letters correctly according to DN methods Space words Use lines and margins correctly Stretch out sounds in words to create a temporary spelling of the word Write sentences with varied beginnings. FIRST GRADE

52. Original Terms New Terms Evaluation Synthesis Analysis Application Comprehension Knowledge Creating Evaluating Analyzing Applying Understanding Remembering (Based on Pohl, 2000, Learning to Think, Thinking to Learn, p. 8)

54. Back to Your Deconstructing Template: Complete the 2 nd column (Category of Learning) for each of the embedded learning targets that you identified earlier. Standard Embedded Learning Targets Type of Target: Knowledge, Reasoning, Skill, or Product Best Assessment Method

55. Quick Check for Learning Whip Around Make a list of 3 items related to your understanding and application of types of targets. Everyone stands. One at a time each person shares one item on his/her list. Others cross off if their own item is duplicated. Sit when your 3 items have been stated. Formative classroom assessments for learning

56. Aligning Learning Targets to Appropriate Assessments Type of Target to Be Assessed Assessment Method Selected Response Constructed Response/ Essay Performance Assessment Personal Communication KnowledgeGood Match NOT a Good Match Partial Match ReasoningPartial MatchGood Match Performance Skills NOT a Good Match Good MatchPartial Match ProductNOT a Good Match Partial MatchGood MatchNOT a Good Match Stiggins 2006

57. Considerations in Creating Assessments… Is it possible for me to create an assessment that everyone in my class can pass? Is it possible for me to create an assessment that virtually no one in my class could pass?

58. Important to Remember…  Include both formative level assessment items as well as several advanced (or summative) items  There is no magic number of items to include at each level of proficiency; simply ensure that multiple opportunities exist for students to demonstrate those varied levels.

59. Quick Check for Learning Formative classroom assessments for learning Bump in the Road Write down something that you find confusing or a skill or concept you find difficult.

60. Quick Check for Learning Formative classroom assessments for learning Application Cards Think of one way to apply your new knowledge or skill in the real world and write it down on an index card.

61. Summative district and state assessments (aggregated, disaggregated; srand, item, and student work) Data about people, practices, perceptions (e.g., demographic, enrollment, survey, interview, observation data, curriculum maps) Benchmark common assessments (e.g., end-of-unit, common grade-level tests reported at item level; aggregated, disaggregated; strand, item, and student work) Formative common assessments (e.g., math problem of the week, writing samples, science journals, other student work) Formative classroom assessments for learning (e.g., student self-assessments, descriptive feedback, selected response, written response, personal communications, performance assessments) Annual 2-4 times a year Quarterly or end of unit 1-4 times a month Daily - Weekly The Data Coach’s Guide: Love, Stiles, Mundry & DiRanna, c. 2008

62. Frequently check for understanding: Are the kids – all of the kids – following what you are saying? Asking “Does anyone have any questions?” does not work. (Ripley, 2010, p.5)

63. Anticipation Guide 5+1, 10+1, 15+1 Pair/Share 3-2-1 Linking Whip Around Graphic Organizer Bump in the Road 5 Seconds Application Cards Formative Classroom Assessments for Learning

64. What Do You Use? Formative Classroom Assessments for Learning

65. Samples A.Student Reflection Chart B.Checklist for Creating Common Assessments C.Checklists for using Formative/Summative Assessments 1.Divide samples at your table 2. Review (2 minutes) and present to the rest of your group (1 minute per sample) A. Benefits of Use B. Ideas for Adaptation

66. Our Targets For Today COMMON Summative & Formative Questions Embedded Learning Targets Categories of Learning Aligning Assessments Formative Classroom Assessments

67. QUESTIONS?????

68. References Bambrick-Santoyo, Paul. (2010). Driven by data. Jossey-Bass Blankstein, A. (2010). Failure is not an option: 6 principles for making student success the only option. Facilitators Guide. Corwin Fisher and Frey. (2007). Checking for understanding: formative assessment techniques for your classroom. ASCD Greenstein, L. (2010). What teachers really need to know about formative assessment. ASCD Gusky, et. al. (2009). The teacher as assessment leader. Solution Tree Press Hattie, John. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge Schmoker, Mike. (2011). Focus: Evaluating the essentials to radically improve student learning. ASCD

69. This document/product/software was supported in whole or in part by the U.S. Department of Education, Office of Special Education Programs, (Award #Q27A090111A, CFDA 84.027A, awarded to the Ohio Department of Education). The opinions expressed herein do not necessarily reflect the policy or position of the U.S. Department of Education, Office of Special Education Programs, and no official endorsement by the Department should be inferred.