1. Professional Development to Practice The contents of this presentation were developed under a grant from the US Department of Education to the Missouri Department of Elementary and Secondary Education (#H323A120018). However, these contents do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government. Professional Development to Practice Writing Constructed Response Items Common Formative Assessment

2. Professional Development to Practice

3. Writing Constructed Response Items Developing Meaningful Learning Targets Quality Assessment Design Performance Events Constructed Response Items Selected Response Items Common Formative Assessment (CFA) Overview and Purpose of CFA

4. Professional Development to Practice What are Constructed-Response (CR) Questions?  Formulate own answers  Construct own responses  Closed and Open-Ended CRs

5. Professional Development to Practice What do you think? What is the difference between an open-ended constructed response item and a closed constructed response item? Jana Scott, University of MO-Columbia, Revised Presentation 2013.

6. Professional Development to Practice Constructed Response Questions CLOSED CRs  Measure knowledge and comprehension  Elicit a right or wrong answer  No tolerance for diversity  Usually allow for only one way to arrive at a correct answer OPEN-ENDED CRs Measure higher-level cognitive processes such as application, inference, analysis and synthesis Elicit diverse answers Mild to high tolerance for diversity Usually allow for more than one way to arrive at an answer

7. Professional Development to Practice Degrees of Open-Endedness Closed CR Minimal Open- Ended CR Medium Open- Ended CR Extended Open- Ended CR

8. Professional Development to Practice Closed 0-1Minimal Open Ended 0-2Medium Open Ended 0-3Extended Open-Ended 0-4 Closed CR items have two distinct levels of performance….right or wrong or acceptable and not acceptable. The range of responses yielded by minimal constructed response items necessitates the use of three distinct levels of performance. The range of responses yielded by medium constructed response items necessitates the use for four distinct levels of performance. The wide range of responses yielded by extended constructed response items necessitates the use for five distinct levels of performance. (Sometimes referred called Extended Responses.) Example: What is the setting of the story Little Red Riding Hood (LRRH)? Example: What is the setting of the story LRRH? How do you know? Example: What is the setting? Explain the impact on the story if the setting was changed?. Use two specific examples from the story to support your answer. Example: Change the setting of LRRH to a city. Explain how the plot, characters, and major story events are impacted and how they are different than the original story. Usual time for completion is about 2-3 minutes. Usual time for completion is about 4-7 minutes. Usual time for completion is about 8-11 minutes. Usual time for completion is about 12-15 minutes. Jana Scott, University of MO-Columbia, 2012. Constructed Response Fall on a Spectrum of Open- Endedness (Handout)

9. Professional Development to Practice Inquiry Activity  What characteristics do open-ended constructed response (OCR) questions possess?  Samples in handout packet.

10. Professional Development to Practice OCR Questions  Application  Complex cognitive processes  Elicit diverse answers  Multiple ways to arrive at an answer  Support or justification,  More than one component piece.

11. Professional Development to Practice OCR Questions  Making an inference  Interpreting information  Logical responses based on givens  Clear concise answer  Completed anywhere from 2-15 minutes (depending on type

12. Professional Development to Practice Different Names for Same Type of Item (Handout) Missouri TermsSmarter Balanced Math Smarter Balanced ELA Assessment For Learning Leadership and Learning Center Closed Constructed Response Items 0-1 Referred to as fill-in-the- blank or short answer questions Not referencedReferred to as short answer written response Does not consider Fill-in the Blank to be CR Items Referred to as fill-in the blank and short answer questions Minimal Open-Ended Constructed Response 0-2 Medium Open-Ended Constructed Response Item 0-3 Referred to as brief constructed response questions Usually part of the computer-adaptive section Referred to as 2 Point CRs or 3-Point CRs Referred to as short written response items Referred to as short constructed response Extended Open-Ended Constructed Response Items 0-4 Performance Events or Tasks Any CR Item that is part of the Performance Task will be referred to as Extended Response (ER) items. Performance tasks are a series of scaffolded CRs (called ERs) related to a theme. Not referenced Part II of Performance Task is a Writing Prompt Referred to as extended written response items Referred to as Performance Task Referred to as extended response

13. Professional Development to Practice Critique Form for OCRs (Handout) Except Mathematics

14. Professional Development to Practice Critique Form for Higher-Level Math CRs (Handout)

15. Professional Development to Practice Constructed Response Items from Smarter Balanced Assessment Consortium

16. Professional Development to Practice The table below shows the number of students in each third-grade class at Lincoln School. There are 105 fourth-grade students at Lincoln School. How many more fourth-grade students than third-grade students are at Lincoln School? Show or explain how you found your answer. The table below shows the number of students in each third-grade class at Lincoln School. There are 105 fourth-grade students at Lincoln School. How many more fourth-grade students than third-grade students are at Lincoln School? Show or explain how you found your answer. Students in Third-Grade ClassNumber of Students Mrs. Roy24 Mr. Grant21 Mr. Harrison22 Ms. Mack25

17. Professional Development to Practice

18. Scaffolded Set of CRs Grade 7

19. Professional Development to Practice Scaffolded Set of CRs Grade 8

20. Professional Development to Practice Constructed Response Extended Response Ms. McCrary wants to make a rabbit pen in a section of her lawn. Her plan for the rabbit pen includes the following: It will be in the shape of a rectangle. It will take 24 feet of fence material to make. Each side will be longer than 1 foot. The length and width will measure whole feet. Part A Draw 3 different rectangles that can each represent Ms. McCrary’s rabbit pen. Be sure to use all 24 feet of fence material for each pen. Use the grid below. Click the places where you want the corners of your rectangle to be. Draw one rectangle at a time. If you make a mistake, click on your rectangle to delete it. Continue as many times as necessary. Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer. [Students will input length, width, and area for each rabbit pen. Students will choose unit from drop down menu.] Pen 1: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet) Part B Ms. McCrary wants her rabbit to have more than 60 square feet of ground area inside the pen. She finds that if she uses the side of her house as one of the sides of the rabbit pen, she can make the rabbit pen larger. Draw another rectangular rabbit pen. Use all 24 feet of fencing for 3 sides of the pen. Use one side of the house for the other side of the pen. Make sure the ground area inside the pen is greater than 60 square feet. Use the grid below. Click the places where you want the corners of your rectangle to be. If you make a mistake, click on your rectangle to delete it. Pen 2: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet) Pen 3: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet) Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer. Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)

21. Professional Development to Practice Grade 4 OCR

22. Professional Development to Practice Grade 4 ELA

23. Professional Development to Practice

24. Handout Packet

25. Professional Development to Practice Items That Are Not Considered to Be Quality Speculate the Reasons! 1.Create a pamphlet to advertise a country. Include the country’s flag, anthem, imports, exports, etc. 2.Write an essay to explain what the country of Canada is like. 3.Create a PowerPoint Presentation about the novel Red Badge of Courage. 4.Draw a chart showing the three branches of government. 5.Draw a picture of a cell. Be sure to add labels. 6.Write five events that happened in the story. 7.Draw a picture of the food pyramid. 8.Explain the causes of the Civil War.

26. Professional Development to Practice What are two major flaws in this OCR question? How might it be repaired?  In class, we learned that when supply goes down, cost goes up. Explain what would happen to the price of hula-hoops if the largest supplier in the United States went out of business. Use four details to support your answer.

27. Professional Development to Practice What are the flaws in these questions? Then, describe how to repair the questions. 1.Did you like the story? Explain why or why not using three details and/or examples from the story. 2.Explain, what you think will happen to John (the main character) after the end of the story. 3.Predict what will happen to Mary if she disobeys her mother? Use two details and/or examples from the story in your answer. 4.What are two historical reasons to explain the present-day appearance of our national flag?

28. Professional Development to Practice Possible Stimulus Materials ELAMathematicsOther Readings video clips audio clips research topic/issue/ problem graphs, charts, other visuals etc. Graphs Pictures Models Tables Figures Scenarios Data Bases Video Clips Maps Spreadsheets Photos Research Reports Any items used in ELA and Mathematics. Any items used in your subject area.

29. Professional Development to Practice Locate Stimulus Materials National Archives Smithsonian Institute Marco Polo website Internet search for charts, graphs, maps, excerpts from documents Digital video clips from U-tube Magazine articles Newspaper articles Reading A to Z Project Gutenberg

30. Professional Development to Practice  Complete the graphic organizer to show the social, political, and economic impact of the Monroe Doctrine. Monroe Doctrine Social Political Economic BeforeAfter

31. Professional Development to Practice  Complete the graphic organizer to show two cause-effect relationships contained in the article. CauseEffect

32. Professional Development to Practice Steps in Writing an Open-Ended CR See Handout

33. Professional Development to Practice 1.Work in a small group. 2.Choose any brochure from the center of your table to use as stimulus material. 3.Write an open-ended constructed response question related to any content area. 4.Be ready to share. Apply Your Knowledge

34. Professional Development to Practice Scoring Open-Ended Constructed Response Items

35. Professional Development to Practice Scoring a Closed CR Generic Rubric for Closed CR 1 Point The response demonstrates the knowledge necessary to complete the prompted purpose and contains the correct or acceptable answer. O Points The response demonstrates little to no evidence of knowledge that is appropriate to the intent of the prompted purpose and does not contain the correct answer.

36. Professional Development to Practice  Requires a different approach than just grading responses right or wrong  OCR questions elicit responses that fall on a wide spectrum of possibilities  Criterion referenced scoring guide or rubric is used.  Generic or Task Specific  Response is rated based in its ability to meet specified criteria Scoring Responses to OCRs

37. Professional Development to Practice Generic Generic Rubric for a Minimal CR 2 Points The response demonstrates proficient evidence of the knowledge necessary to complete the prompted purpose. The student’s strategy and execution meet the content (including concepts, technique, representations and connections), thinking processes and qualitative demands of the task. Minor omissions may exist, but do not detract from the correctness of the response. 1 Point The response demonstrates partial knowledge necessary to complete the prompted purpose. It may contain overlooked issues, misleading assumptions, and/or errors in execution. Evidence in the response demonstrates that that the student can revise the work to accomplish the task with the help of written feedback. The student does not need dialogue or additional instructions. O Points The response lacks any evidence of knowledge that is appropriate to the intent of the prompted purpose.

38. Professional Development to Practice 2 Points: The response contains a main idea and two details as support. 1 Point: The response contains the main idea and one detail as support. 0 Points: The response shows little to no knowledge. Task Specific

39. Professional Development to Practice Steps in Writing an Open-Ended CR See Handout

40. Professional Development to Practice Steps in Writing a Scoring Guide See Handout

41. Professional Development to Practice 2 Points: The response contains an acceptable/correct answer and a valid solution process. The response shows an understanding of the process needed to find the answer. 1 Point: The response contains an acceptable/correct answer. OR 1 Point: The response contains a valid solution process but had minor computational errors. The response shows an understanding of the process needed to find the answer. 0 Points: The response shows severe misunderstanding. Scoring Guide for an OCR Math Problem

42. Professional Development to Practice Sample Scoring Guide for a Mathematics OCR from SBAC

43. Professional Development to Practice Rubric for a 2-Point ELA OCR Item

44. Professional Development to Practice 44 Rubric for a 3-Point ELA OCR Item

45. Professional Development to Practice Generic Rubric for Extended CR

46. Professional Development to Practice Generic Rubric for Medium CR

47. Professional Development to Practice Generic Rubrics for Minimal and Closed CRs

48. Professional Development to Practice Work with a small group to write a scoring rubric for these two questions. Look at the steps in your handout packet. 1.What is the main idea of the article? Use two details and examples from the article to explain why you believe this to be the main idea. 2.Evaluate Sally’s experimental design. Identify two things Sally could have done differently to make her results more valid. Give reasoning for your suggestions. Application Activity

49. Professional Development to Practice Who wants to ask the first question? Qs and As

50. Professional Development to Practice Out of all the things you learned about OCRs, what point stands out as being the “shining star”? Reflection

51. Professional Development to Practice Practice Profile

52. Professional Development to Practice Implementation Fidelity