MCAS COACHING “Preserving Our Past… Focusing on Our Future” March 14, 2008 Deb Donahue Amy Polito Kathy Troy

Agenda Welcome and refreshments Welcome and refreshments Introduction Introduction “Robin Hood” open-response prompt “Robin Hood” open-response prompt Read selection, review benchmarks Read selection, review benchmarks Review rubric Review rubric Score and discuss sample answers Score and discuss sample answers Overview of Open-Response Question Development Overview of Open-Response Question Development Review Handouts Enclosed Review Handouts Enclosed Introduction to NCS Mentor Scoring System Introduction to NCS Mentor Scoring System NCS Mentor Practice – Computer Lab, B116 NCS Mentor Practice – Computer Lab, B116 Evaluation Evaluation

“Robin Hood” Robin Hood Open-Response Robin Hood Open-Response Robin Hood Open-Response Robin Hood Open-Response Massachusetts Department of Education MCAS 2006 Massachusetts Department of Education MCAS 2006 Grade 7 Grade 7 Question #18 Question #18

“Robin Hood” Sample I The character, Marian has many traits that enabled her to trick Robin into winning the fight. Some of her character traits are tall, strong, skilled, and tough. Since she was tall, strong and tough, she could win the fight. I she was weak she probably wouldn’t have won the fight. Marian is skilled in alot of ways. She is skilled, because she made Robin miss his stroke, she would catch him off- balance, and she would use every chance of making him lose the fight, like when the tree was behind him. Those are some of the character traits of Marian that made her enable to trick Robin into winning the fight. The character, Marian has many traits that enabled her to trick Robin into winning the fight. Some of her character traits are tall, strong, skilled, and tough. Since she was tall, strong and tough, she could win the fight. I she was weak she probably wouldn’t have won the fight. Marian is skilled in alot of ways. She is skilled, because she made Robin miss his stroke, she would catch him off- balance, and she would use every chance of making him lose the fight, like when the tree was behind him. Those are some of the character traits of Marian that made her enable to trick Robin into winning the fight.

“Robin Hood” Sample J Marian outsmarts and outfights Robin. Robin didn’t know that Marian was so skilled and tough, that Marian surprised him. She could catch him off-balance and make him miss his stroke. The best trick was that she twice changed hands and fought him left to right as boxers sometimes do. At last Robin had his chance, they knew he would win but Marian saw her chance and he triped over a root and he went back and fell on the tree. At the end they get married. Marian outsmarts and outfights Robin. Robin didn’t know that Marian was so skilled and tough, that Marian surprised him. She could catch him off-balance and make him miss his stroke. The best trick was that she twice changed hands and fought him left to right as boxers sometimes do. At last Robin had his chance, they knew he would win but Marian saw her chance and he triped over a root and he went back and fell on the tree. At the end they get married.

“Robin Hood” Sample K Marian and Robin are both mysterious. And Marian said she has some tricks up her sleeve thats how she outsmarted Robin. Marian and Robin are both mysterious. And Marian said she has some tricks up her sleeve thats how she outsmarted Robin.

“Robin Hood” Sample L The character trait of Marian that let her outsmart robin are observation and intelligence. In the story Marian observes that robin has a root behind him wich if robin was to walk back he would trip. Marian is intelligent by steping on the horn for he can not call for help. That is how Marian out smarted Robin. The character trait of Marian that let her outsmart robin are observation and intelligence. In the story Marian observes that robin has a root behind him wich if robin was to walk back he would trip. Marian is intelligent by steping on the horn for he can not call for help. That is how Marian out smarted Robin.

DOE Rationale for Open-Response Items Open-response items allow for more depth of knowledge to be demonstrated than in multiple- choice items. Open-response items allow for more depth of knowledge to be demonstrated than in multiple- choice items. Students can be asked to demonstrate more complex cognitive behaviors such as comparing, relating, analyzing, inferring, concluding, predicting, generalizing, solving and/or applying. Students can be asked to demonstrate more complex cognitive behaviors such as comparing, relating, analyzing, inferring, concluding, predicting, generalizing, solving and/or applying.

Criteria What criteria does the DOE use to develop MCAS open-response questions? What criteria does the DOE use to develop MCAS open-response questions? FYI: Formerly known as “Language and Literature,” this MCAS component is now referred to as “Reading Comprehension” for ELA FYI: Formerly known as “Language and Literature,” this MCAS component is now referred to as “Reading Comprehension” for ELA

Make sure that the task is actually achievable If the item asks students to read a passage or examine a graphic and then “give three ways” or “explain two reasons” based on the material given, make sure there are three ways or two reasons found in the material provided. If the item asks students to read a passage or examine a graphic and then “give three ways” or “explain two reasons” based on the material given, make sure there are three ways or two reasons found in the material provided.

Directions must be complete, and clearly specify what is expected If you want the students to provide examples, state in the directions that the students should provide examples and how many. If you want the students to provide examples, state in the directions that the students should provide examples and how many. If you want the students to identify information, do not ask them to discuss, describe, or explain. If you want the students to identify information, do not ask them to discuss, describe, or explain.

Suggestions for Item Format Present the prompt in paragraph form. Present the prompt in paragraph form. Use bullets to emphasize the details in the prompt. Use bullets to emphasize the details in the prompt. If the students are required to respond to multiple parts of a question, label each part separately (A, B, C). If the students are required to respond to multiple parts of a question, label each part separately (A, B, C).

Five Types of Open-Response Questions Scaffolded Scaffolded Single Dimension/Component Single Dimension/Component Two or More Relatively Independent Components Two or More Relatively Independent Components Student Choice: Topics/Options Provided Student Choice: Topics/Options Provided Response to Provided Information Response to Provided Information

Scaffolded Sequences increasingly more difficult/complex tasks Sequences increasingly more difficult/complex tasks Success on one part would likely mean there was success on all previous parts Success on one part would likely mean there was success on all previous parts Multipart (simple to complex) Multipart (simple to complex)

Separation of Powers The framers of the U.S. Constitution wanted to prevent the new federal government from becoming a dictatorship. To keep the government from becoming too powerful, they divided its powers among three branches…  A. For each of the three branches of government identify one power given to it by the Constitution.  B. Explain why each power you identified in part A is important in our system of government. Support your answer with real-life examples.

Single-Dimension/Component Straight-forward question Straight-forward question Draw a conclusion or take a position, then support it Draw a conclusion or take a position, then support it Explain a phenomenon or describe procedures Explain a phenomenon or describe procedures

Advantages of Rivers Rivers have always played in important role in the growth of the United States. Many Kentucky cities are located near large rivers. Explain three important advantages that rivers provide to cities.

2 Or More Relatively Independent Components Fairly independent questions addressing the same prompt Fairly independent questions addressing the same prompt Contains multiple parts (A, B, C, etc.) Contains multiple parts (A, B, C, etc.)

Practical Living Example Bob has been told that regular physical activity is important for good health throughout a lifetime. A. Describe THREE ways physical activity helps the body be healthy throughout a lifetime. B. Give THREE physical activities Bob could do that would help keep him healthy throughout his lifetime.

Response to Provided Information Respond to provided information such as data, graphics, or readings Respond to provided information such as data, graphics, or readings

Response To Provided Information - Reading Example In the story, “First Light,” Matthew woke up in another time period, the 1850s. A. Describe FOUR things Matthew discovered that were different from what he was used to in his present life. B. Explain how each of those differences affected him. Use information from the story to support your answer.

Keep in Mind… Measures reading comprehension, not writing skill Measures reading comprehension, not writing skill Almost always requires students to go back to text for examples, details, words or phrases for support Almost always requires students to go back to text for examples, details, words or phrases for support Based on text, not prior knowledge Based on text, not prior knowledge Requires students to demonstrate understanding of question, not just text! Requires students to demonstrate understanding of question, not just text!

A Few Strategies… Preview the test Preview the test Read the italicized print preceding the selection Read the italicized print preceding the selection Read the title – predict Read the title – predict Read pictures, tables, and other graphics Read pictures, tables, and other graphics

Strategies… Set a focus or purpose for reading: Read the open-response question Set a focus or purpose for reading: Read the open-response question Do not read the multiple choice questions Do not read the multiple choice questions Determine what is being asked; label the number of tasks (outline the question) Determine what is being asked; label the number of tasks (outline the question)

Strategies… Read the passage; mark up text Read the passage; mark up text Answer multiple choice questions Answer multiple choice questions Refer to text Refer to text Reread open-response question and formulate response Reread open-response question and formulate response After writing, reread question, and review response After writing, reread question, and review response Label sections to be sure all parts of question were answered Label sections to be sure all parts of question were answered

General Open-Response Rubric ScoreDescription Key Ideas Key Word 4 Response provides a specific argument that is clear and focused. The argument is supported by insightful and illustrative examples. Student demonstrates a thorough understanding of the text. Focused argument Focused argument Insightful examples Insightful examples Thorough understanding Thorough understandingComplete

General Open-Response Rubric ScoreDescription Key Ideas Key Word 3 Response provides a general argument while still retaining focus. Argument is supported by appropriate examples. Student demonstrates an adequate understanding of the text. General argument General argument Appropriate examples Appropriate examples Adequate understanding Adequate understandingAlmost

General Open-Response Rubric ScoreDescription Key Ideas Key Word 2 Response states a vague or unfocused argument. Argument is supported by weak or erroneous examples. Student demonstrates minimal understanding of the text. Vague Vagueargument Weak examples Weak examples Minimal understanding Minimal understandingPartial

General Open-Response Rubric ScoreDescription Key Ideas Key Word 1 Response provides a partial or scattered argument while still retaining focus. Argument is not supported by examples or is supported by irrelevant examples. Student demonstrates little understanding of the text. Scattered argument Scattered argument No or Irrelevant examples No or Irrelevant examples Lacks understanding Lacks understandingGlimmer

General Open-Response Rubric ScoreDescription Key Ideas Key Word 0 Response is irrelevant or blank. Irrelevant argument Irrelevant argument Blank Blank

NCS MENTOR w w wwww wwww.... nnnn cccc ssss mmmm eeee nnnn tttt oooo rrrr.... cccc oooo mmmm Developed by Pearson Education and the Massachusetts Department of Education Online open-response scoring tool designed for teacher and student use Provides a bridge between statewide performance assessment and classroom instruction