A collaboration of 3 nonprofits through a contract award from “If you want to go fast go alone, if you want to go far, go together.” African Proverb

ASSESSMENT DESIGN

Introductions

Understand the CCSS and SBAC connections in terms of Curriculum, Instruction, and Assessment Explore an Assessment Design Process and Sample Assessments Use an Assessment Design Process to construct quality assessments Apply the process and tools for developing interim reading comprehension assessments.

COMMON CORE AND SMARTER BALANCED CONNECTIONS The Curriculum-Instruction-Assessment System

Assessment is an integral part of the teaching process. Teaching may be conceptualized as a process including quality curriculum, instruction/learning and assessment. There should be a reciprocal relationship between curriculum, instruction, and assessment. The Curriculum-Instruction-Assessment System

Types of Assessments SummativeInterimFormative  Intermediate assessment to monitor achievement over a group of standards  Should predict achievement on summative  Data should be used to inform re-teaching prior to summative assessment  Assessment yielded evidence about students’ status that guides instructional decisions and adjustments.  Typically targeted to a single standard  May be embedded in a lesson  Provides immediate evidence of student progress  A comprehensive measurement of learning (outcomes) following instruction  Provides information about what students have and have not mastered  Typically administered at the end of a unit, semester, or year

Green Box>> Sign In>> Choose Grade Level >> Choose Test>> Choose Setting-Select>> Start my Test What do you notice about the structure of the tests? What do you notice about the types of questions? How are the CCSS standards addressed? Sample Assessments

How does assessment fit into CCSS aligned units of instruction ? Curriculum Instruction Assessment  What are the big ideas?  How are text sets used?  How do the unit lessons connect?  How are students assessed? Engage NY

Shifts of the Common Core: English Language Arts 1.Regular practice with complex text and academic language across content areas 2.Using evidence from literary and informational text to support claims and conclusions during reading, writing, and discussion, across content areas 3.Building content knowledge from informational text during reading, writing, and discussion, across content areas

Key Shift 1: Complex Text Reading passages on tests should have appropriate grade complexity Close reading, rather than skimming, is required. Passages should be of high quality and worthy of close reading. Most of the items assessing vocabulary should focus on words that matter most—academic vocabulary--and the use of context to determine their meaning. Source: ccsso.org

Key Shift 2: Evidence Items require students to respond to and draw evidence from texts Items should enable and require students to linger over the specifics, leading back to the text for close reading. Source: ccsso.org

Key Shift 3: Building Knowledge Reading passages should include: strong, cogent examples of fiction, non-fiction, and informational text. Students should be reading like scientists, like historians, etc. Interim/benchmark assessments should include: effective sequences of quality text- dependent questions to build content knowledge and comprehension from the text. Source: ccsso.org

ASSESSMENT DESIGN PROCESS

The Common Core Connection The first step of assessment design is clearly defining the content area (domain) and standards to be measured. For our purposes, we will focus on English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects

Claims

SBAC and CCSS Connections HANDOUT 2

SMARTER BALANCED ELA LITERACY CONTENT SPECIFICATIONS

Clearly define the knowledge, skills, and abilities that students will demonstrate. For Smarter Balanced, these are called assessment targets. These targets are all connected to the clusters we looked at earlier! Assessment Targets

Assessment Targets for Claim 1 Literary 1.Key Details 2.Central Ideas 3.Word Meaning 4.Reasoning and Evidence 5.Analysis within or across texts 6.Text Structures and Features 7.Language Use Informational 8. Key Details 9. Central Ideas 10. Word Meaning 11. Reasoning and Evidence 12. Analysis within or across texts 13. Text Structures and Features 14. Language Use Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts

Assessment Targets for Claim 2 Students can produce effective and well- grounded writing for a range of purposes and audiences.

DETERMINE TEXT COMPLEXITY MEASURES

CCRA.R.10 Read and comprehend complex literary and informational texts independently and proficiently. R.CCR.10 Common Core Connection Remember, assessment selections should be on grade level for students to read independently.

Three Part Model Dimensions of Text Complexity  Quantitative measures  Qualitative measures  Reader and Task considerations

Quantitative Measures

Quantitative Measures Comparison Lexile ATOS Source: HANDOUT 3 (front)

Quantitative Measures For Santorio Santorio Flesch-Kincaid and Reading Maturity Flesch-Kincaid Reading Maturity:

ATOS Analyzer

SANTORIO SANTORIO AND THE THERMOMETER EXAMPLE from HANDOUT 7 READ THE TEXT

Results HANDOUT 7

Qualitative Measures

Background Prior Cultural Vocabulary Standard English Variations Register Genre Organization Narration Text Features Graphics Density and Complexity Figurative Language Purpose Levels of Meaning Structure Knowledge Demands Language Convention and Clarity What other factors make the text complex? Source:

Qualitative Measures Choose appropriate rubric for the type of text Source: ccsso.org HANDOUT 4 ANNOTATE— WHICH COMPONENTS MAKE THIS TEXT COMPLEX?

Qualitative Analysis of Santorio Santorio and the Thermometer HANDOUT 3 (back)

Reader and Task Considerations

Reader and Task Motivation Knowledge and experience Purpose for reading Complexity of task assigned regarding text Complexity of questions asked regarding text

HANDOUT 5

ASSESSMENT QUESTIONS

Selected Response Items Students to select appropriate response(s) from options provided.

Strengths and Weaknesses of Selected Response Items Strengths Allow for large numbers of items Can be scored efficiently, objectively, and reliably Weaknesses Difficult to write Not able to assess all objectives (e.g. writing) Subject to random guessing

Lizards are fascinating creatures. There are over 3,000 known species, including monitors, skinks, geckos, chameleons, and iguanas, and they vary greatly in appearance. The largest lizard, the Komodo dragon, can grow over ten feet long, and the smallest, the Jaragua lizard, can fit on a dime. Skinks usually have smooth scales like snakes, iguanas have mohawk-like crests running down their backs, and the moloch is covered with spikes from head to tail. Lizards vary in color from shades of gray and brown to bright red or green, spotted or striped. Most have four legs but some are legless and easily confused with snakes (Hint: if it has external eardrums and eyelids it’s a lizard). Geckos can walk up walls. Chameleons not only change color but also have prehensile tails, similar to those of monkeys, that wrap around branches and their eyes can move in different directions. What is the best way to revise the highlighted sentence to match the language and style of the paragraph? A.Geckos are able to adhere to flush surfaces because setae on their footpads facilitate van der Waals forces between the setae structures and the surface. B.Geckos are awesome because they have sticky toes that allow them to climb windows like Spiderman. C.Geckos have the remarkable ability to walk up walls thanks to tiny hair-like structures on their toes that cling to smooth surfaces. D.Geckos scurry up walls like tiny dancers gliding effortlessly across a stage, their movements as natural as a well-rehearsed ballet. Formats and Components of Selected Response Items—Traditional Distractor Analysis A. This option uses too much scientific language to fit with the rest of the paragraph. B. This option is more informal than the rest of the paragraph. C. KEY. This option uses formal, non-technical language that fits well with the rest of the paragraph. D. This option uses too much figurative language to fit with the rest of the paragraph. STIMULUS Lizards are fascinating creatures. There are over 3,000 known species, including monitors, skinks, geckos, chameleons, and iguanas, and they vary greatly in appearance. The largest lizard, the Komodo dragon, can grow over ten feet long, and the smallest, the Jaragua lizard, can fit on a dime. Skinks usually have smooth scales like snakes, iguanas have mohawk-like crests running down their backs, and the moloch is covered with spikes from head to tail. Lizards vary in color from shades of gray and brown to bright red or green, spotted or striped. Most have four legs but some are legless and easily confused with snakes (Hint: if it has external eardrums and eyelids it’s a lizard). Geckos can walk up walls. Chameleons not only change color but also have prehensile tails, similar to those of monkeys, that wrap around branches and their eyes can move in different directions. STEM What is the best way to revise the highlighted sentence to match the language and style of the paragraph? OPTIONS A.Geckos are able to adhere to flush surfaces because setae on their footpads facilitate van der Waals forces between the setae structures and the surface. B.Geckos are awesome because they have sticky toes that allow them to climb windows like Spiderman. C.Geckos have the remarkable ability to walk up walls thanks to tiny hair-like structures on their toes that cling to smooth surfaces. D.Geckos scurry up walls like tiny dancers gliding effortlessly across a stage, their movements as natural as a well-rehearsed ballet. (Question)

Selected Response Example OPTIONS

Selected Response Example

OPTIONS/STUDENT RESPONSES

Constructed Response Items Students create or construct an appropriate response.

Strengths and Weaknesses of Constructed Response Items Strengths May be easier to write Can assess higher order cognitive abilities and complex task performance Eliminate random guessing Weaknesses More time for students to complete, so, fewer items can be included on test More difficult to score reliably Vulnerable to construct irrelevance factors

Constructed Response Format The Shepherd’s Boy and the Wolf A Shepherd's Boy was tending his flock near a village, and thought it would be great fun to trick the villagers by pretending that a Wolf was attacking the sheep: so he shouted out, "Wolf! Wolf!" and when the people came running up he laughed at them because they believed him. He did this more than once, and every time the villagers found they had been tricked, for there was no Wolf at all. At last a Wolf really did come, and the Boy cried, "Wolf! Wolf!" as loud as he could: but the people were so used to hearing him call that they took no notice of his cries for help. And so no one came to help the boy, and the Wolf attacked the sheep. In a few sentences, explain what lesson the reader can learn from the shepherd’s boy. Use details from the story to support your response. STEM In a few sentences, explain what lesson the reader can learn from the shepherd’s boy. Use details from the story to support your response. STIMULUS The Shepherd’s Boy and the Wolf A Shepherd's Boy was tending his flock near a village, and thought it would be great fun to trick the villagers by pretending that a Wolf was attacking the sheep: so he shouted out, "Wolf! Wolf!" and when the people came running up he laughed at them because they believed him. He did this more than once, and every time the villagers found they had been tricked, for there was no Wolf at all. At last a Wolf really did come, and the Boy cried, "Wolf! Wolf!" as loud as he could: but the people were so used to hearing him call that they took no notice of his cries for help. And so no one came to help the boy, and the Wolf attacked the sheep.

Constructed Response Example Recall the culminating writing task What was Santorio like as a person? Make a claim about what kind of person he was and defend your claim using evidence from the text. Write your response below. Stimulus Stem

Discuss the types of selected and constructed response items you viewed in the SBAC practice test earlier today.

CONSTRUCTING QUALITY ITEMS BASED ON ESSENTIAL UNDERSTANDING

Text-dependent questions that scaffold learning: Require careful examination of text Require students to respond with support from text Range from simple to complex (but based on importance of building knowledge or interpretations) Go beyond basic facts and recall Promote deep comprehension and retention of content

Words/Phrases Sentences Paragraph Segments Selection Across Texts … to build gradual, overall understanding. A variety of questions on a variety of levels

Variety in levels of cognitive rigor 2009 Karin K. Hess: Hess’ Cognitive Rigor Matrix: Permission to reproduce is given when authorship is fully cited For full article, go to HANDOUT 9

Guidelines for constructing effective text-dependent questions HANDOUT 10

ANALYZING QUESTIONS

Culminating Task/Question related to Essential Understanding What was Santorio like as a person? Make a claim about what kind of person he was and defend your claim using evidence from the text. Write your response below. Text based questions provide scaffolding for students to be able to reach a deep understanding and respond to this question. Which CCSS anchor standards apply?

Read the grade level standards being measured. Remember, these connect to the anchor standards we looked at earlier.

Match Standards to Questions HANDOUT 7 AND 8

Headings in Standards

Peer Review Checklist Peers use checklist to examine items and ensure all criteria are met. HANDOUT 11

What is MTSS? MTSS is a coherent continuum of evidence based, system-wide practices that support: – a rapid response to academic and behavioral challenges, – frequent data-based monitoring – instructional decision-making in order for each Vermont student to achieve high standards.

Why MTSS?

vtMTSS Framework: A Tiered System of Support In a classroom of 30 students MTSS: A tiered system of support

Colorin Colorado The Common Core for English Language Learners What is the role of informal, classroom-based assessment in guiding instruction for students with limited English proficiency? Center on Instruction The Center on Instruction offers reports, professional learning modules, practice guides, presentations and online courses, including research-based practices in instruction and assessment for English Language Learners

Google Translator

Babylon Languages

Reaching ALL Students

This collection of easy to read books provides students with significant cognitive disabilities access to the content of the Exemplar Texts listed in Appendix B of the Common Core State Standards at an appropriate complexity level. The open source, accessible texts were created in Tar Heel Reader (tarheelreader.org)

Lunch

Sample Assessments Cactus Jam (3-5) Nature By Design(6-8) Inaugural Address (9-12)

Read text and determine complexity Dimensions of Text Complexity  Quantitative measures  Qualitative measures  Reader and Task considerations

Take the Test! Analyze the Questions Determine the essential understanding Note the standards and the structure/complexity of the questions

Peer Review Checklist Determine if the assessment questions meet these criteria?

APPLY THE PROCESS TO YOUR OWN TEXT

TRY IT OUT Assessment Design Template Peer Review Text Complexity Analysis Sample Text Selections HANDOUT 12 Developing Effective Text Dependent Questions Guide and DOK levels

Materials Evaluation Toolkit Toolkit for Evaluating Alignment of Instructional and Assessment Materials to the CCSS

Non-Fiction and Informational Text Resources d=1758&use_sec=false&sec_url_var=region1&__uuid=fa36d8e4-96dd-441d-920f-f2dad28ca741 d=1758&use_sec=false&sec_url_var=region1&__uuid=fa36d8e4-96dd-441d-920f-f2dad28ca NY Times Learning Network

Special Thank you to the Professional Learning Team Joey Hawkins Diana Leddy Lindy Johnson Marion Dewey Joe Rivers Maggie Eaton Juliet Halladay Alysia Backman Christina Suarez Janna Osman Jenn Barone Laura King Pat Fitzsimmons Cindy Moran Lori Dolezal Elementary Literacy Assessment Coordinator

PLAN FOR PROFESSIONAL DEVELOPMENT

Join the Exchange Share Discuss Inquire

References Reynolds, Cecil R., Livingston, R.B., & Willson, V. (2009). Measurement and Assessment in Education (2 nd edition). Boston MA: Pearson. (Student Achievement Partners) 20Rubric.pdfhttp://programs.ccsso.org/projects/common%20core%20resources/documents/Informational%20Text%20Qualitative% 20Rubric.pdf (Qualitative Rubric for Informational Text: Council of Chief State School Officers and Student Achievement Partners) bric.pdfhttp://programs.ccsso.org/projects/common%20core%20resources/documents/Literary%20Text%20Qualitative%20Ru bric.pdf (Qualitative Rubric for Literary Text: Council of Chief State School Officers and Student Achievement Partners) (Rubric for interim Assessments) (ATOS analyzer) Hess, K (2009) Hess’ Cognitive Rigor Matrix. (The Common Core Standards)