Selecting and Evaluating Curriculum (K-5) July 2015

Agenda TOOLS FOR SELECTING AND EVALUATING CURRICULUM LEADERSHIP: SELECTING AND EVALUATING CRITERIA Agenda TOOLS FOR SELECTING AND EVALUATING CURRICULUM Leading the implementation of a high quality curriculum What tools do we use to evaluate curriculum? What are the non-negotiables? Deep dive with EQuIP What are next steps? 10 sec Here’s what we’ll do today during this session.

Evaluation Tools TOOL PURPOSE LEADERSHIP: SELECTING AND EVALUATING CRITERIA Evaluation Tools TOOL PURPOSE Instructional Materials Evaluation Tool (IMET) The ELA/Literacy IMET is designed to help educators determine whether instructional materials are aligned to the Shifts and major features of the Common Core State Standards. EQuIP Rubric The EQuIP rubric is a tool for evaluating the alignment of lessons, units and modules to the CCSS (not teachers) Instructional Practice Guide (IPG) The IPG includes coaching and lesson planning tools to help teachers and those who support teachers to make the Key Shifts in Instructional practice required by the Common Core State Standards. 2 min Additional talking points: IMET takes minimum of 30 hours according to SAP. Know as leaders – if considering purchasing curriculum that isn’t already vetted through this process, (like EngageNY’s Core Knowledge, Expeditionary Learning already are), you should use IMET to make sure purchases are worth the money. Make sure to hit point that EQuIP is meant to evaluate lessons and units, not teachers! We will spend more time today on EQuIP process to think about how can apply this versatile tool and structure for multiple purposes in own district or school and what it takes to implement with fidelity for best results.

Using the EQuIP Rubric Ensuring alignment of lessons and units to the Common Core State Standards (CCSS) in English Language Arts (ELA)/Literacy 1 hr. 45 min (NEED FEEDBACK FROM MATH SESSIONS TUESDAY TO ADJUST FOR ELA FRIDAY) 20 sec Reminder that just like in Math, IMET is a tool and process for full curriculum review, while EQuIP is meant for individual lesson and unit review. We look more closely at the materials designed for instructional delivery through this process, rather than an entire curriculum or program. In terms of usability, you can use EQuIP more regularly throughout the year when you are working with teams to evaluate lessons and units, or as a tool for your teachers and content leaders to use when creating or adapting them.

EQuIP Deep Dive During this session, participants will LEADERSHIP: SELECTING AND EVALUATING CRITERIA EQuIP Deep Dive During this session, participants will Develop a common understanding of the EQuIP quality review process Develop a common understanding of the EQuIP Rubric including its criteria and rating scale Practice using the EQuIP quality review process and rubric to evaluate and provide feedback on CCSS-aligned instructional materials 10 sec We have the same session goals as the EQuIP session for Math: Review the process, become familiar with the ELA/literacy rubric, and practice using the process and rubric with some individual lessons. We’ll spend a very brief amount of time reviewing the process and then dive right into going over the criteria and applying it to a sample lesson using all of your learning from the week.

Background: What is EQuIP? LEADERSHIP: SELECTING AND EVALUATING CRITERIA Background: What is EQuIP? Educators Evaluating the Quality of Instructional Products is an initiative designed to: Increase the supply of high quality lessons and units aligned to the CCSS that are available to elementary, middle, and high school teachers as soon as possible; and Build the capacity of educators to evaluate and improve the quality of instructional materials for use in their classrooms and schools. (EQuIP came out of a collaborative effort led by Achieve which was formerly called the Tri-State Rubric and Quality Review Process) 10 sec Again, EQuIP stands for Educators Evaluating the Quality of Instructional Products and is an initiative designed to Increase the supply of high quality lessons and units aligned to the CCSS and build the capacity of educators to evaluate and improve the quality of instructional materials. It EQuIP came out of a collaborative effort led by Achieve and was formerly called the Tri-State rubric.

EQuIP Quality Review: Principles & Agreements LEADERSHIP: SELECTING AND EVALUATING CRITERIA EQuIP Quality Review: Principles & Agreements CCSS: Before beginning a review, all participants are familiar with the standards targeted in the lesson or unit. Inquiry: Emphasis is on inquiry and is organized around a set of guiding questions. Criteria & Evidence: All observations, judgments, discussions and recommendations are criterion and evidence based. Constructive: Lessons/units to be reviewed are seen as “works in progress.” Participants make constructive observations and suggestions based on evidence from the work. Individual to Collective: Each member of a review team independently records his/her observations prior to discussion. Understanding & Agreement: The goal of the process is to compare and eventually calibrate judgments to move toward agreement about quality with respect to the CCSS. 15 sec Again, here are the protocols for EQuIP quality review: After first reviewing the lesson or unit’s targeted standards, participants use a set of guiding questions to make constructive, criterion-based observations and recommendations for improvement, doing so individually first and then discussing and calibrating with the group.

1 2 3 4 EQuIP Quality Review: Dimensions LEADERSHIP: SELECTING AND EVALUATING CRITERIA EQuIP Quality Review: Dimensions 1 2 3 4 Alignment to the Depth of the CCSS Key Shifts in the CCSS Instructional Supports Assessment 5 sec The dimensions are the same in Math and ELA/Literacy: (1) Alignment to the standards, (2) alignment to key shifts, (2) inclusion of instructional supports, and (4) assessment.

EQuIP Rubric LEADERSHIP: SELECTING AND EVALUATING CRITERIA 30 sec (hand out, or have participants pull out, the rubric) We’re going to get into each dimension and its criteria more deeply, but let’s take a quick look at how the rubric is organized. The layout is exactly the same as math. (click through) Four dimensions are placed in the four columns, labeled at the top with criteria in each column. Dimensions 3 and 4 include extra criteria that are specified for longer lessons or full units that a short unit or single lesson would not be held accountable for. There is a place at the bottom to circle the rating for that dimension.

EQuIP Rubric LEADERSHIP: SELECTING AND EVALUATING CRITERIA 5 sec sec For those who prefer, there is an electronic version of the rubric available on Achieve’s EQuIP web page and our site. (http://www.achieve.org/files/EQuIPELAelectronicfeedbackresponse_e.pdf)

EQuIP Quality Review: Steps LEADERSHIP: SELECTING AND EVALUATING CRITERIA EQuIP Quality Review: Steps Step 1 Review Materials (Standards, Lesson/Unit) Step 2 Apply Criteria in Dimension I Step 3 Apply Criteria in Dimensions II – IV Step 4 Apply Overall Rating and Provide Summary Comments Step 5 Compare Overall Ratings and Determine Next Steps 5-10 sec You’ll notice the steps for ELA/Literacy, printed on the back page of the rubric, are the same as for Math. We’ll review them using a flow chart, honoring our more visual/spatial learners, and then dive into practice together. (Next slide)

EQuIP Quality Review: Steps 1 2 3 4 5 Discussion and collaboration occur after Dimension I and again after Dimension IV or… … separately after each dimension 30 sec Review materials, including the targeted standard(s) for the lesson, Apply criteira in Dimension I: Alignment to standards, and decide if the lesson/unit passes If so, move on to apply criteria in Dimensions II, III and IV, (click) choosing to discuss and collaborate when done, (click) or after each dimension, Provide an overall rating and summary comments for improvement. Come together to compare overall ratings and plan next steps. As we move through these steps, think about how you and your teams currently evaluate lessons. How could this process enhance and focus that review, as well as the development of new or adapted materials?

Birth of the Haudenosaunee LEADERSHIP: SELECTING AND EVALUATING CRITERIA Practice K-2 The Great Kapok Tree 3-5 Birth of the Haudenosaunee 2 min You have the option of two example resources: a 1st/2nd grade mini-unit from Newark available on AchieveTheCore’s website for The Great Kapok Tree by Lynne Cherry: http://achievethecore.org/file/2239 or a lesson from Unit 1 of EngageNY’s 4th grade module 1A centered on Birth of the Haudenosaunee, a brief text by Dehowӓhad-dih: Bradley Powless, Eel Clan, Onondaga Nation: https://www.engageny.org/resource/grade-4-ela-module-1a-unit-1 (unit overview) https://www.engageny.org/resource/grade-4-ela-module-1a-unit-1-lesson-3 (Lesson with text) Note on the 1st-2nd Lesson: Scan the unit p. 1-6 but focus on lesson for SECOND READING. Include other lessons or the unit as a whole only if time allows. Note on the 4th grade lesson: Again, scan the unit overview but focus on the individual lesson. The vocabulary activity accompanying this lesson is based on Anita Archer’s work with struggling students, but applies to all students - it takes students through vocabulary quickly using a structure where the teacher and students say the word, use it in a sentence, then define it and provide examples. For more information, google her work - she is very effective with moving quickly and efficiently through vocabulary using research on how our brains take it in. (have participants pull up materials)

(10 minutes, individually) LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 1: Review Materials Record the grade and title of the lesson/unit on the Rubric *If you are not as familiar with the content, review the PARCC Model Content Framework (3-5) for the grade level *If you are not as familiar with the standard(s), review the actual language of each standard Scan to see what the lesson/unit contains and understand how it is organized Carefully read each key material included with the lesson/unit Study any student tasks and the text(s) that serve as the centerpiece for the lesson/unit, analyzing complexity, quality, scope, and relationship to instruction (10 minutes, individually) 10 min 1 min Let’s dive in with step one: review materials. A key part of this step is making sure you have the content knowledge needed to participate in the review. Leaders less familiar with the grade level content should start with the PARCC model content framework for that grade level before looking closely at the standards, using the Common Core app or website. (http://www.parcconline.org/parcc-model-content-frameworks) Take 10 minutes to individually review the standards, scan the materials, and examine the text. Then we’ll bring our attention to the rubric.

Criteria for Dimension I: Alignment to the Depth of the CCSS LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension I: Alignment to the Depth of the CCSS The lesson aligns with the letter and spirit of the CCSS Targets a set of grade-level CCSS ELA/Literacy Standards Includes a clear and explicit purpose for instruction Select text(s) that measure within the grade-level text complexity band and are of sufficient quality and scope for the stated purpose (e.g.: presents vocabulary, syntax, text structures, levels of meaning/purpose, and other qualitative characteristics similar to exemplars in Appendices A & B 1 min Before working on your own to apply Dimension I, let’s look at the criteria together. (Go through both slides...) (Helpful notes) For criterion 1: Does the teacher/developer articulate alignment to a reasonable number of standards? Do the assignments, tasks and activities suggest a set of standards has been targeted for instruction? Does the teacher/developer make a distinction between targeted and supporting standards? Do the instruction, assignments and activities make sense given the standards listed? For criterion 2: Where in the lesson/unit is the instructional purpose communicated? Is the purpose for instruction well-aligned to the standards identified and/or the assignments and activities? Remember there are different ways to capture “clear and explicit purpose”; the rubric is template-agnostic. For criterion 3: Is the text of sufficient quality and scope for the instructional purpose? Does the text present characteristics similar to the exemplars in the CCSS Appendix B? For grade 2 – 12, does the text measure within the grade-level complexity band?

Criteria for Dimension I: Alignment to the Depth of the CCSS LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension I: Alignment to the Depth of the CCSS A unit or longer lesson should also: (K-2 Only) Emphasize the explicit, systematic development of foundational literacy skills (concepts of print, phonological awareness, the alphabetic principle, high frequency sight words, and phonics). (K-2 Only) Regularly include specific fluency-building techniques supported by research (e.g., monitored partner reading, choral reading, repeated readings with text, following along in the text when teacher or other fluent reader is reading aloud, short timed practice that is slightly challenging to the reader). Integrate reading, writing, speaking and listening so that students apply and synthesize advancing literacy skills. Build students’ content knowledge in social studies, the arts, science or technical subjects through a coherent sequence of texts and series of questions that build knowledge within a topic. 1 min Are there any comments or questions about these?

Step 2: Apply Criteria in Dimension I: LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 2: Apply Criteria in Dimension I: Alignment to the Depth of the CCSS INDIVIDUALLY Closely examine the lesson/unit materials through the “lens” of each criterion Put a check by each criterion for which clear and substantial evidence is found Record feedback on specific improvements needed to meet criteria or strengthen alignment overall Enter a rating of 0–3 for Dimension I Determine if lesson/unit is ready to continue to Dimensions II-IV (4 min) 5 min 30 sec (Review steps with any additional notes needed below. Remember that participants have already completed this process in the Math session.) Select the box for each criterion where there is clear and substantial evidence. Leave the box blank if there is insufficient or no evidence of a criterion. Explain that criteria may be checked only if there is clear and substantial evidence of the criterion (there are no “half-checks”). There may be instances when reviewers find clear and substantial evidence of a criterion and constructive suggestions still can be made. In such cases, reviewers may provide feedback related to criteria that have been checked. Remember that feedback should be criterion-based with evidence cited and focus on improvement.   Each team member should engage in the criterion-based analysis of the example’s CCSS alignment individually (and silently) before any discussion occurs. 4 min Let’s take 4 minutes to follow these steps for Dimension 1. (Stay on this slide or click back to the Dimension 1 Criteria slide)

LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 2: Apply Criteria in Dimension I: Alignment to the Depth of the CCSS COLLECTIVELY: Compare and discuss checks and evidence What is the pattern within our team in terms of the criteria we have checked? Do our observations reference the criteria and evidence (or lack of evidence) in the instructional materials? Do our ratings correspond to the rating and descriptors in the rubric? (3 min) 3 min As a group, compare your checks and overall rating and answer these questions.

SHARE OUT: Dimension Rating and Descriptive Scales LEADERSHIP: SELECTING AND EVALUATING CRITERIA SHARE OUT: Dimension Rating and Descriptive Scales Rating Scale for Dimensions I–IV: 3 Meets most to all of the criteria in the dimension 2 Meets many of the criteria in the dimension 1 Meets some of the criteria in the dimension 0 Does not meet the criteria in the dimension Descriptors for Dimensions I–IV: 3 Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations 2 Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations 1 Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations 0 Not representing CCSS Quality — does not address the criteria in the dimension 2 min (Have a few individual volunteers or groups share their ratings and a summary of supporting evidence. Check tables for consensus and if discussion is needed because there are drastic differences, address with evidence.)

Criteria for Dimension II: Key Shifts in the CCSS LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension II: Key Shifts in the CCSS The lesson/unit reflects evidence of key shifts in the CCSS (K-2) Reading Text Closely: Makes reading text(s) closely (including read alouds) a central focus of instruction and includes regular opportunities for students to ask and answer text-dependent questions. (K-2) Text-Based Evidence: Facilitates rich text-based discussions and writing through specific, thought-provoking questions about common texts (including read alouds and, when applicable, illustrations, audio/video and other media). (K-2) Academic Vocabulary: Focuses on explicitly building students’ academic vocabulary and concepts of syntax throughout instruction. (3 minutes, individually) 4-5 min (both slides: K-2 & 3-5) 1-2 min Overall, we find the lesson merits full review, so let’s move on to Dimension II: Key Shifts. The deep knowledge gained over the course of the week will help you apply these to lessons and units. (Go through) Are there any questions or comments about these four criteria? (Helpful notes:) For criterion 1: Is a text and the evidence contained within it the central focus of the lesson? Is it clear from the lesson that a majority of class time is to be spent reading, writing, or speaking directly about a text or texts? Are students being asked to read and reread (or listen to) the text, think deeply about it, participate in thoughtful discussions, and grapple with the particulars of the text? For criterion 2-3: Is there a series of questions that require evidence from text that work together to facilitate rich conversations and writing? If these types of questions are present, reviewers should check the criteria. (Note: There may be instances when reviewers find clear and substantial evidence of this criterion and constructive suggestions can still be made. In such cases, reviewers may provide feedback related to criteria that have been checked.) For criterion 4: Does the lesson/unit explicitly focus on building students’ academic vocabulary and concepts of syntax?” Academic vocabulary is important in K-2 and there may be instances where lessons/units focus on academic vocabulary more overtly than others. If there is clear evidence that academic language is taught within the lesson/unit, this criterion should be checked. 3 min Let’s spend 3 minutes individually evaluating the lesson against each, putting a check, noting evidence, and providing any feedback for improvement.

Criteria for Dimension II: Key Shifts in the CCSS LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension II: Key Shifts in the CCSS The lesson/unit reflects evidence of key shifts in the CCSS: (3-5) Reading Text Closely: Makes reading text(s) closely, examining textual evidence and discerning deep meaning a central focus of instruction (3-5) Text-Based Evidence: Facilitates rich and rigorous evidence-based discussions and writing about common texts through a sequence of specific, thought-provoking and text-dependent questions. (3-5) Writing from Sources: Routinely expects that students draw evidence from texts to produce clear and coherent writing that informs, explains or makes an argument in various written forms (notes, summaries, short responses or formal essays). (3-5) Academic Vocabulary: Focuses on building students’ academic vocabulary in context throughout instruction. (3 minutes, individually) 4-5 min (both slides: K-2 & 3-5) 1-2 min Overall, we find the lesson merits full review, so let’s move on to Dimension II: Key Shifts. The deep knowledge gained over the course of the week will help you apply these to lessons and units. (Go through) Are there any questions or comments about these four criteria? (Helpful notes:) For criterion 1: Is a text and the evidence contained within it the central focus of the lesson? Is it clear from the lesson that a majority of class time is to be spent reading, writing, or speaking directly about a text or texts? Are students being asked to read and reread (or listen to) the text, think deeply about it, participate in thoughtful discussions, and grapple with the particulars of the text? For criterion 2-3: Is there a series of questions that require evidence from text that work together to facilitate rich conversations and writing? If these types of questions are present, reviewers should check the criteria. (Note: There may be instances when reviewers find clear and substantial evidence of this criterion and constructive suggestions can still be made. In such cases, reviewers may provide feedback related to criteria that have been checked.) For criterion 4: Does the lesson/unit explicitly focus on building students’ academic vocabulary and concepts of syntax?” Academic vocabulary is important in K-2 and there may be instances where lessons/units focus on academic vocabulary more overtly than others. If there is clear evidence that academic language is taught within the lesson/unit, this criterion should be checked. 3 min Let’s spend 3 minutes individually evaluating the lesson against each, putting a check, noting evidence, and providing any feedback for improvement.

Criteria for Dimension II: Key Shifts in the CCSS LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension II: Key Shifts in the CCSS A unit or longer lesson should also: Grade-Level Reading: Include a progression of texts as students learn to read (e.g., additional phonic patterns are introduced, increasing sentence length). Provides text- centered learning that is sequenced, scaffolded and supported to advance students toward independent grade-level reading. (3-5 only) Building Disciplinary Knowledge: Provide opportunities for students to build knowledge about a topic or subject through analysis of a coherent selection of strategically sequenced, discipline-specific texts. Balance of Texts: Focus instruction equally on literary and informational texts as stipulated in the CCSS (p.5) and indicated by instructional time (may be more applicable across a year or several units). Balance of Writing: Include prominent and varied writing opportunities for students that balance communicating thinking and answering questions with self-expression and exploration. (3 minutes, individually) 4-5 min 1-2 min Since we’ve gotten a sense of the unit for each of these lessons, let’s try to look at the criteria for longer lessons or units together. (Go through.) Are there any questions or comments about these four criteria? (NEED HELPFUL NOTES/GUIDING QUESTIONS) 3 min Now apply these criteria to the unit on your own, and then assign a rating for alignment to the key shifts.

Step 3: Apply Criteria in Dimension II: Key Shifts in the CCSS LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 3: Apply Criteria in Dimension II: Key Shifts in the CCSS COLLECTIVELY: Compare and discuss checks and evidence What is the pattern within our team in terms of the criteria we have checked? Do our observations reference the criteria and evidence (or lack of evidence) in the instructional materials? Do our ratings correspond to the rating and descriptors in the rubric? (5 minutes) 5 min As a group, compare your checks and overall rating and answer these questions.

SHARE OUT: Dimension Rating and Descriptive Scales LEADERSHIP: SELECTING AND EVALUATING CRITERIA SHARE OUT: Dimension Rating and Descriptive Scales Rating Scale for Dimensions I–IV: 3 Meets most to all of the criteria in the dimension 2 Meets many of the criteria in the dimension 1 Meets some of the criteria in the dimension 0 Does not meet the criteria in the dimension Descriptors for Dimensions I–IV: 3 Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations 2 Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations 1 Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations 0 Not representing CCSS Quality — does not address the criteria in the dimension 4 min (Have groups share their ratings and a summary of supporting evidence. Check tables for consensus and if discussion is needed because there are drastic differences, address with evidence.)

Criteria for Dimension III: Instructional Supports LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension III: Instructional Supports The lesson/unit is responsive to varied student learning needs: Cultivates student interest and engagement in reading, writing and speaking about texts. Addresses instructional expectations and is easy to understand and use. (K-2) Integrates targeted instruction in multiple areas such as grammar and syntax, writing strategies, discussion rules and aspects of foundational reading. (K-2) Provides substantial materials to support students who need more time and attention to achieve automaticity with decoding, phonemic awareness, fluency and/or vocabulary acquisition. Provides all students with extensive opportunities to engage with text of appropriate complexity for the grade level; includes appropriate scaffolding so that students directly experience the complexity of the text. (K-2: including emergent and beginning readers, and with read alouds) (K-2) Focuses on challenging sections of text(s) and engages students in a productive struggle through discussion questions and other supports that build toward independence. (3-5) Focuses on challenging sections of text(s) and engages students in a productive struggle through discussion questions and other supports that build toward independence Integrates appropriate supports in reading, writing, listening and speaking for students who are English language learners, have disabilities or read well below the grade level text band. Provides extensions and/or more advanced text for students who read well above the grade level text band. (5 minutes, individually) 6-7 min (both slides: K-2 and 3-5) 1-2 min Let’s look at Dimension III, Instructional Supports. (Go through). Are there any questions or comments about these criteria? (Helpful notes) For criterion 2: Does this set of materials address instructional expectations? Is it easy to understand and follow? Are the teacher resources (annotated responses, supports for ELLs, SPED, etc.) clear? For criterion 3: Does this lesson/unit integrate targeted instruction in multiple areas such as grammar and syntax, writing strategies, discussion rules and aspects of foundational reading? For criterion 6: Does this lesson/unit focus on sections of rich text(s) (including read alouds) that present the greatest challenge? Do discussion questions and other supports promote student engagement? Rich text(s) are texts that are worthy of rereading, include Tier 2 words, incorporate layers of meaning, and serve as mentor texts for writing. Challenging sections often require scaffolding for close reading. 5 min Individually evaluate the lesson against each, putting a check, noting evidence, and providing any feedback for improvement.

Criteria for Dimension III: Instructional Supports LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension III: Instructional Supports A unit or longer lesson should also: Include a progression of learning where concepts and/or skills advance and deepen over time (may be more applicable across the year or several units). Gradually remove supports, allowing students to demonstrate their independent capacities (may be more applicable across the year or several units). Provide for authentic learning, application of literacy skills and/or student-directed inquiry, analysis, evaluation and/or reflection. Integrate targeted instruction in such areas as grammar and conventions, writing strategies, discussion rules, and all aspects of foundational reading for grades K–5. Indicate how students are accountable for independent engaged reading based on student choice and interest to build stamina, confidence and motivation (may be more applicable across the year or several units). Use technology and media to deepen learning and draw attention to evidence and texts as appropriate. (5 minutes, individually) 6-7 min (both slides: K-2 and 3-5) 1-2 min Again, let’s take a moment for the criteria for longer lessons or units. (Go through.) Are there any questions or comments about these criteria? (NEED HELPFUL NOTES/GUIDING QUESTIONS) 5 min Take 5 minutes to apply individually apply these criteria to the unit, and then give a rating for this dimension overall.

Step 3. Apply Criteria in Dimension III: Instructional Supports LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 3. Apply Criteria in Dimension III: Instructional Supports COLLECTIVELY: Compare and discuss checks and evidence What is the pattern within our team in terms of the criteria we have checked? Do our observations reference the criteria and evidence (or lack of evidence) in the instructional materials? Do our ratings correspond to the rating and descriptors in the rubric? (8 minutes) 8 min As a group, compare your checks and overall rating and answer these questions.

SHARE OUT: Dimension Rating and Descriptive Scales LEADERSHIP: SELECTING AND EVALUATING CRITERIA SHARE OUT: Dimension Rating and Descriptive Scales Rating Scale for Dimensions I–IV: 3 Meets most to all of the criteria in the dimension 2 Meets many of the criteria in the dimension 1 Meets some of the criteria in the dimension 0 Does not meet the criteria in the dimension Descriptors for Dimensions I–IV: 3 Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations 2 Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations 1 Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations 0 Not representing CCSS Quality — does not address the criteria in the dimension 5 min (Have groups share their ratings and a summary of supporting evidence. Check tables for consensus and if discussion is needed because there are drastic differences, address with evidence.)

Criteria for Dimension IV: Instructional Supports LEADERSHIP: SELECTING AND EVALUATING CRITERIA Criteria for Dimension IV: Instructional Supports The lesson/unit regularly assesses whether students are mastering standards-based content and skills Elicits direct, observable evidence of the degree to which a student can independently demonstrate foundational skills and targeted grade level literacy CCSS (e.g., reading, writing, speaking and listening and/or language). Assesses student proficiency using methods that are unbiased and accessible to all students. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for interpreting student performance and responding to areas where students are not yet meeting standards. A unit or longer lesson should Use varied modes of assessment, including a range of pre-, formative, summative and self-assessment measures. (3 minutes, individually) 4-5 min 1-2 min And finally, Dimension IV, Assessment. (Go through). Are there any questions or comments about these four criteria? (Helpful notes:) For criterion 2: Do students have multiple ways to show what they have learned? For criterion 3: Do assessments produce a description of how close students have come to meeting expectations (e.g., annotated student work, descriptive rubrics/checklists). 3 min Take 3 minutes to individually evaluate the lesson/unit against each, putting a check, noting evidence, and providing any feedback for improvement.

Step 3: Apply Criteria in Dimension IV: Instructional Supports LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 3: Apply Criteria in Dimension IV: Instructional Supports COLLECTIVELY: Compare and discuss checks and evidence What is the pattern within our team in terms of the criteria we have checked? Do our observations reference the criteria and evidence (or lack of evidence) in the instructional materials? Do our ratings correspond to the rating and descriptors in the rubric? (5 minutes) 5 min As a group, compare your checks and overall rating and answer these questions.

SHARE OUT: Dimension Rating and Descriptive Scales LEADERSHIP: SELECTING AND EVALUATING CRITERIA SHARE OUT: Dimension Rating and Descriptive Scales Rating Scale for Dimensions I–IV: 3 Meets most to all of the criteria in the dimension 2 Meets many of the criteria in the dimension 1 Meets some of the criteria in the dimension 0 Does not meet the criteria in the dimension Descriptors for Dimensions I–IV: 3 Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations 2 Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations 1 Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations 0 Not representing CCSS Quality — does not address the criteria in the dimension 3 min (Have groups share their ratings and a summary of supporting evidence. Check tables for consensus and if discussion is needed because there are drastic differences, address with evidence.)

Step 4: Apply an Overall Rating and Summary Comments LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 4: Apply an Overall Rating and Summary Comments Review ratings for Dimensions I–IV, adding/clarifying comments as needed Total the dimension ratings and record an overall rating based on total score: (E) Exemplar, (E/I) Exemplar if Improved, (R) Revision Needed, (N) Not Ready to Review Individually write summary comments for the overall rating on 30 sec Now, for step 4, we will determine an overall rating for the lesson/unit and prepare summary comments, first individually and then discuss and calibrate as a group. (Go through slide)

Step 4: Overall Rating for the Lesson/Unit LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 4: Overall Rating for the Lesson/Unit E: Exemplar - Aligned and meets most to all of the criteria in Dimensions II–IV (total 11–12) E/I: Exemplar if Improved - Aligned and needs some improvement in one or more dimensions (total 8–10) R: Revision Needed - Aligned partially and needs significant revision in one or more dimensions (total 3–7) N: Not Ready to Review - Not aligned and does not meet criteria (total 0–2) 30 sec This is the scale for overall ratings, also printed on the back of the rubric. (Helpful Notes) “ALIGNED” in this instance refers to alignment to the criteria of the rubric as opposed to alignment to the CCSS. The level of the need for improvement is emphasized as a deciding factor in the overall rating. If participants are clear about the meaning of the four descriptors click to the next slide for more instructions regarding the overall rating. Remember, EQuIP is never meant to be used for evaluation purposes for an individual teacher, lesson plan, or unit. It is meant to be used as an instructional materials improvement tool, or for development for educators.

Step 4: Apply an Overall Rating LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 4: Apply an Overall Rating INDIVIDUALLY: Go back through dimensions and add up total to initially determine the rating category. Consider how your rating based on the total points matches your overall sense of the quality of the materials. Consider if your judgments and feedback are placed within the appropriate dimensions. Consider how your dimensional feedback supports your judgments. Consider if the lesson falls in the category you feel is appropriate. (3 minutes, individually) 4 min 1 min (Go through slide) 3 min Let’s take 3 minutes to determine our individual overall ratings. If you finish early, start thinking about what summary comments you would submit for this lesson/unit.

Step 4: Summary Comments LEADERSHIP: SELECTING AND EVALUATING CRITERIA Step 4: Summary Comments SUMMARY COMMENTS Highlight the strongest aspects of the lesson/unit Succinctly summarize key areas for improvement articulated in the dimensional comments Are Criteria-based: Written comments are based on the criteria used for review in each dimension. Cite Evidence: Examples are provided that cite where and how the criteria are met or not met. Clarify where improvement is needed: When improvements are identified, specific information is provided about how and where such improvement should be added to the material. (3 minutes, individually) 7 min 2 min (share EQuIP feedback from) Summary comments should highlight the most critical issues that have emerged over the course of the review. Summary comments should acknowledge what the developer has done well, identify the criteria that were not checked, and provide suggestions for improving the alignment and quality of instructional materials. No extraneous or personal comments are included. Written comments suggest that the reviewer looked for evidence in the lesson or unit that address each criterion of a given dimension Not mentioned but assumed: Clarity Provided: Written comments are constructed in a manner keeping with basic grammar, spelling, sentence structure and conventions. 3 min Take a few minutes to jot down your summary comments for this lesson/unit.

LEADERSHIP: SELECTING AND EVALUATING CRITERIA EQuIP Quality Review Process Step 5: Compare Overall Ratings and Determine Next Steps Compare overall ratings and synthesize feedback How do our overall ratings compare? Does this example serve as a model of CCSS instruction? What are its strengths? Areas for improvement? What are the next steps for this material? (who will do what, by when, and with what support?) 3 min As a team, briefly compare your overall ratings and summary feedback using these questions.

Session Take-Aways APPLYING THIS TO OUR OWN CONTEXT Where and when do I see using the EQuIP review process in my school or district, and for what purpose? How can I connect EQuIP to existing systems or processes? What will be needed to implement this tool, in terms of professional development, time, or other considerations? 3 min (Whole group) Let’s think about next steps and how you can leverage this process and tool for your own context. (Go through slide)

References Slide # Source 17 13 http://www.achieve.org/files/EQuIPELAelectronicfeedbackresponse_e.pdf 16 http://achievethecore.org/file/2239 https://www.engageny.org/resource/grade-4-ela-module-1a-unit-1 (unit overview) https://www.engageny.org/resource/grade-4-ela-module-1a-unit-1-lesson-3 (Lesson with text) 17 http://www.parcconline.org/parcc-model-content-frameworks