1. Using EQuIP in Professional Development Ted Coe, Ph.D. Director of Mathematics, Achieve #drtedcoe

2. http://tedcoe.com/math/ade

3. Think about a lesson or unit that flopped. What made it flop? Share. What feedback did you get? What feedback did you need?

4. 4 What do you look for when selecting instructional materials?

5. The EQuIP Initiative

6. The objectives are two-fold: Increase the supply of high quality lessons and units aligned to the CSSS that are available to elementary, middle, and high school teachers as soon as possible; and, Build the capacity of educators to evaluate and improve the quality of instructional materials for use in their classrooms and schools. Overview and Objectives of EQuIP

7. EQuIP = Educators Evaluating the Quality of Instructional Products EQuIP builds on a collaborative effort of education leaders from Massachusetts, New York and Rhode Island that Achieve facilitated. The outcome of that effort was the development of the “Tri-State Rubrics” and a quality review process designed to determine the quality and alignment of instructional lessons and units to the CCSS. Overview and Objectives of EQuIP

8. 8 EQuIP Quality Review Process The EQuIP quality review process is a collegial process that centers on the use of criteria-based rubrics for mathematics. The criteria are organized into four dimensions:

9. 9 The 4 Dimensions Alignment to the depth of the CCSS Key shifts in the CCSS Instructional supports Assessment

10. 10 http://www.achieve.org/files/EQuIPmathrubric-06-17-13_1.pdf http://www.achieve.org/files/EQuIP-ELArubric-06-24-13-FINAL.pdf http://www.achieve.org/files/K-2ELALiteracyEQuIPRubric-07-18-13_1.pdf

11. 11 As educators examine instructional materials against the criteria in each dimension, they are able to use common standards for quality and generate evidence-based commentary and ratings on the quality and alignment of materials.

12. 12 The EQuIP rubrics are designed to evaluate lessons or units. The rubric is NOT designed to evaluate a single task or activity The rubrics do not require a specific template for lesson or unit design

13. Roles & Sample Uses Classroom Teachers Guiding the creation of new materials Reviewing and refining existing materials Instructional Leaders & Curriculum Coaches Guiding professional learning community conversations about developing and/or selecting materials and providing feedback Integrating into professional learning as a way to study the standards Growing leaders by creating opportunities to create, vet and/or share instructional materials Producing useful data about the quality and alignment of existing instructional materials and/or the capacity of educators Superintendents & District Leaders Vetting materials to post on websites Providing quality control/quality assurance with vendors State Education Agency Staff Vetting materials to post on websites Providing quality control/quality assurance with vendors Creating model units

14. Example: Washington OSPI http://digitallearning.k12.wa.us/oer/library/

15. 15 Benefits of the EQuIP Quality Review Process Collaboration – The EQuIP Quality Review Process is a collegial process used to come to common understanding and agreement about the alignment and quality of instructional materials. Consistency – When a set of common criteria are used, all students, regardless of zip code or classroom, have instructional materials that set the same high standards and expectations. Equity – Educators across classrooms, districts, and states identify and use instructional materials that set the same rigorous standards for all students. Efficiency – Instead of each teacher, school, or state developing all of its own instructional materials, states now have multiple partners among whom they can share resources. Innovation – As lessons and units are reviewed, identified, and posted, educators share effective ways to organize and plan aligned instruction.

16. 16 EQuIP Quality Review: Principles & Agreements 1.CCSS: All members of a review team are familiar with the CCSS. 2.Inquiry: Review processes emphasize inquiry rather than advocacy and are organized in steps around a set of guiding questions. 3.Respect & Commitment: Each member of a review team is respected as a valued colleague and contributor who makes a commitment to the EQuIP process. 4.Criteria & Evidence: All observations, judgments, discussions and recommendations are criterion and evidence based. 5.Constructive: Lessons/units to be reviewed are seen as “works in progress.” Reviewers are respectful of contributors’ work and make constructive observations and suggestions based on evidence from the work. 6.Individual to Collective: Each member of a review team independently records his/her observations prior to discussion. Discussions focus on understanding all reviewers’ interpretations of the criteria and the evidence they have found. 7.Understanding & Agreement: The goal of the process is to compare and eventually calibrate judgments to move toward agreement about quality with respect to the CCSS.

17. 17 EQuIP Quality Review Process The Flowchart Discussion and collaboration must occur after Dimension I and then again either for all dimensions after Dimension IV or … … separately after each dimension and … … always during the overall rating process and summary comments.

18. http://www.achieve.org/equip

19. EQuIP and Learning Forward Professional Learning Community Modules

20. 20 The EQuIP / Learning Forward Modules 1.Exploring a Lesson and the EQuIP Rubric: Focus on Dimension I of the EQuIP Rubric 2.Focus on Dimension II: Key Shifts in the CCSS, of the EQuIP Rubric 3.Focus on Dimensions III and IV of the EQuIP Rubric (Instructional Supports and Assessment) 4.Using the EQuIP Rubric with a Second Common Lesson & Qualities of Effective Feedback 5.Using the EQuIP Rubric with a Lesson from the Professional Learning Community 6.Using the Student Work Protocol

21. 21

22. 22 www.achieve.org/equip

23. 23 Module 1

24. 24 Understanding and Applying Dimension I:  Participants silently read the criteria in Dimension I  Full group discusses Dimension I.  Rate the lesson/unit for Dimension I.

25. 25 The 4 Dimensions Alignment to the depth of the CCSS Key shifts in the CCSS Instructional supports Assessment

26. 26 Dimension I Criteria - Math The lesson/unit aligns with the letter and spirit of the CCSS: 1.Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning 2.Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed 3.Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS

27. 27 Dimension I Criteria - Math Discuss: 1.Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning 2.Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed 3.Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS

28. 28 What sort of conversation did these criteria provide? Reflect

29. What is “it”? Is the perimeter a measurement? …or is “it” something we can measure? Perimeter

30. Is perimeter a one-dimensional, two-dimensional, or three- dimensional thing? Does this room have a perimeter? Perimeter

31. 31 http://www.azed.gov/standards-practices/files/2015/04/azccrs-grade3-math-.pdf

32. Progressions http://ime.math.arizona.edu/progressions/ http://commoncoretools.files.wordpress.com/2012/07/ccss_progression_gm_k5_2 012_07_21.pdf p.16. http://commoncoretools.files.wordpress.com/2012/07/ccss_progression_gm_k5_2 012_07_21.pdf

33. 33 Review: Dimension I Criteria - Math The lesson/unit aligns with the letter and spirit of the CCSS: 1.Targets a set of grade-level CCSS mathematics standard(s) to the full depth of the standards for teaching and learning 2.Standards for Mathematical Practice that are central to the lesson are identified, handled in a grade-appropriate way and well connected to the content being addressed 3.Presents a balance of mathematical procedures and deeper conceptual understanding inherent in the CCSS

34. http://www.achieve.org/equip

35. 35 Step 2, Dimension I

36. 36

37. Apply Criteria in Dimension II 37 Alignment to the depth of the CCSS Key shifts in the CCSS Instructional supports Assessment

38. 38 Math Dimension II: Key Shifts The lesson/unit addresses key shifts in the CCSS: 1.Focus: Lessons and units targeting the major work of the grade provide an especially in-depth treatment, with especially high expectations. Lessons and units targeting supporting clusters have visible connection to the major work of the grade and are sufficiently brief. Lessons and units do not hold students responsible for material from later grades. 2.Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions.

39. 39 3. Rigor: Requires students to engage with and demonstrate challenging mathematics with appropriate balance among the following: * Application: Provides opportunities for students to independently apply mathematical concepts in real-world situations and problem solve with persistence, choosing and applying an appropriate model or strategy to new situations Conceptual Understanding: Develops students’ conceptual understanding through tasks, brief problems, questions, multiple representations and opportunities for students to write and speak about their understanding. Procedural Skill and Fluency: Expects, supports and provides guidelines for procedural skill and fluency with core calculations and mathematical procedures (when called for in the standards for the grade) to be performed quickly and accurately. Math Dimension II: Key Shifts

40. 40 During the PLC the teachers will be asked to find consensus on the meanings of the terms in the criteria. What might this criterion require? Coherence: The content develops through reasoning about the new concepts on the basis of previous understandings and provides opportunities for students to transfer knowledge and skills within and across domains and learning progressions.

41. 41

42. 42 Alignment to the depth of the CCSS Key shifts in the CCSS Instructional supports Assessment

43. 43 Math Dimension III: Instructional Supports The lesson/unit is responsive to varied student learning needs: 1.Includes clear and sufficient guidance to support teaching and learning of the targeted standards, including, when appropriate, the use of technology and media. 2.Uses and encourages precise and accurate mathematics, academic language, terminology, and concrete or abstract representations (e.g., pictures, symbols, expressions, equations, graphics, models) in the discipline. 3.Engages students in productive struggle through relevant, thought-provoking questions, problems and tasks that stimulate interest and elicit mathematical thinking. 4.Addresses instructional expectations and is easy to understand and use.

44. 44 Math Dimension III: Instructional Supports 5. Provides appropriate level and type of scaffolding, differentiation, intervention and support for a broad range of learners. *  Supports diverse cultural and linguistic backgrounds, interests and styles.  Provides extra supports for students working below grade level.  Provides extensions for students with high interest or working above grade level. * Note: All three of these components are required in a high quality lesson or unit.

45. 45 Math Dimension IV: Assessment The lesson/unit regularly assesses whether students are mastering standards-based content and skills: 1.Is designed to elicit direct, observable evidence of the degree to which a student can independently demonstrate the targeted CCSS. 2.Assesses student proficiency using methods that are accessible and unbiased, including the use of grade-level language in student prompts. 3.Includes aligned rubrics, answer keys and scoring guidelines that provide sufficient guidance for interpreting student performance. In addition, for units and longer lessons: and longer lessons: 4.Uses varied modes of curriculum-embedded assessments that may include pre-, formative, summative and self-assessment measures.

46. 46 Using Overall Ratings to Summarize Judgment Overall Rating for the Lesson/Unit: E: Exemplar — Aligned and meets most to all of the criteria in Dimensions II–IV (total 11–12) E/I: Exemplar if Improved — Aligned and needs some improvement in one or more dimensions (total 8–10) R: Revision Needed — Aligned partially and needs significant revision in one or more dimensions (total 3–7) N: Not Ready to Review — Not aligned and does not meet criteria (total 0–2)

47. 47 Which Dimension do you think is the toughest to meet? Why?

48. 48

49. 49 In Module 4 teachers will move through a second lesson much more quickly with an added emphasis on providing good feedback.

50. Effective feedback is vital to the EQuIP Quality Review Process. Criterion-based: Written comments are based on the criteria used for review in each dimension. No extraneous comments are included. Evidence Cited: Written comments indicate that the reviewer looked for evidence of each criterion of a given dimension. Examples cite where and how the criteria are met or not met. Improvement Suggested: Improvements are specifically identified to meet criteria or strengthen the lesson or unit. Clear Communication: Written comment are constructed in a manner keeping with basic grammar, spelling, sentence structure and conventions.

51. 51 Sample Feedback: Mathematics This unit targets three CCSS. The overview indicates which Standards for Mathematical Practice are central to the lesson. The activities throughout the unit present a balance of mathematical procedures and deeper conceptual understanding of the standards. The activities reinforce the standards and are well connected to the content. I think the activities might be challenging with a large class with classroom management issues. 1.Criterion-based? 2.Evidence Cited? 3.Improvement Suggested? 4.Clear Communication?

52. 52

53. 53

54. 54 Step one: Analyze the task. Step two: Examine instructional context and CCSS alignment of the task. Step three: Analyze individual student work. Step four: Analyze the collection of student work. Step five: Provide suggestions for improving the materials. http://www.achieve.org/files/Student%20Work%20Protocol_Final_9%2018%2014.pdf

55. Recent Developments and Next Steps EQuIP Tools and Resources EQuIP e-Learning Modules EQuIP Training Materials including ELA K-2 Rubric materials EQuIP Student Work Protocol EQuIP Exemplars Instructional Materials Toolkit with SAP and CCSSO EQuIP NGSS Rubric EQuIP Teacher videos and student work samples for collection of Exemplar materials 55

56. EQuIP Peer Review Panel Materials Submission Process Materials Submission: Individual, state, district, school or organization may submit instructional materials aligned to the CCSS for review by the EQuIP Peer Review Panel in order to provide educators across the country with various models and templates of high quality and CCSS-aligned lesson and unit plans. EQuIP Exemplars: Lessons and units identified as “Exemplars” and “Exemplars if Improved” will be posted on Achieve’s website and shared broadly. http://lessons.achieve.org/

57. 57 www.achieve.org/equip

58. tcoe@achieve.org www.achieve.org/equip Send inquiries to cmarks@achieve.orgcmarks@achieve.org Submit materials for review by the EQuIP Peer Review Panel http://lessons.achieve.org/http://lessons.achieve.org/ #EQuIPchat #drtedcoe