1. Educators Evaluating Quality Instructional Products (EQuIP)
Using the Tri-State Quality Rubric

2. Development of the Tri-State Rubric
Work began by Tri-State Collaborative
Comprised of educational leaders from Massachusetts, New York, and Rhode Island
Developed criterion-based rubrics and review process
Assisted by Achieve
Launched by PARCC in June 2012
The Tri-State Collaborative (comprised of educational leaders from Massachusetts, New York, and Rhode Island) has developed criterion-based rubrics and review processes to evaluate the quality of lessons and units intended to address the Common Core State Standards for ELA/Literacy and Mathematics.

3. Arkansas EQuIP Team ELA/Literacy Mathematics Nancy Papacek
Bryant
Andrea McKenna
Springdale
Vernita Lee
Dumas
Janice Riggs
El Dorado
Felix Maull
Conway
Teresa Martin
Hamburg
The EQuIP team represents one group within the state’s Educator Leader Cadre.
The purpose of the EQuIP team is to learn and apply the tools, rubrics, and processes necessary to determine the quality of instructional materials.
At the first meeting the team In June 2012 the team was trained on how to apply the rubric to assessment lessons brought from different states, with focus on providing constructive feedback.
A follow-up meeting was held in October. At that meeting the team reviewed and evaluated additional lessons/units using the Tri-State Rubric.

4. Why EQuIP and the Tri-State Rubric
Determine quality of existing instructional materials or those under development
Place quality model lessons/units into the hands of the teachers
Build capacity of educators at the classroom, building, district, and state levels to determine quality and alignment

5. An Aligned System Common Core State Standards Model Content Frameworks
Model Lessons/Units
PARCC Assessment

8. State Initiatives Mathematics Design Collaborative (MDC)
Literacy Design Collaborative (LDC)
Cognitively Guided Instruction (CGI)
Understanding By Design (UBD)

9. Tri-State Rubric Purposes
Provides clear, descriptive criteria for CCSS lessons/units
Provides meaningful, constructive feedback to developers of lessons/units, based on common criteria
Identifies lessons/units that can serve as models
Guides collegial review and jurying processes

10. Tri-State Rubric Evaluates
Lessons that include instructional activities and assessments aligned to the CCSS that may extend over a few class periods or days
Units that include integrated and focused lessons aligned to the CCSS that extend over a longer period of time
NOT designed to evaluate a single task
Creates a common language for review and feedback.
The Tri-State Rubric:
Does not require a specific template for lesson or unit design – honors state lead
Provides criteria based feedback to the developer for improvement

11. Evaluating vs. Writing
The Tri-State Rubric can be used to evaluate lessons/units that have already been developed.
New lessons/units could be developed with the rubric in mind.
Creates a common language for review and feedback.
The Tri-State Rubric:
Does not require a specific template for lesson or unit design – honors state lead
Provides criteria based feedback to the developer for improvement

12. Organization of Rubric Criteria
Dimension I
Alignment to the Rigor of the CCSS
Dimension II
Key Areas of Focus in the CCSS
Dimension III
Instructional Supports
Dimension IV
Assessments

13. Tri-State Rubric Formats
One-Page Format
Contains dimensions, criteria, and ratings
Two-Page Format
Used during review process and includes columns for observations, comments, and suggestions
New Long Format
Intended to be completed electronically
One Page Format
Contains the entire rubric including dimensions, criteria and ratings
Two Page Format
Used during the review process and includes a column following each dimension for recording observations, comments and suggestions for improvement.
New Long Format
Intended to be completed electronically

14. The Rubric Organizes Criteria That Describe Quality Lessons/Units
Criteria that define the rubric are organized to describe quality in four dimensions.
**The most critical criteria are considered to be “must have’s” for a quality CCSS lesson/unit.
Example of One-Page Format

15. Two-Page Format Is Used to Check Criteria, Rate, and Provide Feedback:
Page 1 - Dimensions I, II, and Rating Descriptors

16. Two-Page Format Is Used to Check Criteria, Rate, and Provide Feedback:
Page 2 - Dimensions III, IV, and Summary Comments

17. Tri-State Rubric Criteria
Descriptive criteria in each Dimension
Represent a high standard of quality
Describe characteristics found in an exemplary CCSS lesson/unit
Critical criteria designated with double asterisk (**)
Located In Dimensions I & II
Must be checked to be exemplary
The descriptive criteria listed in each Dimension represent a high standard of quality – describing characteristics one would find in an exemplary CCSS lesson/unit.
Critical criteria have been designated with a double asterisk (**).
A criterion is checked when a reviewer believes that the lesson/unit contains clear, substantial evidence of the criterion’s descriptor.
Many “in progress” lessons/units, while representing good instruction, may not be deemed to currently meet the standard.
The pattern of checks in a column should thus indicate both the strengths of the lesson/unit and areas for possible improvement

18. Tri-State Rubric Checked Criteria
A criterion is checked when lesson/unit contains clear, substantial evidence of the criterion’s descriptor.
Many “in progress” lessons/units, while representing good instruction, may not be deemed to currently meet the standard.
Pattern of checks in a column represent both strengths and areas for improvement
A criterion is checked when a reviewer believes that the lesson/unit contains clear, substantial evidence of the criterion’s descriptor.
Many “in progress” lessons/units, while representing good instruction, may not be deemed to currently meet the standard.
The pattern of checks in a column should thus indicate both the strengths of the lesson/unit and areas for possible improvement

19. Tri-State Rubric Initial Review Process
Initial review should focus on
Identifying criteria that are met
Providing feedback on improvements needed
Initial review should not focus on
Assigning ratings
The primary purpose of this process is to provide specific input for the improvement of instructional materials so that teaching and learning are aligned with the CCSS.
Initially the Quality Review sessions should focus on identifying the criteria that are met and providing feedback on improvements needed to meet the criteria.
When the group has reviewed several lessons/units and individuals are interpreting criteria consistently, then ratings may be assigned for each dimension and judgments calibrated.

20. Tri-State Rubric Review Process
Step 1: Record grade and title
Step 2: Scan lesson/unit for content and organization and skim key materials, particularly those related to the Dimensions
Step 3: Compare targeted grade-level standards for alignment to the CCSS
Record the grade and title of the lesson/unit at the top of Rubric.
Scan the lesson/unit packet to see what it contains and how it is organized. Skim key materials in the packet, particularly those related to the Dimensions
Identify the grade-level standards that the lesson/unit targets and compare its focus and expectations to those of the targeted CCSS
Study and measure the text(s) that serves as the centerpiece for the lesson/unit, analyzing text complexity, quality, scope, and relationship to instruction. (Mathematics: Work the student tasks provided with the lesson/unit, keeping in mind all possible strategies students might use.
Analyze the lesson/unit for evidence of Dimension I: Alignment to the Rigor of the CCSS.
Check the criteria in Column I for which there is clear and substantial evidence of meeting the descriptors.
Examine the evidence presented within a lesson/unit through the “lens” of each criterion.
If clear evidence is found, check the criterion on the rubric; if not, use the comment column to provide input on specific improvements that can be made to meet the criterion.
Reference the criteria to make observations and recommendations about this lesson/unit and record in the column next to the Dimension.

21. Tri-State Rubric Review Process
Step 4:
In ELA, study and measure the text(s) that serves as the centerpiece for the lesson/unit, analyzing text complexity, quality, scope, and relationship to instruction.
In mathematics, work the student tasks provided, keeping in mind all possible strategies students might use.
Record the grade and title of the lesson/unit at the top of Rubric.
Scan the lesson/unit packet to see what it contains and how it is organized. Skim key materials in the packet, particularly those related to the Dimensions
Identify the grade-level standards that the lesson/unit targets and compare its focus and expectations to those of the targeted CCSS
Study and measure the text(s) that serves as the centerpiece for the lesson/unit, analyzing text complexity, quality, scope, and relationship to instruction. (Mathematics: Work the student tasks provided with the lesson/unit, keeping in mind all possible strategies students might use.
Analyze the lesson/unit for evidence of Dimension I: Alignment to the Rigor of the CCSS.
Check the criteria in Column I for which there is clear and substantial evidence of meeting the descriptors.
Examine the evidence presented within a lesson/unit through the “lens” of each criterion.
If clear evidence is found, check the criterion on the rubric; if not, use the comment column to provide input on specific improvements that can be made to meet the criterion.
Reference the criteria to make observations and recommendations about this lesson/unit and record in the column next to the Dimension.

22. Tri-State Rubric Review Process
Step 5: Analyze lesson/unit for evidence of Dimension I Alignment to the Rigor of the CCSS
Step 6: Check criteria in Column 1 for which there is clear and substantial evidence of meeting the descriptors and provide input on specific improvements for unchecked criterion
Record the grade and title of the lesson/unit at the top of Rubric.
Scan the lesson/unit packet to see what it contains and how it is organized. Skim key materials in the packet, particularly those related to the Dimensions
Identify the grade-level standards that the lesson/unit targets and compare its focus and expectations to those of the targeted CCSS
Study and measure the text(s) that serves as the centerpiece for the lesson/unit, analyzing text complexity, quality, scope, and relationship to instruction. (Mathematics: Work the student tasks provided with the lesson/unit, keeping in mind all possible strategies students might use.
Analyze the lesson/unit for evidence of Dimension I: Alignment to the Rigor of the CCSS.
Check the criteria in Column I for which there is clear and substantial evidence of meeting the descriptors.
Examine the evidence presented within a lesson/unit through the “lens” of each criterion.
If clear evidence is found, check the criterion on the rubric; if not, use the comment column to provide input on specific improvements that can be made to meet the criterion.
Reference the criteria to make observations and recommendations about this lesson/unit and record in the column next to the Dimension.

23. Tri-State Rubric Review Process
Step 7: Reference criteria when making observations and recommendations in the column next to the Dimension
Repeat Steps 5-7 for each Dimension, examining evidence presented in the lesson/unit through the “lens” of each criterion.
Record the grade and title of the lesson/unit at the top of Rubric.
Scan the lesson/unit packet to see what it contains and how it is organized. Skim key materials in the packet, particularly those related to the Dimensions
Identify the grade-level standards that the lesson/unit targets and compare its focus and expectations to those of the targeted CCSS
Study and measure the text(s) that serves as the centerpiece for the lesson/unit, analyzing text complexity, quality, scope, and relationship to instruction. (Mathematics: Work the student tasks provided with the lesson/unit, keeping in mind all possible strategies students might use.
Analyze the lesson/unit for evidence of Dimension I: Alignment to the Rigor of the CCSS.
Check the criteria in Column I for which there is clear and substantial evidence of meeting the descriptors.
Examine the evidence presented within a lesson/unit through the “lens” of each criterion.
If clear evidence is found, check the criterion on the rubric; if not, use the comment column to provide input on specific improvements that can be made to meet the criterion.
Reference the criteria to make observations and recommendations about this lesson/unit and record in the column next to the Dimension.

24. Evaluating a Lesson or Unit
In each Dimension
Examine dimension through “lens” of the criteria
Check criteria that are met
Provide input on improvements needed to meet the criteria
Examine pattern of checks and use criteria to determine a rating
If it is helpful to provide a criterion-based rating, in addition to feedback:
Examine the pattern of checks in the Dimension’s column. Determine which of the descriptors on the rating scale (found below the rubric, on the left) best characterizes the Dimension.
If all applicable “must have” criteria have been met, as well as other criteria deemed important, the lesson/unit is considered “exemplary” and receives a “3” rating.
If all “must have” criteria are not met, determine where the lesson/unit currently sits along the rating scale for the dimension.
3: Meets all “must have” criteria (**) in dimensions I, II and most of the criteria in III, IV.
2: Meets many of the “must have” criteria in dimensions I, II and many of the criteria in III, IV.
1: Meets some of the criteria in the dimension.
0: Does not meet the criteria in the dimension.
Circle the rating at the bottom of the Dimension’s column.
Make constructive comments/recommendations that explain the rating and indicate how to improve the lesson/unit.

25. Evaluating a Lesson or Unit
Descriptors for the rating scale
3: Meets all ”must have” criteria (**) in dimensions I, II and most of the criteria in III, IV.
2: Meets many of the “must have” criteria in dimensions I, II and many of the criteria in III, IV.
1: Meets some of the criteria in the dimension.
0: Does not meet the criteria in the dimension.
If it is helpful to provide a criterion-based rating, in addition to feedback:
Examine the pattern of checks in the Dimension’s column. Determine which of the descriptors on the rating scale (found below the rubric, on the left) best characterizes the dimension.
If all applicable “must have” criteria have been met, as well as other criteria deemed important, the lesson/unit is considered “exemplary” and receives a “3” rating.
If all “must have” criteria are not met, determine where the lesson/unit currently sits along the rating scale for the dimension.
3: Meets all “must have” criteria (**) in dimensions I, II and most of the criteria in III, IV.
2: Meets many of the “must have” criteria in dimensions I, II and many of the criteria in III, IV.
1: Meets some of the criteria in the dimension.
0: Does not meet the criteria in the dimension.
Circle the rating at the bottom of the Dimension’s column.
Make constructive comments/recommendations that explain the rating and indicate how to improve the lesson/unit.

26. Evaluating a Lesson or Unit
Circle the rating at the bottom of the Dimension’s column
Make constructive comments/recommendations that explain the rating and indicate how to improve the lesson/unit for that Dimension.
If it is helpful to provide a criterion-based rating, in addition to feedback:
Examine the pattern of checks in the Dimension’s column. Determine which of the descriptors on the rating scale (found below the rubric, on the left) best characterizes the dimension.
If all applicable “must have” criteria have been met, as well as other criteria deemed important, the lesson/unit is considered “exemplary” and receives a “3” rating.
If all “must have” criteria are not met, determine where the lesson/unit currently sits along the rating scale for the dimension.
3: Meets all “must have” criteria (**) in dimensions I, II and most of the criteria in III, IV.
2: Meets many of the “must have” criteria in dimensions I, II and many of the criteria in III, IV.
1: Meets some of the criteria in the dimension.
0: Does not meet the criteria in the dimension.
Circle the rating at the bottom of the Dimension’s column.
Make constructive comments/recommendations that explain the rating and indicate how to improve the lesson/unit.

27. Determining an Overall Rating
Review the patterns of check criteria and ratings for each of the for Dimensions.
Make a summary judgment about the overall quality of the lesson/unit, using the Overall Rating Scale.
Record the Overall Rating on the top right of the Tri-State Rubric.
Review the patterns of the checked criteria and the ratings for each of the four dimensions.
Make a summary judgment about the overall quality of the lesson/unit, using the Overall Rating Scale:
E: Exemplar Lesson/Unit - meets all the “must have” criteria (**) and most of the other criteria in all four dimensions (mainly 3’s).
E/I: Exemplar if Improved - needs some improvement in one or more dimensions (mainly 3’s and 2’s).
R: Needs Revision - is a “work in progress” and requires significant revision in one or more dimensions (mainly 2’s and 1’s).
N: Not Recommended - does not meet the criteria in the dimensions (mainly 1’s and 0’s).
N/R: Not ready to review – use rubric to revise and organize lesson/unit then resubmit for a quality review.
Record the Overall Rating on the top right of the Rubric.

28. Final Thoughts
The common descriptive criteria provide a common language for constructive comments, discussions, and evaluations.
The pattern of checked criteria real the perceived strengths and areas for improvement in each dimension.
The four dimensional ratings reveal the current status of the lesson/unit as a model of CCSS instruction within each dimension.
The overall rating indicates whether the lesson/unit has been deemed a CCSS model, or where it is in the process of becoming a model example.
The pattern of checked criteria reveal the perceived strengths of the lesson/unit and areas for improvement.
The comments and recommendations explain the pattern of checks and suggest ways to strengthen the lesson/unit.
The four dimensional ratings reveal its current status as a model of CCSS instruction.
For example, a rating pattern of suggests that the lesson/unit is well-aligned with the CCSS, but could use some additional thinking about Instructional Supports and, particularly, Assessment.
The overall rating indicates whether the lesson/unit has been deemed a CCSS model, or where it is in the process of becoming a model example.

29. Slides developed by and used with permission from the Tri-State Quality Rubric Project:
Achieve
th Street, NW / Suite 510
Washington, DC 20036