1. DDMs for School Counselors RTTT Final Summit April 7, 2014 Craig Waterman & Kate Ducharme

2. Agenda  Overview of DDMs (30 minutes)  Introduction to DDMs  Q&A with Craig Waterman  DDM Considerations for School Counselors – Group Planning Activity (40 minutes)  Selecting Measures (15 minutes)  Closing and Next Steps Massachusetts Department of Elementary and Secondary Education 2

3. The Educator Evaluation Framework Exemplary Proficient Needs Improvement Unsatisfactory High Moderate Low Summative Performance Rating Student Impact Rating  Everyone earns two ratings 3

4. Two Ratings – Intersection of Practice and Impact Summative Rating Exemplary 1-yr Self-Directed Growth Plan 2-yr Self-Directed Growth Plan Proficient Needs Improvement Directed Growth Plan UnsatisfactoryImprovement Plan LowModerateHigh Rating of Impact on Student Learning

5. Identifying DDMs – Key Questions  Is the measure aligned to content?  Does it assess what the educators intend to teach and what’s most important for students to learn?  Is the measure informative?  Do the results tell educators whether students are making the desired progress, falling short, or excelling?  Do the results provide valuable information to schools and districts about their educators? See Technical Guide B for more about these key questions: http://www.doe.mass.edu/edeval /ddm/TechnicalGuideB.pdf See Technical Guide B for more about these key questions: http://www.doe.mass.edu/edeval /ddm/TechnicalGuideB.pdf

6. Key Messages for Stakeholders

7. Difference Between Direct and Indirect Measures Classroom-based educator provides direct instruction to students. Classroom- based Educator Responsibilities Students learn; acquire knowledge and skills Measures of Student Learning  Classroom-based educators can directly measure the impact of their instruction on student learning. Direct Measures Direct Measure

8. Difference Between Direct and Indirect Measures  For SISP who do not directly instruct students, there often is an an intermediary step that exists between SISP responsibilities and student learning, therefore making the SISP contribution indirectly tied to student learning. SISP educator provides specialized support services to students. SISP Responsibilities Students can access general education curriculum. Indirect Measures Students learn; acquire knowledge and skills Measures of Student Learning Indirect Measures Indirect Measure

9. Do DDMs have to be identical?  DDMs must be “comparable across schools, grades, and subject matter district-wide.” (603 CMR 35.09(2)a)603 CMR 35.09(2)a Type 1: Comparable within a grade, subject, or course across schools within a district Identical measures are recommended Investigate Fairness Investigate Fairness for all students and educators Example Elementary 5 th Grade Teacher Music Teacher Another Elementary 5th Grade Teacher Music Teacher Type 2: Comparable across grade or subject level district-wide Impact Ratings should have a consistent meaning across educators; therefore, DDMs should not have significantly different levels of rigor See Investigating Fairness Implementation Brief: http://www.doe.mass.edu/edeval/dd m/Fairness.pdf http://www.doe.mass.edu/edeval/dd m/Fairness.pdf See Investigating Fairness Implementation Brief: http://www.doe.mass.edu/edeval/dd m/Fairness.pdf http://www.doe.mass.edu/edeval/dd m/Fairness.pdf

10. Identify Commonalities  One of the goals of DDMs is to support common measures across a district  Approach to selecting DDMs for SISP educators could involve identifying commonalities across multiple roles  For example, a district-wide DDM involving collecting feedback from stakeholders on the quality, usefulness, and timeliness of communications Massachusetts Department of Elementary and Secondary Education 10

11. DDMs for District “Singletons”  Districts should consider the following options to identify and develop DDMs for “singletons”:  Work with neighboring districts to identify and/or create DDMs.  Group educators within the district who have similar responsibilities, albeit different roles, to collaborate on DDMs.  Look to outside rating entities (such as state or national organization standards).  Use DDMs that can be easily interpreted (e.g., for a school counselor, using number of seniors who applied to college instead of number of students who are prepared to apply to college.) Massachusetts Department of Elementary and Secondary Education 11

12. DDM Implementation Plan – June 1, 2014

13. Resources: http://www.doe.mass.edu/edeval/ddm/ http://www.doe.mass.edu/edeval/ddm/  Implementation Briefs  Technical Guide B  Webinar Series  Commissioner’s Memoranda  Educator Evaluation Newsletter  Technical Assistance and Networking Sessions  Using Current Assessments in DDMs (Curriculum Summit)  Example Assessments  Other ESE documents (Technical Guide A, Part VII, Regulations) 13

14. Questions and Answers

15. Who scores the DDM?  Outside Organizations  Commercial assessments  Automated methods  Paid raters (e.g., college students, retired teachers)  Teams of Teachers (e.g., all 5 th grade teachers)  Team members rate each other’s students’ responses  Multiple raters score each response  Individual Teachers  Random auditing (rechecking) Internal External See Scoring and Parameter Setting Implementation Brief: http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf See Scoring and Parameter Setting Implementation Brief: http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf

16. How do I determine high, moderate, or low growth? QualitativeQuantitative Previous Results Variability Fairness Educator Review Identification Agreement Ask educators how much growth is moderate on this assessment? Does the DDM make meaningful distinctions? Do all students have an equal chance to demonstrate growth? See Scoring and Parameter Setting Implementation Brief: http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf See Scoring and Parameter Setting Implementation Brief: http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf http://www.doe.mass.e du/edeval/ddm/Scoring &PSetting.pdf

17. DDMs for School Counselors

18. Identifying DDMs – Key Questions for Specialized Instructional Support Personnel Massachusetts Department of Elementary and Secondary Education 18  Is the measure aligned to content?  For SISP educators (including school counselors), content should reflect their job functions and responsibilities and should align to what they do to support students, educators, administration, and/or parents.  Instructions: With you table, identify key content (i.e., key job functions and responsibilities) of your role.

19. Identifying DDMs – Key Questions for Specialized Instructional Support Personnel  What does success look like? What would you see if someone was successful at these job functions and responsibilities?  Instructions: With your table, identify what success looks like for each of the job functions and responsibilities you identified. Massachusetts Department of Elementary and Secondary Education 19

20. Identifying DDMs – Key Questions for Specialized Instructional Support Personnel  What is measureable?  Measurement is…  The systematic process to assign a number to an observation  Example: How tall is my friend? 1.Place a measuring tape next to my friend and read the number next to the line that is closest to the top of his or her head. 2.Stand next to my friend and estimate how much taller or shorter they are compared to myself. 3.Place a ruler flat on my friend’s head and mark a line on a wall where the ruler hits the wall. Then measure how far the line is off the ground.  Instructions: With your table, identify approaches could you use to measure the job responsibilities and functions you identified.

21. Identifying DDMs – Key Questions for Specialized Instructional Support Personnel  Is the measure informative?  Chosen measures should provide both educators and districts with actionable information to inform practice and identify both areas of strength and areas where more supports are needed. Massachusetts Department of Elementary and Secondary Education 21

22. Key Messages for Stakeholders