1. Classroom rubric, unannounced observations and summative evaluation
New Educator Evaluation Framework
Classroom rubric, unannounced observations and summative evaluation
May 2, 2012

2. Agenda Educator evaluation overview Dan Warwick 3:00 to 3:20pm
Classroom rubric introduction
District Management Council
3:20 to 4:00pm
Break
4:00 to 4:10pm
Unannounced observations
Kate Fenton
4:10 to 4:00pm
Summative evaluations
5:00 to 5:30pm

3. Agenda Educator evaluation overview Classroom rubric introduction
Unannounced observations
Summative evaluation

4. Springfield Improvement Framework (SIF)
All Schools Will:
1. Identify and implement a schoolwide instructional focus.
2. Develop professional collaboration teams to improve teaching and learning for all students.
3. Identify, learn and use effective evidence-based teaching practices to meet the needs of each student.
4. Create a targeted professional development plan that builds expertise in selected best practices.
5. Re-align resources (people, time, talent, energy and money) to support the instructional focus.
6. Engage families and the community in supporting the instructional focus.
Create an internal accountability system growing out of student learning goals that promote measurable gains in learning for every student and eliminates achievement gaps.
Supported by the district office, each school will work to improve CORE instruction within a climate and culture that is
GOAL FOCUSED, ADAPTIVE AND COHESIVE

5. Springfield Public Schools has undertaken many valuable efforts to develop teachers and raise student achievement MA
Curriculum
Frameworks
Data Warehouse
School Improvement Planning
Teacher Incentive Fund
Springfield Improvement Framework
School Improvement Grants
Dropout Prevention Taskforce
Educator Evaluations
Organizational Health Initiative
Wraparound
Services
Literacy and
Numeracy
Credit Recovery
Night School
Achievement
Network
Partnership
Mass Core

6. The various initiatives fit together with a common purpose of raising student achievement
The work
Implement a consistent, rigorous curriculum built on common standards with common unit assessments
SIF #1,3,4,7
Coach, develop and evaluate educators based on a clear vision of strong instruction
SIF #1,2,3,4
Effective instruction in every class, every day
Shared, high expectations for all students
Students achieve grade level proficiency
Students graduate ready for college and career
Deploy data that is timely, accurate and accessible to make decisions for students, schools and the district
SIF #5,7
Strengthen social, emotional and academic safety nets and supports for all students
SIF #6

7. Currently, the district’s Level 4 schools rarely rate teachers as not meeting expectations
* Source: STEDS ratings collected from Human Resources

8. 37th %ile 96th %ile 3rd %ile 63rd %ile
Effective teaching has a dramatic impact on student achievement and student outcomes in adulthood
Student achievement percentile after 2 years
Least effective teacher
Most effective teacher
50th %ile (average school; average teacher)
37th %ile
96th %ile
3rd %ile
63rd %ile
Student outcomes in adulthood
Most effective school
More likely to attend college
Earn higher salaries
Live in better neighborhoods
Save for retirement
How to understand this table: take a group of similarly performing students and assign them to one of four environments: (1) an ineffectively-led school with ineffective teachers, (2) an ineffectively-led school with effective teachers, (3) an effectively-led school with ineffective teachers, and (4) an effectively-led school with effective teachers. After 2 years, measure the students’ performance and their achievement percentiles (shown in the table).
This shows that effective teaching and effective leadership have significant impact on student achievement
And if students achieve in school, they’ll also much more likely to succeed in life
Least effective school
Source: Marzano (2003) What Works in Schools: Translating Research into Action; Chetty, Friedman, Rockoff (December 2011) The long-term impacts of teachers: teacher value-added and student outcomes in adulthood

9. Teacher effectiveness greatly impacts student achievement
School and teacher scenarios
Student achievement percentile after two years
Most effective school and most effective teacher
96th
Most effective school and average teacher
78th
Least effective school and most effective teacher
63rd
Average school and average teacher
50th
Most effective school and least effective teacher
37th
Least effective school and least effective teacher
3rd
Source: Marzano (2003) What Works in Schools: Translating Research into Action

10. The new system provides an exciting opportunity for Springfield
More honestly assess instruction
Unannounced observations
Use of student data
Increase meaning
Self-assessment by the teacher
Goal setting in collaboration with administrator
Improve practice
Goal setting focused on development
Formative assessments and frequent observations increase feedback
The new evaluation system will be implemented over three years - not all at once
Today will provide an overview of the entire evaluation system, with a focus on what evaluators need to do this year
Other supports, such as technology and templates, will be put in place to help implementation

11. Kim Marshall’s 4 part model
The new system aligns with the state’s regulations and teacher evaluation experts
Kim Marshall’s 4 part model

12. The new teacher evaluation system is based on 5 steps
Every educator uses a rubric and data about student learning
Every educator develops a team student learning goal that aligns with the district and a team professional practice goal that supports the student learning goal
Continuous
Learning
Every educator earns one of four ratings of performance
Every educator and evaluator collects evidence and assesses progress
Every educator has a mid-cycle review

13. The district’s recommended timing for each step:
Educator on a 1-year evaluation cycle
Educator on a 2-year evaluation cycle
Self-assessment
October 1st
Goal setting
November 1st
Formative assessment
February 1st
Not applicable
Formative evaluation
June 1st of year 1
Summative evaluation
June 1st
June 1st of year 2
= tentative contractual agreement

14. Agenda Educator evaluation overview Classroom rubric introduction
Unannounced observations
Summative evaluation

15. What does the new rubric look like?
Old STEDS (43 elements)
New SEEDS (33 elements)
Principle I: Currency in the Curriculum
Principle II: Effective Planning and Assessment of Curriculum and Instruction
Standard I: Curriculum, Planning, and Assessment
Principle III: Effective Management of Classroom Environment
Principle IV: Effective Instruction
Principle V: Promotion of High Standards and Expectations for Student Achievement
Principle VI: Promotion of Equity and Appreciation of Diversity
Standard II: Teaching All Students
Standard III: Family and Community Engagement
Principle VII: Fulfillment of Professional Responsibilities
Standard IV: Professional Culture

16. What’s the purpose of a rubric?
Develop a consistent, shared understanding of what proficient performance looks like in practice,
Develop a common terminology and structure to organize evidence, and
Make informed professional judgments about formative and summative performance ratings on each standard and overall.
Not a classroom observation tool

17. How does the rubric fit into the 5-step evaluation cycle?
Every educator uses a rubric to self-assess against Performance Standards
Professional Practice goals – team and/or individual must be tied to one or more Performance Standards
Continuous
Learning
Rubric is used to analyze performance and determine ratings on each Standard and Overall
Evidence is collected for Standards and Indicators; rubric should be used to provide feedback
Rubric is used to assess performance and/or progress toward goals

18. Exercise #1: Teacher rubric at-a-glance handout
Review the 4 standards, 16 indicators, and 33 elements
Discuss your initial reactions at your table:
Familiarity?
Strengths?
Comments?

19. The rubrics are designed to have alignment vertically, horizontally, and comprehensively (across rubrics)
Vertical alignment (down the rubric)
Standard
Teacher Rubric
Indicator
Unsatisfactory
Needs Improvement
Proficient
Exemplary
Element
(descriptors)
Horizontal alignment (across element)
Comprehensive alignment (across rubrics)

20. Vertical alignment means consistency from standards, to indicators, down to elements
I. Curriculum, Planning, and Assessment
A. Curriculum and Planning
B. Assessment
B1. Variety of Assessment Methods
B2. Adjustment to Practice
C. Analysis
Standard
Indicator
Element

21. Horizontally alignment is established by setting a consistent definition of what is “Proficient”
The educator’s performance fully and consistently meets the requirements of a standard.
It is the expected, rigorous, demanding, but attainable level of performance for educators.
Proficient educators integrate the knowledge, skills and abilities needed for effective content-area instruction
Exemplary: exceeds proficient
Needs Improvement: below proficient
Unsatisfactory: consistently below proficient

22. Performance rating descriptions
The rating descriptions are determined by the quality, scope of impact, and consistency of the educator’s performance
Quality
Scope of impact
Consistency
Performance rating descriptions

23. Exercise #2: Horizontal alignment
Refer to Handout 2
Focus on Element 1
CIRCLE the behavior that is repeated at each level of performance
UNDERLINE the words that distinguish between levels
Discuss your results at your table

24. Comprehensive alignment means consistency between teacher, administrator, and superintendent rubrics
Description
Teacher – Element II-A-3: Meeting Diverse Needs
Uses appropriate practices, including tiered instruction and scaffolds, to accommodate differences in learning styles, needs, interests and levels of readiness, including those of students with disabilities and English learners.
Administrator –
Element I-B-3: Diverse Learners’ Needs
While observing practice and reviewing unit plans, looks for and identifies a variety of teaching strategies and practices that are effective with diverse learners.
Superintendent – Element I-B-3: Diverse Learners’ Needs
While observing principal practice, ensures that principals look for and identify a variety of teaching strategies and practices that are effective with diverse learners when they observe practices and review unit plans.

25. Agenda Educator evaluation overview Classroom rubric introduction
Unannounced observations
Summative evaluation

26. How traditional evaluation fails
It doesn’t discern between levels of teaching:
Excellent
Good
Mediocre
Poor
So it doesn’t affirm the best teachers, use their talents, and keep them on board.
And it doesn’t help less-effective teachers get better or leave the profession.
Source: Kim Marshall

27. But are unannounced visits possible?
Yes, if they are:
Frequent
Short
Systematic
Clear criteria
Coaching feedback each time
Informal tone
Fair
Integrated with unit planning and data analysis
Source: Kim Marshall

28. Unannounced observations: frequent sampling, and coaching
Short visits to fit them in to very busy days
Unannounced to see what kids are experiencing daily
Lots of them to sample all aspects of teaching, blend in
Prompt, thoughtful feedback to each teacher
Early intervention with ineffective teachers, follow-up
Informal to maximize adult learning; formative feedback
Systematic cycling through the whole staff
Integrated with curriculum planning and student learning
Source: Kim Marshall

29. “You can observe a lot by watching”
Slow down, breathe, listen, pay attention
Don’t impose a checklist on the situation; “free range”
Look at the learning task, check in with kids
Look for student understanding vs. superficial stuff
You’ll notice 1-3 main points to discuss.
So as not to miss anything, I suggest a mental checklist
What every lesson, K-12, should contain
Shared with teachers – everyone on the same page
Source: Kim Marshall

30. S - Safety O - Objectives T - Teaching E - Engagement L - Learning
SOTEL
S - Safety O - Objectives T - Teaching E - Engagement L - Learning
Source: Kim Marshall

31. What very effective evaluators look for
Student engagement, learning, and understanding
Participation, thinking, talking, making sense
The rigor and cognitive complexity of the work
Asking students “What are you working on?”
Listening in on turn-and-talks
Climate, tone, and atmosphere
Effective teaching: clarity, modeling, explaining, listening, checking for understanding
And what these evaluators looked for, they got!
Source: Kim Marshall

32. Insights versus time in classroom
Source: Kim Marshall

33. The logic of unannounced observations
Unannounced to see everyday reality.
But it would be unfair to observe just once a year.
So observations must be frequent to sample accurately.
But if frequent, don’t have time to stay for full lessons.
So observations need to be short.
But if visits are short, full write-ups aren’t possible.
But it’s easy to lose track, miss teachers, double up.
So the process must be systematic.
Source: Kim Marshall

34. What does unannounced-observation feedback look like?
Standard II-A-2: Student Engagement
Elementary school example “The teacher was assisting students at their desks. Students were working on four division problems that were on the board. Throughout the class she circulated around the room stood by the students' desks and talked with students about the problems in a soft tone. Hold students responsible for writing the necessary work. Encourage them to copy the information correctly. Also make them responsible for getting the classroom material(s) such as the Math strips which were used to make trains.”
Standard II-B-3: Student Motivation
Standard I-A-2: Child and Adolescent Development
Source: Urban District – Evaluator and Educator is Confidential

35. What does unannounced observation feedback look like?
Standard I-A-3: Rigorous Standards-Based Unit Design
Secondary school example “An essential question is posted and the current activity relates to the e.q. Students are presenting in the front of the class the promotional materials they created for the middle or southern colonies they have chosen. The activity is relevant and all students were engaged in the learning based on individual interests. Research is obvious as students present facts and information. The teacher is evaluating presentations using a pre-designed checklist and is asking students clarifying questions. Teacher is also complimenting student work and effort. Students are attentive.”
Standard II-B-2: Collaborative Learning Environment
Standard II-D-2: High Expectations
Source: Urban District – Evaluator and Educator is Confidential

36. What does unannounced observation feedback look like?
Another example “The teacher was providing instructions and clear examples for each group before transitioning to learning centers.”
How does this example compare to the two examples we looked at earlier?
Source: Urban District – Evaluator and Educator is Confidential

37. So, what have the district, SEA and MTA tentatively agreed upon?
Decision areas
Announced
Unannounced
Number of observations
Minimum of 1 formal per cycle
Minimum of 1 formal per school year
Exception – Minimum of 2 formal per school year for a non-renewal
Observation length
Minimum of 30 minutes
Minimum of 10 minutes
Pre-conference
Required
None
Report documentation
Observation report documented within 10 school days
Post-conference
Required based on:
Teacher receives a Needs Improvement or Unsatisfactory rating on any standard
Teacher request
Not required based on:
Teacher receives an Exemplary or Proficient on all standards
No request made by teacher

38. Exercise #3: unannounced observation practice (see handout)!
Video
Instructions
TIMSS Video – 7th grade Science
Third lesson in a sequence of eight lessons on the circulatory and respiratory systems.
The lesson is 45 minutes in duration
There are 33 students in the class
Minutes 5 to 10
Step 1: provide feedback for the educator in the video
Step 2: evaluate your feedback against SOTEL, rubric standards and Kim Marshall’s four part model

39. Agenda Educator evaluation overview Classroom rubric introduction
Unannounced observations
Summative evaluation

40. Multiple measures of student learning
The summative evaluation will be based on evidence gathered over the school year
Observations
Includes unannounced and announced observations
Multiple measures of student learning
Includes MCAS growth when applicable and district-determined measures of student learning
Effective
Other evidence
Educator’s collection of evidence, including:
Evidence of fulfillment of professional responsibilities and growth
Evidence of family engagement

41. Curriculum, Planning and Assessment Family and Community Engagement
The evidence will be used to determine a rating against the classroom rubric
Curriculum, Planning and Assessment
Family and Community Engagement
Curriculum and Planning
Assessment
Analysis
Engagement
Collaboration
Communication
Standard
Teaching All Students
Professional Culture
Instruction
Learning Environment
Cultural Proficiency
Expectations
Reflection
Professional Growth
Collaboration
Decision making
Shared Responsibility
Professional Responsibilities
Indicator
The new educator rubric is in development but will definitely have four key standards:
Curriculum, Planning and Assessment Standard
Teaching All Students Standard
Family and Community Engagement Standard
Professional Culture Standard

42. Educators will receive one of four performance ratings following the summative evaluation
Consistently exceeds requirements of a standard
Able to model the standard to other educators
Exemplary
Fully and consistently meets the requirements of a standard
Proficient
Below requirements of a standard - improvement is necessary
Needs Improvement
Consistently below the requirements of a standard
Unsatisfactory

43. Next year’s educator plan is determined by the performance rating
Ratings
Educator Plans
PTS educators
Non-PTS educators
Exemplary
Self-Directed Growth Plan
Developing Educator Plan
Proficient
Needs Improvement
Directed-Growth Plan
Unsatisfactory
Improvement Plan

44. Exercise #4: Summative evaluation
Refer to summative evaluation form and classroom rubric
Discuss your initial reactions to the summative evaluation form at your table:
Strengths?
Comments?

45. In closing…POP QUIZ!
If a student has an effective teacher and school for two years, what’s the student achievement impact?
How is the new educator evaluation system an opportunity?
When are summative evaluations due?
Are rubrics an observation tool? Why?
How should I approach an unannounced observation?
If an educator receives a needs improvement in standard two and a proficient rating in all other standards, what is the highest overall rating he/she could receive?
If an educator receives a needs improvement in standard three and a proficient rating in all other standards, what is the highest overall rating he/she could receive?