1. Alignment of Teacher Evaluation to the School Improvement Process Dr. Kathleen M. Smith Almost Former Director of the Office Of School Improvement Retiring July 31, 2014

2. Introduction How were the academic reviews different this year? What were the findings of the academic reviews held throughout the state last year? How can the findings of the academic review be used to implement action steps? What are the barriers to implementing these steps? What will happen if the steps are not implemented?

3. How were the academic reviews different? Focus was on the alignment of the written, taught and tested curricula. Reviews were not on site at each school but held at the division with core division teams selected by the division. Examined and evaluated, against a set of rubrics designed by Stronge and Associates, a core collection of documents from the division’s schools accredited with warning (lesson plans, completed observation forms, observation schedules, curricula guides, unit tests). Why not benchmark tests?

4. How many divisions received an academic review? 97 divisions 403 schools Including divisions with focus schools not accredited with warning or other schools accredited with warning Cost = $790,000 Related Web Links: http://www.doe.virginia.gov/support/school_impr ovement/academic_reviews/index.shtml http://www.doe.virginia.gov/support/school_impr ovement/academic_reviews/index.shtml

5. What “dipstick” was used to gauge that the collection was sufficient enough to draw conclusions throughout the state? Unfortunately, we did not keep track of how many documents were reviewed or how many of each type of documents were collected. We usually finish academic reviews in two to three days, these took at least a week and in some divisions two weeks. We usually finish academic reviews by December, we didn’t finish until March. Generally, the cost is about $580,000.

6. What were the overall findings? Teachers were teaching. Students were learning. Unit tests sampled were frequently not aligned to the new standards. What does this mean?

7. What were the overall findings? There were plenty of summative assessments – benchmark tests. Lessons lacked true classroom day to day formative assessment. What does this mean?

8. What were the overall findings? There was evidence that principals spent time in classrooms completing observations. There was limited evidence on the observation forms reviewed that feedback from principals provided to teachers would clearly enable the teachers’ understanding of how to change practice. What does this mean?

9. What were the overall findings? Professional development was provided. Monitoring of new professional development skills was not evident. What does this mean?

10. What were the overall findings? Lesson plans were reviewed by the principal. When the team reviewed the lesson plans, there was little alignment to the new standards. What does this mean?

11. This is the $1,000,000 question-------- How do we impact the planning and delivery of instruction so that what teachers teach and students learn is aligned to the new standards?

12. Focus on the first five teaching standards. Align school improvement activities to teacher evaluation. 1. Professional Knowledge 2. Instructional Planning 3. Instructional Delivery 4. Assessment of and for Student Learning 5. Learning Environment

13. Strategies & Average Effect Sizes on Achievement* Strategies Percentile Gain Identifying similarities and differences45 Summarizing and note taking34 Reinforcing effort and providing recognition29 Homework and practice28 Nonlinguistic representations27 Cooperative learning27 Setting objectives and providing feedback23 Generating and testing hypothesis23 Questions, cues, and advance organizers22 Building vocabulary20 Interactive games20 Student discussion/chunking17 *Haystead, M. W. & Marzano, R. J. (2009). Meta-Analytic Synthesis of Studies Conducted at Marzano Research Laboratory on Instructional Strategies

14. *Hattie, J (2009). Visible Learning: A synthesis of over 800 meta-analyses relating to achievement. Strategies & Average Percentile Gain on Achievement* Strategies Percentile Gain Feedback to Students37 Instructional Quality34 Instructional Quantity30 Direct Instruction29 Graded homework29 Acceleration27 Remediation/feedback24 Personalized instruction21 Challenge of goals20 Peer Tutoring19 Mastery Learning19 Questioning16 Advance Organizers14 Simulation and games13 Computer-assisted instruction12 Instructional media12

15. The Power of Formative Assessment “…the most powerful single influence enhancing achievement is feedback.” Feedback from teachers to students (e.g., specific feedback on strengths and areas for improvement) Feedback from students to teachers (e.g., what they know and do not know, effectiveness of strategies) *Hattie, J (2009). Visible Learning: A synthesis of over 800 meta-analyses relating to achievement.

16. Basic Component: Assessment (Lesson Observation)

17. Why is Lesson Observation by the Principal Important? “…[T]he purpose of supervision should be the enhancement of teachers’ pedagogical skills, with the ultimate goal of enhancing student achievement.”

18. Teacher evaluation is not an end in itself, but a means to an end — TEACHER IMPROVEMENT. Davis, D. R., Ellett, C. D., & Annunziata, J. (2002). Teacher evaluation, leadership and learning organizations. Journal of Personnel Evaluation in Education, 16(4), 287-301. p. 288. Why Evaluation Only Is Not Enough

19. Making Permanent Changes: The Importance of Leadership Explicit Training and Expectations on Needed Skills Explicit, Sustained Emphasis and Monitoring of New Skills Change in Practice This Same Results as Last Year

20. Focus of Evidence STANDARDS & INDICATORS Teacher Practice Student Learning (Evidence is the available body of facts or information indicating whether a belief or proposition is true or valid.)

21. Main Areas for Evidence Collection during Observations Standard 1: Professional Knowledge Standard 3: Instructional Delivery Standard 4: Assessment of and for Learning Standard 5: Learning Environment Strong Evidence

22. Documenting Evidence Tips Avoid terms that express judgment (“neat classroom,” “fun activity,” “caring attitude”) Avoid words that imply, but do not specify, quantity (“most,” “few,” “several”) Stick to the five senses Remember Who, What, When, Where, How

23. How can this evidence for Instructional Delivery be improved? 3.3Differentiates instruction to meet students’ needs. Spent the majority of the time on whole class instruction for math though several of the students demonstrated proficiency on the long division process in the first few minutes of class.

24. How can this evidence for Learning Environment be improved? 5.1Arranges the classroom to maximize learning while providing a safe environment. Classroom neatly arranged so students can work in small groups.

25. School Leadership Basic Components Evaluation Tool

26. Division/School Professional Development Evaluation Tool

27. Student Behavior vs Teacher Behavior Spend more time providing evidence of what kids are doing and what kids are learning and less time explaining what the teacher is doing

28. Periodically Collect Evidence of How and Why a Lesson was Planned Where is the alignment to the most rigorous part of the standard? What teaching strategies did you select and why? Why these resources? How will you know if a student mastered the objective? Standard 1 Standard 2: Planning the Lesson Standard 3 Standard 4 Standard 5

29. Barriers to Improvement The biggest barriers to improvement are not using the right words to influence change; not evaluating often enough; and, not collecting strong, reliable, and explicit evidence. What happens if I tell you that you work hard? Would it cause you to provide feedback to students more often? What if you only provide me with feedback three times during the year? Would I begin using high- yield teaching strategies? What if you check my lesson plans as “being in on time each week?” Would my lesson plans target more rigorous content? What if you tell me I did a great job of explicitly modeling how to solve a problem, but the students failed the SOL assessment questions with similar problems?

30. What will happen if the barriers aren’t overcome?

31. Questions Dr. Kathleen M. Smith, Retired August 1 kathleeen.smith.petersburg@gmail.com Office of School Improvement Kathleen.Smith@doe.virginia.gov Beverly.Rabil@doe.virginia.gov Michelle.Wallace@doe.virginia.gov (804)225 -2865