The Classroom Analyst & Using Growth Models Heather Tauer-Reid & Val Vogt Education Resource Group, Inc. 21 Waterway Avenue, Suite 300 The Woodlands TX 77380 877-508-6824 www.ERGAnalytics.com Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending Copyright © 2012- Education Resource Group, Inc. All Rights reserved.

Performance Management Functions The “Heart and Soul” of Performance Management How well is your organization doing? Where are improvements necessary? Who are the models of high performance? Are your improvement strategies working? The greatest challenge to performance improvement is the difficulty that organizations have in seeing themselves as they really are. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 2 2

The Campus Analyst – The Purpose Understand and analyze performance of the basic business unit in public education To provide a framework for measuring relative performance To evaluate the effectiveness of campus leaders To set and monitor performance goals All of the learning and most of the costs are at the campuses. We need to understand performance there in great detail. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

ERG Performance Management System Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 4 4

A Level Playing Field - Academic Performance Index Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 5 5

Mesquite ISD Performance Matrix Lets explain the axis. Y is the staff ratio adjusted for campus size. X is the academic index based on met standard and commended for ES and MS and all 5 measures for high schools adjusted for demographics. Within any district there are wide variations in performance. Exercise 3 (4 groups, describe status of schools I your quadrant Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Elementary School Performance Matrix Lets explain the axis. Y is the staff ratio adjusted for campus size. X is the academic index based on met standard and commended for ES and MS and all 5 measures for high schools adjusted for demographics. Within any district there are wide variations in performance. Exercise 3 (4 groups, describe status of schools I your quadrant What can we say about the performance of this ES? Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Activity I Login to the Campus Analyst and choose the “Campus Sample Group”. Look at the “square” that your campus is in and describe the meaning of your campus’ position within the Matrix. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Elementary School Performance Matrix Now we see the year-to-year change in the major performance indices. What else have we learned about this campuses’ performance? Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Activity II Using your campus’ data and the Performance Table to the right of the Matrix, complete the following: 1. Did your EPI move up or down? How much did it change? 2. Did your API move up or down? How much did it change? 3. Did your SPI move up or down? How much did it change? Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Longitudinal Charts Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Activity III Click on the Longitudinal Icon next to your Campus’ API. Using your campus’ API Longitudinal data and the three charts, What do the 3 charts tell you about how your campus been trending? What do you believe are the factors contributing to that trend? HOMEWORK Use the Profile menu option to the left of the screen to view all subject areas (will be found near the bottom of the profile screen. Use the Longitudinals to investigate how each content area contributed to your campus’ overall performance. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Campus Ranking Tables Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Activity IV Using your campus’ data and the Ranking Table to the right of the Matrix, complete the following: 1. What is your API Rank? Which campus is immediately above you? 2. What is your SPI Rank? Which campus is immediately above you? 3. Do these agree with what you expected? Were there any surprises? Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Classroom Focus 15 Now let’s drill down to the classroom level. 15 Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending 15 15

Importance of Measuring Classroom Performance There is no correlation between academic achievement measures and teacher experience We do not have a measure to determine improvement of teacher performance We cannot afford to continue this level of performance If we had a measure, perhaps this relationship would change Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

The Problem with Growth Measures and Hurdle Rates When measuring the year-to-year growth in student performance, the problem with using the Met Standard and Commended Performance hurdle levels is even more acute. Using the example of 7th grade Mathematics, student “A” can gain 1 point in achievement (e.g. from 27 correct to 28 correct) and go from not meeting standard to met standard and the overall pass rate percentage for the campus and district will rise. Student “B” can improve by 15 points (e.g. from 28 correct to 43 correct) and not change the achievement level of the student, campus, or district. Clearly, student “B” shows more growth in achievement than student “A”, but that growth differential is not only hidden by using only hurdle levels of performance, but is actually misleading since the campus with student “A” will appear to improve in achievement while the campus with student “B” will appear to be stagnant. Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

How to Compare Performance Year to Year Same Grade – e.g. 5th grade 2011 to 5th grade in 2012 Student mix may change Same Students – e.g. 5th grade in 2011 to 6th grade in 2012 Better metric for measuring classroom effectiveness Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Explaining Growth in Learning The Learning Growth Index (LGI)

Frequency Curve of Test Scores Student Result Test # 1 Student Result Test #2 LGI Simple frequency distribution of test scores on any test. Every student has a z-score just like they have a raw and scale score. Test scores are distributed in a population (say 5th grade math takers in English) by frequency according to a performance measure which can be raw score or scale score. To make the performance values useful, we utilize a Z-score of NCE which allows us to compare the performance of this population on another measurement instrument. Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Activity V Z-Score Exercise Break into Groups Discuss the concept of Z-Score as a measurement tool What can you gain from creating an “Expected Value” Give an example of how you might use z-scores outside of the classroom. Break into groups (4?) discuss the concept of z-score as a measurement tool What can you gain from creating an expected value? Give an example of how you might use z-score outside of the classroom

What Can We Learn? Change in mean score Lower – test was harder relative to previous Change in standard deviation Did the group cluster more (similar achievement level) or distribute more (dissimilar achievement) Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

What are the Basic Measurements? – Defining Terms Education Performance Index EPI Overall grade 1. Student Achievement SAI Scale Score on a standardized test 2. Student Growth LGI Year-to-year change in Z-score 3. Teacher Achievement TAI Average of student LGIs. Measure of classroom achievement 4. Teacher Growth TGI Year-to-year change in TAI Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending Page 23 23

Measuring Growth and Achievement High Achievement & Low Growth Low Achievement & Low Growth High Achievement & High Growth Low Achievement & High Growth How do we take these basic measurements and make them useful for leadership. A table of numbers is not effective. This view shows performance of your campus within a district for all tests taken. The red dot is the campus’ weighted average. Each of the green dots represents a grade level for all tests taken. Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Using ERG Effectively ERG is not the sole method for personnel evaluation. Protect morale by making decisions based on your mission. Establish the “purpose” of ERG. Conduct autopsies without blame. Confront the brutal facts. Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Final Thoughts!! ERG is a tool NOT another program. It tells the story of what is really going on – all the way down to the classroom level. Many tools reveal performance data, ERG reveals dynamic movement of students’ learning. Which teachers “grow” students the most, i.e. which resources are being deployed successfully for the most benefit. ERG helps to set CAMPUS GOALS that are based on a total picture. How a campus achieves those goals is up to the principals’ leadership. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Measuring Growth and Achievement Campus Weighted Average Selected Detail level: Grade Level Subject Teacher How do we take these basic measurements and make them useful for leadership. A table of numbers is not effective. This view shows performance of your campus within a district for all tests taken. The red dot is the campus’ weighted average. Each of the green dots represents a grade level for all tests taken. Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

What is Next?

Let’s Look at Smith Elementary While Ms. Adams students had low achievement, they were greatly improved from the prior year. In looking at her students’ results, it was easy to see they were on a trajectory for success in the coming year. What could the principal do with this information? Low Growth/ High Achievement Low Growth/ Low Achievement Mr. Atkins is a top teacher; his students have high growth and high achievement. What could the principal do with this information? High Growth/ High Achievement High Growth/ Low Achievement Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 29

Mr. Atkins Mr. Atkins had a class of traditionally average students. In examining the reports from ERG, the principal found that his students were strong across the board in all of the sixth grade math Student Expectations. The principal asked Mr. Atkins what his key strategies were. Rather than start class by taking roll, Mr. Atkins had students immediately do a “warm-up” activity of previously taught skills. Mr. Atkins had designed these himself to ensure that they covered the S.E.’s. He used the Information Guide from TEA to design his questions. He had also developed his own calendar to make sure that each S.E. was regularly reviewed. Mr. Atkins had a study group for students during homeroom. He worked with his fellow teachers to allow the students to come in and work on skills they had difficulty with. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 30

Ms. Adams The new principal also interviewed Ms. Adams. She found that Ms. Adams had a disproportionate number of At-Risk students in her room, and yet had done an outstanding job in helping them improve. Like Mr. Atkins, Ms. Adams had a routine for review and reinforcement of skills and was very knowledgeable about the TAKS (soon to be STAAR) Information Booklets. The new principal was concerned that, in reviewing PDAS, Ms. Adams was ranked in the lower quartile in the building in Domains III and VIII. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 31

ERG Provides Specific Measures of “Output” PDAS Active, Successful Student Participation in the Learning Process  Learner-centered Instruction  Evaluation and feedback on Student Progress  Management of Student Discipline, Instructional Strategies, Time/Materials  Professional Communication  Professional Development  Compliance with Policies, Operating Procedures and Requirements   Improvement of All Students' Academic Performance  ERG Provides Specific Measures of “Output” Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending

PDAS Comparison Mrs. Iron’s Domain III Ms. Adam’s Domain III 33 Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 33

Mrs. Irons and the Gifted Class Ms. Adams and the At-Risk Class Ms. Adams/Mrs. Irons Mrs. Irons and the Gifted Class Ms. Adams and the At-Risk Class Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 34

The New Principal has Determined Several Action Steps Based on the Information from ERG She will be providing a copy of the campus matrix to each teacher with all names anonymous but the individual’s. This will give the teachers an opportunity to see their students’ progress relative to the rest of the campus. She will set the stage for implementing some of the best practices that Mr. Atkins and Ms. Adams are using with such success. She will let the faculty know that this is the first step. They will be using their ERG data to “drill down” to the student expectation level each child is at, then using new benchmarking to monitor progress. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending 35

Teacher Performance - Growth At the overall test level, teachers for this subject and grade perform at a similar level. Students are achieving one year of growth relative to the district. Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Teacher Performance by TAKS Objective Note that each teacher has different strengths and weaknesses in TAKS objectives. What does this mean for PD? What impact does this have on tutoring and scheduling? Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Student Growth – By TAKS Objective Students Variability increases as you move from the district through the campus and classroom to the student. Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Campus Performance Measures What qualities are desirable in a campus? Students achieve a year of growth All teachers have all students achieve a year of growth All teacher improve from year to year How would we define measureable qualities of a high performing campus? Copyright © 2005-2012 Education Resource Group, Inc. All Rights Reserved. Patent Pending

Final Thoughts!! ERG is a tool NOT another program. It tells the story of what is really going on – all the way down to the classroom level. Many tools reveal performance data, ERG reveals dynamic movement of students’ learning. Which teachers “grow” students the most, i.e. which resources are being deployed successfully for the most benefit. ERG helps to set campus goals that are based on a total picture. How a campus achieves those goals is up to the principals’ leadership. Copyright © 2005-2011 Education Resource Group, Inc. All Rights Reserved. Patent Pending