Interpreting Effectiveness Making Use of EVAAS Data So what is this EVAAS system I have been hearing about and how does it tell me if I am a good teacher? My principal says it will be part of my evaluation this year, but I don’t know anything about it. Could you help me?

What is EVAAS Education Value Added Assessment System Dr. William Sanders Well, EVAAS is actually the brainchild of Dr. William Sanders and his colleagues from the University of Tennessee. He had an idea that he could analyze the effects of learning in a single classroom by using complicated algorithms and mixed methods to create a statistical model. He eliminated factors that could contribute to learning, such as student background and school equipment and tried to provide a baseline for predicting student performance. In the last few years, there has been quite a bit of buzz about this new model of predicting growth in our schools. NCDPI has allotted $1 million in the annual budget to allow teachers to make use of EVAAS data. Legislation now requires every school in North Carolina to make use of EVAAS data or a comparable system to create school improvement goals. However, no one has given teachers much training on what these numbers mean or how we are being evaluated based on these numbers. I am going to provide you with an overview of how the system works and what you need to know about making the most use of this information.

Where do they get these numbers? 8th Grade Reading test score gives English I EOC predicted score AC=CS-(.92 * ATPA) AC= Academic Change CS=Current Score ATPA= Average of two previous assessment scores Becomes .82 when only one pre-test score is available English I is the 8th Grade EOG First we need to know where this model comes up with these numbers. Let’s use English I as our model. EVAAS uses previous test history to predict where students should perform based on the average performance of students with similar results across the state of North Carolina. In order to determine a ninth grader’s English I score, EVAAS uses the 8th grade reading test score. The computed c-scale score for each grade is correlated with the corresponding predicted value. For English I, that value is .79. The 8th grade reading score is converted into a predicted English I score.

What Do We Do with the Information? Low performing scores=119-137 Medium performing scores=138-156 High performing scores=157-176 EVAAS is slow about putting the information out to teachers in a timely manner. Teachers are waiting now for how they performed last year. But we all know teachers want to use the information now to make adjustments for the next year. In order to have access to the EVAAS system, one must be granted access to the system by the administrator to get access to the information. However, you can do some preliminary figuring yourself. Student growth is broken down into thirds (Low, medium, high). Low scores would range from 119-137. Medium scores from 138-156. High scores from 157-176. Each year, your accountability person at the central office should be sending your principal a file that has these projections already calculated.

What Do We Do with the Information? Student Predicted Score Anticipated Group Johnny 136 Low Suzie 155 Medium Beevis 139 Frankie 165 High Ashley 131 Let’s put those calculations to use. First, we divide our students up into thirds, depending on the expected score range. This is how the students will be grouped based on teacher effect.

What Do We Do with the Information? Student Predicted Score Anticipated Group Actual Score Academic Change Johnny 136 Low 140 +4 Suzie 155 Medium Beevis 139 Frankie 165 High 160 -5 Ashley 131 +8 Next, we look at academic change once we get student test scores back. How did this teacher do with the low group? In this case, the teacher was highly effective in assisting students in the low group to achieve greater academic growth. In the medium group, this teacher did not make a discernable difference in the growth expectancy. With the high group, the teacher actually had a negative growth rate in the students’ performance.

What do all these numbers mean? Yay! We just found a new teacher for the lower level kids! Administrators are very quick in making judgments based on the teacher effect data. A knee-jerk reaction would be to place the previous teacher with the lower level kids to get the most teacher effect. However, this is a practice that should not be used based solely on the information provided by one year of data analysis.

Factors to consider Were there excessive absences in the year by the teacher or the students? Were there any changes in the student’s life that could have influenced academic growth? Is this the normal performance of the teacher or are there changes in the teacher’s situation that may have influenced the results? Are there any new programs at the school being implemented that could have had an effect on the student? Administrators must look at several factors that could impact the data. For instance, what if the teacher had a student teacher for a semester. Surely that would change the results. Read statements

Can we trust this information? .8 Reliability Coefficient – Provided by Sanders NCDPI suggests that the English I scores have a standard error of estimation of .586 EOC Correlations between Predicted values and Residuals = -.006