1. TEACHER EFFECTIVENESS INITIATIVE VALUE-ADDED TRAINING Value-Added Research Center (VARC) October 2012

2. Minneapolis Milwaukee Racine Chicago Madison Tulsa Atlanta New York City Los Angeles Hillsborough County NORTH DAKOTA SOUTH DAKOTA MINNESOTA WISCONSIN ILLINOIS Districts and States Working with VARC Collier County NEW YORK

3. The Power of Two & A more complete picture of student learning AchievementValue-Added Compares students’ performance to a standard Does not factor in students’ background characteristics Measures students’ performance at a single point in time Critical to students’ post- secondary opportunities Measures students’ individual academic growth longitudinally Factors in students’ background characteristics outside of the school’s control Critical to ensuring students’ future academic success Measures the impact of teachers and schools on academic growth Adapted from materials created by Battelle for Kids

4. How does Value-Added fit into the Teacher Effectiveness Initiative? Overview

5. Where does VARC get data? VARC State Departments of Education (Student Test Scores, Demographics, Enrollment; Teacher Licensure and Assignments) Institutions of Higher Education (Graduate Information) Pilot Districts (student-teacher linkages, non- NCLB test scores)

6. Where does VARC usually send School-Level and Grade-Level output? District Schools IHE? If District Shares Teachers VARC

7. Where will VARC send Teacher-Level output? VARC Teacher IHE (Graduates Only) Principals? Districts?

8. How will this inform your IHE’s Program Improvement? Teacher Education Program Improvement Graduate Value-Added IHE-Level Aggregate Value-Added Surveys Observational Data

9. varc.wceruw.org/welcome The Oak Tree Analogy

10. The Oak Tree Analogy Review

11. This method is analogous to using an Achievement Model. Gardener A Gardener B 61 in. 72 in. Method 1: Measure the Height of the Trees Today (One Year After the Gardeners Began)  Using this method, Gardener B is the more effective gardener.

12. This is analogous to a Simple Growth Model, also called Gain. 61 in. 72 in. Gardener A Gardener B Oak A Age 4 (Today) Oak B Age 4 (Today) Oak A Age 3 (1 year ago) Oak B Age 3 (1 year ago) 47 in. 52 in. +14 in. +20 in. Method 2: Compare Starting Height to Ending Height  Oak B had more growth this year, so Gardener B is the more effective gardener.

13. +20 Average + 3 for Rainfall - 3 for Soil + 2 for Soil - 8 for Temp + 5 for Temp _________ +12 inches During the year _________ +22 inches During the year 59 in. 74 in. Gardener A Gardener B 47 in. 52 in. - 5 for Rainfall Controlling for Non-Gardener Factors  The predicted height for trees in Oak A’s conditions is 59 inches.  The predicted height for trees in Oak B’s conditions is 74 inches. Predicted Oak A Predicted Oak B

14. This is analogous to a Value-Added measure. Above Average Value-Added Below Average Value-Added Predicted Oak A Predicted Oak B Actual Oak A Actual Oak B 59 in. 74 in. Gardener A Gardener B 61 in. 72 in. +2 -2 Method 3: Compare the Predicted Height to the Actual Height  By accounting for last year’s height and environmental conditions of the trees during this year, we found the “value” each gardener “added” to the growth of the trees.

15. Sample Report Review

16. Page 1 Color Coding Explanation Table of Contents Reporting Period and Context

17. Page 2 School-Level Value-Added Estimates Grade-Level Value-Added Estimates

18. Page 2 Top School-Level Value-Added NUMBER OF STUDENTS (WEIGHTED) NUMBER OF STUDENTS (WEIGHTED) VALUE-ADDED ESTIMATES Past Academic Year 2010-2011Up-To-3-Year Average 12 3 4 5 12 3 4 5 READING MATH School-Level Value-Added 182.9 559.4 Subject Level of Analysis Value-Added Estimate Point Estimate (number in color-coded bubble) 95% Confidence Interval (black line) Number of students included in the analysis 2.5 2.4 1.6 1.7 Past Academic Year Up-To-3-Year Average

19. Page 2 Bottom Grade-Level Value-Added Grade 3 Grade 4 Grade 5 Grade 3 Grade 4 Grade 5 FAQ 1: Which school year is this? 2.1 1.9 3.3 4.3 2.62.1 0.7 1.1 1.6 1.8 3.84.1

20. Value-Added on the NDSA Grade 3 Summer Grade 4 Summer Grade 5 Summer Grade 6 Nov 3 rd Grade Value-Added 4 th Grade Value-Added 5 th Grade Value-Added  4 th grade example:  “Starting knowledge” is the November 2010 4 th grade test.  “Ending knowledge” is the November 2011 5 th grade test.  This aligns to growth in the 2010-2011 4 th grade school year.  Why don’t we have 8 th grade Value-Added in North Dakota?

21. Page 2 Bottom Grade-Level Value-Added FAQ 2: How do I interpret the “Up-To-3-Year Average”? Grade 3 Grade 4 Grade 5 Grade 3 Grade 4 Grade 5 2.1 1.9 3.3 4.3 2.62.1 0.7 1.1 1.6 1.8 3.84.1

22.  NOT Jimmy as he goes through three consecutive school years  3 rd grade to 4 th grade  4 th grade to 5 th grade  5 th grade to 6 th grade  3 rd grade team with  2008-2009 cohort (3 rd grade to 4 th grade)  2009-2010 cohort (3 rd grade to 4 th grade)  2010-2011 cohort (3 rd grade to 4 th grade)  Keep teacher mobility in mind Does not follow individual students for 3 years Represents the 3 rd grade teaching team over three cohorts of students What Does “Up-To-3-Year Average” Mean for the 3 rd Grade?

23. NUMBER OF STUDENTS (WEIGHTED) NUMBER OF STUDENTS (WEIGHTED) VALUE-ADDED ESTIMATES Past Academic Year 2010-2011Up-To-3-Year Average 12 3 4 5 12 3 4 5 READING Grade-Level Value-Added 20 60 20 60 Grade 3 Grade 4 Grade 5 What Does “Up-To-3-Year Average” Mean? 2010-2011 3 rd Graders  The “Past Academic Year” represents longitudinal growth over a single school year. 10-11 3 rd Gr. 09-10 3 rd Gr. 08-09 3 rd Gr. 2010-2011 4 th Graders 2010-2011 5 th Graders 10-11 4 th Gr. 10-11 5 th Gr. 09-10 4 th Gr. 09-10 5 th Gr. 08-09 4 th Gr. 08-09 5 th Gr.  The “Up-To-3-Year Average” represents average longitudinal growth of three different groups of students at each grade level.

24. NUMBER OF STUDENTS (WEIGHTED) NUMBER OF STUDENTS (WEIGHTED) VALUE-ADDED ESTIMATES Past Academic Year 2010-2011Up-To-3-Year Average 12 3 4 5 12 3 4 5 READING Grade-Level Value-Added What Does “Up-To-3-Year Average” Mean?  Which grade-level teaching team…  Was most effective in the 2010-2011 school year?  Was most effective over the past three school years?  Was more effective in 2010-2011 than in the past? 48.5 44.5 146.0 141.1 46.0147.8 Grade 3 Grade 4 Grade 5 3.4 4.1 4.4 3.5 0.9 2.8

25. Grade 3 Grade 4 Grade 5 Grade 3 Grade 4 Grade 5 2.1 1.9 3.3 4.3 2.62.1 0.7 1.1 1.6 1.8 3.84.1 FAQ 3: Does this show student growth to go from red to yellow to green over time? Page 2 Bottom Grade-Level Value-Added

26. Value-Added, Not Achievement Grade 3 61 READING Grade 4 63 Grade 5 60 3 Grade 3 61 MATH Grade 4 63 Grade 5 60 3  In your groups:  Describe this school’s math performance  Describe this school’s reading performance 3.8 3.9 4.8 3.0 1.1

27. Page 2 Bottom Grade-Level Value-Added Grade 3 Grade 4 Grade 5 Grade 3 Grade 4 Grade 5 2.1 1.9 3.3 4.3 2.62.1 0.7 1.1 1.6 1.8 3.84.1 FAQ 4: Why are there non-integer numbers of students?

28. Mobile Students  If a student is enrolled in more than one school between the November NDSA administration and the end of the school year, each school gets credit for a portion of the student’s growth. Grade 3 Nov NDSA School ASchool B 45% Attributed to B 55% Attributed to A End of School Year

29. Student Group Interpretation

30. Pages 3 & 4 School-Level Value-Added Estimates by Student Groups: Special Ed Low Income Gender LEP

31. Example Student Group Interpretation  At this school which student group is growing faster than their similar peers from across the state?  Does that mean the “Special Ed” group grew more scale score points on the test than “Not Special Ed” group? NUMBER OF STUDENTS (WEIGHTED) NUMBER OF STUDENTS (WEIGHTED) VALUE-ADDED ESTIMATES Past Academic Year 2010-2011Up-To-3-Year Average 12 3 4 5 12 3 4 5 READING By Special Ed 22.0 160.9 72.5 486.9 Special Ed Not Special Ed 3.9 2.3 3.6 2.2

32. Student Group Interpretation What Would You Do?  What do these results mean?  If this was your school, how could you use these results to monitor instructional improvement? NUMBER OF STUDENTS (WEIGHTED) NUMBER OF STUDENTS (WEIGHTED) VALUE-ADDED ESTIMATES Past Academic Year 2010-2011Up-To-3-Year Average 12 3 4 5 12 3 4 5 By LEP 52.0 130.9 153.0 406.9 LEP Not LEP 0.8 2.2 0.9 2.4 MATH

33. Page 5 School-Level Value-Added and Achievement Scatter Plot Interpretation

34. Page 5 Scatter Plots

35. How to Read the Scatter Plots 12354 0 20 40 60 80 100 Value-Added (2010-2011) Percent Prof/Adv (2010) These scatter plots are a way to represent Achievement and Value-Added together Achievement Value-Added

36. How to Read the Scatter Plots 12354 0 20 40 60 80 100 Value-Added (2010-2011) Percent Prof/Adv (2010) Schools in your district A A. Students know a lot and are growing faster than predicted B B. Students are behind, but are growing faster than predicted C C. Students know a lot, but are growing slower than predicted D D. Students are behind, and are growing slower than predicted E E. Students are about average in how much they know and how fast they are growing

37. Page 6 (or 6 & 7 for large grade span schools) Grade-Level Value-Added and Achievement

38. Page 6 Example (Grade 4)

39. Last Pages (Part 1) 1-5 Value-Added Scale Note Regarding Comparison of Student Groups Number of Students (Weighted)

40. Last Pages (Part 2) List of Control Variables Note Regarding the Bush Foundation List of Reasons for Result Suppression