1. Copyright ©2006. Battelle for Kids. Understanding & Using Value-Added Analysis

2. Learning Targets Develop confidence in accessing, navigating and interpreting value-added information. Understand the connections between value-added information and school improvement.

3. Assumptions Related to this PD Participants are DVAS, elementary and middle school administrators Participants know very little about value- added information Participants want to get better at their jobs.

4. Introductions Name Role in Building and/or District Draw a face on a post it to indicate your feelings about Value Added

5. Setting the Stage: Over the last 15 years we have seen: Proficiency Testing 4th, 6th and 9th grade accountability Standards Achievement Testing State and National Accountability Systems for grades 3-10.

6. Essential Questions Over the last 15 years how have external forces shaped what happens in schools? How have these changes affected the work of Teachers and School Administrators? What tools and training have been made available to do the work?

7. The Work of the Day Morning will be spent learning about value-added information: –Conceptual understanding of value-added –Overview of the information available in value- added reports Afternoon will be spent: –Navigating through value-added reports –Learning about people and tools that can support for your learning –Connecting value-added to school improvement

8. Copyright ©2006. Battelle for Kids. Getting Started Value-Added Analysis

9. Copyright ©2006. Battelle for Kids. Presentation Targets Gain an introductory understanding of value-added analysis. Acquire an awareness of information available in value-added reports. Explore how value-added information can inform school improvement. Understand the difference between progress and achievement.

10. Copyright ©2006. Battelle for Kids. Stair-Step Expectations Grade Three Grade Four Grade Five Grade 3 Test Grade 4 Test Grade 5 Test In a perfect world scores on year-end achievement tests would be enough to document growth. Students would start school at the same place, and move together toward proficiency. Copyright © 2003. Battelle for Kids

11. Copyright ©2006. Battelle for Kids. Differentiated Reality Grade Three Grade Four Grade Five Students begin school at different places and progress at different rates. This complicates accountability. We need more than scores on a single test to tell the story of a school’s effectiveness. Grade 3 Test Grade 4 Test Grade 5 Test Copyright © 2003. Battelle for Kids

12. Copyright ©2006. Battelle for Kids. Achievement measures alone provide a partial picture: School improvement perspective Birth Kindergarten 4th Grade Test Achievement measures are snapshots in time. They provide information on what a child has learned since birth. If we want to improve schools we need a metric that provides reliable feedback on the work that schools produce. 1st 2nd 3rd

13. Copyright ©2006. Battelle for Kids. A View of Student Progress: Accountability perspective We must expect progress for all students. Student A Student B Proficient

14. Copyright ©2006. Battelle for Kids. School L School F School H School B School A School K School D School G School E School P School I School J School O School C School M School N The Power of Two: Achievement & Progress How do value-added measures support what we know about schools?

15. Copyright ©2006. Battelle for Kids. The question for educators today Regardless of where students begin, how do we maximize the progress each makes in any given year? 84% of the American public believe student progress is the best measure of a school’s effectiveness ! Copyright © 2003. Battelle for Kids

16. Copyright ©2006. Battelle for Kids. What is the difference between progress and achievement?

17. Copyright ©2006. Battelle for Kids. Value-Added Analysis in Ohio Project SOAR Ohio’s Value-Added System

18. Copyright ©2006. Battelle for Kids. Operated by the Ohio Department of Education. Began as a grade 4 pilot in 2006 in all districts and community schools. Provides analysis in math & reading, grades 4-8. Uses only state achievement test data. Uses the mean gain model. Expected growth is set by the State Board of Education. Ohio’s System

19. Copyright ©2006. Battelle for Kids. Operated by Battelle for Kids. Began in 2002 with 42 school districts. Now includes 100+ districts. Provides analysis in all core subjects, grades 3-10. Uses state and non-state test data. Uses the mean predicted model. Expected growth is normative (Average Growth). Project SOAR (Schools’ Online Assessment Reports)

20. Copyright ©2006. Battelle for Kids. Why EVAAS ™ ? Applies the most sophisticated statistical methodology available to ensure reliability. Allows for the use of all student test data. Provides valuable diagnostic information. Accommodates different types of test data. Used statewide in Tennessee for more than 10 years.

21. Copyright ©2006. Battelle for Kids. Ohio’s Scale Up Plan R M Sc SS

22. Value-Added in the State Accountability System Copyright ©2006. Battelle for Kids. If a district or school demonstrates more than expected growth over two consecutive years it moves up one level in the system. If a district or school demonstrates less than expected growth over three consecutive years it moves down one level in the system. Value-added measures are significant at one standard error.

23. Copyright ©2006. Battelle for Kids. How is value-added used in the state accountability system?

24. Copyright ©2006. Battelle for Kids. How is student assessment data used to produce a value-added growth metric? Value-Added Analysis* Conceptual Overview

25. Copyright ©2006. Battelle for Kids. What is a Growth Measure? Growth = Observed Score – Baseline Score

26. Copyright ©2006. Battelle for Kids. Value-Added Example 4th5th6th7th8th A value-added metric measures the average growth that is produced in one subject in a given year. This growth is then compared to the amount of growth that would be expected for that group of students. Growth above (or below) expectation is the “value-added.”

27. Copyright ©2006. Battelle for Kids. Tests must: Be highly correlated to curricular objectives. Have sufficient stretch. Have appropriate levels of reliability. Test Properties

28. Copyright ©2006. Battelle for Kids. School Effect ≈ Mean Gain – Expected Growth Used when all tests across grade levels have consistent scales. Mean Gain Approach

29. Copyright ©2006. Battelle for Kids. Mean Gain Example A crude measure of the growth for this group is 10 NCEs. Growth = Mean Observed – Mean Baseline Growth = 35.2 - 25.2 = 10.0 Math scaled scores for the same students. Year 1 Grade 3 Student 1 377 = 26 (NCE) Student 2 380 = 27 Student 3 360 = 19 Student 4 377 = 26 Student 5 381 = 28 Mean Baseline = 25.2 Year 2 Grade 4 Student 1 400 = 37 (NCE) Student 2 385 = 30 Student 3 395 = 35 Student 4 405 = 41 Student 5 390 = 33 Mean Observed = 35.2

30. Copyright ©2006. Battelle for Kids. Mean Gain Approach (Converted to NCEs) High Progress Positive School Effects High Achievement High Mean NCE or Mean Scaled Score School Effect Observed Baseline Sample School Value-Added Report Math

31. Copyright ©2006. Battelle for Kids. Mean Gain Approach (Converted to NCEs)

32. Copyright ©2006. Battelle for Kids. Describe, in conceptual terms, the “mean gain” value-added model.

33. What New Diagnostic Information Do Value-Added Reports Provide?

34. Copyright ©2006. Battelle for Kids. Value-Added Summary Report Where are the areas of strength and weakness?

35. Copyright ©2006. Battelle for Kids. Are the curriculum and instruction working for all students? School Diagnostic Report

36. Copyright ©2006. Battelle for Kids. What pattern would represent optimum growth? Performance Diagnostic Report

37. Copyright ©2006. Battelle for Kids. How does a student’s growth rate compare to other groups of students? Student Report

38. Copyright ©2006. Battelle for Kids. With average growth, where will this student likely score in the future? Student Projection

39. Student Search Copyright ©2006. Battelle for Kids. How can Predictions help identify at risk students?

40. Copyright ©2006. Battelle for Kids. How could this information change the way educators approach intervention? Student Search Results

41. Copyright ©2006. Battelle for Kids. In Summary, Value-Added Information Shows… The students’ progress made in each subject area and grade level. The progress students made who enter the year at different achievement levels. How students’ progress in one curricular area or program compares to their progress in another. Whether individual students are making adequate progress toward state standards. Without data, all we have are opinions!

42. Copyright ©2006. Battelle for Kids. What new information do value-added reports provide?

43. Copyright ©2006. Battelle for Kids. Connections with School Improvement Efforts How can value-added information be used in an annual school improvement cycle?

44. Copyright ©2006. Battelle for Kids. Comparison of Effect Sizes

45. Copyright ©2006. Battelle for Kids. Beginning-of-the-Year Planning Grade Level/Department Teams Examine value-added and other school performance information. Assess strengths and weaknesses and their potential causes. Celebrate strengths. Set goal(s) that build on strengths and address weaknesses. Create action plans, timelines and responsibilities.

46. Copyright ©2006. Battelle for Kids. Using Value-Added Information in an Annual School Improvement Cycle Beginning-of-the-Year Planning Teams meet to work on goals Individual teachers working on the work Teams meet to work on goals

47. Copyright ©2006. Battelle for Kids.

48. Why is it important to focus school improvement strategies at the individual grade level and/or subject area?

49. For more information, contact: www. BattelleforKids.org (866) KIDS-555