1 Getting Up to Speed on Value-Added - An Accountability Perspective Presentation by the Ohio Department of Education

2 Agenda 1)Value-Added and the Accountability System 2)2007 – Overall Value-Added Results 3)Analysis of Individual Districts/Schools 4)AYP Growth Model 5)Data Availability

3 Value-Added and the Accountability System

4 Former Accountability System Architecture Indicators Met (District #s in parentheses) Performance Index Score Change in PI Score AYP Status Excellent 94% - 100% (29 or 30) oror 100 to 120 andand Met/Not Met Effective 75% % (23 to 28) oror 90 to 99.9 andand Met/Not Met Continuous Improvement 0% % (0 to 22) oror 0 to 89.9 PI Score Improvement Criteria Met andand Met 50% % (15 to 22) oror 80 to 89.9 oror andand Not Met Academic Watch 31% % (10 to 14) oror 70 to 79.9 oror andand Not Met Academic Emergency 0% % (9 or fewer) andand 0 to 69.9 andand Not Met

5 Up to 2007, Performance Index Growth has been used Starting in 2008, Value-Added data will be used for districts and schools with tested grades 4-8 Change in Improvement Measure: Implementing Value-Added

6 Ohio’s Value-Added Measure Measures the contribution of a school or district to the progress of its students on test scores Requires a Scaled Score metric –Current Alternate Assessments are not measured on a Scaled Score

7 Ohio’s Value-Added Measure Scores are measured in “Normal Curve Equivalent” gains “0” gain represents the typical or “expected” gain “Value-Added” is based on the 2006 – 07 distribution of scores Scores also use a measure of precision (1 “Standard Error”) to help describe the Value-Added classification

8 There are three classification “bands” using “gain score” and 1 Standard Error + Above expected growth (Green) Met expected growth (“one year of growth in one year of time”) (Yellow) - Below expected growth (Red) Ohio’s Value-Added Measure

9 Scores calculated for:  Grades (4-8)  Subjects (Reading and Math only)  Grade and subject composites  School composite  District composite Only District and School composite scores will be used for ratings Ohio’s Value-Added Measure

Meet ( ) Exceed (+) Meet ( ) Not Meet (-) Gain Score, Standard Error and Classifications Relative to Value-Added Standard Exceed (+)

11 Value-Added Will Affect Ratings first year VA can change LRC designation Reward / sanction –Reward enough growth –Penalize insufficient growth

12 Impact on Designation Above expected gain is rewarded Each rating category is rewarded based on above expected gain, including an “Excellent with Distinction” rating At least two years of above expected gain

13 Below expected gain results in lower ratings Each rating category is impacted by below expected gain Rating is lowered if you have three years of below expected gain Academic Emergency will not be lowered Impact on Designation

New Accountability System Architecture Including VA Impact Indicators Met Performanc e Index Score AYP Status Preliminary Designatio n 94% - 100% oror 100 to 120 andand Met or Not Met Excellent 75% % oror 90 to 99.9 andand Met or Not Met Effective 0% % oror 0 to 89.9 andand Met Continuous Improvemen t 50% % oror 80 to 89.9 andand Not Met 31% % oror 70 to 79.9 andand Not Met Academic Watch 0% % andand 0 to 69.9 andand Not Met Academic Emergency Did the Preliminary Designatio n increase or decrease based on the AYP Status? IF YES STOP HERE No additional change to the designatio n can occur based on the value added calculation IF NO CONTINU E Value- added MAY affect a designatio n when it has not been changed by the AYP Status Preliminary Designatio n Amount of growth using value-added calculation Final Designation Excellentandand Above expected growth for at least 2 consecutive years Excellent with Distinction Below expected growth for at least three consecutive years Effective Otherwise no effect on ratingExcellent Effectiveandand Above expected growth for at least 2 consecutive years Excellent Below expected growth for at least three consecutive years Continuous Improvement Otherwise no effect on ratingEffective Continuous Improvement andand Above expected growth for at least 2 consecutive years Effective Below expected growth for at least three consecutive years Academic Watch Otherwise no effect on ratingContinuous Improvement Academic Watch andand Above expected growth for at least 2 consecutive years Continuous Improvement Below expected growth for at least three consecutive years Academic Emergency Otherwise no effect on ratingAcademic Watch Academic Emergency andand Above expected growth for at least 2 consecutive years Academic Watch Otherwise no effect on ratingAcademic Emergency

15 Impact on Designation - Examples District A: PI = 85 (Effective Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating ?

16 Impact on Designation - Examples District A: PI = 85 (Effective Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating - Excellent

17 Impact on Designation - Examples District B: PI = 77 (Academic Watch Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating - ?

18 Impact on Designation - Examples District B: PI = 77 (Academic Watch Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating - Continuous Improvement (Why?)

19 Proposed Report Card Graphic Grade Reading Mathematics Subject Composite Total Composite* + + Grade Composite *Used in LRC rating

– Overall Value-Added Results

21 Value-Added 2007 Data Scores based on four years of data: Grades 4-8 results (Reading, Math, Writing, Science and Social Studies) Grades 3-7 results (Reading and Math in addition to Grade 4 Writing) Grade 3 results (Reading and Math); Grades 4-5 results (Reading only) Grade 3 results (Reading only)

22 Achievement Test Results: NCE Means based on 2007 results Math Grade

23 Relationship of Gains to Grade Level

24 Achievement Test Results: NCE Means based on 2007 results Reading Grade

25 Relationship of Gains to Grade Level

26 Value-Added 2007 Composite Results

27 Relationship of LRC Designation to District Typology *Note: Urban 21 are a subset of All Urban category

28 Relationship of Composite VA Gains to District Typology *Note: Urban 21 are a subset of All Urban category

29 Relationship of VA Gains to Achievement

30 Relationship of VA Gains to Math Achievement

31 Relationship of VA Gains to Reading Achievement

32 Performance (Status) by % Poverty

33 Performance (VA Gain) by % Poverty

34 Performance (Status) by % Minority

35 Performance (VA Gain) by % Minority

36 Analysis of Individual Districts/Schools

37 Value-Added Gains and Performance Index High Performance – Low Value-Added

38 Value-Added Gains and Performance Index Average Performance – Low Value-Added

39 Value-Added Gains and Performance Index Very Low Performance and Value-Added

40 Value-Added Gains and Performance Index - Very Low Performance – “Green” VA But is it enough?

41 Value-Added Gains and Performance Index “Green”- But will it be enough to improve?

42 Value-Added Gains and Performance Index Low Performance – High Value-Added

43 Value-Added Gains and Performance Index High Performance – High Value-Added

44 Value-Added Gains and Performance Index High Performance – High Value-Added

45 Within District Variability School level VA Gains vs. Performance

46 AYP Growth Model

47 Changes in 2008 New Goals Making minimum N uniform Include a Growth Model criterion

48 New Goals Reading Reading Math Math Elementary Grade 371.2%77.0%60.6%68.5% Grade 468.3%74.6%67.1%73.7% Grade 568.3%74.6%49.6%59.7% Middle Grade 675.8%80.6%55.1%64.1% Grade 768.6%74.9%47.3%57.8% Grade 873.8%79.0%47.5%58.0% HighOGT (Grade 10)71.8%77.4%60.0%68.0%

49 Uniform Minimum N Size Minimum size for evaluation was: 30 for all groups except students with disabilities 45 for students with disabilities Minimum N size starting with : 30 for all groups Minimum N size change for report card data to meet federal requirement

50 Up to , districts and schools could meet AYP achievement in one of three ways: 1)By meeting or exceeding all AYP targets; 2)By meeting or exceeding AYP targets with a two- year average of previous and current year’s reported data; 3)Via the AYP safe harbor provision – district/school achieves a 10% reduction in the percentage of non-proficient students from the previous year and also meets graduation or attendance rate goal. Meeting Adequate Yearly Progress

51 Growth Model and AYP 4 th way of meeting AYP Individual student projections Projected out to first year of “next school” (3 years maximum) Includes “next school” effects Only counts “Full Academic Year” students in computation

52 Growth Model and AYP AYP% for subgroup = Numerator (count subgroup students only): Sum of 3 rd graders proficient or better + Sum of “on track” students (those projected to reach proficiency) + Sum of other proficient students with no growth computation.

53 Growth Model and AYP AYP% for subgroup = Denominator: Total number of full academic year students in combined tested grades (including 3 rd where applicable)

54 Meeting AYP – Status Only Percent of Students Proficient A D B C MEET AYP MISS AYP Students On-Track to Proficiency Students Not Moving To Proficiency MEET AYP MISS AYP Magnitude of Individual Student Gains

55 Meeting AYP – Growth Only Percent of Students Proficient A D B C MISS AYP Students On-Track to Proficiency Students Not Moving To Proficiency MEET AYP Magnitude of Individual Student Gains

56 Meeting AYP – Status and Growth Percent of Students Proficient A D B C MEET AYP MISS AYP Students On-Track to Proficiency Students Not Moving To Proficiency MEET AYP Magnitude of Individual Student Gains

57 AYP Growth vs. Value-Added Gains AYP Growth _______________ Student Help meet AYP OAT Projected gain Students “on-track” to proficiency Value-Added Gains _______________ School / District LRC rating OAT Composite mean gain Value schools add to students’ starting points Level Use Data Source Measure Purpose

58 Value-Added Data Availability

59 School and District Value-Added Data Publicly available ODE home page – Accountability Menu – Value Added and click on “Value-Added Data and Reports” Data Available through Power User Reports (iLRC) in “Ratings” folder Data limited to Gain Scores, Standard Errors, and “VA – N” - by subject and grades level and composites

60 Value-Added Data Availability – EVAAS Diagnostic Reports Hosted on SAS web site at: Login and password required – District superintendent has administrative authority

61 Value-Added Data Availability – EVAAS Diagnostic Reports Reports –Are interactive –Contain one year of data –Provide student information by SSID –List gain scores only for students who have contiguous grade scores in a subject - Includes only full academic year students (WKC)

62 In the Works: Value-Added Tools for Teachers and Parents Explaining the Value-Added measure in the Accountability System to educators Practical knowledge for teachers to talk to parents about value-added Disk format – available before school year

63 Getting Up to Speed on Value-Added - An Accountability Perspective Questions?