Value-Added: Then, Now & in the Future Ohio RttT Webinar Presented by Battelle for Kids June 7, 2011 Race to the Top

THEN…in the beginning Race to the Top ODE: Uses SAS® EVAAS® for Value-Added BFK: Uses SAS® EVAAS® for Value-Added School Improvement Collaborative (SOAR) Began in 2002 with 42 districts (LEAs) Currently 100+ districts (LEAs) get expanded value-added reporting Uses multiple test data (OAA & norm- referenced tests) from grades 2-8 for value-added reports in grades 3-8 Uses all students with three prior data points Employs URM (univariate response model) statistical approach to determine student growth Local Report Card Component (LRC) Statewide accountability metric Uses OAA test data from grades 3-8 for value-added reports in grades 4-8 Originally, only the Where Kids Count students were used in the analysis Employs MRM (multivariate response model) statistical approach to determine student gains to a standard

Why Two Systems in Ohio? Race to the Top  The MRM approach:  Is the preferred model when there is consistent, consecutive testing from year to year  Measures growth to a predetermined standard  Allows gains to be averaged over time  Uses new data to make prior estimates more precise  The URM approach:  Accommodates multiple kinds of tests and differing testing regimens; therefore, it is re-normed each year  Provides a comparative measure of growth based on the pool

The System Evolved... Race to the Top BFK SOAR Project (100+ districts/LEAs) Currently use TerraNova™ to augment state tests BFK SOAR Project (100+ districts/LEAs) Currently use TerraNova™ to augment state tests 2007: BFK T-CAP project launch, 50+ SOAR districts (LEAs) received teacher-level value-added reporting 2007: BFK T-CAP project launch, 50+ SOAR districts (LEAs) received teacher-level value-added reporting ODE: SAS® EVAAS® reporting BFK Ohio Value- Added High Schools project launch, 30+ schools Use ACT End-of- Course Assessments BFK Ohio Value- Added High Schools project launch, 30+ schools Use ACT End-of- Course Assessments

NOW…a unified system and expansion of teacher-level reports statewide Race to the Top  One Value-Added (VA) System for Ohio:  Ohio VA EVAAS ® reporting  Transparency for LEA accountability on report cards  School improvement  Expanded reporting  LEAs in BFK projects receive additional reports (no duplication of grades/subjects that ODE provides)  Teacher-level reporting statewide

Our Unified Approach Race to the Top ODE: Teacher-Level Value- Added Reporting Grades 4-8 reading and math ODE: Teacher-Level Value- Added Reporting Grades 4-8 reading and math ODE: SAS® EVAAS® Value- Added Reporting Grades 4-8 reading & math Grades 5 & 8 science BFK SOAR & High School: SAS® EVAAS® Value- Added Reporting Grade 3, reading & math Grades 3,4,6,7 science Grades 3-8, social studies High school: 9 core subjects BFK Extended Teacher-Level Reporting Grade 3, reading & math Grades 3,4,6,7 science Grades 3-8, social studies High school: 9 core subjects BFK Extended Teacher-Level Reporting Grade 3, reading & math Grades 3,4,6,7 science Grades 3-8, social studies High school: 9 core subjects Ohio VA Reporting

Ohio’s Value-Added System Accountability Measures and Reports Technical Assistance and Support SAS® Data Processing Maintained by SAS®  Single Limited Access Password Protected Data:  District/LEA and school  Student information  Analytic tools  Teacher-level reports  Limited Use Public Access  Includes BFK SAS® EVAAS® reporting  Enhanced reporting features Developed & supported by BFK  Regional System  Trained VAL’s support districts/LEAs through DVALT training  Support to teacher- teams  Focus on school improvement  Toolkits  Online courses Maintained by ODE  LRC  Valued-Added  AYP Growth Measure  ODE Reports-School and District (LEA) Measures Diagnostic Tools EVAAS® Diagnostic Tools EVAAS® ODE- BFK Partnership

Report Enhancements for 2011 Race to the Top  Inclusion of students' historical data across LEAs  Previously not included in analysis due to migration across LEAs  Will increase school improvement utility for LEAs with high mobility  Tested & attributed students in ODE reports  Previously, only Where Kids Count (WKC) students that were used for accountability purposes were available in EVAAS®  Report views of both the WKC students and all tested students with sufficient data will now be available through EVAAS®.

Report Enhancements for 2011 (cont’d) Race to the Top  Continuation of student names for EVAAS® reports  In July, your district and ITC will be able to submit student names for the SSID crosswalk file  An additional benefit of getting this done each year is that teachers are able to view reports with re-rostered student information.  Watch for instructions next month.

Report Enhancements for 2011 (cont’d) Race to the Top  Science reports in grades 5 & 8  Will now include school-level science results for grades 5 & 8 statewide  Merged EVAAS® site – SOAR & ODE  All users will have only one EVAAS® account  All LEAs will receive the state's value-added results for grades 4-8, math and reading and grades 5 & 8 science  SOAR districts/LEAs will receive expanded value-added results through the same EVAAS® site  Includes high school level if involved in the Ohio Value- Added High Schools project  All teacher-level reports will be interactive and available through the EVAAS® site

Recap of Fall 2010 Enhancements Race to the Top In :  Names added to diagnostic reports to make these reports more useful for school improvement  Technical Advances  Reset the baseline to 2010 results  Stabilization of the state system  Projections for 1 & 2 years and pie charts  Integration into Ohio Improvement Process (OIP)  Data access for regional support

Accountability Rule Changes Race to the Top  Increase the standard for determining “a year's growth in a year's time" from one standard error of measure to two; and  Reduce the time period for value-added impact on the report card designation from the current “2 years plus (green) to go up, and 3 years minus (red) to go down” to “1 year plus to go up, and 2 years minus to go down.”

Teacher-Level Value-Added Reporting Race to the Top  Rollout Schedule  30% of LEAs Link in Year 1 RttT (reports received fall 2011)  60% of all RttT LEAs in Year 2  100% of all LEAs in Ohio in Years 3 & 4  Requirements—Accuracy of Reporting  Must conduct linkage  Minimum number of students and time enrolled  Access to Reporting  Online via EVAAS® accounts  Password protected  Grades/Subjects Available  ODE: grades 4-8, math & reading  BFK: grade 3, math & reading; grades 3-8, science & social studies; high school—algebra I & II, geometry, pre-calculus, biology, chemistry, English 9, 10 & 11

Value-Added & Teacher Evaluation Race to the Top  Race to the Top  House Bill 153  Teacher Incentive Fund  Measuring Effective Teaching (MET)  Bill & Melinda Gates Foundation Study

Value-Added & Race to the Top Race to the Top  Teacher-Level Reporting  Mini Grants for Extended Value-Added Reporting (fall 2011)  Support Resources  Value-Added Toolkits—will be updated (fall 2011)  Problem-based approach  Key reports organized by audience  Includes URM examples  Refresh of Value-Added On-line Courses (occurred June 1)  Greater interactivity  Condensed and practical  Face-to Face Regional Support  Network support: regional Value-Added Leaders & District Value- Added Leadership teams  Focus on both teacher-level reporting & value-added uses in school improvement  Webinar Series

2011 Budget Bill Discussions Race to the Top Requires school districts/LEAs to establish new policies relative to teacher evaluation that include these requirements:  At least 50% of each evaluation is to be based on measures of student academic growth  Evidence-based and use multiple measures of a teacher's use of knowledge and skills and of students' academic progress  Alignment with the standards for teachers  Statements of expectations for professional performance  Observation of the teacher on at least two occasions for not less than thirty minutes on each occasion  Each teacher be provided with a written report of the results of the teacher's evaluation and recommendations

Race to the Top  When applicable to a teacher, those measures shall include student performance on the assessments and the value-added progress metric  For teachers of grade levels and subjects for which those measures are not applicable, student assessments that measure mastery of the course content for the appropriate grade level, which may include nationally normed standardized assessments, industry certification examinations, end-of-course examinations developed or selected by the board, or assessments may be used  Shall include student growth data for students assigned to the teacher during the three most recent school years  If less than three years of growth data are available, the board shall use the growth data for all of the school years that are available and the student academic growth portion of the teacher evaluation may be reduced to 40% 2011 Budget Bill Discussions (cont’d)

Teacher Incentive Fund (TIF) Race to the Top  Value-added use in 24 Ohio TIF districts/LEAs as one part of multiple measures within award designs (in grades/subjects available)  Participating districts/LEAs will:  Define what it means to be an effective educator  Utilize quality, reliable data to inform instructional practices  Enhance the evaluation systems for principals and teachers to improve student achievement  Design award programs that recognize educators for their contributions to student achievement and leadership in the district/LEA  Make data-driven decisions about professional development opportunities

Multiple Measures Race to the Top  MET resources (  White paper: Working with Teachers to Develop Fair and Reliable Measures of Effective Teaching  Teachers Talk About Multiple Measures Video  MET Project Preliminary Findings Research Report MET Project Preliminary Findings Research Report  Home Runs, RBIs and Batting Averages: How Today's Educators Measure Up

In conclusion, where is it all going? Race to the Top  Expanded use of value-added analysis for:  School Improvement  Program Effectiveness  Teacher Reflection  Educator Evaluation

Additional Webinars Race to the Top June 3 p.m. Value-Added: Then, Now and in the Future June 3 p.m. Link Before You Leap June 2:30 p.m. Building Capacity Using Value-Added in School Improvement June 2:30 p.m. Implications of Teacher-Level Value-Added Data

Additional Resources Race to the Top Contacts: Ohio Department of Education Matt Cohen, Battelle for Kids Mary Peters, Help Desk, or (866)