VAM Primer

Agenda History Statute Files provided to Districts Model Performance and Results 2

History 3

Florida’s Value-Added Model Was Developed by Florida Educators The Student Growth Implementation Committee (SGIC) was originally composed of 27 members from across the state, selected from over 250 volunteers, including: Teachers (across various subjects and grade levels, including exceptional student education, and union) School-level administrators District-level administrators (assessment, HR, superintendent, school board) Postsecondary teacher educators Representative from the business community Parent representative The SGIC met regarding the FCAT model from March-June 2011 Meetings were webcast live. See all materials and videos/recordings of committee proceedings at http://www.fldoe.org/committees/sg.asp The SGIC’s recommended model for FCAT data was fully adopted by the Commissioner of Education in June 2011 as Florida’s FCAT Value-added Model with no additions, deletions, or changes

Florida’s Value-Added Model Was Developed by Florida Educators After exploring eight different types of value-added models, the SGIC recommended a model from the class of covariate adjustment models. This model begins by establishing expected growth for each student which is based on: Historical data each year The typical growth, by grade and subject, among students who have earned similar test scores the past two years, and share the other characteristics controlled for by the model To isolate the impact of the teacher on student learning growth, the model developed by the SGIC and approved by the Commissioner accounts for: Student Characteristics Classroom Characteristics School Characteristics 5

Changes Since VAM was Created Originally, the performance of students component of a teacher’s evaluation was required to comprise at least 50% of the evaluation. In 2015, HB 7069 reduced this requirement to 1/3. From 2011-12 to 2014-15, districts were required to use VAM data for teachers who received scores, but could decide for themselves how to do so During 2015-16, districts were required to use the measures and performance level standards adopted by the State Board under Rule 6A-5.0411, FAC for teachers who received VAM scores In 2017, HB 7069 amended s. 1012.34, FS to make use of VAM data optional

Statute 7

Overview Performance of students is required by law to be included in educator evaluations (s. 1012.34, FS) For about 1/3 of classroom teachers, a VAM score is produced Districts have locally approved evaluation systems that describe how VAM scores will be used in their teachers’ and administrators’ evaluations

Evaluation Procedures Section 1012.34, F.S., Personnel evaluation procedures and criteria Section 3 – Evaluation procedures and criteria; Must be based upon the performance of students assigned to the educator’s classrooms (teachers) or schools (administrators); Must be conducted at least annually; Must be based upon sound educational principals and contemporary research in effective educational practices; and 9

Evaluation Procedures (Continued) Section 1012.34, F.S., Personnel evaluation procedures and criteria Section 3 – Evaluation procedures and criteria; Must include Performance of students (at least 1/3) Instructional practice/leadership (at least 1/3) Other indicators of performance 10

Differentiation Section 1012.34(2)(e), F.S. specifically requires that evaluation systems: Differentiate among four levels of performance as follows: Highly Effective. Effective. Needs improvement or, for instructional personnel in the first 3 years of employment who need improvement, developing. Unsatisfactory. 11

VAM Models Currently, VAM models are generated annually for the following subjects and grades: English language arts (4th, 5th, 6th, 7th, 8th, 9th, 10th) Mathematics (4th, 5th, 6th, 7th, 8th) Algebra 1 (8th & 9th) 12

Covariates ELA/Mathematics Number of subject relevant courses Up to 2 prior test scores Disabilities English language learner status Gifted status Attendance Mobility Difference from modal age of peers in the same grade Class size Similarity of prior test scores among students in the class 13

Covariates (Continued) Algebra I covariates – same as ELA and Mathematics except 3 more are added: Average prior test score on most recent test among students in the class Percent of students in the class who are gifted Percent of students in the class who are younger or older than the majority of students in the class 14

3 year aggregate combined VAM score - ELA/Mathematics Primary VAM Measures 3 year aggregate combined VAM score - ELA/Mathematics Individual grade and subject VAM score - Algebra I Standard errors Percent meeting expectations 15

Value-Added Results and Scores The formula produces a value-added score for a teacher, which reflects the average amount of learning growth of the teacher’s students above or below the expected learning growth of similar students in the state, using the variables accounted for in the model. A score of “0” indicates that, on average, students performed no better or worse than expected based on the factors in the model A positive score indicates that students, on average, performed better than expected A negative score indicates that students, on average, performed worse than expected 16

Standard Error An estimate of a teacher’s impact on student learning (the score) contains some Variability. The standard error is a statistical term that describes that variability. Using the standard error to construct a confidence interval around a score (like the +/-3 points in an opinion poll) is a good statistical practice that can assist in increasing the accuracy of classification decisions. 17

Value-Added Performance-Level Standards 18

Files Provided to Districts 19

VAM Files Generally available by the end of the 1st week in August. Uploaded to sharefile for retrieval by district users authorized to view PII at both teacher and student level. Each district receives a package of 74 data files containing student, teacher, school, district, statewide and model-level results. There is also a “Read Me” file. The “FileContents” tab provides an overview of all of these files. Other tabs in the “Read Me” file contain layouts and descriptions of the variables contained in each of the major file types. 20

Primary VAM Files These include: Teacher_Aggregation_3yr Teacher_Alg_Grade9 Teacher_Alg_Grade8 School_Aggregation_3yr (1yr and 2yr scores may be appropriate for administrators who have not been at the school for all 3 years) 21

Teacher_Aggregation_3yr VAM File Primary fields of interest include: Agg_vam_combined_cattxt Agg_vam_combined_catscr Fewer_than_10 HOSS Flag_1314, Flag_1415, Flag_1516 22

Teacher_Alg VAM files Primary fields of interest include: Teacher_vam_estimate_catttxt Teacher_vam_estimate_catscr Fewer_than_10 HOSS 23

School_Aggregation VAM files Primary fields of interest include: Agg_vam_combined_cattxt Agg_vam_combined_catscr HOSS Flag_1314, Flag_1415, Flag_1516 24

After reviewing if you still have questions about how to use these files, Call (850) 245 - 0411 or email VAM@fldoe.org 25

Model Performance & Results 26

R-squared Measures 27

Historical Percent of Variance Explained Trends 28

Share of Students Scoring at or Above Expected Score 29

Share of Students Scoring at or Above Expected Score- English Language Arts 30

Share of Students Scoring at or Above Expected Score – Mathematics and Algebra 1 About 7 in 10,000 students has a predicted score above the test score ceiling. About 1 in 5,000 students has a predicted score above the ceiling and scored at the ceiling. 31

Average Growth (Scale Score Units) 32

Score Classification 33

Score Classification Two Year Comparison 34

Historical Impact Analysis Trends - ELA 35

Historical Impact Analysis Trends - Math 36

Historical Impact Analysis Trends – Algebra 1 (9th Grade) 37

VAM & Mean Prior Test Score 38

VAM & Economically Disadvantaged 39

VAM & % with Disabilities

VAM & % English Language Learners 41

VAM & % Non-White 42

VAM & % Gifted 43

VAM & % With Expected Scores Above HOSS 44

2015-16 Final Evaluation Rating Compared to VAM Score Classification 45

https://app1.fldoe.org/rules/default.aspx OR ARM@fldoe.org 46