© 2014, Florida Department of Education. All Rights Reserved Annual District Assessment Coordinator Meeting VAM Update

© 2014, Florida Department of Education. All Rights Reserved. 2 Topics HB 7069 Rule Timeline for Questions

© 2014, Florida Department of Education. All Rights Reserved. HB 7069

© 2014, Florida Department of Education. All Rights Reserved. 4 House Bill 7069 Changes to component proportions Validity study Annual report Rules

© 2014, Florida Department of Education. All Rights Reserved. Student Learning Growth Measurement Using Statewide Assessment Data for Use in School Personnel Evaluations 6A , FAC Student Learning Growth Measurement Using Statewide Assessment Data for Use in School Personnel Evaluations

© 2014, Florida Department of Education. All Rights Reserved. 6 Rule Adoption Timeline April 14, 2015: HB 7069 Became law, requiring rules be adopted by the State Board of Education by August 1, 2015 April 27, 2015: Notice of Rule Development posted May , 2015: Rule Development workshops and web conference conducted for public input on draft text May 15 – June 16, 2015: Review public comments and revise rule text based on feedback submitted June 17, 2015: Additional rule development workshop conducted for public input on revised draft text June 19, 2015 Rule submitted for consideration by State Board of Education July 23, 2015: State Board meeting for consideration of rule for adoption August 20, 2015: Rules filed with Department of State September 9, 2015 – Rule becomes effective

© 2014, Florida Department of Education. All Rights Reserved. 7 Sections of Proposed Rule 6A Purpose of the Rule 2.Definitions 3.Formulas for Measuring Student Learning Growth 4.Data Collected and Reported for Calculation of Student Learning Growth Formulas 5.Performance Level Standards

© 2014, Florida Department of Education. All Rights Reserved. 8 Section 1 - Purpose The purpose of this rule is to provide the formulas for measuring student learning growth, the statewide standards for determining each performance level for use in each school district instructional personnel and school administrator evaluation systems developed under section , Florida Statutes, and procedures associated with implementing the formulas and standards.

© 2014, Florida Department of Education. All Rights Reserved. 9 Section 2 - Definitions Confidence interval Courses associated Covariate Covariate adjustment model Expected score Observed score Staff information system Standard error Student information system. Value-added model

© 2014, Florida Department of Education. All Rights Reserved. 10 Section 3 - Formulas for Measuring Student Learning Growth Two subsections a)ELA/Mathematics b)Algebra I Describes the covariates used and directs the reader to forthcoming documentation drawn from technical reports and analysis specifications used to produce annual model results

© 2014, Florida Department of Education. All Rights Reserved. 11 Section 3 - Formulas for Measuring Student Learning Growth ELA/Mathematics Covariates a.Number of subject relevant courses b.Up to 2 prior test scores c.Disabilities d.English language learner status e.Gifted status f.Attendance g.Mobility h.Difference from modal age of peers in the same grade i.Class size j.Similarity of prior test scores among students in the class

© 2014, Florida Department of Education. All Rights Reserved. 12 Section 3 - Formulas for Measuring Student Learning Growth Algebra I covariates – same as ELA and Mathematics except 3 more are added: k.Average prior test score on most recent test among students in the class l.Percent of students in the class who are gifted m.Percent of students in the class who are younger or older than the majority of students in the grade

© 2014, Florida Department of Education. All Rights Reserved. 13 Section 3 - Formulas for Measuring Student Learning Growth For ELA and Mathematics : VAM score = Teacher effect + (0.5 x School Component) For Algebra I: VAM score = Teacher effect

© 2014, Florida Department of Education. All Rights Reserved. 14 Section 4 - Section 4 - Data Collected and Reported for Calculation of Student Learning Growth Formulas 1.3 year aggregate combined VAM score - ELA/Mathematics 2.Individual grade and subject VAM score - Algebra I 3.Standard errors

© 2014, Florida Department of Education. All Rights Reserved. 15 Exclusions Use of VAM becomes optional in the following situations: A teacher has one or more assessments included in their score where the expected score for the student exceeded the highest score it was possible to achieve on the assessment. Number of assessments included in the calculation is fewer than 10. Teacher is not in the classroom for more than 50% of the days in the course. The teacher teaches Advanced Academics

© 2014, Florida Department of Education. All Rights Reserved. 16 Value-Added Results and Scores The formula produces a value-added score for a teacher, which reflects the average amount of learning growth of the teacher’s students above or below the expected learning growth of similar students in the state, using the variables accounted for in the model. A score of “0” indicates that, on average, students performed no better or worse than expected based on the factors in the model A positive score indicates that students, on average, performed better than expected A negative score indicates that students, on average, performed worse than expected

© 2014, Florida Department of Education. All Rights Reserved. 17 Standard Error An estimate of a teacher’s impact on student learning (the score) contains some variability. The standard error is a statistical term that describes that variability. Using the standard error to construct a confidence interval around a score (like the +/-3 points in an opinion poll) is a good statistical practice that can assist in increasing the accuracy of classification decisions.

© 2014, Florida Department of Education. All Rights Reserved. 18 Section 5 - Performance-Level Standards

© 2014, Florida Department of Education. All Rights Reserved. 19 Section 5 - Performance Level Standards Use the standard error to construct confidence intervals. Use these confidence intervals to establish performance standards. Give teachers the benefit of the doubt when there is below-threshold statistical uncertainty. Place a larger share of teachers statewide into higher performance categories than student-level metrics using the same assessment results.

© 2014, Florida Department of Education. All Rights Reserved. Impact and Simulation Data

© 2014, Florida Department of Education. All Rights Reserved VAM Model Impact Data

© 2014, Florida Department of Education. All Rights Reserved. 22

© 2014, Florida Department of Education. All Rights Reserved. 23

© 2014, Florida Department of Education. All Rights Reserved. 24

© 2014, Florida Department of Education. All Rights Reserved. 25

© 2014, Florida Department of Education. All Rights Reserved Year Aggregate Combined VAM Score Averages by Performance Level

© 2014, Florida Department of Education. All Rights Reserved. 27 Comparison of Distribution Between 3 Year Aggregate Combined VAM Score and Student Achievement Levels

© 2014, Florida Department of Education. All Rights Reserved VAM Process Timeline

© 2014, Florida Department of Education. All Rights Reserved. 29 Timeline DateActivity September  Vendor selection. October  RVT UAT for migration to SSO. November  Fall Survey 2 RVT opens. December 31, 2015  All evaluations due via Survey 5 Staff Database submission for inclusion in annual legislative report. Mid-March  Spring Survey 3 RVT opens. April 15, 2015  Survey 2/3 match request list files from posted to sharefile for districts to update. April 29, 2015  All districts requesting any changes from the prior year to their Survey 2-3 match conditions for VAM calculation purposes must have updated the document in their sharefile directory to reflect changes. May 13, 2015  Fall Survey 2 and Spring Survey 3 Roster Verification Tools close.  Districts electing to use Survey 2 and 3 data instead of RVT for VAM purposes must have those data files finalized. Week of June 3, 2015  FDOE posts files to sharefile site with Survey 2-3 requests applied for district review.  Districts verify that Survey 2-3 requests were applied correctly.  The district accountability contact in every district replies to FDOE that files were compiled accurately. July 2015  Processes VAM calculations for FSA ELA,FSA Mathematics, and Algebra I. September, 2015  FLDOE posts VAM results for teachers to school districts via sharefile website.

© 2014, Florida Department of Education. All Rights Reserved.