1. Student Growth Measurements and Accountability
Virginia Association of School
Superintendents Conference
October 23, 2017

2. Our Goals for this session
Provide an overview of growth measures examined and adopted for Virginia.
Consider a conceptual model for thinking about measuring growth.
Review recent Innovation Committee recommendations and actions related to measuring growth.

3. Why Measure Growth?
Considering both achievement and growth may provide a more complete picture of student learning.
Achievement – Did students pass the statewide tests?
Growth – Did students demonstrate progress based on where they started the school year?

4. Growth Measures Reviewed
Student Growth Percentiles
Progress Tables
Predictive Models (2017 Growth Pilot – SAS and Education Analytics)
Vertical Scale

5. Student Growth Percentiles
In 2011 VDOE examined Student Growth Percentiles and their use as a measure of academic growth for its reading and mathematics tests.
A student growth percentile ranges from 1-99 and compares a student’s current SOL score with the scores of students throughout the state with similar score histories.

6. Progress Tables

7. Progress Tables
In the Progress Tables student growth is determined by comparing the student’s test score in the current year to his/her prior test score.
To facilitate the measurement of student growth, each of the performance levels used for the Standards of Learning tests in reading and mathematics, Below Basic, Basic, Proficient, and Advanced, is divided in half to create two sublevels.
The resulting sublevels are: Low Below Basic, High Below Basic, Low Basic, High Basic, Low Proficient, High Proficient, Low Advanced, and High Advanced.

8. Performance Sub-Levels for Reading
Virginia Reading SOL Sub-Level Cut Scores
Grade
Below Basic
Basic
Proficient
Advanced
Low
High 3
0-280 4
0-277 5
0-270 6
0-291 7 8
0-293
The interpolated cuts for sub-performance levels were calculated based on the standard setting forms from the most recent standard setting year.
Each performance category has a cut sub-performance level cut score that divides Low and High

9. Performance Sublevels for Mathematics
Virginia Mathematics SOL Cut Scores
Test Level
Fail/Below Basic
Fail/Basic
Proficient
Advanced
Low
High 3
0-305 4
0-310 5
0-312 6
0-331 7
0-303 8
0-323
This table lists the specific performance sublevel cuts for each performance level in grades 3 through 8 mathematics and Algebra I.
Test
Level
Fail
Proficient
Advanced
Low
High
Algebra I
0-375

10. Determination of Student Growth Using Progress Tables
Student progress is measured by whether a student who failed the reading and/or mathematics SOL test the previous year has moved at least one sublevel based on the current year’s data.
For example, a student whose score on the grade 3 reading test in was “high below basic” has moved one sublevel if his score on the grade 4 reading test in was “low basic.”

11. Progress Table Examples

12. Use of Progress Tables in Combined Rate for Accountability Calculations for Reading and Mathematics
Integrates achievement, growth, and progress for EL students towards gaining proficiency in reading.
A student will be counted in the numerator of the reading or mathematics combined rate if:
The student passes the assessment*; or
The student does not pass the assessment but demonstrates growth using the progress tables; or
For the reading assessment only, the student does not pass the assessment or demonstrate growth, but is an EL and demonstrates progress as measured by the ACCESS for ELLs 2.0 assessment.
*Includes recovery
Reading combined rate
8 / 10 = 80%
Pass rate on state assessments:
Student growth:
English Learner progress:
Example of Reading
Combined Rate

13. 2017 Growth Model Pilots

14. Growth Pilot
The 2016 General Assembly directed the Department of Education to conduct a pilot to investigate a student growth model that could be applied for school and division accountability.
Contracts were awarded to:

15. Growth Pilot
The proposed growth measures considered students’ prior achievement on SOL tests and the performance of similar students
Statistical predictions were developed for how each student would score on the current year’s test (predicted score) based on the student’s performance on the previous year’s test.
A student’s growth was then determined by whether the student fell below, met, or exceeded the predicted score and by how much.

16. An Example…
After the statewide assessments are administered, the data are used to calculate a statistical prediction for each student based on his or her characteristics and similar students across Virginia.

17. An Example…
Students’ actual scores and predicted scores are compared to determine whether each student met or exceeded the prediction and by how much.
Student 1:
Student 2:

18. Proposed Growth Measures Using Predictive Models
Growth results for a school could be determined by considering whether, on average, the school’s students tended to exceed or not meet their predictions and by how much.

19. Potential Use of a Vertical Scale

20. Current SOL Scale
Scores on each SOL test are currently reported on a scale that is specific to that test.
0 -600
0 -600
0 -600
0 -600
Grade 8
0 -600
Grade 7
0 -600
Grade 6
Grade 5
Grade 4
Grade 3

21. Vertical Scale
A vertical scale would allow for the reporting of scores across SOL tests.
For example, a vertical scale for reading could allow for the reporting of scores for the grade 3 reading test through the grade 8 reading test.

22. Sample Vertical Scale 8,000 2,000 Grade 8 Grade 7 Grade 6 Grade 5
Grade 3

23. Vertical Scaling Study
Creating a vertical scale requires that we “link” the tests together.
In spring 2016 a study was conducted to “link” the content of the reading tests for grades 3-8 and the mathematics tests for grade 3 – Algebra I together so that a vertical scale could be created.
Results are promising but must be re- evaluated based on the change in the mathematics SOL.

24. Potential Growth Measures with Vertical Scale and CAT
Across-year growth: growth from test taken the previous year to test taken in the current year.
Within-year growth: could allow for students to test in the fall and again in the spring to measure growth within the year.

25. Use of Computer Adaptive Testing to Support the Measurement of Growth
Customized selection of items for students supports more precise measurement of student achievement and growth.
Combined with the vertical scale, CAT supports off-grade level testing.

26. ESSA and Off-Grade Level Testing
Under ESSA we must provide information regarding a student’s proficiency in reading and mathematics in grades 3-8.
However, unlike NCLB, ESSA also allows for “off-grade level” testing if a computer adaptive model is used.

27. Excerpt from ESSA
(J) ADAPTIVE ASSESSMENTS.— (i) IN GENERAL.—Subject to clause (ii), a State retains the right to develop and administer computer adaptive assessments as the assessments described in this paragraph, provided the computer adaptive assessments meet the requirements of this paragraph, except that—
subparagraph (B)(i) shall not be interpreted to require that all students taking the computer adaptive assessment be administered the same assessment items; and
such assessment—
(aa) shall measure, at a minimum, each student’s academic proficiency based on the challenging State academic standards for the student’s grade level and growth toward such standards; and
(bb) may measure the student’s level of academic proficiency and growth using items above or below the student’s grade level, including for use as part of a State’s accountability system under subsection (c).

28. Timeline for Implementation of Vertical Scales
Mathematics
Additional linking studies will be needed in spring 2018 because of content changes in mathematics SOL.
Reading
Once curriculum framework is approved this fall, content changes in SOL will be evaluated so that a timeline can be determined.

29. Vertical Scale Characteristics
Growth is based on comparing a student’s performance to his/her own previous performance
Expectations for growth can be defined so that the teacher and student know what is needed at the beginning of the year
More sensitive measure than progress tables
Understandable and transparent
Supports off-grade level testing to measure growth

30. Use of Computer Adaptive Testing to Support the Measurement of Growth
Without CAT, the number of testing opportunities for students is limited by number of test forms available
With CAT, computer algorithm creates tests so number of test forms not applicable
Customized selection of items for students supports more precise measurement of student achievement and growth
Supports off-grade level testing
May also be used for “pre-test”/”post-test” model to allow for the measurement of student growth within the same year

31. School Accountability
Different Measures, Different Purposes
School Accountability
Student Learning

32. Measuring Growth: Principles and Key Question
Balanced Assessment System: The assessment system should include traditional achievement tests, growth measures and performance assessments across students’ school careers.
What’s Possible and What’s Desirable: The system should include growth measures that are currently feasible, and should adapt as more direct measures become available.
Can currently available SOL tests be used to measure growth in 3-8 reading and math?

33. IC Subcommittee’s First Recommendation on Growth
Recommend the use of linked SOL item banks to measure growth in grades 3-8 reading and mathematics, with assumptions.
This recommendation passed the full Committee.

34. IC Subcommittee’s Second Recommendation on Growth
Recommend ongoing research leading to implementation of more direct measures of student growth.
This recommendation failed to pass the full Committee.

35. Questions?