Custom Reports: SCGs and VCGs

Standard Comparison Group (SCG)

What is an SCG?  Custom Report  Evaluates your students’ observed growth against normative group  Same grade & initial score  Same number of instructional weeks  Allows you to compare growth patterns across different groups of students 3

Regular reports give you high level view of growth data SCGs break growth data down for deeper analysis Pivot tables allow you to sort data by – Gender – Ethnicity – School – Subject Digging Deeper into Growth Data 4

Six Pivot Reports: – Growth by Achievement – Growth by School – Growth by Subgroup (ethnicity) – Growth Figure (bar graph) – Testing Conditions – Testing Conditions & Growth SCG Reports Reports are best used in relation to a research question that needs to be answered with data.

Using Questions to Guide Data Collection Dylan Wiliam calls it: “Question-driven data collection.” Not…. “Data-driven decision-making.”

Report Tabs

Tab: Growth by Achievement Question: How did our kids grow in various percentile ranges of achievement compared to other kids with similar scores and similar amounts of instructional time? – Or, “Are our high-achieving students growing as much as our low-achieving students?” Questions to Drive the Data

Growth by Achievement

Filter your Data  Filters help administrators to dig into the data

Tab: Growth by Achievement Question: How did our kids grow in various percentile ranges of achievement compared to other kids with similar scores and similar amounts of instructional time? Answer: Most students in grades 1-8 grew below average compared to other students that had similar amounts of instructional time. Students in grade K, 9 & 10 grew average or above average compared to students with similar amounts of instructional time. Questions to Drive the Data

Tab “Growth By School” Question: How did students in my building grow compared to other students nationally by grade, and with similar amounts of instructional time? Questions to Drive the Data

Growth by School

Tab “Growth By School” Question: How did students in my building grow compared to other students nationally by grade, and with similar amounts of instructional time? Answer: In elementary building #1, all grades except 6 th grew below average compared to other students with similar starting scores and instructional time. Grade 6 showed above average growth. Questions to Drive the Data

Tab “Growth By Ethnicity” Question: Which subgroups of students grew average or above average compared to other subgroups nationally that had the similar scores and similar amounts of instructional time? Questions to Drive the Data

Growth by Ethnicity District Data: Filters always available:

Tab “Growth By Ethnicity” Question: Which subgroups of students grew average or above average compared to other subgroups nationally that had the similar scores and similar amounts of instructional time? Answer: Native Hawaiian and Pacific Islander and Asian students showed average growth compared other students, while all other groups showed below average growth. Questions to Drive the Data

Tab “Growth Figure” Question: What was the observed actual growth of our students compared to students nationally that had similar scores and instructional time? Questions to Drive the Data

Growth Figure

Tab “Growth Figure” Question: What was the observed actual growth of our students compared to students nationally that had similar scores and instructional time? Answer: Grades K, 4, 7, 9 & 10 showed more observed growth compared to the average normative growth of students with similar scores and instructional time. All other grades showed below average growth. Questions to Drive the Data

Growth Graph Grades K, 4, 7, 9 & 10 showed higher growth than average normative growth.

Tab “Testing Conditions” Shows the percentages of students within each school whose MAP assessments triggered one or more “flags” Indicates that the student was not fully engaged when taking the assessment – Low accuracy = significantly less than 50% correct (higher SEM) – Short duration = little time on test – Unusual time increase/decrease = differences between fall to spring test duration Questions to Drive the Data

Tab “Testing Conditions” Question: Throughout the district, are there any flags related to testing conditions that might be cause for further investigation? Questions to Drive the Data

Testing Conditions

Tab “Testing Conditions” Question: Throughout the district, are there any flags related to testing conditions that might be cause for further investigation? Answer – Only 9 out of 52 buildings showed no flags for testing conditions. Many buildings had flags for multiple categories, especially related to low accuracy and short duration. Schools with multiple flags may want to investigate the reasons for these flags. Questions to Drive the Data

Tab – “Testing Conditions & Growth” Question – What impact did low accuracy and short test duration have on the growth rates of these students? Questions to Drive the Data

Testing Conditions and Growth

Pricing and Support  30¢ per student  $1,000 minimum  Includes one hour of live online support 29

Tab – “Testing Conditions & Growth” Question – What impact did low accuracy and short test duration have on the growth rates of these students? Answer: The district wide aggregate growth percentile is 41%ile. For schools that had significant flags for low accuracy and short test duration, the aggregate growth percentiles were significantly lower. This may indicate that high numbers of disengaged students may have depressed the district-wide growth percentile. Questions to Drive the Data

Virtual Comparison Group (VCG)

Since my students and my school aren’t typical, how can you expect my students to make typical progress? – Need an apples to apples comparison – A proof point to demonstrate what is possible That’s great, but… 32

What is a VCG?  Custom Report  Evaluates individual students’ observed growth against a virtual comparison group  Same grade & initial score  Same number of instructional weeks  Similar F/R eligibility  Same urban/rural classification 33

What is a VCG?  One step further….  We apply the filters to create a group that matches each of your students  Virtual Comparison Group = virtual peer group 34

How we create a VCG 35 We identify included students Identify all matching students from GRD School Income, Urban vs. Rural Classification Grade, Subject, Starting Achievement, Instructional Weeks Randomly select comparison group

Class Reports – Includes questions to guide interpretation School Reports District Reports with pivot tables What do you get?

Class Report

Questions to guide interpretation

School Report 6 th Grade 7 th Grade 8 th Grade

District Report

Growth by Achievement Growth by School Growth by Ethnicity Testing Conditions and Growth Hybrid Success Metric VCG & Hybrid Growth Pivot Tables

Hybrid Success Goal – For students above state proficiency target = average growth from norming study – For students below state proficiency target = amount of growth needed to be proficient by 10 th grade Hybrid Success Metric

Pricing and Support  90¢ per student  $1,500 minimum  Includes two hours of live online support 45