Assessment & Evaluation, October 2012

District Accountability Handbook CDE – School and District Accountability guidelines alTraining.asp SPF Technical Guide manceFrameworks.asp Video/training materials on SPF training/SPF_Online_Tutorial/playe r.html

Creating Meaning: How will I recognize “notable change” in performance over time?

Many factors can influence the interpretation of a difference between two numbers. Leaders must consider all before ascribing noteworthy status to a change in a percent, mean, or median. How Certain We Need To Be Statistical conventions in scientific research often call for high levels of certainty (90%-99%) that require very large differences for “significance.” If we are satisfied being "fairly certain," then lesser changes can be regarded as “notable.” Pre-Existing Differences Between Groups Statistical comparisons begin with an assumption that groups to be compared are the same in every way except for the intervention. If two groups begin with very different characteristics, then differences in % P/A or growth may be due pre-existing difference. Size of Comparison Groups A difference of 5 percentage points can be significant when both groups being compared are large but insignificant when the groups being compared are small. Multiple Points over Time A 2% or 3% change may not be statistically significant from one year to the next. However, if percent values change by 2 or 3 points over several successive points in time, then the overall change may be significant.

General “rules of thumb” for identifying percent P/A changes that may be significant between a moderate to high level of certainty: Most Classrooms Most Elementary Grades Most Middle School Grades Most High School Grades District Results Approx. Number of Students About students About students per grade About 400 students per grade About 600 students per grade About 4000 students per grade Approx. Percent change percentage points 6-12 percentage points 4-8 percentage points 2-5 percentage points 1-3 percentage points These are approximate guidelines. The larger the difference, the more certain we can be that it is not due to chance.

Achievement Percent Proficient and Advanced Reading (TCAP, Lectura, and CoAlt) Writing (TCAP, Escritura, and CoAlt) Math (TCAP and CoAlt) Science (TCAP and CoAlt) Growth Actual Growth & Target Growth TCAP Reading, Writing and Math Median Student Growth Percentiles Adequate Median Student Growth Percentiles Gaps Growth Gaps Median Student Growth Percentiles for disaggregated groups: Poverty (FRL) Race/Ethnicity Disability English proficiency Below proficient Postsecondary and Workforce Readiness Colorado ACT Graduation Rate Dropout Rate

To define the type of plan your school must create for submission to CDE I.Performance II.Improvement III.Priority Improvement IV.Turnaround PerformanceImprovement Priority Improvement Turnaround  All Cherry Creek Schools were assigned to either Performance or Improvement plans (most to Performance).

SPF Sample

CELA growth is added to the growth section. Distinguish between adequate growth targets and actual median percentile growth. The State defines minority students as “non-white.” District KPIs provide a better picture of growth for students of color.

KEY CONCEPTS FOR UNDERSTANDING GROWTH Actual Growth and Adequate Growth Catching Up and Keeping Up For Students SGP – Student Growth Percentile AGP – Adequate Growth Percentile target For Schools MGP -Median Growth Percentile Median Adequate Growth target

Office of Assessment & Evaluation  For each student, the State calculates a Student Growth Percentile (SGP) for the current year The SGP is based on the comparison of a student’s growth to growth in his or her State “academic peer group”  SGPs are normative  Some students’ SGP calculations use TCAP/CSAP data from several years, while others reflect only 2 data points

Student Pctls Sorted Pctls Median Pctls Median Growth for this school is 55 Search for the middle value… Student growth percentiles

Adequate Growth Percentiles: Are Students Growing Enough? For each student, the State projects growth paths in order to calculate an AGP for the current year. The AGP is based on the calculated projection of how much a student needs to grow. Students Below Proficient need to grow enough to “Catch Up,” or become Proficient in 3 years or by Grade 10 Students Proficient or above need to grow enough to “Keep Up,” or remain Proficient or above for 3 years or until Grade 10

Each year for each student, the State projects the growth necessary for that student to score at or above Proficient in 3 years or by grade 10 Not Proficient Proficient 7 th grade8 th grade9 th grade10 th grade6 th grade 76 is the minimum-this student’s adequate growth TARGET

Each year for each student, the State projects the growth necessary for that student to continue to score at or above Proficient for 3 years or until grade th grade8 th grade9 th grade10 th grade6 th grade Not Proficient Proficient is the maximum - this student’s adequate growth TARGET

What does the “Adequate Growth Percentile” calculation mean for classrooms with a high proportion of very Unsatisfactory or low scoring Partially Proficient students? What does the AGP calculation mean for classrooms with high proportions of Advanced or high Proficient students?

BObj growth reports offer administrators and instructional leaders a variety of ways to explore and disaggregate CSAP/TCAP growth results from the Colorado Growth Model. Most BObj growth reports are designed in a “crosstabular” form that allows the user to view results by row for students in each prior year’s CSAP/TCAP performance level (A, P, PP, US) and for all students (bottom row).

Student Growth Percentiles (SGP) from the Colorado Growth Model tell us how a student grew compared to other students in the State with the same CSAP performance starting point.

Understanding Cherry Creek School’s

The new federal racial/ethnic categories prompt a change in district reporting of results for our two racial/ethnic groupings (“White/Asian” and “Black/Hispanic/Native American”): Students in the Multi-racial category will be grouped: White/Asian: Students with any combination of White, Asian, and/or Pacific Islander Black/Hispanic/Native American Indian: Any students with a racial/ethnic identification that includes Black and/or Native American NOTE: Regardless of how many races are chosen, Hispanic students are not considered multi-racial

Achievement Status & Achievement Growth CSAP achievement status on most recent tests 10 points CSAP achievement growth over 2 most recent consecutive tests 30 points Equity Among Student SubGroups CSAP achievement status differences 5 points CSAP achievement growth differences 15 points Post-Secondary Readiness Colorado ACT performance~ most recent tests 10 points High School graduation rates~ prior year 10 points 45 total points for growth KPI Reports

TCAP Status TCAP Growth Excellence TCAP Status Equity TCAP Growth Equity Equity ES/MS: 3 rd Reading/ EXPLORE HS: COACT, Graduation Rate College Prep

Total points are summed across all elements. The final percent of points earned is the result of total points earned divided by the total points possible. For middle & elementary schools, the total number of points possible is 70. For high schools, the total possible is 80. KPI Reports, Summary page

The State longitudinal growth percentile allows us to compare the average growth of our students with average growth for students in the State with similar starting points. The objective is for CCSD schools to meet or exceed the State median growth percentile for students with similar performance starting points (i.e., median growth percentiles at or above 50). Cherry Creek Schools Office of Assessment & Evaluation CSAP_Longitudinal Growth on Data Access System from CDE student data file Results include all students with valid CSAP scores in 2007 & 2008 EXCEPT students who participated in CSAP-Alternate, were expelled or withdrew during testing, or were new to the school after October 1. KPI Reports

Excellence: CSAP Performance Indicator How is this KPI similar to or different from the SPF performance indicator? What areas are “red” or “blue” (weakest or strongest) on each report? Excellence: CSAP Growth Indicator How is this KPI similar to or different from the SPF performance indicator? Note any similarities or differences in the degree to which your school meets or exceeds growth expectation on both systems. What areas are “red” or “blue” (weakest or strongest) on each report? SPF, KPI Reports, Notes Page

The objective is for all student subgroups in CCSD to meet or exceed the State median growth percentile for students with similar performance starting points (i.e., median growth percentiles at or above 50). Cherry Creek Schools Office of Assessment & Evaluation CSAP Longitudinal Growth on Data Access System from CDE student data file Results include all students with valid CSAP scores in 2007 & 2008 EXCEPT students who participated in CSAP-Alternate, were expelled or withdrew during testing, or were new to the school after October 1. Calculations are included for all four ethnic groups regardless of numbers of students. Native American Indian student groups are not summarized due to very small N's across all grades & schools. Student program labels are taken from the CSAP test booklets and preprinted labels. (Reading shown. Rubric Points totaled across Reading, Writing, and Math.) KPI Reports

District Performance Plan

Creating Meaning: What types of information should I look for on reports?

Remember the fable of the 6 men in the dark trying to describe an elephant? What do we need to see in order to get a “complete picture” of summary performance and growth?

Views of Results Overall More Specific Details Levels of Analysis

TCAP BObj reports provide several “lenses” through which to examine school results. From different views, you can create a complete picture of summary school performance and growth. Average/Median or overall %P/A. Distributions. Trends. Disaggregations. Comparisons

All Schools Change Report (pdfs ed and on COLE) and 5-Year Performance Summary (BObj)

5-Year Ethnic Summary (BObj) and Disaggregated Summary (BObj)

Standards & SubContent Area Summaries

Overall medians over time for all schools Performance for Subgroups Overall medians, crosstabs, % in high, middle, and low growth levels, some disaggregations In what ways do the following report “picks” provide: 1.Multiple Views Of Summary Results Overall %P/A, Distributions, Trends, Disaggregations, Comparisons 2.Multiple Levels of Summary Analysis