PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment System (PVAAS) The Power of School District School District

PDE Data Tools PSSA Data Interactive by eMetric Interactive tool for analyzing PSSA data by district, school, grade, subgroup and student – Reporting category information is also available PVAAS Value-added tool that reports growth of cohorts of students and projections for individual students for performance on future PSSAs PA AYP Public site that reports all AYP related data on district, school, grade and subgroup - Reporting category information is also available 4Sight Member Center Reporting site for benchmark data by grade, class, subgroup and student – intended to be formative and used often throughout the school year

Stair-Step Expectations Grade Three Grade Four Grade Five Grade 3 Test Grade 4 Test Grade 5 Test In a perfect world students would start school at about the same academic level. Proficiency levels could simply be achieved by students gaining age-appropriate knowledge and skills each year.

Differentiated Reality Grade Three Grade Four Grade Five Students begin school at different places; they often progress at different rates. Yet all students are expected to achieve the same level of academic proficiency each year. Grade 3 Test Grade 4 Test Grade 5 Test

The Need for Progress Measures Grade Three Grade Four Grade Five To measure school effectiveness in this “differentiated” world, we need to pay attention not only to proficiency levels, but also to how much progress students make in a given year. Grade 3 Test Grade 4 Test Grade 5 Test

PVAAS IS : Another tool for continuous school improvement A statistical analysis of existing assessment data (PSSA). A realistic measurement of a school’s progress/growth over time. Not to be used in isolation Progress data to add to achievement data as schools are analyzing their work with students and making improvement plans.

PVAAS is NOT: Another Test A measure for teacher accountability

The PVAAS Methodologies Today Looking Back/Evaluation… Value-added Growth Reports For Cohorts of Students Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Cohorts of Students

PVAAS Fall 2008 Reporting (math, rdg; wrtg; science: coming late fall) Value-added growth reports for grades 4 through 8, and 11 (Last’s years cohorts) Projections to basic, proficient, advanced   Grade last tested - 3  4 or 5 (future PSSAs)   Grade last tested - 4  5 or 6 (future PSSAs)   Grade last tested - 5  6 or 7 (future PSSAs)   Grade last tested - 6  7 or 8 (future PSSAs)   Grade last tested - 7  8 (future PSSAs)   Grade last tested - 8  11 (future PSSAs)

Student Projections Wouldn’t it be great to know the likelihood that a student will be proficient on a future PSSA?

Student Projection Methodology A separate longitudinal modeling process focused on the performance scores of individual students. A separate longitudinal modeling process focused on the performance scores of individual students. The model uses all of the available data (both math and reading) to project future PSSA performances in Math and Reading. The model uses all of the available data (both math and reading) to project future PSSA performances in Math and Reading. Projections can be calculated for an individual student or groups of students. Projections can be calculated for an individual student or groups of students.

PVAAS Student Projection Report

Identified Students Meeting Search Requirements

Data Packets: Student Level TimingSourceReport Initial Meeting – Looking Back PSSA: eMetric/GROW Individual Student Report Initial Meeting – Today 4SightBaseline Looking Forward PVAAS PVAAS Projection Periodic Meetings (based on subject area and grade level) Examples:DIBELS4SightGRADEGMADE Core Phonics Local Formative Assessment

PVAAS Projections with Other Data: Elementary Scenario/Reading Student NOW in 5 th Grade PSSA Grade 4 Proficiency Range: Sight Baseline Grade 5 Proficiency Range: PVAAS Projections To Basic: To Proficient: To Advanced: 93.6%64.8%5.5% DIBELSORF Grade 5 – Low Risk 104+ at beginning of year 124+ at end of year 89 Some Risk

PVAAS Projections with Other Data: High School Scenario/Math Student NOW in 9 th Grade PSSA Grade 8 Proficiency Range Sight Baseline Grade 9 Proficiency Range PVAAS Projections (11 th Grade) To Basic: To Proficient: To Advanced: 98.7%89.9%45.7% Grade 8 Final Grades Final Exam: 91% Course Grade: 87%

What Does This Mean to You? The district could provide you each with the probability of students in your classrooms (current grades 4-9) The district could provide you each with the probability of students in your classrooms (current grades 4-9) Grade level/course specific teams can collaborate to make sure adequate supports are in place for all students to reach success. Grade level/course specific teams can collaborate to make sure adequate supports are in place for all students to reach success.

How could we use this information? For students with each level of probability of proficiency, what do we need to keep doing to keep at this level? What do we need to do to increase the level of probability? For students with each level of probability of proficiency, what do we need to keep doing to keep at this level? What do we need to do to increase the level of probability? % likelihood of proficiency70-100% likelihood of proficiency – what is our action plan? – what is our action plan? 40-70% likelihood of proficiency40-70% likelihood of proficiency – what is our action plan? Goal date for proficiency? – what is our action plan? Goal date for proficiency? 0-40% likelihood of proficiency0-40% likelihood of proficiency – what is our action plan? Goal date for proficiency? – what is our action plan? Goal date for proficiency?

Wouldn’t it be great to know if we made one year’s growth with last year’s 4 th, 5 th, 6 th, 7 th, and 8 th graders ? 11 th graders? Value-Added

Value-Added Measure of Cohort Growth Comparison to a State Growth Standard Comparison to a State Growth Standard The Growth Standard specifies the minimal designated academic gain from grade to grade for a cohort of students. The Growth Standard specifies the minimal designated academic gain from grade to grade for a cohort of students. The use of a Growth Standard creates the possibility that ALL schools can demonstrate appropriate growth. The use of a Growth Standard creates the possibility that ALL schools can demonstrate appropriate growth.

Caution Indicator Estimated gain below growth standard but by less than one standard error. Students in this cohort have grown less than the standard. Stronger Caution Estimated gain below growth standard by more than one but less than two standard errors. Students in this cohort have fallen behind their peers. Strongest Warning Estimated gain well below growth standard by more than two standard errors. Students in this cohort have made little progress. PVAAS Value-added Cohort Growth Descriptors Favorable Indicator Estimated gain at or above growth standard. Students in this cohort have made at least one year growth. All schools can achieve this rating.

PVAAS School Report Grades 4 through 8 Every grade in the school has met or exceeded a year’s worth of growth? (“Getting Results!”™, p. 12)

11 th Grade Growth Descriptors Below Observed performance is significantly BELOW the expected performance. NDD Observed performance is NOT DETECTABLY DIFFERENT than the expected performance. Above Observed performance is significantly ABOVE the expected performance.

Grade 11 School Report

Every performance levels (below basic, basic, proficient, advanced) in each grade has met or exceeded a year’s worth of growth? (“Getting Results!”™, p. 12) Performance Diagnostic Report

What the whiskers tell us… Growth Standard (One Year’s Growth) Met the Growth Standard; Made One Year’s Growth Yellow Below Growth Standard; Less than One Year’s Growth Rose Exceeded Growth Standard; More than One Year’s Growth Green

Every performance level (below basic, basic, proficient, advanced) in each grade has met or exceeded a year’s worth of growth? (“Getting Results!”™, p. 12) Patterns: Performance Diagnostic Report

Patterns of Growth A B C

What Does This Mean to You? The district could provide each grade level/course specific team with their growth reports The district could provide each grade level/course specific team with their growth reports Grades 4-8, and 11 (Last’s years cohorts)Grades 4-8, and 11 (Last’s years cohorts) Subgroup reportingSubgroup reporting

How could we use this information? At a grade level/course specific team, discuss why and how you may have gotten the growth results on the report?At a grade level/course specific team, discuss why and how you may have gotten the growth results on the report? Which students made a year’s worth of growth?Which students made a year’s worth of growth? Which did not?Which did not? Why? Why not?Why? Why not? What is our team’s plan of action as a grade level/course specific team?What is our team’s plan of action as a grade level/course specific team?

Where do we go from here? 1. How do we plan to use our PVAAS data this next month? This school year? Value-added reportsValue-added reports Diagnostic reportsDiagnostic reports Student projectionsStudent projections 2. How do we plan to share our PVAAS data? With other Administrators?With other Administrators? With Teachers?With Teachers? With Parents?With Parents? With Others?With Others? District Goal: PVAAS will be used by teachers and administrators for annual school evaluation and planning, grade/subject level evaluation and planning, and for student level planning.

Data Packet: Annual School-Level Analysis QuestionToolReport 1) School Met AYP?PAAYPPerformance Chart 2) Every Grade Met AYP?eMetric3-Year Portrait 3) Every Grade Met Growth Standard?PVAASSchool Value-added Report 4) Every Grade 3 Year Trend Change in Proficiency? eMetric3-Year Portrait 5) Every Predicted Performance Level Met Growth Standard? PVAASPerformance Diagnostic Report 6) Every Subgroup Met AYP?eMetric3-Year Portrait 7) Achievement Gap Narrower?eMetric3-Year Portrait 8) Every Grade Projected To Meet AYP Target? PVAASGrade Projection Summary 9) School and Subgroups Met Participation Targets? PAAYPData Table 10) Every Subgroup Met Growth Standard? PVAASPerformance Diagnostic Report 11) Subgroups Performed Similarly in Reporting Categories? eMetric3-Year Portrait

Data Packets: Grade Level TimingSourceReport Initial Meeting – Looking Back eMetriceMetricPVAASPVAAS eMetric/PVAAS Performance Levels Reporting Categories Value Added Report Performance Diagnostic Report PSSA/ PVAAS Scatterplots Initial Meeting– Looking Forward PVAAS 4Sight Baseline Projection Summary Report Performance Levels Subscale Reports Initial Meeting– Subgroup Comparisons Same as above Periodic Meetings 4Sight4Sight Local Formative Assessment Performance Levels Subscale Reports

Data Packets: Student Level TimingSourceReport Initial Meeting – Looking Back PSSA: eMetric/GROW Individual Student Report Initial Meeting – Today 4SightBaseline Looking Forward PVAAS PVAAS Projection Periodic Meetings (based on subject area and grade level) Examples:DIBELS4SightGRADEGMADE Core Phonics Local Formative Assessment

Statewide PVAAS Professional Development, Support & Materials Contact Information PVAAS Website

Gerald L. Zahorchak, D.Ed. Secretary of Education Commonwealth of Pennsylvania