PPT Presentation Template: This PPT includes all slides to present a district or building level overview of PVAAS. This was used with a district- wide.

Slides:



Advertisements
Similar presentations
Pennsylvania Value-Added Assessment System (PVAAS) High Growth, High Achieving Schools: Is It Possible? Fall, 2011 PVAAS Webinar.
Advertisements

PVAAS (The Pennsylvania Value Added Assessment System )
PPT Presentation Template: This PPT includes all slides to present a district-level overview of PVAAS. This was used with a district-wide elementary faculty.
PVAAS School Consultation Fall 2011 PVAAS Statewide Core Team
Pennsylvania Value-Added Assessment System Overview: PVAAS
Pennsylvania Value-Added Assessment System (PVAAS): A Compass for Improvement Oxford Area School District February 14, 2013.
Upper Darby School District Growth Data
Pennsylvania Department of Education PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment System.
PVAAS + Other Data Consultation 2013 PVAAS AND OTHER DATA TOOLS SCHOOL CONSULTATION FALL 2013.
PVAAS Overview: Evaluating Growth, Projecting Performance
Enquiring mines wanna no.... Who is it? Coleman Report “[S]chools bring little influence to bear upon a child’s achievement that is independent of.
PAIU CC Meeting PVAAS Update November 2014 Kristen Lewald, Ed.D. Statewide PVAAS Director for PDE Lancaster-Lebanon IU13.
Using Growth Models for Accountability Pete Goldschmidt, Ph.D. Assistant Professor California State University Northridge Senior Researcher National Center.
Performance Diagnostic Report PVAAS Overview 2013 Blue Bar – Current Year Missing Bar – Insufficient Number of Students Whisker – Margin of Error on Growth.
Delaware’s Accountability Plan for Schools, Districts and the State Delaware Department of Education 6/23/04.
Pennsylvania Value-Added Assessment System (PVAAS): PVAAS Public Site Tour PVAAS Statewide Team for PDE Fall 2011.
Common Questions What tests are students asked to take? What are students learning? How’s my school doing? Who makes decisions about Wyoming Education?
Pennsylvania Value-Added Assessment System (PVAAS): PVAAS Public Data Release Update to PDE and PAIU 2 Sessions March 9 and 10, 2011 Kristen Lewald, Ed.D.
Introduction to the Georgia Student Growth Model Student Growth Percentiles 1.
PVAAS Overview: Evaluating Growth, Projecting Performance PVAAS Statewide Core Team Fall 2008.
Adequate Yearly Progress (AYP) Academic Performance Index (API) and Assessing California Standards Test (CST) Data.
Department of Research and Evaluation Santa Ana Unified School District 2011 CST API and AYP Elementary Presentation Version: Elementary.
Introduction to the Georgia Student Growth Model Understanding and Using SGPs to Improve Student Performance 1.
1 Paul Tuss, Ph.D., Program Manager Sacramento Co. Office of Education August 17, 2009 California’s Integrated Accountability System.
Effectively Communicate PVAAS with your Board and Community Pennsylvania Value-Added Assessment System (PVAAS): Effectively Communicate PVAAS with your.
Copyright ©2006. Battelle for Kids. Understanding & Using Value-Added Analysis.
SY PVAAS Scatterplots State to IU Region to LEA Public Schools/Charter Schools/Comprehensive CTCs Math, Reading, Writing, Science Template Contact.
SY PVAAS Scatter Plots State to IU Region to School District Grades 4-8, 11 Math & Reading PVAAS Statewide Team for PDE Contact your IU PVAAS contact.
PVAAS – Pennsylvania Value-Added Assessment System added_assessment_system_(pvaas)/8751.
California Assessment of Student Performance and Progress (CAASPP) 1 California Department of Education, September 2015 EL SEGUNDO UNIFIED SCHOOL DISTRICT.
Supporting Students with IEPs: They CAN and ARE Making Progress! January 2013 Jennifer Ross, PVAAS Statewide Team for PDE Bonnie Dyer, AIU3 Curriculum.
Comparison PVAAS Scatter Plots State to IU Region to School District Grades 4-8, Math & Reading PVAAS Statewide Team for PDE Contact your IU PVAAS.
Slide 1 Estimating Performance Below the National Level Applying Simulation Methods to TIMSS Fourth Annual IES Research Conference Dan Sherman, Ph.D. American.
School Performance Framework Sponsored by The Colorado Department of Education Summer 2010 Version 1.3.
Pennsylvania Value-Added Assessment System (PVAAS) Updates to Curriculum Directors September 2013.
Introduction to GREAT for ELs Office of Student Assessment Wisconsin Department of Public Instruction (608)
TVAAS Tennessee Value-Added Assessment System
PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment System (PVAAS) The Power of Derry Township School District School Board Presentation Sept.
Adequate Yearly Progress (AYP) Academic Performance Index (API) and Analysis of the Mathematics Section of the California Standards Test (CST) Data Elementary.
PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment System (PVAAS) The Power of School District School District.
Santa Ana Unified School District 2011 CST Enter School Name Version: Intermediate.
PVAAS Public Data Release January January 20,
Annual Measurable Objectives (trajectory targets).
DVAS Training Find out how Battelle for Kids can help Presentation Outcomes Learn rationale for value-added progress measures Receive conceptual.
Using EVAAS to Improve Student Performance Donna Albaugh Rachel McBroom Heather Stewart Region 4 PD Leads NCDPI.
Evaluation Requirements for MSP and Characteristics of Designs to Estimate Impacts with Confidence Ellen Bobronnikov February 16, 2011.
Copyright © 2010, SAS Institute Inc. All rights reserved. How Do They Do That? EVAAS and the New Tests October 2013 SAS ® EVAAS ® for K-12.
ACT ASPIRE GROWTH REPORTS. DISTRICTS AND SCHOOLS THAT PARTICIPATED IN ACT ASPIRE ASSESSMENTS (READING, MATH, ENGLISH, SCIENCE AND WRITING) WITH AN N COUNT.
PVAAS Overview: Evaluating Growth, Projecting Performance PVAAS Statewide Core Team Fall 2008.
PVAAS School Consultation Guide Fall 2010 Session C: 9-12 High School – All Data Tools PVAAS Statewide Core Team
1 Getting Up to Speed on Value-Added - An Accountability Perspective Presentation by the Ohio Department of Education.
October 25, 2012 Jonathan Wiens, PhD Office of Assessment and Information Services Oregon Department of Education.
More than One Year’s Growth in One Academic Year Key PVAAS Reports for Special Education Kristen Lewald, Ed. D. PVAAS Statewide Project Director for PDE.
PVAAS – The Thanksgiving Edition Did your Thanksgiving Dinner Meet the minimum expectations this year? Whether it met the expectations or not was it better.
TRANSITION OF PA’S ASSESSMENT SYSTEM: IMPACT ON PVAAS?
PVAAS Overview: Evaluating Growth, Projecting Performance PVAAS Statewide Core Team Fall 2011 August
Using PVAAS for Gifted Learners Tanya Morret. How effectively is our system meeting the needs of our advanced learners? Are we projected to grow our advanced.
Connecticut Mastery Test Fourth Generation Grades 3, 4, 5, 6, 7 and 8 Connecticut Academic Performance Test Third Generation Grade 10 Presented to the.
PVAAS Overview: Evaluating Growth, Projecting Performance PVAAS Statewide Core Team for PDE Fall
1 Testing Various Models in Support of Improving API Scores.
Evaluation Requirements for MSP and Characteristics of Designs to Estimate Impacts with Confidence Ellen Bobronnikov March 23, 2011.
PVAAS: One of the Data Tools in a Standards-Aligned System to Support Data-Informed Decision-Making in K-12 Schools PVAAS Core Team June, 2007.
EVAAS Overview.
PVAAS Statewide Core Team
Wade Hayashida Local District 8
PVAAS Statewide Project Director for PDE
PVAAS Statewide Core Team August, 2008
Transition of PA State Assessments
Tom Tomberlin Director, Educator Recruitment and Support
So what is PVAAS? PVAAS measures a group of students as they move from one grade level to the next. PVAAS is not concerned with “Achievement” PVAAS is.
Presentation transcript:

PPT Presentation Template: This PPT includes all slides to present a district or building level overview of PVAAS. This was used with a district- wide K-12 faculty group in a small school district, where one grade level represented one building versus multiple buildings on a district level report.

PVAAS: Evaluating Growth, Projecting Performance

Today’s Session Concept of PVAAS Concept of PVAAS Overview of Reporting Overview of Reporting Math and Reading, Grades 4-8Math and Reading, Grades 4-8 Math and Reading, Grade 11Math and Reading, Grade 11 Science, Grades 4, 8 and 11Science, Grades 4, 8 and 11 Writing, Grades 5, 8 and 11Writing, Grades 5, 8 and 11 School District Data School District Data Questions and Answers Questions and Answers

What Is PVAAS?

PVAAS IS : Analysis of existing assessment data –Longitudinal PSSA scores in available content areas Math, Reading, Science, Writing –Provides reliable estimates A measure of a school’s effectiveness Critical Relationship: Growth Data vs. Achievement Data

PVAAS: Is not another test Is not a comparison of single test scores Critical concept: Does not provide a measure of growth for an individual student

High Achievement Average Achievement Low Achievement Current Data Collection Approaches Achievement is based on each student’s ending point.

PVAAS Adds Another Variable To Achievement…Progress Below Growth Standard At or Above Growth Standard Progress is based on each student’s starting point.

Viewing Achievement and Progress Together Scatter Plots: Grades 4-8 Reading and Math

The Scatter Plot Coordinate System Vertical Axis PSSA Percent Proficient or Advanced Horizontal Axis PVAAS Growth Measure AYP Percent Proficient Target Growth Standard Boundary One Year Growth

Scatter Plot Quadrants and Their Meanings Quad 1 Adequate Achievement Positive Growth Quad 4 Below Adequate Achievement Positive Growth Quad 2 Adequate Achievement Negative Growth Quad 3 Below Adequate Achievement Negative Growth

SD Scatter Plots Grade 4 ReadingMath Insert District Scatter Plots Here

SD Scatter Plots Grade 5 ReadingMath Insert District Scatter Plots Here

SD Scatter Plots Grade 6 ReadingMath Insert District Scatter Plots Here

SD Scatter Plots Grade 7 ReadingMath Insert District Scatter Plots Here

SD Scatter Plots Grade 8 ReadingMath Insert District Scatter Plots Here

Viewing Achievement and Progress Together Scatter Plots: Grade 11 Reading and Math

Grade 11 Scatter Plots The PSSA-PVAAS Scatter Plots visually display the following data from the statewide distribution of PSSA results for the most current school year. Achievement Achievement The mean predicted performance of the cohort.The mean predicted performance of the cohort. The actual mean performance of all students in the state is displayed for referenceThe actual mean performance of all students in the state is displayed for reference Growth Growth School Effect ≈ Observed Mean – Predicted MeanSchool Effect ≈ Observed Mean – Predicted Mean Standardized School Effect = School Effect relative to its error – displays degree of confidenceStandardized School Effect = School Effect relative to its error – displays degree of confidence The following two slides explain the meaning of each of the regions of an 11 th grade scatter plot.

The Scatter Plot Coordinate System for Grade 11 Horizontal Axis Growth Measure in Standard Errors Vertical Axis Mean Predicted Scaled Score Growth Descriptor Boundaries (±2SEs) 2010 Mean Scaled Score

Scatter Plot Regions and Their Meanings for Grade 11

SD Scatter Plots Grade 11 ReadingMath Insert District Scatter Plots Here

The TWO Types of PVAAS Information Today Looking Back/Evaluation… Value-added Growth Reports For Cohorts of Students Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Cohorts of Students NO individual measure of student growth.

PVAAS Reporting

Student Projections Wouldn’t it be great to know the likelihood that a student will be proficient on a future PSSA?

Student Projection Methodology A separate longitudinal modeling process focused on the performance scores of individual students. A separate longitudinal modeling process focused on the performance scores of individual students. The model uses all of the available data (both math and reading) to project future PSSA performances in Math, Reading, Science and Writing. The model uses all of the available data (both math and reading) to project future PSSA performances in Math, Reading, Science and Writing. Projections can be calculated for an individual student or groups of students. Projections can be calculated for an individual student or groups of students.

How do I know that the projection is accurate? The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office). The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study. ( Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score.

PVAAS Student Projection Report

PVAAS Projections for Groups of Students Student Search

Identified Students Meeting Search Requirements

PVAAS Reporting Projections to Basic, Proficient, and Advanced Grade Last Tested Math and Reading:  3  4 or 5  4  5 or 6  5  6 or 7  6  7 or 8  7  8  8  11 Writing:  3  5  4  5  5  8  6  8  7  8  8  11 Science :  3  4  4  NA  5  8  6  8  7  8  8  11 Grade Last Tested

Projection Summary Reports What is it? What is it? Report that summarizes the numbers and percents of students in likelihood ranges of performing at the proficient level on a future PSSA exam.Report that summarizes the numbers and percents of students in likelihood ranges of performing at the proficient level on a future PSSA exam. How might a school use this report? How might a school use this report? Intervention PlanningIntervention Planning Resource AllocationResource Allocation Strategic PlanningStrategic Planning School Improvement PlanningSchool Improvement Planning Cautions Cautions This report provides another indicator about likelihood of future performance.This report provides another indicator about likelihood of future performance.

School (Single Grade) Projection Summary Report

Wouldn’t it be great to know if your students made one year’s growth for the year of schooling they experienced last year? In Math and Reading (4-8 and 11)? In Science (4, 8 and 11)? In Writing (5, 8 and 11)? Value-Added

PVAAS Growth Methods TestingSubjects – GradesMethodology PSSA in consecutive years Math & Reading – 4-8 Growth Standard PSSA not in consecutive years Writing – 5, 8, 11 Science – 4, 8, 11 Math & Reading – 11 Predictive

Value-added Measure of Growth Math and Reading (Grades 4-8) Comparison to a State Growth Standard Comparison to a State Growth Standard The Growth Standard specifies the minimal acceptable academic gain from grade to grade for a cohort of students. The Growth Standard specifies the minimal acceptable academic gain from grade to grade for a cohort of students. The use of a Growth Standard creates the possibility that ALL schools can demonstrate appropriate growth. The use of a Growth Standard creates the possibility that ALL schools can demonstrate appropriate growth.

PVAAS Value-Added Growth Descriptors – Grade 4 – 8 Math and Reading The PA Growth Standard specifies the minimal acceptable academic gain from grade to grade for a cohort of students. The Growth Standard is met when the students in a school or district on average maintain their relative position when compared to the 2006 statewide distribution of student scores (2006 is the base year). Estimated mean NCE gain equal or greater than zero. There is evidence that the school has met or exceeded the Growth Standard. All schools can achieve this rating. Estimated mean NCE gain less than zero by one standard error or less. There is slight evidence that the school has not met the Growth Standard. Estimated mean NCE gain less than zero by between one and two standard errors. There is moderate evidence that the school has not met the Growth Standard. Estimated mean NCE gain more than two standard errors less than zero. There is significant evidence that the school has not met the Growth Standard.

PVAAS School Report Grades 4 through 8 - Math and Reading

What Value-added Gain Means! NCE Scale State Mean = 50 Cohort’s Achievement 6 th grade 60.7 Cohort’s Achievement 7 th grade 66.6 PVAAS Gain ≈ Change in position from 6 th to 7 th grade ≈ 66.6 – 60.7 ≈ 5.9 NCE points

PVAAS Growth Methods TestingSubjects – GradesMethodology PSSA in consecutive years Math & Reading – 4-8 Growth Standard PSSA not in consecutive years Writing – 5, 8, 11 Science – 4, 8, 11 Math & Reading – 11 Predictive

Value-added Measure of Cohort Growth Math and Reading (Grade 11) Science (Grades 4, 8, and 11) Writing (Grades 5, 8, and 11) Based on a predictive methodology… A predicted score is calculated for each student; A predicted score is calculated for each student; Mean of predicted scores is calculated for students in a cohort; Mean of predicted scores is calculated for students in a cohort; Mean of observed scores of students in the cohort; Mean of observed scores of students in the cohort; School Effect is the difference between the Observed and the Predicted Scores. School Effect is the difference between the Observed and the Predicted Scores.

PVAAS Predictive Analysis Year 1 Year 2 Year 3 PSSA PSSA PSSA (11th) 7th 8th Predicted Actual 1.We collect all of the individual student data that is available for a minimum of two years. 2.All of these data are used to develop individual student profiles. 3.PVAAS then calculates a Predicted Performance Value based on the student profile for each student. Reading Math ?? 1320??

PVAAS Ratings of School Effect Remember: School Effect  Mean of Student Scores – Mean of Predicted Scores So, if the students perform as expected, School Effect  0. The ratings are in terms of the expected value of SEs 0 +2 SEs Not Detectably Different AboveBelow Standard Error (SE) is a measure of variation in the prediction.

Writing, Science & Grade 11 Reading/Math Growth Descriptors Progress is defined by the district or school effect, which is a measure of the difference between observed performance and predicted performance. Progress is significantly above the progress of the average district in the state. Progress is not detectably different from the progress of the average district in the state. Progress is significantly below the progress of the average district in the state.

PVAAS School Report Grades 11- Math and Reading Grades 4, 8 & 11 – Science Grades 5, 8 & 11 – Writing

Summary of PVAAS Growth Methods TestingSubjects – GradesMethodology PSSA in consecutive years Math & Reading – 4-8 Growth Standard PSSA not in consecutive years Writing – 5, 8, 11 Science – 4, 8, 11 Math & Reading – 11 Predictive

Summary: One Year’s Growth Growth Standard Method Math and Reading – Grades 4 through 8 Cohort maintains its achievement position. Predictive Method Math & Reading – Grade 11 Writing – Grades 5, 8, 11 Science – Grades 4, 8, 11 Cohort’s actual performance is as expected.

PVAAS Performance Diagnostic Report The Key is the RED Whisker!

Growth Standard Methodology What the whiskers tell us… Growth Standard (One Year’s Growth) Met the Growth Standard; Made One Year’s Growth Yellow Below Growth Standard; Less than One Year’s Growth Rose Exceeded Growth Standard; More than One Year’s Growth Green

Predictive Methodology What the whiskers tell us… Observed – Predicted (One Year’s Growth) NDD – No Detectable Difference Between Observed Score and Predicted Score BELOW Expectations -- Observed Score Was Less Than Predicted Score ABOVE Expectations -- Observed Score Was Greater Than Predicted Score

Performance Diagnostic Report

Greater Detail – based on % of students

Click on Hot Keys for lists of students.

Student Lists from Performance Diagnostic Reports

Your District and School Data

PA’s Intent for PVAAS Another Tool for Continuous Improvement Another Tool for Continuous Improvement Measure of Growth to Add to Achievement/Status Measures Measure of Growth to Add to Achievement/Status Measures Not to Be Used in Isolation Not to Be Used in Isolation

Questions and Feedback

Questions: PVAAS Materials or Statewide Implementation PVAAS Report Web Site

333 Market Street Harrisburg, PA 17126