1. PVAAS Overview: Evaluating Growth, Projecting Performance PVAAS Statewide Core Team Fall 2008

2. What Is PVAAS?

3. PVAAS IS : A statistical analysis of existing assessment data. A realistic measurement of a school’s progress/growth over time. Progress data to add to achievement data as schools are analyzing their work with students and making improvement plans.

4. PVAAS: Is not another test. Is not being used for teacher- level accountability in Pennsylvania. Does not use teacher-level data elements. Does not yield a metric on the effectiveness of a teacher.

5. High Achievement Average Achievement Low Achievement Current Data Collection Approaches Achievement is based on each student’s ending point.

6. PVAAS Adds Another Variable To Achievement…Progress Below Growth Standard At or Above Growth Standard Progress is based on each student’s starting point.

7. The Scatter Plot Coordinate System Vertical Axis PSSA Percent Proficient or Advanced Horizontal Axis PVAAS Growth Measure AYP Percent Proficient Target Growth Standard Boundary

8. Scatter Plot Quadrants and Their Meanings Quad 1 Adequate Achievement Positive Growth Quad 4 Below Adequate Achievement Positive Growth Quad 2 Adequate Achievement Negative Growth Quad 3 Below Adequate Achievement Negative Growth

9. Example: Regional Scatter Plot

10. PVAAS Fall 2008 Reporting: Reading and Mathematics Projections to basic, proficient, advanced Projections to basic, proficient, advanced  3  4 or 5  4  5 or 6  5  6 or 7  6  7 or 8  7  8  8  11 Grade Last Tested

11. PVAAS Fall 2008 Reporting: Writing and Science Projections to basic, proficient, advanced Projections to basic, proficient, advanced Writing: Writing:  5  8  8  11 Science: Science:  4  8  8  11 Grade Last Tested

12. Wouldn’t it be great to know if you made at least one year’s growth with students last tested in grades 4-8? grade 11(NEW)? Value-Added

13. The Assessment Dilemma True Student Achievement Any test is just a snapshot in time!

14. Four Important Issues 1. Measuring Gain 2. Growth Standard 3. Comparing Scores from year to year 4. Estimating Achievement Difference in scores in consecutive years. Combine current observed scores with all previous scores in students’ histories. Convert all scores to same scale – NCE scale Student NameQuiz 1Test 1Quiz 2Project 1Test 2Test 3 Cum Possible25100307570100 Percent Joe209723726555 83 Betty1476276556 73.5 Elizabeth219625606478 86

15. PVAAS Value-added Cohort Growth Descriptors – Grade 4 – 8 Favorable Indicator Estimated gain at or above growth standard. Students in this cohort have made at least one year growth. All schools can achieve this rating. Caution Indicator Estimated gain below growth standard but by less than one standard error. Students in this cohort have grown less than the standard. Stronger Caution Estimated gain below growth standard by more than one but less than two standard errors. Students in this cohort have fallen behind their peers. Strongest Warning Estimated gain well below growth standard by more than two standard errors. Students in this cohort have made little progress.

16. PVAAS School Report Grades 4 through 8 Every grade in the school has met or exceeded a year’s worth of growth? (“Getting Results!”™, p. 12)

17. Your Turn! Interpret this report!

18. PVAAS School Report High School Based on a predictive methodology A predicted score is calculated for each student; A predicted score is calculated for each student; Mean of predicted scores is calculated for students in a cohort; Mean of predicted scores is calculated for students in a cohort; Mean of observed scores of students in the cohort; Mean of observed scores of students in the cohort; School Effect is the difference between the Observed and the Predicted Scores. School Effect is the difference between the Observed and the Predicted Scores.

19. 11 th Grade Growth Descriptors Below Observed performance is significantly BELOW the expected performance. NDD Observed performance is NOT DETECTABLY DIFFERENT than the expected performance. Above Observed performance is significantly ABOVE the expected performance.

20. School Report

21. PVAAS Performance Diagnostic Report The Key is the RED Whisker!

22. What the whiskers tell us… Growth Standard (One Year’s Growth) Met the Growth Standard; Made One Year’s Growth Yellow Below Growth Standard; Less than One Year’s Growth Rose Exceeded Growth Standard; More than One Year’s Growth Green

23. Every performance levels (below basic, basic, proficient, advanced) in each grade has met or exceeded a year’s worth of growth? (“Getting Results!”™, p. 12) Performance Diagnostic Report

25. Click on Hot Keys for lists of students.

26. Student Lists from Performance Diagnostic Reports

28. Student Projections Wouldn’t it be great to know the likelihood that a student will be proficient on a future PSSA?

29. Student Projection Methodology A separate longitudinal modeling process focused on the performance scores of individual students. The model uses all of the available data (both math and reading) to project future PSSA performances in Math, Reading, Science and Writing. Projections can be calculated for an individual student or groups of students.

30. PVAAS Student Projection Report

31. Identified Students Meeting Search Requirements

32. Projection Summary Reports What is it? What is it? Report that summarizes the numbers and percents of students in likelihood ranges of performing at the proficient level on a future PSSA exam.Report that summarizes the numbers and percents of students in likelihood ranges of performing at the proficient level on a future PSSA exam. How might a school use this report? How might a school use this report? Intervention PlanningIntervention Planning Resource AllocationResource Allocation Strategic PlanningStrategic Planning School Improvement PlanningSchool Improvement Planning Cautions Cautions This report provides another indicator about likelihood of future performance.This report provides another indicator about likelihood of future performance.

33. Grade Projection Summary Report

34. School Projection Summary Report

35. District Projection Summary Report

36. What do you think… What are some examples of how PVAAS Value-added Reports can be used in your school?

37. Questions: PVAAS Materials or Statewide Implementation pdepvaas@iu13.org 717-606-1911 PVAAS Report Web Site https://pvaas.sas.com

38. Gerald L. Zahorchak, D.Ed. Secretary of Education Commonwealth of Pennsylvania www.pde.state.pa.us 333 Market Street Harrisburg, PA 17126