1. PPT Presentation Template: This PPT includes all slides to present a district-level overview of PVAAS. This was used with a district-wide elementary faculty group.

2. PVAAS Overview: Evaluating Growth, Projecting Performance Faculty Introduction

3. PVAAS IS : Analysis of existing assessment data –Longitudinal PSSA scores in available content areas Math, Reading, Science, Writing –Provides reliable estimates A measure of a school’s effectiveness Critical Relationship: Growth Data vs. Achievement Data

4. PVAAS: Is not another test Is not a comparison of single test scores Critical concept: Does not provide a measure of growth for an individual student

5. High Achievement Average Achievement Low Achievement Current Data Collection Approaches Achievement is based on each student’s ending point.

6. PVAAS Adds Another Variable To Achievement…Progress Below Growth Standard At or Above Growth Standard Progress is based on each student’s starting point.

7. Viewing Achievement and Progress Together Scatter Plots: Grades 4-8 Reading and Math

8. The Scatter Plot Coordinate System Vertical Axis PSSA Percent Proficient or Advanced Horizontal Axis PVAAS Growth Measure AYP Percent Proficient Target Growth Standard Boundary One Year Growth

9. Scatter Plot Quadrants and Their Meanings Quad 1 Adequate Achievement Positive Growth Quad 4 Below Adequate Achievement Positive Growth Quad 2 Adequate Achievement Negative Growth Quad 3 Below Adequate Achievement Negative Growth

10. District Scatter Plots District vs. IU Region vs. State Math and Reading Grades 4 – 8

11. District Scatter Plot – Grade 4

12. District Scatter Plot – Grade 5

13. District Scatter Plot – Grade 6

14. District Scatter Plot – Grade 7

15. District Scatter Plot – Grade 8

16. Viewing Achievement and Progress Together Scatter Plots: Grade 11 Reading and Math

17. Grade 11 Scatter Plots The PSSA-PVAAS Scatter Plots visually display the following data from the statewide distribution of PSSA results for the most current school year. Achievement Achievement The mean predicted performance of the cohort.The mean predicted performance of the cohort. The actual mean performance of all students in the state is displayed for referenceThe actual mean performance of all students in the state is displayed for reference Growth Growth School Effect ≈ Observed Mean – Predicted MeanSchool Effect ≈ Observed Mean – Predicted Mean Standardized School Effect = School Effect relative to its error – displays degree of confidenceStandardized School Effect = School Effect relative to its error – displays degree of confidence The following two slides explain the meaning of each of the regions of an 11 th grade scatter plot.

18. The Scatter Plot Coordinate System for Grade 11 Horizontal Axis Growth Measure in Standard Errors Vertical Axis Mean Predicted Scaled Score Growth Descriptor Boundaries (±2SEs) 2010 Mean Scaled Score

19. Scatter Plot Regions and Their Meanings for Grade 11

20. District Scatter Plot – Grade 11

21. The TWO Types of PVAAS Information Today Looking Back/Evaluation… Value-Added Growth Reports For Cohorts of Students Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Cohorts of Students NO individual measure of student growth..

22. Student Projections

23. PVAAS Reporting Projections to Basic, Proficient, and Advanced Grade Last Tested Math and Reading:  3  4 or 5  4  5 or 6  5  6 or 7  6  7 or 8  7  8  8  11 Writing:  3  5  4  5  5  8  6  8  7  8  8  11 Science :  3  4  4  NA  5  8  6  8  7  8  8  11 Grade Last Tested

24. How do I know that the projection is accurate? The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office). The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study. (http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html)http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score.

25. PVAAS Student Projection Report

26. PVAAS Projections for Groups of Students Student Search

27. Identified Students Meeting Search Requirements

28. District (Single Grade) Projection Summary Report – Grade 3 District (Single Grade) Projection Summary Report for Grade 3 to be inserted here

29. District (Single Grade) Projection Summary Report – Grade 4 District (Single Grade) Projection Summary Report for Grade 4 to be inserted here

30. District (Single Grade) Projection Summary Report – Grade 5 District (Single Grade) Projection Summary Report for Grade 5 to be inserted here

31. District (Single Grade) Projection Summary Report – Grade 6 District (Single Grade) Projection Summary Report for Grade 6 to be inserted here

32. District (Single Grade) Projection Summary Report – Grade 7 District (Single Grade) Projection Summary Report for Grade 7 to be inserted here

33. District (Single Grade) Projection Summary Report – Grade 8 District (Single Grade) Projection Summary Report for Grade 8 to be inserted here

34. Value-Added Wouldn’t it be great to know if you made at least one year’s growth with students last tested in grades 4-8 and in grade 11?

35. Value-added Measure of Cohort Growth Grades 4 through 8 in Reading and Math Comparison to a State Growth Standard The Growth Standard specifies the minimal acceptable academic gain from grade to grade for a cohort of students. The use of a Growth Standard creates the possibility that ALL schools can demonstrate appropriate growth.

36. Four Important Issues 1. Measuring Gain 2. Growth Standard 3. Comparing Scores from year to year 4. Estimating Achievement Difference in scores in consecutive years. Combine current observed scores with all previous scores in students’ histories. Convert all scores to same scale – NCE scale Student NameQuiz 1Test 1Quiz 2Project 1Test 2Test 3 Cum Possible25100307570100 Percent Joe209723726555 83 Betty1476276556 73.5 Elizabeth219625606478 86

37. PVAAS Value-Added Growth Descriptors – Grade 4 – 8 Math and Reading The PA Growth Standard specifies the minimal acceptable academic gain from grade to grade for a cohort of students. The Growth Standard is met when the students in a school or district on average maintain their relative position when compared to the 2006 statewide distribution of student scores (2006 is the base year). Estimated mean NCE gain equal or greater than zero. There is evidence that the school has met or exceeded the Growth Standard. All schools can achieve this rating. Estimated mean NCE gain less than zero by one standard error or less. There is slight evidence that the school has not met the Growth Standard. Estimated mean NCE gain less than zero by between one and two standard errors. There is moderate evidence that the school has not met the Growth Standard. Estimated mean NCE gain more than two standard errors less than zero. There is significant evidence that the school has not met the Growth Standard.

38. PVAAS District Report Grades 4 through 8 - Math and Reading

39. What Value-added Gain Means! 0 10 20 30 40 50 60 70 80 90 100 NCE Scale State Mean = 50 Cohort’s Achievement 5 th grade 60.7 Cohort’s Achievement 6 th grade 64.6 PVAAS Gain ≈ Change in position from 5 th to 6 th grade ≈ 64.6 – 60.7 ≈ 4.0 NCE points

40. PVAAS School Report High School Math and Reading Science and Writing Based on a predictive methodology: A predicted score is calculated for each student; A predicted score is calculated for each student; Mean of predicted scores is calculated for students in a cohort; Mean of predicted scores is calculated for students in a cohort; Mean of observed scores of students in the cohort; Mean of observed scores of students in the cohort; School Effect is the difference between the Observed and the Predicted Scores. School Effect is the difference between the Observed and the Predicted Scores.

41. Writing, Science & Grade 11 Reading/Math Growth Descriptors Progress is defined by the district or school effect, which is a measure of the difference between observed performance and predicted performance. Progress is significantly above the progress of the average district in the state. Progress is not detectably different from the progress of the average district in the state. Progress is significantly below the progress of the average district in the state.

42. PVAAS District Report Grades 11- Math and Reading Grades 4, 8 & 11 – Science Grades 5, 8 & 11 – Writing

43. PVAAS Performance Diagnostic Report The Key is the RED Whisker!

44. Growth Standard Methodology What the whiskers tell us… Growth Standard (One Year’s Growth) Met the Growth Standard; Made One Year’s Growth Yellow Below Growth Standard; Less than One Year’s Growth Rose Exceeded Growth Standard; More than One Year’s Growth Green

45. Predictive Methodology What the whiskers tell us… Observed – Predicted (One Year’s Growth) NDD – No Detectable Difference Between Observed Score and Predicted Score BELOW Expectations -- Observed Score Was Less Than Predicted Score ABOVE Expectations -- Observed Score Was Greater Than Predicted Score

46. Performance Diagnostic Report

47. Greater Detail – based on % of students

48. Click on Hot Keys for lists of students.

49. Student Lists from Performance Diagnostic Reports

51. PA’s Intent for PVAAS Another Tool for Continuous Improvement Another Tool for Continuous Improvement Measure of Growth to Add to Achievement/Status Measures Measure of Growth to Add to Achievement/Status Measures Not to Be Used in Isolation Not to Be Used in Isolation

52. PVAAS Help Menus PVAAS Help Menus are designed around Frequently Asked Questions that the Core Team and IU staff often receive.

53. PVAAS Help Menus Each Help Menu is organized into 5 common sections: Navigating: designed to help users navigate through the report and access all available features. Understanding: designed to increase users’ understanding of the key pieces of the report, as well as general assistance with correct interpretation. Interpreting: designed to help users gain a deeper understanding of the interpretation of the reports, including tips on analyzing the report for patterns and trends. Using: designed to help users gain an understanding of how the report can be used with other data sources for various decision- making purposes. Assistance: provides general information on the viewing and printing of reports, as well as state and IU points of contact.

54. Questions and Feedback

55. Questions: PVAAS Materials or Statewide Implementation pdepvaas@iu13.org 717-606-1911 PVAAS Report Web Site https://pvaas.sas.com

56. www.pde.state.pa.us 333 Market Street Harrisburg, PA 17126