1. PVAAS + Other Data Consultation 2013 PVAAS AND OTHER DATA TOOLS SCHOOL CONSULTATION FALL 2013

2. PURPOSE OF CONSULTATION  Goal: To model a process for the inspection, analysis and interpretation of your 2013 data. This session is NOT designed to serve as a PVAAS overview or to teach the PVAAS concepts.  Focus: PVAAS Data and eMetric  This process is designed to give a school the “30,000 foot view” of the status of achievement and growth  Whole school data  Grade level data  Subgroup data PVAAS + Other Data Consultation 2013

3. Two key questions to keep in mind as you examine your data! Based on the system’s prior year achievement goals/targets: Is the system (school/district) serving all students adequately? Is the system (school/district) serving all students equitably? Enter last year’s achievement goals into the box on the cover of the Consultation Guide. PURPOSE OF CONSULTATION PVAAS + Other Data Consultation 2013 C. G. – Cover

4. AGENDA PVAAS Review a Conceptual Model of Measuring Growth…...15 min The Power of Two: Web-Based Scatterplots………......15 min PVAAS Reports…………………………………………45 min NEW!!!! PVAAS Launchpad - Value-Added Reports, Performance Diagnostic Reports, School Projection Summary Reports PSSA Data Interaction Reports, eMetric ………..............30min Performance Level Summary, Reporting Categories Summary, Matched Comparison Report Summary and Next Steps………………………………...15 min PVAAS + Other Data Consultation 2013

5. CONSULTATION GUIDE Decide on the focus for your team to make the most use of today’s time. On what subject will you focus? What grade level(s) within the subject? What subgroup within the grade and subject? Purpose is to model a process that can be used to determine the status of achievement & growth in your school. The next step is to begin to investigate WHY you are getting these results. PVAAS + Other Data Consultation 2013

6. PENNSYLVANIA VALUE ADDED ASSESSMENT SYSTEM PVAAS PVAAS + Other Data Consultation 2013

7. MEASURING GROWTH IN PVAAS PVAAS CONSULTATION GUIDE PVAAS + Other Data Consultation 2013

8. Measuring Growth, or Progress Students don’t all start the year at the same place academically PVAAS + Other Data Consultation 2013

9. Measuring Growth or Progress Standard for PA Academic Growth – Regardless of their entering achievement level, students should not lose ground academically Is it not a reasonable goal to maintain the achievement level of a group of students? – Not necessarily increase their achievement level, but maintain their achievement level. PVAAS + Other Data Consultation 2013

10. PVAAS Growth Measure & Color Indicator The Growth Measure represents the amount of growth a group of students has made. PVAAS yields a Growth Measure AND a Growth Color Indicator. PVAAS + Other Data Consultation 2013 The Growth Color Indicator represents the level of evidence we have in the data to say whether or not the group of students increased, decreased, or maintained their achievement levels on average.

11. Prior Achievement for Group Most Recent School Year for Group Higher Achievement Concept of Measuring Growth in PVAAS – Grades 4-8 Math & Reading - Represents the mean achievement level of the group of students Lower Achievement Higher Achievement Lower Achievement PVAAS + Other Data Consultation 2013

12. Prior Achievement for Group Higher Achievement Concept of Measuring Growth in PVAAS Science, Writing, Keystones Lower Achievement Predicted Achievement Mean Predicted Achievement - Represents the mean achievement level of the group of students - Represents the mean predicted achievement level of the group of students PVAAS + Other Data Consultation 2013 Most Recent School Year for Group

13. Schools with Lower Achievement - Represents the mean achievement level of the group of students Is a “green” in PVAAS sufficient for moving students to higher levels of achievement? Higher Achievement Lower Achievement Prior Achievement for Group Most Recent School Year for Group Higher Achievement Lower Achievement PVAAS + Other Data Consultation 2013

14. (100% Advanced) Schools with High Achievement - Represents the mean achievement level of the group of students Can this group of high achieving students still meet the standard for PA Academic Growth? Higher Achievement Lower Achievement Prior Achievement for Group Most Recent School Year for Group (100% Advanced) PVAAS + Other Data Consultation 2013

15. Measuring Growth at a Deeper Level PVAAS does NOT use the percentages of students at various PSSA performance levels to measure growth. Instead, estimated growth measures for groups of students in PVAAS are in terms of NCE units (grades 4-8 Math & Reading) or scaled score points (Science, Writing, & Keystone Algebra 1 or Literature). PVAAS + Other Data Consultation 2013

16. Regardless of the entering achievement level of the group… Students should not lose ground academically A reasonable goal would be to at least maintain the achievement level of the group All groups of students can make progress or growth PVAAS + Other Data Consultation 2013

17. Practice! Turn to a partner Explain the concept of growth Previous School Year Current School Year Higher Achievement Lower Achievement Higher Achievement Lower Achievement C. G. – pg. 2 PVAAS + Other Data Consultation 2013

18. THE POWER OF TWO! CONSIDERING ACHIEVEMENT AND GROWTH TOGETHER! PVAAS + Other Data Consultation 2013

19. SCATTERPLOT QUADRANTS AND THEIR MEANINGS PVAAS + Other Data Consultation 2013 Quad 1 - Excelling Higher Achievement, Higher Growth Quad 2 - Slipping Higher Achievement, Lower Growth Quad 4 - Improving Lower Achievement, Higher Growth Quad 3 - Underperforming Lower Achievement, Lower Growth 1 2 3 4

20. WEB-BASED SCATTERPLOTS https://pvaas.sas.com PVAAS + Other Data Consultation 2013

21. QUESTIONS FOR ANALYSIS 1.Based upon the position of the school(s) in the district, is the system serving all students adequately in the chosen subject area? 2.How are the schools in the district performing in comparison to all schools in the state? C.G. Pg. 2 PVAAS + Other Data Consultation 2013 C. G. – pg. 3

22. PVAAS REPORTS https://pvaas.sas.com PVAAS + Other Data Consultation 2013

23. LOOKING BACK/EVALUATION PVAAS + Other Data Consultation 2013 What progress have groups of students made? Value-Added Reports & Performance Diagnostic Reports

24. THE PVAAS GROWTH LAUNCHPAD PVAAS + Other Data Consultation 2013

25. PVAAS GROWTH LAUNCHPAD GRAPHICS PVAAS + Other Data Consultation 2013

26. A GRADE/SUBJECT GRAPHIC REPORT PVAAS + Other Data Consultation 2013

27. WHAT DO THE COLORS MEAN? PVAAS + Other Data Consultation 2013 Average achievement level of the group increased ↑ 0 Maintained the average achievement level of the group ↓ Average achievement level of the group decreased Group exceeded their predicted achievement level Group met their predicted achievement level Group did not meet their predicted achievement level Math & Reading – Gr 4-8 Keystones Science & Writing

28. THE PVAAS GROWTH LAUNCHPAD PVAAS + Other Data Consultation 2013

29. THE TABULAR REPRESENTATION PVAAS + Other Data Consultation 2013

30. THE LEGEND OF THE LAUNCHPAD TABLE SHAPES AND COLORS COUNT! PVAAS + Other Data Consultation 2013

31. GRAPHICS AND TABLE CONNECTIONS PVAAS + Other Data Consultation 2013

32. OPTIONS FOR THE LAUNCHPAD Organized by Subject Organized by Grade PVAAS + Other Data Consultation 2013

33. FROM LAUNCHPAD TO REPORTS The Launchpad provides a visual and tabular overview of the data reports. We now consider the actual reports in detail PVAAS + Other Data Consultation 2013

34. QUESTIONS FOR ANALYSIS VALUE-ADDED REPORTS PVAAS + Other Data Consultation 2013 C. G. – pg. 4 1.Is there a consistent pattern of positive growth (blues and green) in each assessed PSSA grade level or Keystone content area? What grade levels are a strength/concern? 2.Is there a consistent pattern of positive growth (blues and green) in a subject area across state assessed subject/content areas? What subject areas are a strength/concern?

35. QUESTIONS FOR ANALYSIS PERFORMANCE DIAGNOSTIC REPORTS PVAAS + Other Data Consultation 2013 C. G. – pg. 4 1.Is there a consistent pattern of positive growth for all performance levels across PSSA grade levels or Keystone content areas? Which performance levels are a strength/concern? 2.Is there a consistent pattern of positive growth for all performance levels across assessed subject/content areas? Which performance levels are a strength/concern? 3.By viewing the proportion of students falling into each predicted performance level, what is the impact of the growth pattern for your students?

36. PERFORMANCE DIAGNOSTIC SUMMARY REPORT - SUBGROUPS Allows you to view growth patterns disaggregated by predicted PSSA performance levels (Below Basic, Basic, Proficient, and Advanced). C.G. Pg. 8 PVAAS + Other Data Consultation 2013 C. G. – pg. 5

37. PERFORMANCE DIAGNOSTIC SUMMARY REPORT - SUBGROUPS PVAAS + Other Data Consultation 2013 C. G. – pg. 5

38. QUESTIONS FOR A SUBGROUP 1.Is there a consistent pattern of positive growth for all performance levels for this subgroup across assessed PSSA grade levels or Keystone content areas? Which performance levels are a strength/concern? 2.Does the pattern of growth for the selected subgroup mirror the pattern of growth for the all student group at this grade or in this Keystone content area? Make note of any concerns. 3.What do these patterns suggest about the system’s success at meeting the needs of students at varying achievement levels within each subgroup? PVAAS + Other Data Consultation 2013 C. G. – pgs. 5-7

39. LOOKING FORWARD/PLANNING PVAAS + Other Data Consultation 2013 Projection Summary Reports Where are our students headed? C. G. – pg. 8

40. PVAAS PROJECTION REPORTING: PROJECTIONS TO PSSA (BASIC, PROFICIENT, ADVANCED) Grade PSSA Last Taken Math and Reading:  3  4 or 5  4  5 or 6  5  6 or 7  6  7, 8  7  8 Writing:  3  5  4  5  5  8  6  8  7  8 Science:  3  4  4  NA  5  8  6  8  7  8 Grade PSSA Last Taken Grade PSSA Last Taken Fall Overview 201340

41. PVAAS PROJECTION REPORTING: PROJECTIONS TO KEYSTONES (BASIC, PROFICIENT, ADVANCED) Fall Overview 201341 Anticipating projections to future Keystone exams will be available to students who last tested in grade 5 Remember, projections to a Keystone exam are NOT grade specific. The projection probability indicates the probability of a student reaching the indicated performance level on a future Keystone exam – not necessarily at a specific grade level.

42. HOW OFTEN DID STUDENTS, WITH PVAAS PROBABILITIES OF 70% OR HIGHER, SCORE AT THE PROFICIENT OR ADVANCED LEVEL ON THE NEXT PSSA? PVAAS + Other Data Consultation 2013

43. SCHOOL (SINGLE GRADE) PROJECTION SUMMARY REPORT This report provides educators with the probability of students scoring at or above a Proficient level on a future PSSA or Keystone Exam. PVAAS + Other Data Consultation 2013 C. G. – pg. 8

44. QUESTIONS 1.What percentage of students fall in the 70%-100% probability range of reaching proficiency at each assessed PSSA grade level and Keystone content area? What does this suggest about the degree to which the core program is meeting the needs of your students? 2.What percentage of students fall in the 40%-70% probability range of reaching proficiency at each assessed PSSA grade level and Keystone content area? What does this suggest about the needs for supports or interventions for students that are at risk of not reaching proficiency? What percentage of students fall in the 0-40% probability range of reaching proficiency at each assessed PSSA grade level and Keystone content area? What does this suggest about the need for intensive supports and interventions for students in this range? PVAAS + Other Data Consultation 2013 C. G. – pg. 8

45. SCHOOL (SINGLE GRADE) PROJECTION SUMMARY REPORT PVAAS + Other Data Consultation 2013 Navigate & Investigate C. G. – pg. 8

46. SCHOOL (SINGLE GRADE) PROJECTION SUMMARY REPORT - SUBGROUPS PVAAS + Other Data Consultation 2013 C. G. – pgs. 9-11 To Select Subgroups

47. QUESTIONS FOR A SUBGROUP 1.In what probability range do most of the students in this subgroup fall in each assessed PSSA grade level and Keystone content area? Greater than 70%, Between 40% to 70%, or Less than 40%? 2.What do the data suggest about the degree to which your core program is meeting the needs of the selected subgroup of students? 3.What interventions, supports, and/or services have been provided for these students? PVAAS + Other Data Consultation 2013 C. G. – pgs. 9-11

48. PSSA DATA INTERACTION – EMETRIC https://solutions1.emetric.net/pssa/ PVAAS + Other Data Consultation 2013 C. G. – pg. 12

49. EMETRIC Password-protected site PSSA Reports and Keystone Reports Grade level data Multiple years Student groups within grade levels Ethnicity Gender ED, IEP, LEP Title I PVAAS + Other Data Consultation 2013 C. G. – pg. 12

50. PSSA OR KEYSTONE EXAMS? PVAAS + Other Data Consultation 2013

51. EMETRIC FOR PSSA, PASA, PSSA-M PVAAS + Other Data Consultation 2013 C. G. – pg. 12

52. GRAPHICAL SUMMARY: PERFORMANCE LEVELS Provides summary reports in the form of graphs, including bar graphs, pie charts, histograms, or line graphs. Allows for selection of subject areas, demographic variables, multiple grades, and multiple years. PVAAS + Other Data Consultation 2013 C. G. – pg. 12

53. Navigate & investigate your Graphical Summary: Performance Levels report for each grade level and subgroup and answer the questions in the guidebook. Graphical Summary: Performance Levels PVAAS + Other Data Consultation 2013 C. G. – pg. 12

54. QUESTIONS 1.Over time, is the percentage of students scoring in the proficient and advanced performance levels increasing in each subject area? Make note of any concerns. 2.Over time, is the percentage of students in relevant subgroups (n≥ 40) scoring in the proficient and advanced performance levels increasing in each subject area? Make note of any concerns. 3.Does the pattern of performance for the selected subgroup mirror the pattern of performance for the grade as a whole in each subject area? C. G. – pg. 12

55. GRAPHICAL SUMMARY: REPORTING CATEGORIES Provides a bar graph which displays the percentage of points earned for each reporting category. Allows for selection of different subject areas and demographic variables at each grade level. PVAAS + Other Data Consultation 2013 C. G. – pg. 13

56. Navigate & investigate your Graphical Summary: Reporting Category report for each grade level and subgroup and answer the questions in the Consultation Guide. GRAPHICAL SUMMARY: REPORTING CATEGORIES PVAAS + Other Data Consultation 2013 C. G. – pg. 13

57. QUESTIONS 1.At each grade level, did all students perform satisfactorily (at least 60% correct) in all reporting categories in the targeted subject area? 2.Does the pattern of performance across reporting categories for the selected subgroups mirror the pattern of performance across reporting categories for the grade as a whole? PVAAS + Other Data Consultation 2013 C. G. – pg. 13

58. MATCHED COMPARISON Compares PSSA performance levels for two years to determine if students have moved to higher PSSA performance levels in the subject area this past year. PVAAS + Other Data Consultation 2013 C. G. – pg. 14

59. MATCHED COMPARISON PVAAS + Other Data Consultation 2013 Navigate & Investigate

60. QUESTIONS 1.In each grade level, have more students moved to higher PSSA performance levels, remained in the same performance levels, or moved to lower performance levels? 2.How do these results impact your school’s achievement goals? PVAAS + Other Data Consultation 2013 C. G. – pg. 14

61. EMETRIC FOR KEYSTONE EXAMS PVAAS + Other Data Consultation 2013 C. G. – pg. 15

62. THE KEYSTONE REPORT MENU PVAAS + Other Data Consultation 2013 The three reports listed on the left of the menu focus on a)Performance levels – both in table form and in graphical form; and b)Assessment anchors – in table form. C. G. – pg. 15

63. GROUP SUMMARY: PERFORMANCE LEVELS PVAAS + Other Data Consultation 2013 Data Reported: School name Administration Number Tested Percentage of students in each performance category C. G. – pg. 15

64. QUESTIONS 1.What percentage of students scored in the proficient and advanced performance levels in each Keystone content area? 2.What percentage of students scored in the proficient and advanced performance levels for every relevant subgroup (n≥ 40) in each Keystone content area? Make note of any concerns. 3.Does the pattern of performance for the selected subgroups mirror the pattern of performance for the school as a whole? C. G. – pg. 15

65. THE KEYSTONE REPORT MENU PVAAS + Other Data Consultation 2013 Next: the Group Summary: Assessment Anchors C. G. – pg. 16

66. GROUP SUMMARY: ASSESSMENT ANCHORS PVAAS + Other Data Consultation 2013 Data Reported: School names Number Tested Percentage of students above the passing mark for each assessment anchors C. G. – pg. 16

67. QUESTIONS 1.For each Keystone content area, did all students perform satisfactorily in all anchor areas for the targeted subject area? 2.Does the pattern of performance across the anchors for the selected subgroups mirror the pattern of performance across the anchors for the entire group as a whole? PVAAS + Other Data Consultation 2013 C. G. – pg. 16

68. SUMMARY OF ESSENTIAL FINDINGS Based on your system’s achievement goals and analysis of your PVAAS data, how would you answer the two main questions? Is the system (school/district) serving all students adequately? Is the system (school/district) serving all students equitably? List three findings from today’s session for each subject area. PVAAS + Other Data Consultation 2013 C. G. – pg. 17

69. NEXT STEPS What about other data? What other data sources are available for staff? Benchmark Assessments (4-Sight, MAPS, Acuity) CDT – Classroom Diagnostic Tool DIBELS, AIMSweb, G-MADE, GRADE, Study Island, STAR District/School Common Assessments Who has access to these data? How often is it shared and discussed? PVAAS + Other Data Consultation 2013 C. G. – pg. 18

70. WHY ARE WE GETTING THESE RESULTS? PVAAS + Other Data Consultation 2013

71. QUESTIONS: PVAAS MATERIALS OR STATEWIDE IMPLEMENTATION PDEPVAAS@IU13.ORG 717-606-1911 PVAAS REPORT WEB SITE HTTPS://PVAAS.SAS.COM

72. www.pde.state.pa.us PVAAS + Other Data Consultation 2013