1. PVAAS School Consultation Fall 2011 PVAAS Statewide Core Team pdepvaas@iu13.org

2. Purpose of Consultation Goal: To model a process for the inspection, analysis and interpretation of your 2011 data Focus: PAAYP, eMetric, and PVAAS Data for Reading and Math This process is designed to give a school the “30,000 foot view” of the status of achievement and growth –Whole school data –Grade level data –Subgroup data Consultation Guide Pg. 1 2PVAAS 2011 Consultation Guide9/19/2011

3. Two key questions to keep in mind as you examine your data! Based on the system’s prior year achievement goals/targets: –Is the system (school/district) serving all students adequately? –Is the system (school/district) serving all students equitably? Purpose of Consultation 3PVAAS 2011 Consultation Guide9/19/2011

4. Agenda The Power of Two: Web-Based Scatter Plots……15 min PA AYP Report………………………………………..5 min PSSA Data Interaction Reports, eMetric ………...20 min –Performance Level Summary, Reporting Categories Summary, Matched Comparison Report PVAAS Reports………………………………………35 min –Value-Added Reports, Performance Diagnostic Reports, School Projection Summary Summary and Next Steps………………………….15 min 4PVAAS 2011 Consultation Guide9/19/2011

5. The Power of Two! Considering Achievement and Growth TOGETHER! C.G. Pg. 2

6. Quad 2 - Slipping Higher Achievement, Lower Growth Scatter Plot Quadrants and Their Meanings August 20116 Quad 1 - Excelling Higher Achievement, Higher Growth Quad 4 - Improving Lower Achievement, Higher Growth Quad 3 - Underperforming Lower Achievement, Lower Growth C.G. Pg. 2

7. NEW! Web-Based Scatter Plots 7 C.G. Pg. 2 https://pvaas.sas.com PVAAS 2011 Consultation Guide9/19/2011

8. Questions For Later Analysis 1.Based upon the position of the schools in district, is the system serving all students adequately in the chosen subject area? 2.How are the schools in the district performing in comparison to all the schools in the state? C.G. Pg. 2 8PVAAS 2011 Consultation Guide9/19/2011

9. Consultation Guide Decide on focus for your team to make the most of the time: –What subject you/others will focus on – Reading or Math? –What grade level(s) within the subject? –What subgroup within the grade and subject? Purpose is to model a process that can be used to determine the status of achievement & growth in your school. The next step is to begin to investigate WHY you are getting these results. C.G. Pg. 2 9PVAAS 2011 Consultation Guide9/19/2011

10. PA AYP http://paayp.emetric.net/ C.G. Pg. 3 10PVAAS 2011 Consultation Guide9/19/2011

11. PA AYP Report Public Site Has a school met the AYP targets? –Academic Performance in Reading & Math –Test Participation in Reading & Math –Attendance or Graduation Which student groups count for AYP purposes? – A total of 40 or more students in the tested grades in the school C.G. Pg. 3 11PVAAS 2011 Consultation Guide9/19/2011

12. PAAYP Data Table Navigate & investigate your PAAYP Data Table and answer the questions in your guidebook. 2011 AYP Targets Reading – 72% Math – 67% C.G. Pg. 3 12PVAAS 2011 Consultation Guide9/19/2011

13. Questions 1.Did your school make AYP for performance for the whole school and all measurable subgroups? (72% Reading, 67% Math) 2.Did your school make AYP for participation for the whole school and all measurable subgroups? 3.Did the school make AYP for Attendance or Graduation? C.G. Pg. 3 13PVAAS 2011 Consultation Guide9/19/2011

14. PSSA Data Interaction – eMetric https://solutions1.emetric.net/pssa/ C.G. Pg. 4 14PVAAS 2011 Consultation Guide9/19/2011

15. eMetric Password-protected site Grade level data –Multiple years Student groups within grade levels –Ethnicity –Gender –ED, IEP, LEP –Title I C.G. Pg. 4 15PVAAS 2011 Consultation Guide9/19/2011

16. eMetric C.G. Pg. 4 16PVAAS 2011 Consultation Guide9/19/2011

17. Graphical Summary: Performance Levels Provides summary reports in the form of graphs, including bar graphs, pie charts, histograms, or line graphs. Allows for selection of subject areas, demographic variables, multiple grades, and multiple years. C.G. Pg. 4 17PVAAS 2011 Consultation Guide9/19/2011

18. Navigate & investigate your Performance Levels Summary for each grade level and subgroup and answer the questions in the guidebook. Graphical Summary: Performance Levels C.G. Pg. 4 18PVAAS 2011 Consultation Guide9/19/2011

19. Questions 1.Did every grade meet the AYP performance targets for reading and math? (72% Reading, 67% Math) 2.Did every relevant subgroup within each grade meet the AYP performance targets for reading and math? (72% Reading, 67% Math) 3.Does the pattern of performance for the selected subgroup mirror the pattern of performance for the grade as a whole? C.G. Pg. 4 19PVAAS 2011 Consultation Guide9/19/2011

20. Graphical Summary: Reporting Categories Provides a bar graphs which display the percentage of score points earned for each reporting category. Allows for selection of different subject areas and demographic variables at each grade level. C.G. Pg. 5 20PVAAS 2011 Consultation Guide9/19/2011

21. C.G. Pg. 5 Navigate & investigate your Reporting Category Summary for each grade level and subgroup and answer the questions in the guidebook. Graphical Summary: Reporting Categories 21PVAAS 2011 Consultation Guide9/19/2011

22. Questions 1.At each grade level, did all students perform satisfactorily (at least 60% correct) in all reporting categories in the targeted subject area? 2.Does the pattern of performance across reporting categories for the selected subgroups mirror the pattern of performance across reporting categories for the grade as a whole? C.G. Pg. 5 22PVAAS 2011 Consultation Guide9/19/2011

23. Matched Comparison Compares PSSA performance levels for two years to determine if students have moved to higher PSSA performance levels in the subject area this past year. C.G. Pg. 6 23PVAAS 2011 Consultation Guide9/19/2011

24. Matched Comparison Navigate & investigate your Matched Comparison reports for each grade levels and answer the questions in the guidebook. C.G. Pg. 6 24PVAAS 2011 Consultation Guide9/19/2011

25. Questions For Later Analysis 1.In each grade level, have more students moved to higher PSSA performance levels, remained in the same performance levels, or moved to lower performance levels? 2.How do these results impact your school’s achievement goals? C.G. Pg. 6 25PVAAS 2011 Consultation Guide9/19/2011

26. PVAAS Reports C.G. Pg. 7 https://pvaas.sas.com 26PVAAS 2011 Consultation Guide9/19/2011

27. TWO Types of PVAAS Information Today Looking Back/Evaluation… Value-added Growth Reports For Groups of Students Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Groups of Students C.G. Pg. 7 27PVAAS 2011 Consultation Guide9/19/2011

28. School Value-Added Reports These reports provide information on how effective a school was in supporting groups of students to at least meet the standard for PA Academic Growth. C.G. Pg. 7 28PVAAS 2011 Consultation Guide9/19/2011

29. NEW! NEW! Legends and Color Codes Alignment of colors and interpretative legends across all Value-Added Reports (all subjects and grade levels) –New 5 color-coding scheme Same for Reporting 4-8 and 9-11! –All legends are consistent in color and language –Use of the data becomes easier as there is no confusion about what the colors indicate New 5 Color Codes –Allow for more differentiation among schools exceeding the standard for PA Academic Growth –Allow for more differentiation in Science and Writing reporting, as well as grades 9-11 Reading and Math reporting 29PVAAS 2011 Consultation Guide9/19/2011

30. 30PVAAS 2011 Consultation Guide9/19/2011

31. NEW! NEW! PVAAS Value-added Growth Descriptors 31PVAAS 2011 Consultation Guide9/19/2011

32. Meeting the standard for PA Academic Growth Growth Measure= 0 Grades 4-8 - A group meets the standard for PA Academic Growth when they maintain their relative achievement level from one year to the next in relation to the statewide distribution (2006 baseline). Grades 9-11 – A group meets the standard for PA Academic Growth when their actual achievement meets their predicted achievement- based on the average schooling experience. 32PVAAS 2011 Consultation Guide9/19/2011

33. School Value-Added Reports Navigate & investigate your Value-Added Reports and answer the questions in your guidebook. C.G. Pg. 7 33PVAAS 2011 Consultation Guide9/19/2011

34. Questions 1.Are the students in your school making the growth needed to meet the school’s/districts achievement goals/targets? 2.Describe the pattern of growth within each grade over the last three years. 3.What patterns do you see when examining the entering achievement level of students at each grade level over the past years (looking at achievement vertically for the grade)? 4.What do the Growth Measures tell you about the system’s success at meeting the needs of students at their achievement level? C.G. Pg. 7 34PVAAS 2011 Consultation Guide9/19/2011

35. Performance Diagnostic Report The Performance Diagnostic Report allows you to view growth patterns disaggregated by predicted PSSA performance levels (Below Basic, Basic, Proficient, and Advanced). C.G. Pg. 8 35PVAAS 2011 Consultation Guide9/19/2011

36. Interpretation of the Performance Diagnostic Report Zero (0) Line Growth Met standard for PA Academic Growth Green Less Than standard for PA Academic Growth Pink Exceeded standard for PA Academic Growth Blue What the whiskers tell us… C.G. Pg. 8 36PVAAS 2011 Consultation Guide9/19/2011

37. WB P. 35 A B C 37 Patterns of Growth C.G. Pg. 8 PVAAS 2011 Consultation Guide9/19/2011

38. Performance Diagnostic Report Navigate & investigate your Performance Diagnostic Reports for each grade level and complete the questions in your guidebook. C.G. Pg. 8 38PVAAS 2011 Consultation Guide9/19/2011

39. Questions 1.How does the growth pattern for the current year (blue bars) compare to the growth pattern for previous years (gold bars) in same grade and subject? 2.What patterns of growth do you see across grades in each subject area? 3.What do these patterns suggest about the system’s success at meeting the needs of students at varying achievement levels? 4.What might these patterns suggest about the system’s curriculum, instruction, materials & resources, or interventions? C.G. Pg. 8 39PVAAS 2011 Consultation Guide9/19/2011

40. Performance Diagnostic Report - Subgroups Navigate & investigate your Performance Diagnostic Reports by subgroup for each grade level. Complete the table and answer the questions in your guidebook. C.G. Pg. 9 40PVAAS 2011 Consultation Guide9/19/2011

41. Questions for a Subgroup 1.What patterns of growth do you see for the selected subgroup across grades in each subject area? 2.Does the pattern of growth for the selected subgroup mirror the pattern of growth for the grade as a whole? Make note of any concerns. 3.What do these patterns suggest about the system’s success at meeting the needs of students at varying achievement levels within each subgroup? C.G. Pg. 9 41PVAAS 2011 Consultation Guide9/19/2011

42. TWO Types of PVAAS Information Today Looking Back/Evaluation… Value-added Growth Reports For Groups of Students Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Groups of Students 42PVAAS 2011 Consultation Guide9/19/2011

43. Projections to Proficient or Advanced 43 Grade PSSA Last Taken 3  4 or 5 4  5 or 6 5  6 or 7 6  7 or 8 7  8 8  11 Project to these Grades PVAAS 2011 Consultation Guide9/19/2011

44. School (Single Grade) Projection Summary Report This report provides educators with the probability of students scoring at or above a Proficient level on a future PSSA. C.G. Pg. 10 44PVAAS 2011 Consultation Guide9/19/2011

45. School (Single Grade) Projection Summary Report Navigate & investigate your Single Grade Projection Summary Reports for each tested grade and answer the questions in your guidebook. C.G. Pg. 10 45PVAAS 2011 Consultation Guide9/19/2011

46. Questions 1.What percentage of students fall in 70%-100% probability range of reaching proficiency? What does this suggest about the degree to which the core program is meeting the needs of the majority of your students? 2.What percentage of students fall in the 40%-70% probability range of reaching proficiency? What does this suggest about the needs for supports or interventions for students that are at risk of not reaching proficiency? 3.What percentage of student fall in the 0- 40% probability range of reaching proficiency? What does this suggest about the need for intensive supports and interventions for students in this range? C.G. Pg. 10 46PVAAS 2011 Consultation Guide9/19/2011

47. School (Single Grade) Projection Summary Report - Subgroups Navigate & investigate your Single Grade Projection Summary Reports by Subgroup for each tested grade and answer the questions in your guidebook. C.G. Pg. 11 47PVAAS 2011 Consultation Guide9/19/2011

48. Questions 1.In what probability range do most of the students in this subgroup fall in each tested grade? Greater than 70%, Between 40% to 70%, or Less than 40%? 2.What do the data suggest about the degree to which your core program is meeting the needs of the selected subgroup of students? 3.What interventions, supports, and/or services have been provided for these students? C.G. Pg. 11 48PVAAS 2011 Consultation Guide9/19/2011

49. How do I know that the projections are accurate? The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office). The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study. (http://www.ed.gov/admins/lead/account/growthmodel/pa/inde x.html)http://www.ed.gov/admins/lead/account/growthmodel/pa/inde x.html Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score. 49PVAAS 2011 Consultation Guide9/19/2011

50. Summary of Essential Findings Based on your system’s achievement goals/target, how would you answer the two main questions? –Is the system (school/district) serving all students adequately? –Is the system (school/district) serving all students equitably? List three findings from today’s session under the subject area you investigated to share with the group. C.G. Pg. 12 50PVAAS 2011 Consultation Guide9/19/2011

51. Next Steps Share the findings from this session with staff Or, even better…. Replicate the process with staff –Whole school – half focusing on math; half focusing on reading –Grade level teams – focusing on math and reading for their grade only –Departmental – focusing on subject area C.G. Pg. 13-14 51PVAAS 2011 Consultation Guide9/19/2011

52. What about other data? What other data sources are available for staff? –Benchmark Assessments (4-Sight, MAPS, Acuity) –CDT – Classroom Diagnostic Tool –DIBELS, AIMSweb, G-MADE, GRADE, Study Island, STAR –District/School Common Assessments Who has access to these data? How often is it shared? C.G. Pg. 13 52PVAAS 2011 Consultation Guide9/19/2011

53. Why are we getting these results? C.G. Pg. 13-14 53PVAAS 2011 Consultation Guide9/19/2011

54. How can we support you in the future? Discuss with your team what types of support you might find useful from your IU PVAAS contact(s), as well as any other resources or materials needed from the PDE PVAAS Statewide Team. Detach and return page 19 of the Consultation Guide to your session facilitator. C.G. Pg. 19 54PVAAS 2011 Consultation Guide9/19/2011

55. Questions: PVAAS Materials or Statewide Implementation pdepvaas@iu13.org 717-606-1911 PVAAS Report Web Site https://pvaas.sas.com 55PVAAS 2011 Consultation Guide9/19/2011

56. www.pde.state.pa.us 56 PVAAS 2011 Consultation Guide 9/19/2011