PVAAS Statewide Core Team

PVAAS Statewide Core Team pdepvaas@iu13.org
11/28/2018 PVAAS School Consultation Guide Fall 2010 Session B: School Level – All Data Tools PVAAS Statewide Core Team

PVAAS 2010 Consultation Guide - Session B
Agenda Purpose of the Consultation The Power of Two The Guidebook PA AYP Data Table Report PSSA Data Interactive Reports – Getting Results Data Packet PVAAS Reports Student Search and Projection Reports Next Steps Future Considerations Questions 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Purpose of Consultation
11/28/2018 Consultation Guide Pg. 1 Purpose of Consultation Focus: Multiple Source Data Reports for Math and Reading in grades 4 through 8 including PAAYP, eMetric, and PVAAS. Intended to facilitate the inspection, analysis and interpretation of your 2010 data. To inform your investigation into root causes and assist in developing action plans to increase student achievement. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Considering Achievement and Growth TOGETHER!
11/28/2018 The Power of Two! Considering Achievement and Growth TOGETHER! Before we look at your actual eMetric, AYP and PVAAS data, we are going to start by considering the concepts of achievement and growth together.

The Scatter Plot Coordinate System
11/28/2018 The Scatter Plot Coordinate System Vertical Axis PSSA Percent Proficient or Advanced Growth Standard Boundary AYP Percent Proficient Target On this screen you should see a blank scatter plot. [Click] The Horizontal Axis is for displaying the PVAAS Growth Measure. A Growth Measure value of 0 means that the cohort of students has met the Growth Standard for its grade and has maintained its position in the distribution of all scores in that grade across the Commonwealth relative to the previous year using the base year 2006 distributions. In other words, the Growth Standard value of 0 means the cohort of students made one year’s growth. We have highlighted the Growth Standard boundary of 0. If Growth is positive, that is to the right of 0, then we have an indication that the cohort exceeded the Growth Standard and has increased its position in the statewide distribution of scores (i.e., the cohort of students made more than one year’s growth). Similarly, if the Growth value is negative, then we have an indication that the growth of the cohort was below the Growth Standard and the position of the cohort in the statewide distribution of scores has decreased during the recently completed academic year (i.e., the cohort of students made less than one year’s growth). The vertical axis displays the percent Proficient or Advanced for each cohort. The AYP Proficiency Target for Math is currently 56% - we have highlighted that target in the graph. Note that the Target for Reading is different and that all targets will increase as we approach the % proficiency deadline set by NCLB. Each of the regions isolated by the AYP Target level and the PVAAS Growth Standard splits the scatter plot into 4 quadrants that we number just as you did in Algebra 1! Horizontal Axis PVAAS Growth Measure 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Scatter Plot Quadrants and Their Meanings
11/28/2018 Scatter Plot Quadrants and Their Meanings Quad 2 Adequate Achievement Negative Growth Quad 1 Adequate Achievement Positive Growth Quad 4 Below Adequate Achievement Positive Growth We will now discuss the interpretations of the locations of schools in each quadrant. [Click] If a school has their dot in the first quadrant, indicated by Green in the graphic, it means that they have accomplished adequate achievement and positive growth. The farther the dot for the school is away from the intersection of the boundaries the higher the achievement and/or the greater the evidence of growth. If a school has its dot in the second quadrant, displayed in yellow, this indicates that they have adequate achievement but negative growth. This negative growth should be of great concern since continued adequate achievement may be in jeopardy. If a school has its dot in the fourth quadrant displayed in blue, this indicates the school has not achieved at or above the AYP target level; however, their growth is positive. This situation may indicate that interventions implemented are producing positive results. If a school has its dot in the third quadrant colored red, this indicates that the school has both below adequate achievement and negative growth. This situation suggests that substantial investigation and remediation is necessary since the students are not achieving and are headed the wrong direction. Quad 3 Below Adequate Achievement Negative Growth 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Statewide to IU Region to District Grade 4 Math
11/28/2018 Statewide to IU Region to District Grade 4 Math Statewide Grade 4 Math IU XX On this slide, you see scatter plots corresponding to Grade 4 Math for the State and the IU region. Comment about each plot regarding: Growth, Achievement, and The combination of Growth and Achievement Finally, comment on the comparison of the plots. [Click] 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Let’s Look At Your Scatterplots!
11/28/2018 Let’s Look At Your Scatterplots! Display the district ppt presentation showing state, IU, and district plots for this district or provide to the team in hard copy for the session. If actual scatter plots are not available move on to next slide. Encourage participants to discuss each plot, as well as the comparison of the 3 plots. [Click]

Instructions – See Guide Book
11/28/2018 Instructions – See Guide Book Session focuses on Math and Reading, Grades 4 through 8. Options: Divide the team with half focusing on math, half focusing on reading Choose to focus on one subject during the session and complete other subjects at a later time. About your inspection, analysis and interpretation of YOUR data! Purpose is to use the data to explore root causes and move to action planning to enhance student achievement. Remind the participants of the information on this slide. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

PA AYP Report http://paayp.emetric.net/
11/28/2018 PA AYP Report We will now examine your PA AYP Report. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 1 – PAAYP Data Table
11/28/2018 C.G. Pg. 2 Report # 1 – PAAYP Data Table Guiding Statement: At least (63% Reading, 56% Math) of the students in the school were Advanced or Proficient in the targeted subject area. The school met the AYP target for participation in the targeted subject area for all relevant student groups. Assist the team to examine their own data and answer the questions for this report. You should encourage the participants to document their responses in their guide books. Questions Did your school make AYP for performance for the whole school and all measurable subgroups? Make note of any provisions used to meet AYP and/or any concerns. Did your school make AYP for participation for the whole school and all measurable subgroups? Make note of any concerns. What does this report suggest as possible foci for further investigation and discussion? Investigate your PAAYP Data Table and answer the questions in your guidebook. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 Getting Results Data Packet PSSA Data Interactive – eMetric Now we will examine your Getting Results Data Packet which is available on the Home Page from PSSA Data Interactive (eMetric). 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 2 – Getting Results Data Packet
11/28/2018 C.G. Pg. 3 Report # 2 – Getting Results Data Packet Guiding Statement: Each grade level, as well as each subgroup at each grade level, in the school met or exceeded the NCLB READING/MATH target (63% Reading, 56% Math) of Proficient or Advanced. Similarly, here are the questions for the Getting Results Data Packet: Most Recent PSSA Results. Questions Did every grade in your school make AYP for performance for all students and all measurable subgroups? Make note of any concerns. What does this report suggest as possible foci for further investigation and discussion? Investigate your Most Recent PSSA Results Report for each grade level and subgroup and answer the questions in the guidebook. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 C.G. Pg. 4 Report # 3 – Getting Results Data Packet Guiding Statement: Every grade level trend for the past 3 years for all students and all subgroups reflects an increase each year in the percent of students Proficient or Advanced in the targeted subject area. Questions Did every grade in your school show a positive 3-year trend for all students and all measurable subgroups? Make note of any concerns. What does this report suggest as possible foci for further investigation and discussion? Investigate your Multi-year Report for each grade level and subgroup and answer the questions in your guidebook. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 C.G. Pg. 5 Report # 4 – Getting Results Data Packet Guiding Statement: All students and all measurable subgroups are indicating consistently good performance (at least 60% correct) in each of the reporting categories in the targeted subject area. Questions At each grade level, did all students and all measurable subgroups perform satisfactorily (at least 60% correct) in all reporting categories in the targeted subject area? Make note of any concerns. Did all students and all measurable subgroups perform consistently in all reporting categories in the targeted subject area? That is, are the levels of performance of all students in each of the reporting categories relatively the same as the levels of performance for students in all subgroups? Make note of any concerns. What does this report suggest as possible foci for further investigation and discussion? Investigate your Reporting Categories Report at each grade level and subgroup and answer the questions in your guidebook. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 5 – Matched Comparison
11/28/2018 C.G. Pg. 6 Report # 5 – Matched Comparison Guiding Statement: All grade levels moved more students to higher PSSA performance levels in the targeted subject area this past year. Questions Did all grade levels move more students to higher PSSA performance levels in the targeted subject area this past year? Make note of any concerns. Do these data confirm your hypothesis of how your students performed last year? What does this report suggest as possible foci for further investigation and discussion? Investigate your Matched Comparison Report and answer the questions in your guidebook. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

PVAAS Reports https://pvaas.sas.com
11/28/2018 C.G. Pg. 6 PVAAS Reports Finally, let us consider your PVAAS reports… 9/15/2010 PVAAS 2010 Consultation Guide - Session B

TWO Types of PVAAS Information
11/28/2018 C.G. Pg. 6 TWO Types of PVAAS Information Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Cohorts of Students Looking Back/Evaluation… Value-added Growth Reports For Cohorts of Students Facilitator says: A quick review of the two types of PVAAS information: The two PVAAS methodologies really look at two different issues. [CLICK] The Value-added, or growth, information “looks back”…it helps schools to evaluate the effectiveness of the school. How much growth did students make in the past school year? The Projection information “looks forward”…it helps schools plan for the future. Are students on a path to proficiency or higher? Both serve different purposes, and both are equally important for continuous school improvement. Again – it is important to be clear - PVAAS does not provide a growth estimate for individual students. We will share why in the next section. Today PVAAS does NOT calculate growth of an individual student. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Legend for Growth Methodology
11/28/2018 Legend for Growth Methodology The PA Growth Standard specifies the minimal acceptable academic gain from grade to grade for a cohort of students. The Growth Standard is met when the students in a school or district on average maintain their relative position when compared to the 2006 statewide distribution of student scores (2006 is the base year). Estimated mean NCE gain equal or greater than zero. There is evidence that the school has met or exceeded the Growth Standard. All schools can achieve this rating. Estimated mean NCE gain less than zero by one standard error or less. There is slight evidence that the school has not met the Growth Standard. Estimated mean NCE gain less than zero by between one and two standard errors. There is moderate evidence that the school has not met the Growth Standard. Estimated mean NCE gain more than two standard errors less than zero. There is significant evidence that the school has not met the Growth Standard. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 6 – School Value-added
11/28/2018 C.G. Pg. 7 Report # 6 – School Value-added Guiding Statement: All grades in your school met or exceeded the Growth Standard in Reading/Math. Questions In what grades do you see positive growth for all students in your school? Has the pattern of growth for these grades been consistently positive over the last three reporting years? In what grades do you see negative growth for all students in your school? Has the pattern of growth for these grades been consistently negative over the last three reporting years? What questions does this report suggest to you for further investigation? Investigate your School Value-added Report and answer the questions in your guidebook. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 7 – Performance Diagnostic
11/28/2018 C.G. Pg. 8 Report # 7 – Performance Diagnostic Guiding Statement: Every predicted PSSA performance category met or exceeded the Growth Standard in Reading/Math. For this report, the team should complete the table in the Consultation Guide and use the questions to search for patterns across grades and subjects. Questions In what grades do you see positive growth for students in the majority of the predicted PSSA performance categories? In what grades do you see negative growth for students in the majority of the predicted PSSA performance categories? What do these reports suggest as possible foci for further investigation and discussion? Investigate your Performance Diagnostic Reports for each grade level. Complete the table and answer the questions in your guidebook. There are multiple reports available depending on the grade levels tested in your school. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Interpretation of the Performance Diagnostic Report
11/28/2018 Interpretation of the Performance Diagnostic Report + ─ 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 Patterns of Growth A C B Facilitator says: The POWER of the PVAAS Performance Diagnostic Reports are the PATTERNS that they can display about your school/district. These patterns serve as an indicator of where you are headed in the right direction in terms of growing students and where you may have areas for improvement. The meaning of the pattern of growth should be viewed through the context of the achievement goals of your school. What are your school’s goals for achievement? And, as a result where do you need to make more than a year’s growth or make at least a year’s growth? This slide displays some common patterns that we see in Performance Diagnostic reports: A: Tent Pattern This pattern occurs when school learning experiences benefit students in the middle of its population more than their lower-achieving or higher achieving peers. This trend can be reversed by attending to students at the extremes of the grade level’s population. [CLICK] B: Uniform Pattern This pattern occurs when school learning experiences benefit all students in approximately the same way. In this example, all students in all predicted PSSA performance levels demonstrated positive and significant growth. A uniform pattern with all bars below the green Zero line would obviously indicate that none of the learners demonstrated positive growth. C: Downward Shed Pattern This pattern occurs when school learning experiences benefit lower achieving students more than their higher-achieving peers. For many reasons, the system is not appropriately addressing the needs of all students, but rather has a focus on the middle- and lower- achieving students. 9/15/2010 PVAAS 2010 Consultation Guide - Session B 23

Report # 7 – Performance Diagnostic
11/28/2018 C.G. Pg. 8 Report # 7 – Performance Diagnostic Guiding Statement: Every predicted PSSA performance category met or exceeded the Growth Standard in Reading/Math. For this report, the team should complete the table in the Consultation Guide and use the questions to search for patterns across grades and subjects. Questions In what grades do you see positive growth for students in the majority of the predicted PSSA performance categories? In what grades do you see negative growth for students in the majority of the predicted PSSA performance categories? What do these reports suggest as possible foci for further investigation and discussion? Investigate your Performance Diagnostic Reports for each grade level. Complete the table and answer the questions in your guidebook. There are multiple reports available depending on the grade levels tested in your school. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 8 – Performance Diagnostic - Subgroups
C.G. Pg. 9-11 11/28/2018 Report # 8 – Performance Diagnostic - Subgroups Guiding Statement: Every predicted PSSA performance category for this subgroup met or exceeded the Growth Standard in Reading/Math. The Consultation Guide contains sheets for up to 3 subgroup Performance Diagnostic investigations. In each case, we recommend that teams complete the table for the grade levels and subgroups of interest (if not all) and answer the following questions: Questions In what grades do you see positive growth for students in the majority of the predicted PSSA performance categories? In what grades do you see negative growth for students in the majority of the predicted PSSA performance categories? What do these reports suggest as possible foci for further investigation and discussion? Investigate your Performance Diagnostic Reports by subgroup for each grade level. Complete the table and answer the questions in your guidebook. There are multiple reports available depending on the grade levels tested in your school and the appropriate subgroups based on your student population. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

TWO Types of PVAAS Information
7/7/10 TWO Types of PVAAS Information Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Cohorts of Students Looking Back/Evaluation… Value-added Growth Reports For Cohorts of Students Facilitator says: A quick review of the two types of PVAAS information: The two PVAAS methodologies really look at two different issues. [CLICK] The Value-added, or growth, information “looks back”…it helps schools to evaluate the effectiveness of the school. How much growth did students make in the past school year? The Projection information “looks forward”…it helps schools plan for the future. Are students on a path to proficiency or higher? Both serve different purposes, and both are equally important for continuous school improvement. Again – it is important to be clear - PVAAS does not provide a growth estimate for individual students. We will share why in the next section. Today PVAAS does NOT calculate growth of an individual student. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 9 – School (Single Grade) Projection Summary Report
11/28/2018 C.G. Pg. 12 Report # 9 – School (Single Grade) Projection Summary Report Guiding Statement: Each tested grade has at least (SY10-11 AYP targets: 72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level. Questions: (Note that 72% and 67% are the 2011 AYP targets for Reading and Math, respectively.) In what tested grades do you see at least (72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level? In what tested grades do you NOT see at least (72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level? What do these reports suggest as possible foci for further investigation and discussion? Investigate your Single Grade Projection Summary Reports for each grade level tested and answer the questions in your guidebook. There are multiple reports available depending on the grade levels tested in your school. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 C.G. Pg Report # 10 – School (Single Grade) Projection Summary Report - Subgroups Guiding Statement: Each tested grade has at least (SY10-11 AYP targets: 72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level. Questions: (Note that 72% and 67% are the 2011 AYP targets for Reading and Math, respectively.) In what tested grades do you see at least (72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level? In what tested grades do you NOT see at least (72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level? What do these reports suggest as possible foci for further investigation and discussion? Investigate your Single Grade Projection Summary Reports for each grade level tested and answer the questions in your guidebook. There are multiple reports available depending on the grade levels tested in your school. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

See Data Tool Reference Guide
11/28/2018 See Data Tool Reference Guide Examples & Non-Examples of Effective Use of PVAAS Student Projection Reports! Instruct participants to reference the back-to-back reference document titled Effective Use of Student Projection Reports: Examples and Non-Examples that is located in the back of the data tool reference guide document. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Report # 10 – Projection Summary (Single Grade) - Subgroups
11/28/2018 C.G. Pg Report # 10 – Projection Summary (Single Grade) - Subgroups Guiding Statement: Each tested grade has at least (72% Reading, 67% Math) of the students in this subgroup in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level. The Guide includes sheets for up to 3 subgroup Projection Summary reports. The same procedure should be followed throughout. Questions In what tested grades do you see at least (72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level? In what tested grades do you NOT see at least (72% Reading, 67% Math) of its students in the 70% -100% probability range of reaching proficiency in the targeted subject area at the next tested grade level? What do these reports suggest as possible foci for further investigation and discussion? Investigate your Single Grade Projection Summary Reports by subgroup for each grade level and answer the questions in your guidebook. There are multiple reports available depending on the grade levels tested in your school and the appropriate subgroups based on your student population. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

How do I know that the projections are accurate?
11/28/2018 How do I know that the projections are accurate? The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office). The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study. ( Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score. The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office). The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study. ( Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Student Search Capabilities
11/28/2018 C.G. Pg. 16 Student Search Capabilities The Student Search capability allows you to search and investigate the probabilities of students reaching selected PSSA performance level ranges on future PSSA examinations. This feature also allows you to narrow or expand your search based on various student demographic criteria of interest. Information on Student Search is included for completeness and as an encouragement for teams to perform searches for a variety of user-defined subgroups. For example, IEP students who have less than a 70% chance of reaching proficiency on next PSSA. ED students who have less than a 40% chance of reaching proficiency on next PSSA. For accounts with student access! 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 C.G. Pg. 17 Student Search Report Here is a picture of the Student Search report. Note the columns at the far right detail the selected probabilities. Note also that each column is “sortable” by clicking on the column headers. We recommend that you experiment with the different search options to focus on groups of students that are of interest to you. You may wish to reflect back on the reports and questions already addressed in this session to guide your search on groups of students you wish to investigate and discuss. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 C.G. Pg. 18 Next Steps Discuss with your team the next steps in using these data to enhance educational programming for your students. Use the table in the guidebook to document your ideas for further actions. Actions: Involving Who? Other Data to Consider? Timeline Examples: Share results with staff Plan for sharing Reconvene this team and go deeper Discuss our results with SIP team John/Sally Full team Fred /Spot eMetric eMetric, 4sight local assessment 1 month Next Tuesday at 2PM Next Friday at midnight Next SIP team meeting 9/15/2010 PVAAS 2010 Consultation Guide - Session B

Considerations for the Future
11/28/2018 C.G. Pg. 19 Considerations for the Future Discuss with your team what types of support you might find useful from your IU PVAAS contact(s), as well as any other resources or materials needed from the PDE PVAAS Statewide Team. When you are ready to discuss PVAAS with your faculty, you might want to consult the Faculty Roll-out Guide in the Data Tool Reference Guide. Detach and return page 19 of the Consultation Guide to your session facilitator. Here are two more considerations for the future… Think about what supports you might need as you pursue your data investigation; We recommend consulting the Faculty Roll-out documents at the end of the Data Tool Reference Guide. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

See Data Tool Reference Guide
11/28/2018 See Data Tool Reference Guide District and building level faculty roll-out guides! Instruct participants to reference the back-to-back reference documents for rolling out PVAAS to a district-wide audience and building-wide audience that is located in the back of the data tool reference guide document. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

11/28/2018 Questions: PVAAS Materials or Statewide Implementation PVAAS Report Web Site Questions related to the statewide implementation of PVAAS or PVAAS materials can be directed to or Remember to go to to access a district/school’s PVAAS reports. 9/15/2010 PVAAS 2010 Consultation Guide - Session B

333 Market Street Harrisburg, PA 17126
11/28/2018 PVAAS is provided by the Department of Education as a data tool for continuous school improvement. Thomas Gluck, Secretary of Education 333 Market Street Harrisburg, PA 17126 Thomas Gluck Acting Secretary of Education Commonwealth of Pennsylvania 9/15/2010 PVAAS 2010 Consultation Guide - Session B

PVAAS Statewide Core Team

Similar presentations

Presentation on theme: "PVAAS Statewide Core Team"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

PVAAS Statewide Core Team

Similar presentations

Presentation on theme: "PVAAS Statewide Core Team"— Presentation transcript:

Similar presentations

About project

Feedback