Ready, Set, Grow! PVAAS Teacher Specific Reporting

Ready, Set, Grow! PVAAS Teacher Specific Reporting
Fall 2014 Ready, Set, Grow! PVAAS Teacher Specific Reporting Questions?

A Framework for Teaching: Components of Professional Practice
Fall 2014 A Framework for Teaching: Components of Professional Practice Domain 4: Professional Responsibilities Reflecting on Teaching Maintaining Accurate Records Communicating with Families Participating in a Professional Community Growing and Developing Professionally Showing Professionalism Domain 3: Instruction Communicating with Students Using Questioning and Discussion Techniques Engaging Students in Learning Using Assessment in Instruction Demonstrating Flexibility and Responsiveness Domain 1: Planning and Preparation Demonstrating Knowledge of Content and Pedagogy Demonstrating Knowledge of Students Setting Instructional Outcomes Demonstrating Knowledge of Resources Designing Coherent Instruction Designing Student Assessments Domain 2: The Classroom Environment Creating an Environment of Respect and Rapport Establishing a Culture for Learning Managing Classroom Procedures Managing Student Behavior Organizing Physical Space As a reminder for everyone, this slide shows the Framework for Teaching. This framework contains four domains and 22 components, all of which describe the qualities of teaching that are tied to student learning. The “pink” Domains (1 and 4) are considered the “off-stage” domains. That is, these domains and competencies are not observable during a lesson being taught. The “blue” Domains (2 and 3) are the “on-stage” domains. These domains are directly experienced during the act of teaching. This is important because teaching is much more than what we can see during the teaching of the lesson. Our goal is to have teachers walk away with an understanding of how to investigate their reporting relevant to what we know is effective teaching or good professional practice. The Framework for Teaching here was developed many years ago as a way of looking at good teaching practices BEFORE it was used as any kind of observational piece in a teacher’s evaluation. That is what we want to encourage…the use of this information BEYOND the evaluation. How can we use what we see on our PVAAS Teacher Specific reports to look carefully and objectively at how our professional practices are supporting students’ academic growth or not? And, as a result, what can we reflect upon and improve upon related to those practices relative to this framework? T Questions?

Domain 4: Professional Responsibility
Fall 2014 Domain 4: Professional Responsibility Questions Domain Am I open to participating in a culture of inquiry with colleagues and administration? Can I use PVAAS Teacher Specific reports in my post-conference or summative evaluation conference with my principal(s)? 4a & 4d (Professional Responsibility) Do I access PVAAS reports and use them along with other data and information to plan, deliver, and assess instruction? 4b & 4e We will be talking about the Framework for Teaching and how the PVAAS Teacher Specific reports can be used to help our thinking in terms of teachers’ own professional practices. For example, the whole process we are discussing about the use of PVAAS Teacher Specific reports and how it can help frame our thinking about our professional practice can be thought about in terms of domain 4 (Professional Responsibility) in the Framework for Teaching. In other words, am I open to participating in a culture of inquiry with my colleagues and administrators? Do I access PVAAS reports and use them along with other data/information to plan, deliver, and assess instruction? How can I improve my professional practice in this domain by addressing these things? T Questions?

Types of Teacher Specific Reports
Fall 2014 Types of Teacher Specific Reports PVAAS Teacher Specific Reporting (growth of a teacher’s group of students) Value-Added (growth for an overall subject/grade/course) Value-Added Summary (multiple subjects/grades for single year) (single subject/grade for multiple years) Diagnostic (growth across achievement levels, and by subgroups) Diagnostic Summary (multiple subjects/grades for single year Custom Diagnostic (user-defined subgroup of students for single subject/grade for single year) PVAAS Teacher Specific reporting provides you with an estimate of growth of a teacher’s group of students. There are two overall types of reports that provide this information to you. There are Value-Added types of reports indicated by the rose colored rectangles and Diagnostic types of reports indicated by the green colored rectangles. First, the Value-Added reports provide information about the growth for an overall subject and grade, or course. When looking at Value-Added reports, you will have a choice of viewing a Value-Added Summary that looks at multiple subjects/grades for any single year…or a Value Added that looks at a single subject/grade across multiple years. Second, the Diagnostic reports provide information about the growth for smaller groups of students…for various achievement levels, and by subgroups. When looking at the Diagnostic reports, you will have a choice of viewing a Diagnostic Summary that looks at multiple subjects/grades for any single year…or a Diagnostic that looks at a single subject/grade across multiple years. Additionally, you can create a Custom Diagnostic that looks at a user-defined group of students for a single subject/grade for a single year. We will be delving deeply into each of these reports in this session. T Questions?

What is PVAAS? PVAAS: Ready, Set, Grow! T Fall 2014
So, let’s begin by discussing what PVAAS is and how it is different from achievement which is traditionally how student performance has been considered for years. What is PVAAS? T Questions?

What is PVAAS? PVAAS = Pennsylvania Value-Added Assessment System
Fall 2014 What is PVAAS? PVAAS = Pennsylvania Value-Added Assessment System Value-added analysis measures the academic growth of a district's or school's or teacher’s group(s) of students Growth ≈ Current Achievement compared to all Prior Achievement First point we always want to emphasize is that there are no individual student measures of growth! Rather, PVAAS provides a measure of academic growth of a district’s, school’s, or teacher’s GROUP of students…where the group of students serve as their own baseline, or control. The growth of the group of students is conceptually measured by comparing the current achievement of the group of students to their prior achievement. We will be talking more about measuring growth later in the session. Again, we want to emphasize that the PVAAS Teacher Specific reports are about the growth of a teacher’s group of students…not about impact of the teacher. For that, we need multiple measures. T Questions?

Achievement versus growth
Fall 2014 Achievement versus growth Achievement Measures student performance at a single point in time Typically related to socio-economic and demographic characteristics Often measured by percent reaching proficiency Might not improve all students’ achievement Educators cannot influence entering achievement Growth Compares students’ performance to their own prior performance, students are their own control More closely related to teaching and schooling effectiveness Focus placed on all students Educators can have an impact on the progress, or growth, students make Together A more complete picture of student learning Both achievement and growth are important to consider as they focus on different aspects of student performance – achievement is about a student’s performance against a standard (such as proficiency) at a single point in time, while growth compares students’ performance to their own prior performance as a measure of progress. Ultimately, high achievement should be an outcome of exceptional progress or academic growth, so we must look at both if we want to truly reduce the achievement gap. Research has shown that the socio-economic and demographic characteristics of students is typically related to their achievement; however, those same characteristics have little to no relationship with the academic growth of students when their prior achievement is adequately accounted for. What is important to consider is that while teachers cannot influence the entering achievement of the group of students in their classroom, their professional practices can have an impact on the academic growth that their group of students makes. T Questions?

How Does PVAAS Work? PVAAS
Fall 2014 How Does PVAAS Work? PVAAS Keystones PSSA Science Writing* PSSA Math Reading LOOKING AHEAD Planning for Students’ Needs: Student Projections to Future Tests LOOKING BACK Measuring Academic Growth of Groups of Students: Value-Added & Diagnostic Reports District School Teacher So how does PVAAS work? All of the standardized test results from PSSA and Keystones exams go through rigorous analyses by SAS EVAAS. From this information, we get two key pieces of information. First, we can look back to see how our programs are supporting students in making academic progress, and Second, we can look ahead by seeing student projections to future state assessments and plan for their needs. The results from the looking back part of the analyses are available at the district, school, and teacher level. Note, however, that teacher specific reporting is not available in the area of writing. Teacher specific reports are available in math, reading, science, and the 3 Keystone content areas. Student *Writing used in District & School Reporting Note: All Prior Data are Used T Questions?

How is PVAAS used in Pennsylvania?
Fall 2014 How is PVAAS used in Pennsylvania? School accountability through the School Performance Profile (SPP) Teacher evaluation (PVAAS 3 Year Rolling Avg.) PVAAS has been and continues to serve as a tool for LEA/district and school improvement Now, annual tool for teachers to reflect on professional practices PVAAS has been used in Pennsylvania in a variety of ways. First, it is used as an overall school measure for school accountability through the School Performance Profile. Additionally, the PVAAS 3-year rolling average is a part of the teacher evaluation process where teachers with instructional responsibility in state assessed grades and subjects will receive an annual growth measure reflecting the growth of their students, with the exception of grade 3 and writing. As part of the bigger picture, however, PVAAS has been and will continue to serve as a tool for district and school improvement…a way for educators to reflect annually on how their programs (curriculum, instruction, assessments, and structures) are supporting the academic growth of students. Similarly, PVAAS is now an annual tool for teachers to also reflect upon their professional practices. In other words, an annual reflection on students’ growth is a critical component of a teacher’s efforts in striving for continuous professional growth. T Questions?

Why is PVAAS Important? PVAAS: Ready, Set, Grow! T Fall 2014
So, why is having the PVAAS information important? Why is PVAAS Important? T Questions?

Philosophy Behind PVAAS
Fall 2014 Philosophy Behind PVAAS All students can learn. All students deserve to make appropriate academic progress each year. As you strive to use PVAAS information, it is key to recognize the two key philosophies behind PVAAS. First, ALL students can learn. And second, ALL students deserve to make appropriate academic progress or growth each year. This is something most, if not all, educators would agree with. T Questions?

Common Questions About PVAAS
Fall 2014 Common Questions About PVAAS Is PVAAS fair if my students are… Low-achieving? High-achieving? Economically Disadvantaged? Other subgroups? The answer to all of these questions is YES! Remember, we are measuring growth, not just looking at achievement! [SLIDE CONTAINS ANIMATIONS] At this point, many questions like these start to surface. Is PVAAS fair if my students are low-achieving, high-achieving, or from a certain demographic background? [CLICK] The answer is YES, because PVAAS measures the GROWTH of a group of students, NOT their achievement. Let’s look at some evidence. T Questions?

Teacher Specific Reports
Fall 2014 PVAAS: Ready, Set, Grow! Now let’s turn back to the content of the teacher session and start looking at and discussing PVAAS Teacher Specific reporting. Teacher Specific Reports T Questions?

Learning Module for Teacher Specific Reports

PVAAS Teacher Specific Reporting
Fall 2014 Types of Reports PVAAS Teacher Specific Reporting (growth of a teacher’s group of students) Value-Added (growth for an overall subject/grade/course) Value-Added Summary (multiple subjects/grades for single year) (single subject/grade for multiple years) Diagnostic (growth across achievement levels, and by subgroups) Diagnostic Summary (multiple subjects/grades for single year Custom Diagnostic (user-defined subgroup of students for single subject/grade for single year) Remember this slide from earlier in the session. We want to talk about these reports. However, before we talk specifically about each report, let’s first discuss how to navigate through the system to access the various reports listed here. T Questions?

Fall 2014 Types of Information PVAAS Teacher Specific Reporting (growth of a teacher’s group of students) Value-Added (growth for an overall subject/grade/course) Value-Added Summary (multiple subjects/grades for single year) (single subject/grade for multiple years) Diagnostic (growth across achievement levels, and by subgroups) Diagnostic Summary (multiple subjects/grades for single year Custom Diagnostic (user-defined subgroup of students for single subject/grade for single year) Let’s start digging into the Value-Added reports. T Questions?

Value Added reports PVAAS: Ready, Set, Grow! T Fall 2014
Questions?

Terminology Growth Measure Standard Error (SE) Growth Index
Fall 2014 Terminology Growth Measure Math & Reading - how much movement a group of students has made on the NCE scale as compared to the previous year (in NCE units) Science & Keystones - difference between the group of students’ observed scores and their predicted scores (in scale score units) Standard Error (SE) All measures have error. PVAAS takes into account the error around the growth measure before a color is determined. The SE establishes a confidence band around the Growth Measure. Growth Index The Growth Measure divided by the SE This provides a more comparable look across subjects, grades, and courses for a teacher as it takes into account the SE. Composite (SY13-14 reporting ONLY) A combined growth measure across a teacher’s PVAAS reported subjects/grades/courses within SY13-14 This slide a reference slide so that you can walk away with the definitions of some of the key terminology used in PVAAS reporting. T Questions?

Fall 2014 Composite Score Note that the Composite Score is a combined growth measure across a teacher’s PVAAS reported subjects/grades/courses For SY13-14, we only have one year of reporting so the Composite Score is a combined growth measure across a teacher’s PVAAS reported subjects/grades/courses for SY13-14 only. In future years, the Composite Score will represent a combined growth measure across a teacher’s PVAAS reported subjects/grade/courses across years, up to and including 3 consecutive school years. Note that the Composite Score is a combined growth measure across a teacher’s PVAAS reported subjects/grades/courses For SY13-14, we only have one year of reporting so the Composite Score is a combined growth measure across a teacher’s PVAAS reported subjects/grades/courses for SY13-14 only. In future years, the Composite Score will represent a combined growth measure across a teacher’s PVAAS reported subjects/grade/courses across years, up to and including 3 consecutive school years. T Questions?

Fall 2014 Remember the 4 Domains??? …from A Framework for Teaching: Components of Professional Practice Domain 4: Professional Responsibilities Reflecting on Teaching Maintaining Accurate Records Communicating with Families Participating in a Professional Community Growing and Developing Professionally Showing Professionalism Domain 3: Instruction Communicating with Students Using Questioning and Discussion Techniques Engaging Students in Learning Using Assessment in Instruction Demonstrating Flexibility and Responsiveness Domain 1: Planning and Preparation Demonstrating Knowledge of Content and Pedagogy Demonstrating Knowledge of Students Setting Instructional Outcomes Demonstrating Knowledge of Resources Designing Coherent Instruction Designing Student Assessments Domain 2: The Classroom Environment Creating an Environment of Respect and Rapport Establishing a Culture for Learning Managing Classroom Procedures Managing Student Behavior Organizing Physical Space Let’s look at the Framework for Teaching which as you recall contains four domains and 22 components, all of which describe the qualities of good teaching that are tied to student learning. The Framework for Teaching suggests that teacher evaluation is much more than just watching a lesson; it includes gathering information about a teacher’s planning, and also about the professional responsibilities that the teacher regularly performs. What we want you to think about is…is the Value Added Report creating any thoughts or questions for you about your professional practice in these 22 components found here? T Questions?

Making the Connections
Fall 2014 Making the Connections As a teacher, what do you do? Plan and prepare Create an environment conducive to optimizing learning Instruct and assess using evidence-based practices Looking at your PVAAS Teacher Specific reporting AND thinking about the components within the 4 domains, what practices might you further reflect upon and explore more deeply? REMEMBER, PVAAS reports are only ONE source of data. Multiple data sources should be used! [SLIDE CONTAINS ANIMATIONS] Let’s think about this a bit more. As a teacher, what do you strive to do in providing instruction for students? It is more than just delivering a lesson. You plan and prepare for instruction. You work to create an environment in your classroom that is conducive to optimizing learning for all students. And, finally not only do you instruct and assess your students using evidence-based practices. You do all of this to support student learning and student growth. [CLICK] The Framework for Teaching defines those components of what you do in a more concrete way. So thinking about these 22 components and thinking about what you see on your Value Added Report, what practices might you further reflect upon and explore more deeply? As you do this, remember that PVAAS is ONE source of data; you should use other sources of data as well in this process. T Questions?

What might account for what you see on your Value-Added reports?
Fall 2014 What might account for what you see on your Value-Added reports? Questions Domain Is my content knowledge solid? Am I aware of the important concepts, big ideas, eligible content in my subject area? Am I knowledgeable about the possible misconceptions around specific topics in this subject? 1a (Planning & Preparation) Am I using the most appropriate pedagogical approaches for this subject/content? Am I using the allocated time for teaching my subject/course efficiently, using every available minute for instruction? Are my routines and procedures intact so as to minimize transition time and discipline time in order to use allocated time efficiently? 2c (Classroom Environment) Am I clearly communicating to students during instruction about the purpose of the lesson, and providing scaffolding to support student learning? 3a (Instruction) Do I have students engaged in self-assessing and monitoring their own progress? 3d We have started this line of inquiry, or line of questioning, for you. Here you see 5 sets of questions that you could use to help frame your line of inquiry…each of these sets of questions aligns to a specific domain and component in the Framework for Teaching, and we’ve identified that domain/component for you in the column next to the questions. Let’s take a moment to read through some of these questions. Do you see the connection of the question to the domain/component listed? Note that while we are currently discussing the Value-Added Summary report where you can see all grades/subjects/courses, this same line of inquiry applies to the Value Added report for a single grade/subject/course. It is important to remember that these questions provide a guided discussion of self-reflection that probes at what might account for growth of students, whether it be low growth or high growth…so in other words, this line of inquiry should happen no matter if you see yellows and reds on a report, or if you see light blues and dark blues. As these questions are a bit general to guide you in your thinking, you will want to ask more specific questions like this related to your practices and to your Value-Added results. Let’s take a look at an example. T Questions?

Possible Related Questions
Fall 2014 What might account for what you see on your Value-Added reports (multiple subjects)? Questions Possible Related Questions Is my content knowledge solid? Am I aware of the important concepts, big ideas, eligible content in my subject area? Am I knowledgeable about the possible misconceptions around specific topics in this subject? (Domain 1a: Planning & Prep) Do I understand the science standards (as well as I understand reading/math)? Have I worked on unpacking them? Do I have the content knowledge regarding the specific eligible content in “nature of science”? Am I using the most appropriate pedagogical approaches for this subject/content? (Domain 1a: Planning & Prep) Is my knowledge base on how to teach Science via the inquiry method at the level it needs to be? Is my lack of confidence in Science impacting my planning? What, if anything, is different about my pedagogical approaches in math and reading from science? Am I using the allocated time for teaching my subject/course efficiently, using every available minute for instruction? Are my routines and procedures intact so as to minimize transition time and discipline time in order to use allocated time efficiently? (Domain 2c: Classroom Environment) Thinking about science, am I using lab time efficiently? Are we spending too much time on projects and not enough on specific content knowledge? Am I clearly communicating to students during instruction about the purpose of the lesson, and providing scaffolding to support student learning? (Domain 3a: Instruction) Thinking about science, am I clearly communicating to students “what they are to know and be able to do” as a result of a group project/experiment? Do I have students engaged in self-assessing and monitoring their own progress? (Domain 3d: Instruction) Are there additional ways that I can engage students in monitoring their progress on the science standards? On all subjects? When looking at this report, it is easy to go right to “what’s wrong with Science?” Would you agree? However, my self-reflection and probing might be best facilitated by comparing what I know and am doing in Science relative to Reading and Math to see if there are any differences. For example, is my content knowledge weaker in Science than in Reading and Math? Is that what might account for the differences I see? Since this is my first year teaching Science in many years, do I need to re-think how I am planning for labs and classroom/group project work? Let’s walk through each of these general questions on the left to more specific questions for this teacher on the right. [Facilitator – walk through as many of these as needed for the participants.] T Questions?

What might account for what you see on your Value-Added reports?
Fall 2014 What might account for what you see on your Value-Added reports? Questions My Questions Is my content knowledge solid? Am I aware of the important concepts, big ideas, eligible content in my subject area? Am I knowledgeable about the possible misconceptions around specific topics in this subject? (Domain 1a: Planning & Prep) Am I using the most appropriate pedagogical approaches for this subject/content? (Domain 1a: Planning & Prep) Am I using the allocated time for teaching my subject/course efficiently, using every available minute for instruction? Are my routines and procedures intact so as to minimize transition time and discipline time in order to use allocated time efficiently? (Domain 2c: Classroom Environment) Am I clearly communicating to students during instruction about the purpose of the lesson, and providing scaffolding to support student learning? (Domain 3a: Instruction) Do I have students engaged in self-assessing and monitoring their own progress? (Domain 3d: Instruction) T Questions?

I did roster verification. Why do I not have any reports?
Fall 2014 I did roster verification. Why do I not have any reports? Must have at least 11 students Must have an “Active N Count” of at least 6 students/6 FTE Students Let’s take some time to dig deeper into the Value Added reports and address some of the questions that may have been raised here. First question that often is raised is “why do I not have any reports if I did roster verification?” In order to receive Teacher Specific Value Added reporting, you first must have at least 11 students on your roster for that grade/subject/course. This n count of 11 is NOT across grades/subjects/courses, but rather for EACH tested grade/subject/course. Additionally, you must have what we call an “active N count” of at least 6 students. We will explain this shortly. T Questions?

Would this teacher receive reporting? Actual N Count ≥ 11
Fall 2014 Would this teacher receive reporting? Actual N Count ≥ 11 100% 100% 100% 100% 50% 100% 100% 100% 100% 100% YES 50% 100% 50% 100% 100% 100% 100% 50% [SLIDE CONTAINS ANIMATIONS] So, let’s take a look at how this happens. SAS EVAAS looks at the actual N count first. Did the teacher have at least 11 students in that grade/subject/course? [CLICK] Yes, this teacher had 22 students. This is the first N count, meaning there must be at least 11 different students. Now let’s look at the second N count. 100% 50% 100% 50% Actual N=22 T Questions?

Teacher Value Added Single Subject/Grade/Course
Fall 2014 Teacher Value Added Single Subject/Grade/Course New info only on Value Added for single subject/grade/course Same info as Value Added Summary [SLIDE CONTAINS ANIMATIONS] So, let’s look at a Teacher Value Added report for a single subject/grade/course. You can get this report by clicking on any of the blue hyperlinks for a specific grade/subject/course in the table below the graph on the Value Added Summary Report. [CLICK] You will notice that the information you see on the left of the Value Added report is the same information for that subject/grade/course as on the Value Added Summary. There is no difference in the information. The only difference is an option you see between the graph and the table that allows you to change the type of graph you are viewing. If you wish to do so, you can use the drop-down menu to view the Growth Measure graph instead of the default Growth Index Graph. The information on the right is different however from the Value Added Summary. The information displayed here provides the teacher with data on where they fall for that grade/subject/course in relation to other teachers statewide with instructional responsibility in that same grade/subject/course. T Questions?

Teacher Value Added Single Subject/Grade/Course Multiple Years
Fall 2014 Teacher Value Added Single Subject/Grade/Course Multiple Years When multiple years are available Although this is our first year of reporting and teachers will only see one year of reporting on their own reports, we wanted to show you what this single subject/grade/course Value-Added report will look like when it eventually includes multiple years of data. This report, as you can see, shows this subject/grade/course for this teacher for up to 3 years. In this case, 2014, 2015, and So this teacher can view the growth of his/her students in this subject/grade/course over the last several years. Remember, these are most likely different groups of students each year…each year is a reflection of the growth of the students whom this teacher had instructional responsibility for in that year. T Questions?

Students Included in Reporting
Fall 2014 Students Included in Reporting To actually be included in PVAAS Teacher Specific reporting, a student MUST: Have a PSSA or Keystone score from the most recent year NOT be a foreign exchange student NOT be a first-year ELL student NOT be Proficient or Advanced on a PRIOR Keystone exam NOT be claimed at less than 10% instructional responsibility NOT have tested with the PASA (alternate assessment) Here is some additional information to keep in mind. To be included in Teacher Specific reporting, a student must… have taken the state assessments -- in other words, have a PSSA or Keystone score; NOT be a foreign exchange student or first-year ELL student; NOT be Proficient or Advanced on a prior Keystone exam in that same content area; NOT be claimed at less than 10% instructional responsibility; and NOT have tested with PASA – Pennsylvania’s Alternate System of Assessment. Remember, it was important for the teacher and school to roster ALL students, even those with who might appear as one of the students above. SAS EVAAS applies the business rule of removing any students for any of these reasons. T Questions?

Fall 2014 Are there students to be removed (based on PA business rules)? Per Subject/Grade/Course [SLIDE CONTAINS ANIMATIONS] So, let’s see how this works. The first thing SAS EVAAS looks at is…are there any students that should be removed? One student did not take the test (there is no score for this student); one student took the PASA. [CLICK] These 2 students would be removed. T Questions?

Are there any students with < 10% Instructional Responsibility?
Fall 2014 Are there any students with < 10% Instructional Responsibility? 100% 100% 100% 100% 50% 100% 100% 100% 100% 100% 50% 100% 50% 100% 100% 100% 100% 50% [SLIDE CONTAINS ANIMATIONS] Remember the roster verification process you completed in the spring. The % instructional responsibility listed in the roster for each student is shown here next to their picture. These 3 students were claimed with <10% Instructional Responsibility on this teacher’s roster. [CLICK] Therefore, these 3 students are dropped from the analyses by SAS EVAAS and not included. Remember, it was important for the teacher and school to roster ALL students, even those with whom the teacher had less than 10% instructional responsibility. SAS EVAAS applies the business rule of removing any students for any of the reasons we stated just a few moments ago. 100% 50% 100% 5% 5% 50% 5% T Questions?

Measuring Growth: Students’ Starting Point
Fall 2014 Measuring Growth: Students’ Starting Point PA Data [SLIDE CONTAINS ANIMATIONS] So now SAS EVAAS has the group of students for a teacher (after removing students as indicated in the last few slides). When the state assessment is administered in consecutive grade levels as it is in reading and math for grades 3-8, PVAAS uses a growth standard method to measure the growth or progress of a group of students. When the state assessment is not administered in consecutive grade levels as in science or the Keystone content areas, PVAAS uses a predictive method to measure the growth or progress of a group of students. However, while the reports are slightly different and the behind the scenes methodology is different, we will think about these conceptually in the same way for our purposes today. Let’s consider this group of students. These students are going to score along the achievement spectrum from low to high achievement, in any given year. SAS EVAAS begins by getting the starting or entering achievement for that group of students. [CLICK] It considers all prior data, across grades and subjects, for those students by looking in the longitudinal database we have in PA for state assessment scores. So now we get an estimated prior or starting achievement for that group of students. What happens next? Students’ Prior Data Across Grades & Subjects Group of Students Estimated Prior (or Entering) Achievement of the Group of Students T Questions?

Concept of Measuring Growth
Fall 2014 Concept of Measuring Growth Higher Achievement Higher Achievement In District & School Reporting, students are weighted equally. For Teacher Specific Reporting, students are weighted by taking into account the % Total Instructional Responsibility for each student. [SLIDE CONTAINS ANIMATIONS] Students are then tested again towards the end of the subject/grade, or course. This new assessment data is added to what we already know about the students. [CLICK] The ending achievement of the group of students is then calculated. If the ending achievement of the group is at about the same as the entering achievement of the group, then this will be indicated as a green in PVAAS. If there is moderate evidence that the ending achievement of the group is higher than the entering achievement of the group, then this will be indicated as a light blue in PVAAS. If there is significant evidence that the ending achievement of the group is higher than the entering achievement of the group, then this will be indicated as a dark blue in PVAAS. The opposite can happen as well. If there is moderate evidence that the ending achievement of the group is lower than the entering achievement of the group, then this will be indicated as a yellow in PVAAS. If there is significant evidence that the ending achievement of the group is lower than the entering achievement of the group, then this will be indicated as a red in PVAAS. Now remember we discussed the weighting of students based upon the % of instructional responsibility we looked at earlier? In district and school level reporting, students are weighted equally. BUT for teacher specific reporting, students are not necessarily weighted equally. Instead, SAS EVAAS takes into account the % of Instructional Responsibility for each student and weights the students accordingly. In this way, students with 50% IR are weighted less than those with 100% IR, for example. Lower Achievement Lower Achievement Prior (or Entering) Achievement of the Group of Students Current Achievement of the Group of Students T Questions?

Going Back: Roster Verification Summary
Fall 2014 Going Back: Roster Verification Summary If you would like to see how each student is weighted in the analyses, you can go back to the roster verification summary (link is shown here) to view that information. It is retained in the system. T Questions?

Going Back: Roster Verification Summary
Fall 2014 Going Back: Roster Verification Summary So which students are weighted in the analysis at 100%? Which students are weighted at less than 100% in the analysis? Look at the column for total % Instructional Responsibility. In this example we see that some students are weighted more than others. For any students with the orange caution triangle next to their %, these were students who were over-claimed across teachers in the state. For these students, their % is proportionately adjusted by SAS EVAAS so that the total % instructional responsibility across all teachers in the state for this student does not total more than 100%. The questions that going back to this roster verification summary will not help you answer include the following: Which students did not take the assessment? Which students took PASA? Which students were 1st year ELA? Which students were foreign exchange students? What students reached proficiency on a prior Keystone in this content area? T Questions?

Fall 2014 Types of Information PVAAS Teacher Specific Reporting (growth of a teacher’s group of students) Value-Added (growth for an overall subject/grade/course) Value-Added Summary (multiple subjects/grades for single year) (single subject/grade for multiple years) Diagnostic (growth across achievement levels, and by subgroups) Diagnostic Summary (multiple subjects/grades for single year Custom Diagnostic (user-defined subgroup of students for single subject/grade for single year) Remember this graphic that showed you the two different types of Teacher Specific reporting. We’ve talked a lot about the Value Added reports. We are now going to talk about the Diagnostic reports…much in the same way we did with the Value Added reports. T Questions?

Diagnostic reports PVAAS: Ready, Set, Grow! T Fall 2014
Questions?

Fall 2014 Diagnostic Reports Do NOT factor in the percent of Instructional Responsibility Reflect the growth of a group of students who may have had more than one teacher with instructional responsibility in that specific subject/grade or course. Keep in mind that on the Value Added reports, students are weighted in the analysis based upon the total % of Instructional Responsibility for that student. However, for Diagnostic reports, students are weighted equally. The Diagnostic reports reflect the growth of a group of students who may have had more than one teacher with instructional responsibility in that specific subject/grade, or course. These are to be used for diagnostic purposes only, not evaluative purposes. T Questions?

Diagnostic Summary Report
Fall 2014 Diagnostic Summary Report [SLIDE CONTAINS ANIMATIONS] First, let’s consider how the diagnostic report looks at the growth of smaller subgroups of students rather than an entire grade level or course of students. In this report, growth is looked at by entering achievement level. [CLICK] So in other words, it is looking at the growth of students who are in the lowest third of the state in terms of achievement, then those who are in the middle third of the student in terms of achievement, and then finally those who are in the highest third of the state in terms of achievement. T Questions?

Diagnostic Summary Report
Fall 2014 Diagnostic Summary Report So now, let’s look more closely at the other pieces of a Diagnostic Summary Report. [Facilitator: You may choose to use this screen shot to explain this report, or you may go to the demo site to explain there.] Here, you can see how students at various levels of achievement are growing academically across all grades and subjects, or courses. Students are divided into 3 achievement groups based on where they fall in the statewide distribution of achievement. [Facilitator: be sure to explain the following How to navigate to this report The meaning of the four colors – blue, green, pink, white The meaning of the size of pie slice [CLICK] Let’s look at this report together, what Growth Color Indicators are you seeing, and what do those colors mean to you? If you have more than one subject/grade or course, do you see a different pattern across the subjects/grades/courses? What can you learn from this report to help you reflect upon your instruction and planning this year? What Growth Color Indicators are you seeing, and what do those colors mean to you? If you have more than one subject/grade or course, do you see a different pattern across the subjects/grades/courses? What can you learn from this report to help you reflect upon your instruction and planning this year? T Questions?

Fall 2014 Looking at Subgroups [SLIDE CONTAINS ANIMATIONS] The Diagnostic reports are also available on even smaller groups of students, such as demographic subgroups of students. Clicking on the “Subgroup” button as shown here will reveal a new window that allows you to select a report based on a subgroup of students. [CLICK] As you can see, there are a number of subgroups to choose from. An important item to note is that two new subgroups will be added this year – Historically Underperforming and Non-Historically Underperforming. These are unduplicated counts of students who fall into the subgroups of ED, IEP, and ELL – just as they are on the School Performance Profile. Note: Two new subgroups will be added this year – (1) Historically Underperforming and (2) Non-Historically Underperforming T Questions?

What might account for what you see on your Diagnostic reporting?
Fall 2014 What might account for what you see on your Diagnostic reporting? Questions Domain Do I know my students to the degree that I plan/prepare lessons based on my students and their special needs, individual and groups of students? 1b & 1c (Planning & Preparation) Am I planning coherent instruction that differentiates for groups of students? Am I using flexible grouping to meet the needs of specific groups of students? 1e Am I using questioning and discussion techniques that stretch advanced learners? Am I promoting higher level thinking with all students? 3b (Instruction) Am I responsive to the needs of all students by seeking approaches for students who have difficulty learning? Do I have a repertoire of strategies to use with students who are low achieving? 3e Just like we did with the Value Added reporting, we have started this line of inquiry, or line of questioning, for you for your Diagnostic reporting. Here you see 4 sets of questions that you could use to help frame your line of inquiry…each of these sets of questions aligns to a specific domain and component in the Framework for Teaching, and we’ve identified that domain/component for you in the column next to the questions. Let’s take a moment to read through some of these questions. Do you see the connection of the question to the domain/component listed? Note that while we are currently discussing the Diagnostic Summary report where you can see all grades/subjects/courses, this same line of inquiry applies to the Diagnostic report for a single grade/subject/course. It is important to remember that these questions provide a guided discussion of self-reflection that probes at what might account for growth of students, whether it be low growth or high growth…so in other words, this line of inquiry should happen no matter if you see pink or green or blue on the report. As these questions are a bit general to guide you in your thinking, you will want to ask more specific questions like this related to your practices and to your Diagnostic results. Let’s take a look at an example. T Questions?

Possible Related Questions
Fall 2014 What might account for what you see on your Diagnostic reporting (Keystone Biology)? Questions Possible Related Questions Do I know my students to the degree that I plan/prepare lessons based on my students and their special needs, individual and groups of students? (Domains 1b & 1c: Planning & Prep) Am I aware of which students have reading difficulties and may need support/scaffolding for reading text? Am I planning coherent instruction that differentiates for groups of students? Am I using flexible grouping to meet the needs of specific groups of students? (Domain 1e: Planning & Prep) Are there opportunities for grouping students and providing tiered assignments to students? Am I using questioning and discussion techniques that stretch advanced learners? Am I promoting higher level thinking with all students? (Domain 3b: Instruction) Are my unit assessments aligned with the level of thinking required for each standard/assessment anchor? Am I responsive to the needs of all students by seeking approaches for students who have difficulty learning? Do I have a repertoire of strategies to use with students who are low achieving? (Domain 3e: Instruction) Do I use multiple learning paths to meet the needs of all students? (example: use visual organizers to accompany text) With these two examples, it is possible to see that some questions can be applied to widely differing results or reports. For example, asking about tiered assignments in differentiated instruction can be applied to students who did not demonstrate growth, whether those students were considered in the lowest third or highest third achievement levels. In other situations, a teacher may see a group with a lower achievement levels not growing, and want to ask himself/herself if he/she was aware of which students were in need of scaffolding and whether or not sufficient scaffolding was provided to those students. Let’s walk through each of these general questions on the left to more specific questions for this teacher on the right. [Facilitator – walk through as many of these as needed for the participants.] We have provided two more examples in the next 4 slides that you can also use as a reference or resource. These examples are for (1) grades 6 and 7 reading and for (2) middle school math. [Facilitator: you may want to assess if participants need or want to go through another example as a large group or even as smaller table groups.] T Questions?

Fall 2014 What might account for what you see on your Diagnostic reporting? Questions My Questions Do I know my students to the degree that I plan/prepare lessons based on my students and their special needs, individual and groups of students? (Domains 1b & 1c: Planning & Prep) Am I planning coherent instruction that differentiates for groups of students? Am I using flexible grouping to meet the needs of specific groups of students? (Domain 1e: Planning & Prep) Am I using questioning and discussion techniques that stretch advanced learners? Am I promoting higher level thinking with all students? (Domain 3b: Instruction) Am I responsive to the needs of all students by seeking approaches for students who have difficulty learning? Do I have a repertoire of strategies to use with students who are low achieving? (Domain 3e: Instruction) T Questions?

Fall 2014 What might account for what you see on your Diagnostic reporting? Questions Possible Related Questions Do I know my students to the degree that I plan/prepare lessons based on my students and their special needs, individual and groups of students? (Domains 1b & 1c: Planning & Prep) Am I planning appropriately for the students who are in my highest achievement level? Am I aware of which students receive Tiers 2 and 3 supports, and what are their specific challenges? Am I planning coherent instruction that differentiates for groups of students? Am I using flexible grouping to meet the needs of specific groups of students? (Domain 1e: Planning & Prep) Do I provide text at varying levels of complexity to address the instructional levels of all students? Am I aware of the text complexity levels of texts I’m using with my highest achieving groups of students? Am I using questioning and discussion techniques that stretch advanced learners? Am I promoting higher level thinking with all students? (Domain 3b: Instruction) Have I analyzed the questions I pose during discussions and on assessments to determine their DOK level? Am I responsive to the needs of all students by seeking approaches for students who have difficulty learning? Do I have a repertoire of strategies to use with students who are low achieving? (Domain 3e: Instruction) Do I know how to scaffold instruction as well as provide background knowledge and advanced organizers for students who have difficulties? T Questions?

Diagnostic Report Single Subject/Grade/Course
Fall 2014 Diagnostic Report Single Subject/Grade/Course Same info as Diagnostic Summary New info only on Diagnostic for single subject/grade/course [SLIDE CONTAINS ANIMATIONS] So, let’s look at a Diagnostic report for a single subject/grade/course. You can get this report by clicking on any of the blue hyperlinks for a specific grade/subject/course in the table below the graph on the Diagnostic Summary Report. [CLICK] You will notice that the information you see on the left of the Diagnostic report is the same information for that subject/grade/course as on the Diagnostic Summary. There is no difference in the information. The only difference is an option you see between the graph and the table that allows you to change the type of graph you are viewing. If you wish to do so, you can use the drop-down menu to view the Bar graph instead of the default Pie graph. The information on the right is different however from the Diagnostic Summary. The information displayed here provides the teacher with a look at which students fall into each of the achievement groups for the current year – lowest, middle, and highest. Additionally you will see two groups labeled “Students Not Used in Report” and “Student Not Used in Analysis”. We will go over those groups and what they mean in just a moment. Notice that the number in parentheses reflects the number of students represented in each of those groups. T Questions?

Diagnostic Report Single Subject/Grade/Course Multiple Years
Fall 2014 Diagnostic Report Single Subject/Grade/Course Multiple Years When multiple years are available Although this is our first year of reporting and teachers will only see one year of reporting on their own reports, we wanted to show you what this single subject/grade/course Diagnostic report will look like when it eventually includes multiple years of data. This report, as you can see, shows this subject/grade/course for this teacher for multiple years. In this case, 2015 and So this teacher can view the growth of his/her students in this subject/grade/course over multiple years. Remember, these are most likely different groups of students each year…each year is a reflection of the growth of the students whom this teacher had instructional responsibility for in that year. T Questions?

Diagnostic Report Single Subject/Grade/Course
Fall 2014 Diagnostic Report Single Subject/Grade/Course Students Not Used in Report Students with not enough data to be included in diagnostic reporting Math & Reading: two most recent years of data Science, Writing, & Keystones: 3 prior data points Exception 4th grade science – 2 prior data points [SLIDE CONTAINS ANIMATIONS] Let’s look closer at the two groups labeled, “Students Not Used in Report” and “Students Not Used in Analysis” that are found in the right side of the report. [CLICK] The students who are not used in the report would be students for whom SAS EVAAS does not have enough data to include them in diagnostic reporting. To be used in this report for math or reading, students must have the most recent two consecutive years of scores in the selected subject. To be included in this report for Science and Keystones, students must have at least 3 prior data points (only exception is grade 4 science in which a student must have 2 prior data points). The students who are not used in the analysis would be students who are also excluded from the value-added analyses. This would match the students with an “N” listed on the Student List in the Value Added reports. Students Not Used in the Analysis Students who are also not included in Value Added analyses Students with an “N” on the student list T Questions?

Custom Diagnostic reports
Fall 2014 PVAAS: Ready, Set, Grow! Now, let’s quickly look at one last type of Diagnostic report available to teachers. Just like the other Diagnostic reports, PVAAS Custom Diagnostic reports are not used for evaluative purposes. This report will allow you to look deeper into the data at a specific grade and subject, or course for which you are responsible - for a user-defined groups of students. Custom Diagnostic reports T Questions?

Custom Diagnostic Report
Fall 2014 Custom Diagnostic Report Purpose – to explore the growth of groups of students that YOU define Must have a minimum of 15 students to create a Custom Diagnostic Report The purpose of the Custom Diagnostic Report is to explore the growth of groups of students that YOU define. The only caveat to using this report is that you must have a minimum of 15 students in the grade and subject, or course, you are investigating to create a Custom Diagnostic Report. The reason for this is that you need at least 5 students in a student group to estimate growth. As there are 3 groups on a diagnostic report, you would then need a minimum of 5 students for each of the 3 groups, or in other words 15 total students. T Questions?

What Custom Diagnostic Reports Might YOU Create?
Fall 2014 What Custom Diagnostic Reports Might YOU Create? Growth data by class section or period Students with similar educational opportunities Interventions, Enrichment, Curricular experiences, Instructional delivery approaches Examples: Elementary: students in different flexible reading groups Middle School: 8th grade Algebra I students who had Pre-Algebra in 7th grade vs. those who had regular track 7th grade Math High School –Biology period 1 class vs. period 8 class So, why would you want to use this report? What are some examples of Custom Diagnostic reports you might want to create? Here are some thoughts we had. You could look at growth by a specific class period or section. This is particularly useful for secondary teachers who may be teaching several periods of Algebra I or Biology throughout the day. You may want to look at growth for students who have similar educational opportunities. For example, what about the growth of 8th grade Algebra I students who had differing progression of math courses prior to 8th grade? T Questions?

Planning for Today PVAAS: Ready, Set, Grow! T Fall 2014
What other data do you have available in PVAAS to help you with your current students…the students in your classroom today? Planning for Today T Questions?

Fall 2014 So What, Now What? You have analyzed the growth of your students from last school year You have asked yourself questions to reflect on your professional practices BUT, it is a new school year with different students! So, now what? [SLIDE CONTAINS ANIMATIONS] So what have we done so far? We’ve analyzed the growth of your students from last school year. You’ve also asked yourself probing questions in order to reflect on your professional practices as an educator. [CLICK] But, what does that mean to you now that we are in a new school year and you have different students? T Questions?

Growth to Projections: Planning for Today
Fall 2014 Growth to Projections: Planning for Today Create a Custom Student Report (PVAAS Student Projections) List of projections for just MY students By subject/grade/course By section/period By subgroups Which of your current students are: Lower-achieving (projected percentile between 1-33)? What did the growth of students like this look like last year? Middle-achieving (projected percentile between 34-66)? Higher-achieving (projected percentile between 67-99)? [SLIDE CONTAINS ANIMATIONS] How can you plan for today? First, you will want to create a Custom Student Report containing projected percentiles for your students you have in this current school year. [CLICK] You will then go back to your Custom Student Report and sort them from lowest to highest on the Projected Percentile column in order to have your list sorted from lower to higher achieving students. You will then divide your list into 3 groups: Those at significant risk (projected percentile between 1 and 33) Those at moderate risk (projected percentile between 34 and 66) Those at lower risk (projected percentile between 67 and 99) For each of those 3 groups of students, how did your similar students last year grow? Before we demonstrate how to create a Custom Student Report, let’s review a student projection report. T Questions?

What is a Student Projection?
Fall 2014 What is a Student Projection? So let’s start by looking at an individual student projection report. The red dots on the graph show the student’s observed performance in the indicated subject area in previous years. The yellow square on the graph indicates the projection for the indicated grade/subject/course. The red, blue and green lines on the graph indicate the percentile boundaries for the Basic, Proficient and Advanced categories for the specified subject area based on the most recent state testing. These values are utilized in the calculation of the probability of a student score being in an indicated performance level or higher. The first table below the graph displays two types of information. The first is the Projected State Percentile. This number (in this example it is 62) is the same as what the yellow square displays in the graph. This says that based on this student’s history and his/her achievement pattern (across subjects and years) the student is projected to be in the 62nd state percentile on the 8th grade Reading assessment. The calculation of this projection assumes too that the conditions that were in place for the data that are used to create the model continue to be in place. This table also provides the probabilities of this student scoring in each of the following performance levels or higher given the student’s Projected State Percentile. In this example, the probability of scoring in the Basic level or above (99.9%), in the Proficient level or above (99.1%), or in the Advanced level (79.2%). T Questions?

Fall 2014 Custom Student Report: Projection Summaries to Proficiency and Beyond for a Teacher’s Current Students [SLIDE CONTAINS ANIMATIONS] This is what a Custom Student Report looks like. Note that this report is NOT the same as the Custom Diagnostic Report that we discussed earlier. [CLICK] You will notice that this report provides to you a full list of the students you selected, along with both their projected state percentile and the probability of reaching the indicated performance level or higher (often times you will choose this to reflect the probability of reaching proficiency or higher). It is important to note that this report CAN be saved and referenced at a later time! T Questions?

Options to Access PVAAS Student Projections
Fall 2014 Options to Access PVAAS Student Projections **Access to Student Level Data is Required Student Search LEA provides access to teacher Custom Student Report School Admin provides access to Teacher Leader in building; Teacher Leader creates lists for each teacher Export by LEA LEA/District Admin School Admin (if permission given to School Admin) Access through Local Student Data Management System In order to have access to projections, you as a PVAAS account holder must have access to student level data. So what are your options in gaining access to this type of information? There are several options. You as an account holder can be granted access to the Student Search and Custom Student Report features in PVAAS by your LEA/district granting you access to student level data. Another option to having access to a Custom Student Report would be having a designated teacher leader in the school have access to student level data and creating these Custom Student Reports for each teacher. A new feature, just recently released, is an export feature in which the LEA/District Admin (or the School Admin if permission to this feature is granted by the LEA/District Admin) can export student projection data from the system. Some districts choose to do this and import that information into the LEA’s/district’s local student data management system. T Questions?

Accessing support & resources
Fall 2014 PVAAS: Ready, Set, Grow! In closing, we want to be sure everyone is aware of the following resources and how you can access them. Accessing support & resources T Questions?

Admins and Teachers: Virtual Learning Modules
Fall 2014 Admins and Teachers: Virtual Learning Modules Free, online professional development Access through PVAAS website TWO Modules Teacher Reports (45 min) Admin Reports (15 min) We have two updated virtual learning modules on Teacher Specific reporting which can be found by clicking on the e-Learning button in the blue menu bar within the reporting system. There is a 45-minute module focusing on teacher specific reports and a 15-minute module focusing on admin reports. T Questions?

Misconceptions Addressed
Fall 2014 Misconceptions Addressed Contains common misunderstandings Available on PVAAS login page Critical to address, so do not perpetuate misunderstandings No answer or a wrong answer continues the misunderstanding Consistent answers across the LEA/district is critical As mentioned earlier, a document on the common misunderstandings of teacher reporting can be found on the PVAAS login page. The scatterplots shown much earlier in the session are included in this document. T Questions?

PVAAS Help Menus Detailed content and explanations of:
Fall 2014 PVAAS Help Menus Detailed content and explanations of: Concept of Growth Terminology Use of Information There are detailed help menus within the web-based system for all PVAAS reports including the new teacher reports and Admin reports. T Questions?

PVAAS Login Page T Fall 2014 Questions? Email pdepvaas@iu13.org
You will find a number of resources, including a number of those mentioned today, on the PVAAS login page. T Questions?

Ready, Set, Grow! PVAAS Teacher Specific Reporting

Similar presentations

Presentation on theme: "Ready, Set, Grow! PVAAS Teacher Specific Reporting"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Ready, Set, Grow! PVAAS Teacher Specific Reporting

Similar presentations

Presentation on theme: "Ready, Set, Grow! PVAAS Teacher Specific Reporting"— Presentation transcript:

Similar presentations

About project

Feedback