PAIU CC Meeting PVAAS Update November 2014 Kristen Lewald, Ed.D. Statewide PVAAS Director for PDE Lancaster-Lebanon IU13

Topics  Guide for Reflection on Teacher Specific Reporting  Demo Access Accounts  Common Questions from Admins and Teachers  Feedback from Teachers  Feedback from 499 Superintendents/Designee  Help Menu System  PVAAS Reporting/Website Updates

NEW: Guide for Reflection PVAAS Teacher Specific Reporting

Guide for Reflection: Process for Review and Planning Steps to Review Reports and Develop Plan of Action

Guide for Review and Planning Two Sets of Guiding Questions:  Organized Based on Domains 1. Planning and Preparation 2. Classroom Environment 3. Instruction  Organized Based on CIAO! 1. Curriculum 2. Instruction 3. Assessment 4. Organization

Demo Account Access PVAAS Teacher Specific Reporting

Demo Account Access Share with LEAs

Common Questions from Admins and Teachers PVAAS Teacher Specific Reporting

Why No Report for a Teacher Who Rostered? PVAAS Teacher Specific Reporting Common Questions

Students Included in Reporting To actually be included in PVAAS Teacher Specific reporting, a student MUST: Have a PSSA or Keystone score from the most recent year NOT be a foreign exchange student NOT be a first-year ELL student NOT be Proficient or Advanced on a PRIOR Keystone exam NOT be claimed at less than 10% instructional responsibility NOT have tested with the PASA (alternate assessment)

Minimum N Counts  Must have at least 11 students  Must have an “Active N Count” of at least 6 students/6.0 FTE Students

Why No Individual Student Growth Measure? PVAAS Teacher Specific Reporting Common Questions

Why No Individual Student Growth Measure? Student History Report  Growth is about a GROUP of students NOT an individual student.  Error would be too large for an individual student to know if they made growth or not. This report should NOT be used or interpreted as growth for an individual student!

Why No Predicted Score NOW? PVAAS Teacher Specific Reporting Common Questions

Why No Predicted Score Available NOW for SY14-15?  A student’s predicted score is based on the student’s own testing history and on the average performance of students in the same cohort statewide who have a similar testing history.  In other words, for each assessment we analyze with the predictive model, we look at the past testing histories of students across the state who took that assessment in the most recent year.  Because the predicted score is based on the performance of other students statewide who took the assessment in the same year, it’s not possible to generate the predicted score prior to the assessment.  What would the score tell you that a PVAAS projection/% of likelihood does not?

Why Different? School Value-Added vs. Teacher Value-Added Reporting PVAAS Teacher Specific Reporting Common Questions

Why Different? School Value-Added vs. Teacher Value-Added Reporting 2 Primary Reasons:  PVAAS School Reporting uses full academic year/Oct 1 as a requirement for including students  PVAAS Teacher Value-added reporting uses % Student + Teacher Enrollment and % Shared Instruction to determine the weighting of each student on each teacher’s PVAAS teacher Specific Reporting

Why Different? Composite v. Teacher Value- Added PVAAS Teacher Specific Reporting Common Questions

What is the Composite?  A combined growth measure across a teacher’s PVAAS reported subjects/grades/courses within SY  For SY13-14, we only have one year of reporting so the Composite Score is a combined growth measure across a teacher’s PVAAS reported subjects/grades/courses for SY13-14 only.  In future years, the Composite Score will represent a combined growth measure across a teacher’s PVAAS reported subjects/grade/courses across years, up to and including 3 consecutive school years.  1 year Composite  2 Year Composite  3 Year Composite/3 Year Rolling Average 1 Pager in Develop- ment

Teacher Value Added Summary The composite is a combined measure of all tested subjects, grades, and Keystone content areas in which a teacher received a Value Added report.

Composite: How Calculated  To calculate the composite, a simple average is taken of all of the teacher's individual index values for up to three years.  Then the average is multiplied by the square root of the number of individual index values that went into the average.  This step is a necessary step that accounts for the fact that more data was used to generate the average than was used to generate each individual index, which affects the standard error for the composite.

Why Different? Composite v. Teacher Value-Added  The growth color indicators are categories or ranges of indicators of growth. We cannot “average” categories. In other words, green + red ≠ yellow; likewise green + dark blue ≠ light blue.  For PVAAS, the assumption is that the achievement level of the teacher's group of students is maintained (green) UNLESS there is enough evidence in the assessment data to say otherwise.

Why Different? Composite v. Teacher Value-Added  It is all about the amount of evidence in the student assessment results – the assumption is that the achievement level of the teacher's group of students is maintained (green) UNLESS there is enough evidence in the assessment data to say otherwise.  The more data available, the more evidence we have to see if the group of students exceeded the growth standard OR not.  There is more evidence when all data was combined for a composite score. With data from multiple subjects (or multiple years when that’s available) included in the Composite, there’s more evidence/more data.  The composite is not really an “average,” but rather an accumulation of the evidence/data towards meeting, exceeding, or falling short of the standard for PA Academic Growth.

Why Different? Teacher Value-Added v. Diagnostic Reporting PVAAS Teacher Specific Reporting Common Questions

 Diagnostic Reports do NOT factor in the percent of Instructional Responsibility  Students are weighted equally  Reflect the growth of a group of students who may have had more than one teacher with instructional responsibility in that specific subject/grade or course Why Different? Teacher Value-Added vs. Diagnostic Reporting

Methodology Questions PVAAS Teacher Specific Reporting Common Questions

More Info About Methodology  PVAAS Methodologies, Measuring Growth & Projecting Performance – PVAAS login page, under Professional Development  Virtual Learning Modules – eLearning Link  Value Added Reports – Math/Reading  Value Added Reports – Science/Keystones  Teacher Specific Reporting  EVAAS Statistical Models in PA – PDE PVAAS site, Methodology Page

Full Day TOT Sessions for Admins PVAAS Teacher Specific Reporting

Participation  450 LEAs  Over 1700 Participants

Misunderstanding  PVAAS 3 Year Rolling Average  3 Consecutive Years of PVAAS Teacher Reporting  Any state assessed subject/grade/course  Does not need to be in the same subject/grade/course

Half-Day Teacher Sessions at IUs PVAAS Teacher Specific Reporting

Half-Day Sessions with Teachers – We Need Your Support!  Before session  Setup of session – registration, handouts, technology  Make sure all participants can get online  Make sure all participants log onto PVAAS site - check username and password and PPID before training starts  Download session PPT  Welcome Participants to your IU/Orient to Facility  During Session  Rotate around room during online work  Gather questions if not comfortable answering  Engage with participants  Ideas for Follow-up IU services  After Session  Gathering and Packing Up Materials  Copies of Session Feedback Forms  Final Registrant List

Feedback from Teachers PVAAS Teacher Specific Reporting

What will you tell your colleagues about PVAAS?

s to 499 Superintendents/Designees PVAAS Teacher Specific Reporting

Asked 4 Questions  How is it going overall?  Were most of the reports in sync with what principals expected?  Any/many surprises for the principals?  Any questions or information needed by you or your team?

“Best” Teacher  Qualities such as: students engaged in teacher’s classroom, using technology with students, positive feedback from students and/or parents, meets with students before/after school, goes to a student’s activities and sporting events, contributes to the school community, contributes to the district community, etc.  Could a teacher do all of these things listed above, but students may not be academically growing? is that possible?  Why?  Could a teacher do all of these things listed above AND their students ARE academically growing? is that possible?  Why?

NEW: PVAAS Help Menu System PVAAS Teacher Specific Reporting

New PVAAS Help Menu System System

Updates to PVAAS Keystone Reporting PVAAS District/School Reporting

Updates to Both PVAAS Reporting Sites:  PVAAS Restricted/Password Protected Site: Thursday, Nov. 20 th  Updates to Keystone Reporting  PVAAS Public Site: Thursday, Dec. 4 th

Updates: PVAAS Password Protected Site Keystones district/school value-added reports Keystones district/school performance diagnostic reports Keystones district/school quintile diagnostic reports PVAAS SPP/AGI reports PVAAS DPP/AGI reports Changes to students included on drill down in above reports based on new business rules for Keystone reporting

Questions? Feedback?