Kristen Lewald PVAAS Statewide Director for PDE PENNSYLVANIA VALUE-ADDED ASSESSMENT SYSTEM (PVAAS) UPDATE TO PAIU CC NOVEMBER 14, 2013

TOPICS o Feedback from Fall Sessions o PILOT Teacher Specific Reporting o Preparing for Roster Verification AND Statewide Teacher Specific Reportin g o Updates to Reporting

FEEDBACK FROM 99 FALL SESSIONS

WHO RESPONDED TO SURVEY? o Sent day of/after fall session o Consistent representation across IUs respondents o By LEA role: Superintendents/Asst Superintendents – 5% Director of Curriculum & Instruction – 5% Director of Special Education – 2% Principals/Asst Principals – 32% Teachers – 32% Other Roles – 24% –Coaches & Specialists –Guidance Counselors –Psychologists o Participated in PVAAS professional development prior to SY13-14? Yes – 47% No, this was my first training – 38%

VIEWED VIRTUAL LEARNING MODULES? Report-based VLMsConceptual VLMs VLM Feedback: o Easy to use and navigate o Helpful to have it broken into sections that you can view or repeat as needed o Thorough and informative o Activities are good!

USING PVAAS FOR A PURPOSE? My district or school uses Value-Added reports as part of grade/subject/course level team meetings with teachers to analyze growth of grade-level groups of students. My district or school uses Performance Diagnostic reports as part of grade/subject/course level team meetings with teachers to determine if educational programming is meeting the needs of all levels of learners.

USING PVAAS FOR A PURPOSE? My district or school uses Student Projection data to plan enrichment supports for students with a HIGH likelihood of being Advanced on a future PSSA/Keystone. Of the 26% who indicated YES above, 53% indicated that teachers are included in those discussions.

USING PVAAS FOR A PURPOSE? My district or school uses Student Projection data to plan interventions for students at risk of NOT reaching proficiency on a future PSSA/Keystone. Of the 49% who indicated YES above, 68% indicated that teachers are included in those discussions.

INDIVIDUAL PVAAS ACCOUNTS? Teacher Perspective: Have you been assigned your own individual PVAAS School User account? Admin Perspective: Have you assigned PVAAS School User accounts to teachers in tested subjects/grades/courses?

INDIVIDUAL PVAAS ACCOUNTS? Admin Comments on Assigning School User Accounts: o In process of doing this now o Have not been given permission to create these accounts o Don’t know how to do this! PIMS Administration Manual “How to Create Accounts” document on PVAAS login page

SESSION SUGGESTIONS Should this session be offered again? o Offer for new Administrators o Offer for teachers specifically o Differentiate sessions for: –Novices –Veterans o Differentiate sessions for: –Elementary –Middle School –High School o Make into a full-day session o Make into a course (possibly offered during the summer)

SESSION SUGGESTIONS This session provided enough time to review and discuss my district’s or school’s PVAAS data. o MORE TIME! –Need more time to look at and interpret data & discuss as a team o Need more support/resource facilitators o Need more time to spend discussing data with facilitator(s) o Limit number of districts per session to allow more one-on- one time with facilitator(s) o Limit session/audience size

OTHER RESOURCES, MATERIALS, SUPPORTS TO MAXIMIZE USE OF PVAAS? o Time to interpret/review the data with PVAAS support o Release PVAAS data sooner (July/August) – working on o More training on crosswalk between PVAAS and School Performance Profile o Who is included in % P/A; Who is included in PVAAS o Provide export option within PVAAS for student projection data – working on o More information on mathematical calculations and formulas for growth and projections– working on

 Survey Results by IU  Provided to all IUs in January  Will Send to IU PVAAS Contact FINAL SURVEY RESULTS BY IU

EXTRA GUIDES TO REPORTING

PILOT TEACHER SPECIFIC REPORTING

WEBINARS FOR ADMINISTRATORS: HOW ARE SESSIONS GOING? o First TWO sessions Tuesday, November 12 th Wednesday, November 13 th o Remaining FOUR sessions – We encourage you & others from your IU to participate! Monday, Nov 18 th – 1:00-3:30pm Tuesday, Nov 19 th – 1:00-3:30pm Thursday, Nov 21 st – 9:00-11:30am Friday, Nov 22 nd – 9:00-11:30am

WEBINARS FOR ADMINISTRATORS: HOW ARE SESSIONS GOING? o Averaged sites joining each session –Some sites had multiple participants (up to 20 people) o Some still have questions related to Concept of Growth and how growth is measured o Examples of other questions: –Do teachers need access to school or student reports to view their PILOT teacher reporting? –How do we update our District and School Admin accounts? –Do the Admin Summary reports include all teachers, or just those that submitted rosters (versus those who did NOT submit)? –Why can we not yet see a Teacher Value Added report?

BECOME VERY FAMILIAR WITH CONCEPT OF GROWTH TWO Virtual Learning Modules Introduction to Measuring Academic Growth (12-13 min) Concept of Growth: Connections to Value- Added Reporting (30 min)

BECOME FAMILIAR WITH PILOT REPORTING  Administrator reports  Teacher reports

GUIDE TO PILOT TEACHER SPECIFIC REPORTING Coming Soon! Available by release of PILOT Teacher Specific reporting on December 2 nd

MISCONCEPTIONS ABOUT PVAAS TEACHER SPECIFIC REPORTING Related to:  the student population served by educators  the test used in value-added modeling  the value-added modeling approach itself

GUIDED INQUIRY CONVERSATIONS WITH TEACHERS  Preview the teacher self-reflection document for guidance  Teacher can use prior to meeting with principal  Use it to guide the conversation

PILOT LEA S - KNOW WHICH TEACHERS SUBMITTED AND WHICH TEACHERS DID NOT SUBMIT If you do not have your Excel file to share with your LEAs, contact PVAAS Statewide Core Team at

DECISIONS SINCE SPRING 2013 ROSTER VERIFICATION  Actual N Count = 11 Students  Cumulative Count - Per tested grade/subject/course  “Active” N Count = 6 Students  Similar in concept to an “FTE”; 6 “active” students  Example: 11 actual students, each claimed for 50% Instructional Responsibility, would equal 5.5 “active” students” – not enough “active” students for a teacher to get PVAAS teacher specific reporting in this subject/grade  Minimum Threshold for % Instructional Responsibility: 10%  All students need to be rostered  <10% will not be included in a teacher’s analyses

WHAT IF AN ERROR IS FOUND? There will be NO changes made after the release of PILOT Teacher Specific reporting!

 Planning templates Thank you for confirming!  Still waiting on templates from 2 IUs  IU 15  IU 17 ON-SITE PROFESSIONAL DEVELOPMENT AT LOCAL IU S Put them here because I’m thinking some of them would be wondering did we complete? Should verify with Jen before session to see if still outstanding

o Monitor registration –Need to consolidate sessions if less than 15 participants –Need to limit registration so audience is not too large for room, nor too large for number of available facilitators o Encourage IU team participation! –PVAAS staff –Educator Effectiveness staff –Math consultants –TAC ON-SITE PROFESSIONAL DEVELOPMENT AT LOCAL IU S

IN SUMMARY…FOR PILOT LEA S o Encourage your PILOT LEAs to be prepared to support PILOT teachers as of Dec 2 nd o Advertise on-site PILOT training to your PILOT LEAs AND encourage them to register a team o PILOT Admin sharing experience with other Admin at job- alike meetings o Supt o Curriculum Directors o Principals o PILOT Teachers sharing experiences o Teacher Leaders/Content Leaders

PREPARING FOR ROSTER VERIFICATION & STATEWIDE TEACHER SPECIFIC REPORTING

QUESTIONS & SUPPORT ON EDUCATOR EFFECTIVENESS & PVAAS  Brian Truesdale and/or Kristen Lewald can join Q&A sessions at your IU, virtually  Videoconference  Skype  LYNC  Blackboard Collaborate  Some IUs have done this or have them scheduled- examples (not all inclusive):  IU9 – Supt retreat; one for elementary & one for secondary  IU3 – Curriculum Coordinators meeting  IU20 - Curriculum Coordinators meeting  IU24- Curriculum Coordinators meeting/PILOT Panel  IU18 - Curriculum Coordinators meeting  IU11- Curriculum Coordinators meeting

 PIMS/SIS work well under way OR PIMS/SIS work not in process at all  Many on the “program side” not in process yet/teachers do not know, etc.  LEAs saying they are excluding specific types of teachers – ex. only attributing to reg ed teacher; we give them questions to ask themselves  Literal interpretation of PDE guidance WHAT WE ARE FINDING

FOR ALL LEA S  Communicate the dates for “A Look at Teacher Specific Reporting” webinars for SY13-14 statewide implementation for admin and teachers NOT in the PILOT  View LIVE or archived webinar  January 9 th 2:00-3:30pm  January 17 th 9:00-10:30am  January 23 rd 9:00-10:30am

QUESTIONS TO ASK EACH OF YOUR LEAS  Have the LEA staff responsible for EE met with your LEA staff responsible for PIMS/SIS?  Has your SIS vendor provided your LEA an update tot her SIS system aligned to SY13-14 PIMS?  All teachers eligible for PVAAS in the SIS?  Scheduling Elementary Students in SIS?  All Keystone courses identified?  Structure for % Student + Teacher Enrollment in SIS?  Structure for % Full/Partial Instruction in SIS?  District Level Planning – to Ensure Consistency Across LEA  % Instructional Responsibility – Discuss LEA wide

 Awareness?  Do all district level admin know about Spring 2013 roster verification?  Do all school admin know about Spring 2013 roster verification?  Do the teachers eligible for PVAAS teacher specific reporting know about Spring 2013 roster verification?  Training?  When/how will district admin/school admin receive training on roster verification process and system?  When/how will teachers receive training on roster verification process and system?  Time to Complete Roster Verification in Spring 2014  When will teachers complete the roster verification process?  How will they be supported? Questions to Ask Each of Your LEAs

 Who to Ask?  Superintendents  Assistant Superintendents/ Director of C & I  How did IU5 get their LEAs engaged in this discussion? ASK EACH OF YOUR LEAS?

UPDATES TO REPORTING

Performance Diagnostic Reports SEMESTER LABELING FOR REPORTS ON KEYSTONE EXAMS

Student Projection Report

SEMESTER LABELING FOR REPORTS ON KEYSTONE EXAMS Student History Report

PROJECTION SUMMARY TO KEYSTONE EXAMS Before Now

PA SECURE ID ON STUDENT SEARCH RESULTS

QUESTIONS