Are You reEEEady to rosterRRR?! SY16-17 PIMS Staff-Student-Subtest: PVAAS Roster Verification Slides to update each year 2 – SAS EVAAS facilitator and pic 3 – links for session recording and additional materials, if changed 4 – update with new screen shot of Google page for RV resources 6 – release of e-Learning modules 7 – Update to sections of e-Learning modules 12 – Update to any additional account management resources 16 – Update PIMS submission deadline 17 – Update with PIMS submission deadline info 18 – Update PIMS help desk information 29-31 – Update RV phase dates and info 34 – Update year in which summer is represented for RV 37 – Update RV dates 41, 43-44 – Update RV dates 70 – Update year in which summer is represented for RV 83 – Update link and where to go for resources 84 – update with new screen shot of Google page for RV resources 89 – Same updates as slide 6 (release of e-Learning Modules) 90 – Same updates as slide 7 General considerations for updates: After slide 9 – include any new features to the RV system January 18, 2017 Data Quality Network PVAAS Statewide Team for PDE • pdepvaas@iu13.org PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Welcome! Session Facilitator: Session Facilitator: <Introduce session facilitator(s)> Session Facilitator: Kristen Lewald, Ed.D. PVAAS Statewide Project Director for PDE Session Facilitator: Candice Felton Educator Effectiveness Data Administrator PDE/PIMS PVAAS Statewide Team for PDE - pdepvaas@iu13.org

PIMS Staff Student Subtest What is PVAAS RV? PIMS Staff Student Subtest

What is PVAAS Roster Verification? Roster Verification is a process in which teachers and administrators ENSURE and VERIFY that the right students are linked to the right teachers for EACH state assessment, and for the right proportion of instructional responsibility! Teacher Student Proportion of Instructional Responsibility State Assessment It’s important for a teacher’s rosters to accurately reflect all students in each tested grade and subject or course for a given school year AND the proportion of responsibility for each state assessment, because the verified rosters and the percent of instructional responsibility for each student are used to generate teacher-specific PVAAS reports. PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Ensuring Accurate Information: PVAAS Roster Verification Provide PVAAS teacher specific reporting based on an accurate representation of a teacher’s responsibility and influence on the academic growth of students. Copyright © 2010, SAS Institute Inc. All rights reserved.

What is PVAAS Teacher Specific reporting? Estimates the academic growth of a teacher’s group of students. PVAAS Teacher Specific Reporting Part of Pennsylvania’s new Educator Effectiveness System Teacher Evaluation System Principal Evaluation System Employment decisions are linked to evaluation Accurate Data is Essential! Copyright © 2010, SAS Institute Inc. All rights reserved.

PIMS-PVAAS Timelines PVAAS Statewide Team for PDE - pdepvaas@iu13.org

PIMS template information: PVAAS enrollment /Gap enrollment Student, School Enrollment, Programs Fact PVAAS staff Staff PVAAS Student, School Enrollment, Staff, and Staff Student Subtest

PIMS exports to PVAAS: The PIMS Data Collection Team runs reports that behave like an internal snapshot. The data is pulled after the NOON cutoff date listed on the Elementary-Secondary Data Collection calendar for all PVAAS PIMS submissions. Data in the PIMS data warehouse is exported to prepopulate the PVAAS system. Staff Student Subtest data can NOT be changed via PIMS after March 20th. All changes must occur in the PVAAS roster system after this date.

PIMS submissions: C6 is accessible throughout the year. Use C6 to submit all PVAAS data. Use can also use the PIMS Sandbox to start submitting preliminary data for the purpose of error checking.

PIMS Staff-Student-Subtest: Pre-population of Rosters Staff-Student-Subtest (SSS) data submission is dependent on the Student, Enrollment, and Staff templates. If you have a staff member listed in the SSS template, that staff member has to exist in the Staff template. If you have a student listed in the SSS template, the student must exist in both the Student and Enrollment templates. It will provide an error if you try to submit something in SSS that is not aligned to the other templates!  PVAAS Statewide Team for PDE - pdepvaas@iu13.org

PIMS Staff-Student-Subtest: Pre-population of Rosters Staff-Student-Subtest template: PIMS Collection “C6 PVAAS 16-17” Deadline to submit is March 20, 2017 at 12:00 noon Hard deadline! There will NOT be an additional correction window for this submission! This submission window will NOT be extended! Prepopulated rosters CANNOT be added from PIMS after this date RV system is “linked and live” statewide Cannot take RV system offline once it is open Only option for “changes” is through the RV system itself during the designated phases OR adding students to the district via Gap enrollment It is important to note that the deadline for districts and LEAs to submit data into PIMS for pre-population of the PVAAS rosters is March 20 at 12:00 pm (NOON). There will be NO additional correction window for this submission AND the window will NOT be extended. PVAAS Statewide Team for PDE - pdepvaas@iu13.org

PIMS Staff-Student-Subtest: Pre-population of Rosters Prepopulated from your LEA into PIMS into PVAAS PIMS Staff-Student-Subtest: Submission Deadline March 20, 2017 at 12:00 noon The rosters in the PVAAS system are pre-populated with data from PIMS - the Pennsylvania Information Management System – which is the data submitted to PDE by districts and LEAs. PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Updates to PIMS manual 1 for Staff Student Subtest template: PIMS manual #1 pdf page 71- field 10- Name “PERCENTAGE OF SHARED INSTRUCTION (INSTRUCTIONAL RESPONSIBILITY WEIGHT)” Added description: “PVAAS-Roster Verification field-“Full or Partial % of Instruction” PIMS manual #1 pdf page 72- field 11- Name “PERCENTAGE OF CONCURRENT ENROLLMENT (INSTRUCTIONAL RELATIONSHIP WEIGHT)” Added description: “PVAAS-Roster Verification field- “Student + Teacher Enrollment” to Name description cell.

% Instructional Responsibility % Student + Teacher Enrollment Full or Partial % of Instruction What proportion of the grade/subject/course are the student and teacher concurrently enrolled? Of time the student & teacher were enrolled together, is this teacher the only PA-certified educator providing content-specific instruction (Domains 1 and 3) of the assessed eligible content for a state assessment, or is it shared with other teacher(s)? % Instructional Responsibility Let’s think about this another way – The % of Total Instructional Responsibility is equal to % student + teacher enrollment multiplied by the full/partial % of instruction. You do not have to do this calculation. The PVAAS system does this for you. You only need to verify the two %s of % student + teacher enrollment and full/partial % of instruction – the system will then multiply it for you to get the total % instructional responsibility. PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Verifying Instructional Responsibility The percentage of instructional responsibility claimed determines how much each student is weighted in generating the individual teacher value-added measures! The total percentage of instructional responsibility claimed determines how much each student is weighted in generating the individual teacher value-added measures. This is why verifying these percentages and verifying all students with integrity and accuracy is important. % Instructional Responsibility PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Verifying Instructional Responsibility In the PVAAS roster verification system, there are two columns for claiming and verifying instructional responsibility: one labeled percentage of Student and Teacher Enrollment and the other labeled Full or Partial % of Instruction. Let’s start by talking about the percentage of Student plus Teacher Enrollment. % Student + Teacher Enrollment Full or Partial % of Instruction PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Staff Student Subtest Example 1:

Staff Student Subtest Example 2:

PIMS template verification reports: Run verification reports after each successful submission of data to PIMS. Staff Template Details Student Template Details Student Enrollment Template Details Staff Student Subtest Template Details Location: Public Folders > eScholar Framework for Cognos - Verify > Validation Reports > Template Verification >

PIMS Production Error reports: Run production reports after the noon or 5:30 a.m. refresh. Run these reports to see errors in data that need to be corrected. Staff Student Subtest-PSSA Grade Level Error Report Staff Student Subtest Over-Claimed Students Location: Public Folders > eScholar Framework for Cognos - Production > PVAAS – Teacher Specific > Error Reports >

PIMS Production Possible Error Reports: Run production reports after the noon or 5:30 a.m. refresh. Run these reports to see possible errors in data that might need to be corrected. Staff Student Subtest - Students Missing PSSA Subtest Records Staff Student Subtest Count of Students by Assessment Type Staff Student Subtest Under- Claimed Students Location: Public Folders > eScholar Framework for Cognos - Production > PVAAS – Teacher Specific > Validation Reports

Submit Early… Update later! Submit and correct as early as possible. This gives you time to deal with potential complications related to: Submission formats SIS report bugs DQE rules Reports analysis Update near the end of the window to get the data as accurate/current as possible before RV data population. Again, there are no extensions for these data collections.

Final Submission of All Rosters from the LEA/District Admin “The submission of all rosters from your LEA as of the closing date of the LEA Verification phase below is indication that all roster data has been reviewed and verified, and is true and correct to the best of your knowledge. This is also an indication that any and all disputes related to an individual educator’s roster data have been resolved prior to submission.” “The FINAL submission of all rosters from your LEA as of the closing date of the LEA Verification Phase below is indication that all roster data has been reviewed and verified, and is true and correct to the best of your knowledge.  This is also an indication that any and all disputes related to an individual educator’s roster data have been resolved prior to submission.” PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Who Participates in PVAAS RV?

District Administrators School Administrators Who Participates in RV? District Administrators School Administrators Teachers Permanent or temporary professional employees with a valid PA teaching certificate who have full or partial responsibility for assessed eligible content as assessed by the PSSA or Keystone exams During the roster verification process, teachers, school administrators, and district administrators will all review and verify the roster for each state assessment. For PSSA tested grades and subjects, there will need to be one roster per tested subject and grade. For Keystone-tested courses, there will need to be one roster each for students tested in the Summer, students tested in the Winter (students typically enrolled in 1st semester courses), and students tested in the Spring (students typically enrolled in 2nd semester courses or students enrolled in year-long courses). In other words, students tested in EACH of these three testing windows will need to be on separate rosters.   But which teachers need to participate? Teachers who are permanent or temporary professional employees who hold a valid Pennsylvania teaching certificate AND who have full or partial responsibility for content-specific instruction of assessed eligible content as measured by the PSSA or Keystone exams, with or without accommodations. Guidance for content-specific may also include teachers working with students not yet proficient on the eligible content; rather, the teacher is providing instruction with the goal of the student working towards proficiency on the eligible content. This includes teachers who teach ELA and math in Grades 4-8; science in Grades 4 and 8; and/or teachers responsible for the instruction of students enrolled in a Keystone course as identified by the district or LEA. PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Teacher Eligibility: To Guide Your Thinking Act 82 regulations include the following language: “A professional or temporary professional employee who provides direct instruction to students related to a specific subject or grade level.” Responsibility and eligibility for PVAAS teacher-specific reporting is a determination made locally by the LEA. The LEA should consult with its solicitor regarding any such determination because the application of PVAAS teacher-specific data has employment implications. Teachers Pennsylvania defines the teacher of record as “a professional or temporary professional educator assigned by a school in entity as the primary instructor for a group of students.” Responsibility and eligibility for PVAAS teacher-specific reporting is a local determination – made by the LEA (locally). PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Teachers and Administrators RV Phases Teachers and Administrators

PVAAS Roster Verification Phases Make sure the system is ready for your teachers! Teachers can Preview BUT cannot edit during this phase! The process is divided into several phases of time based on your role. These phases help to ensure that the process flows smoothly, and that teachers’ rosters are reviewed and verified within the allotted time. As the process moves from one person to the next, the primary responsibility for roster verification shifts from District and School Admins, to Teachers, back to School Admins, and then to the District Admins. The goal is for each person or Roster Approver to complete their part of the process and pass the rosters on to the next person/Roster Approver before each phase closes. These are HARD deadlines. The phases lock at 11:59pm on the designated dates and cannot be accessed after that time! Each phase will close on a Friday, at 11:59 pm, and roll into the next phase Saturday at midnight. E-mail notifications to the appropriate users for the next phase, informing them of the opening of the phase, will be sent on the Monday of each phase. HARD DEADLINES! (each phase ends at 11:59pm on the Friday ending date) PVAAS Statewide Team for PDE - pdepvaas@iu13.org

PVAAS Roster Verification Phases The LEA Preview phase will open mid-week this year! Begins on WEDNESDAY, May 3 at 12:01am PVAAS Statewide Team for PDE - pdepvaas@iu13.org

What is the “Copy Roster” option? The copy roster functionality allows users to copy students from one roster to another roster: from one school to another school within the same district from one teacher to another within the selected school from one roster to another roster for the selected teacher The copy roster functionality allows users to copy students from one roster to another roster: District admin and district users with approver permission will be able to copy from one school to another school within the same district School admin and school users with approver permission will be able to copy from one teacher to another within the selected school Teachers will be able to copy from one of their rosters to another roster Copy Percentages 100% PVAAS Statewide Team for PDE - pdepvaas@iu13.org

FINAL PHASE: LEA/District Verification Phase Saturday, June 10 – Friday, June 23, 2017 @ 11:59pm Review and verify each school’s rosters Help resolve underclaiming and overclaiming issues, if needed Pass rosters back to School Admin for additional changes, if needed Submit all schools’ rosters to SAS® EVAAS® Once the School Admin has reviewed/verified and approved all rosters in the school, the rosters are submitted to the District Admin for another round of review. If the School Admin does not submit the rosters by the end of the phase, the rosters are considered verified and will be passed to the District Admin automatically by the roster verification system. During the LEA Verification Phase, the District Admin reviews the rosters submitted by each school in the district. In addition, the District Admin may need to help resolve issues of overclaimed students WITHIN the district or LEA. All overclaiming WITHIN the LEA is to be resolved prior to the final submission of rosters by the district or LEA. Districts/LEAs do NOT need to resolve overclaiming across districts or LEAs (unless LEAs share a student simultaneously, such as co-teaching between a district teacher and an IU teacher). Inaccurate underclaiming will need to be addressed as well. There are expected reasons for underclaiming – the simplest being that the student or teacher was new to Pennsylvania public schools part of the way through the subject and grade or course. The District Admin may return a school’s rosters back to the School Admin for additional changes. The rosters can be passed back and forth as many times as needed to resolve all issues, within the timeframe allotted for the LEA Verification Phase. [click] After rosters are submitted to SAS EVAAS, they are considered final and cannot be returned to your district/LEA. We encourage LEAs to wait until the end of the phase to submit, in case changes are needed! After rosters are submitted to SAS EVAAS, they are final and cannot be returned to your district/LEA. We encourage LEAs to wait until the end of the phase to submit, in case changes are needed! PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Resources and Supports

RV Resources on the PVAAS Google Site https://sites.google.com/a/iu13.org/pvaas-pl-resources/roster-verification-resources PVAAS Statewide Team for PDE - pdepvaas@iu13.org

PIMS Help Desk Don’t wait until the last week Need assistance with PIMS data submission? PIMS Help Desk: 800-661-2423 PIMS email: RA-DDQDataCollection@pa.gov Don’t wait until the last week for Staff-Student-Subtest submission to ask for help! PVAAS Statewide Team for PDE - pdepvaas@iu13.org

Additional Trainings RV e-Learning Module: Part 1 and 2 https://evaas.sas.com/videos/PA/RosterVerification/story.html February 15: Are Yooooouuu Ready to Roster?! (Training for NEW Administrators) Date and Time: Wednesday, 2/15/2017, 8:30 – 11:30 AM Link to Join Live: https://attendee.gotowebinar.com/register/8648028766622985475 Recording will be posted at http://tinyurl.com/PVAAS-RV-Resources. Recordings are typically posted within 24 hours of the live session. If your LEA does not allow for the use of tiny urls, please contact pdepvaas@iu13.org for the full hyperlink. March 2017: Session at PDE Data Summit: Are Yooooouuu Ready to Roster?!

PVAAS Statewide Team for PDE - pdepvaas@iu13.org

PVAAS Materials or Statewide Implementation pdepvaas@iu13 PVAAS Materials or Statewide Implementation pdepvaas@iu13.org PVAAS Report Web Site https://pvaas.sas.com PVAAS Statewide Team for PDE - pdepvaas@iu13.org

333 Market Street Harrisburg, PA 17126 PENNSYLVANIA Value Added Assessment System ______________________________________________________________________ PVAAS 333 Market Street Harrisburg, PA 17126 www.education.pa.gov PVAAS Statewide Team for PDE - pdepvaas@iu13.org