1. Learning More About PVAAS Teacher Specific Reporting PVAAS Statewide Team for PDE January 2014 Webinars

2. Welcome! Session Needs: Jen Kurtz, pdepvaas@iu13.orgpdepvaas@iu13.org PVAAS Statewide Team for PDE Session Facilitator: Kristen Lewald, Ed.D. PVAAS Statewide Project Director for PDE SY12-13 PVAAS PILOT2 Session Facilitator: Joan Perekupka PVAAS Statewide Team for PDE Session Facilitator: Jennifer Ross PVAAS Statewide Team for PDE

3. Session Focus PVAAS: 2002 to Today Roster Verification Access to Roster Verification and Teacher Reporting Concept of Growth & Measuring Growth Types of Reports for Teachers Effective Use and Support for Teacher Specific Reporting Timeline for PVAAS Teacher Specific Reporting Resources and Supports -To receive a copy of today’s PPT in PPT format, email: pdepvaas@iu13.org pdepvaas@iu13.org -Session archived

4. PVAAS: 2002 TO TODAY Learning More About PVAAS Teacher Reporting

5. PVAAS: 2002 to Today 100 Pilot LEAs District & School Reporting Statewide Implementation: District & School Reporting Student Projections 273 Pilot LEAs Teacher PILOT Reporting NOW: Statewide Implementation of District, School, and Teacher Reporting & Student Projections

7. Copyright © 2013, SAS Institute Inc. All rights reserved. Purpose of PVAAS Teacher Specific Reporting: To inform teaching practice in order to guide replication, changes, or adjustments of teaching practices for continual professional growth.

8. ROSTER VERIFICATION: LEA DATA TO PVAAS TEACHER SPECIFIC REPORTING Learning More About PVAAS Teacher Reporting

9. Prior to the Release of Teacher Specific Reporting… Teachers, School Admins, and District Admins will complete the PVAAS Roster Verification Process

10. Prior to the Release of Teacher Specific Reporting… PVAAS Roster Verification Process – Teachers, School Admins, and District Admins verify that the right students are linked to the right teachers for the right subject/grade/course for the right % of Instructional Responsibility – The total instructional responsibility percentage for each student is comprised of two percentages. % student-teacher enrollment  the percentage of days that the teacher and the student have been concurrently enrolled; from the first school day of the tested grade/subject/course through the last school day before the LEA’s testing window opens for the tested grade/subject/course % full/partial instruction (shared instruction)  the percentage of instruction for which the teacher has responsibility for the student. – Responsibility for planning, delivering, and assessing instruction – Responsibility for components of Domains 1 and 3

11. PIMS Course/HQT into PVAAS Roster Verification System PIMS Course/HQT Collection Windows Purposes of this Collection: – HQT – PVAAS Teacher Specific Reporting – Courses of Rigor for SPP SY13-14 Schedule: – Jan 6 – Feb 3 Collection window – Feb 10 – March 21 Correction window March 28 File sent from PIMS to PVAAS

12. From your LEA into PIMS into PVAAS 12 PVAAS Roster Verification Process

13. PVAAS Roster Verification Timelines Teachers (May 5-16) School Admins (May 19-30) District Admins (June 2-6) SAS EVAAS® Strongly Recommended Make sure it is ready for your teachers! Strongly Recommended Make sure it is ready for your teachers! LEA Start-Up (April 28-May 2)

14. Release of Teacher Specific Reporting April-June 2014 LEAs complete PVAAS Roster Verification Process Summer 2014 DRC sends final assessment files to SAS EVAAS® Summer/Fall 2014 SAS EVAAS® completes PVAAS Teacher Specific Analyses and Web- Based Reporting Fall 2014 SY13-14 reports released to LEAs (Admins and Teachers) Email Notification

15. PVAAS Roster Verification Training Opportunities Webinars – live or recorded; same as March Face to Face Sessions Dates/Times: – February 4, 2014 12:30-3:30pm – February 5, 2014 8:30-11:30am – February 10, 2014 12:30-3:30pm – February 13, 2014 8:30-11:30am – February 18, 2014 12:30-3:30pm – February 20, 2014 8:30-11:30am – February 24, 2014 12:30-3:30pm – February 27, 2014 8:30-11:30am No Registration Necessary for Webinars! Instructions for joining webinar are found in SY2013-14 PVAAS Teacher Specific Reporting Guide to Implementation booklet found on PVAAS login page

16. PVAAS Roster Verification Training Opportunities Face-to-Face Sessions – same as Webinar Sessions Locations/Dates/Times: – PaTTAN Pittsburgh Office March 6, 2014 - 8:30-11:45am OR 12:30-3:45pm March 7, 2014 – 8:30-11:45am – PaTTAN King of Prussia Office March 11, 2014 - 8:30-11:45am OR 12:30-3:45pm March 12, 2014 – 8:30-11:45am – PaTTAN Harrisburg Office March 13, 2014 - 8:30-11:45am OR 12:30-3:45pm March 14, 2014 – 8:30-11:45am Registration is REQUIRED for Face-to-Face Sessions! – Registration Deadline: February 28, 2014 – Instructions for registration are found in SY2013-14 PVAAS Teacher Specific Reporting Guide to Implementation booklet found on PVAAS login page

17. Roster Verification Virtual Learning Module Self- Guided, Web-based Module Available in February 2014 Anticipate time of 45-60 min Learn about Roster Verification (RV) – What is it? – Why do we need to do RV? – % Instructional Responsibility – How to do RV?

18. ACCESS TO PVAAS ROSTER VERIFICATION AND PVAAS TEACHER REPORTING Learning More About PVAAS Teacher Reporting

19. Access to Teacher Specific Reporting Access to PVAAS – PVAAS Username – PVAAS Password – Double level of authentication is required to access PVAAS Teacher Specific Reporting Administrators need to know their PPID Teachers need to know their PPID No account sharing!! – 1 Person per PVAAS Account

20. Will PVAAS teacher-specific reporting be reported publicly? Act 82 states that “An employee's individual rating form shall not be subject to disclosure under the act of February 14, 2008 (P.L. 6, No. 3), known as the "Right-to-Know Law." PDE will follow the law on non-disclosure.

21. What does my District or School Admin see? The same PVAAS data and reporting for an individual teacher that a teacher can see No additional data or reporting on any individual teacher! School Admins can also: – See a District/School Teacher Summary for quick access by school/grade/subject/course – Search to locate an individual teacher’s report

22. PVAAS User Accounts.XLS of all of your LEA User accounts will be sent to PVAAS District Admin account holder in January from IU PVAAS Contact – If you do not receive by the end of January, contact IU PVAAS Contact. Make sure everyone who needs an account has one by the end of March 2014 – Use PIMS Staff Template for account creation – Modify/Delete accounts locally Teachers who will do Roster Verification must have a PVAAS SCHOOL USER account!

23. Account Management for PVAAS Roster Verification Training Opportunities Webinars – live or recorded Dates/Times: – January 28, 3:00-4:00pm – January 29, 9:00-10:00am – January 30, 3:00-4:00pm No Registration Necessary for Webinars! – PVAAS Account Administrator; PIMS Admin Instructions for joining webinar are found in SY2013-14 PVAAS Teacher Specific Reporting Guide to Implementation booklet found on PVAAS login page

24. CONCEPT OF GROWTH Learning More About PVAAS Teacher Reporting

25. PVAAS Reporting: Growth and Projections PVAAS Teacher Specific Reporting is about GROWTH!

26. Is it at least reasonable…. To be expected to maintain the achievement level for a group of students??? – If a higher achieving group of students: to keep them high achieving? – If an average achieving group of students: to keep them at least at that level? – If a lower achieving group of students: to keep them at least at that level, and not slip as a group? We may need to yield more growth if increased achievement is the goal.

27. Achievement vs. Growth Measures students’ performance at a single point in time Highly correlated with students’ demographic background Measures students’ progress across time (i.e., across years and grade levels) Not related to demographic background when a student’s prior achievement is adequately accounted for; dependent on what happens as a result of schooling

28. Teacher Specific Evidence #1: Growth and % Economically Disadvantaged Higher % Econ Disadv Lower % Econ Disadv

29. Teacher Specific Evidence #2: Growth and % Minority Higher % Minority Lower % Minority

30. Teacher Specific Evidence #3: Growth and % LEP Higher % LEP Lower % LEP

31. Teacher Specific Evidence #4: Growth and % Special Education Higher % Special Ed Lower % Special Ed

32. Teacher Specific Evidence #5: Growth and Achievement Higher Avg Achievement Lower Avg Achievement

33. Teacher Specific Evidence #6: Growth and Number of Students per Roster More Students Fewer Students

34. Previous Year’s Group Grade 8 Most Recent Year’s Group Grade 8 Measuring Academic Growth in PVAAS

35. Grade 3 Grade 4 Grade 5 Grade 6 Measuring Academic Growth in PVAAS

36. Measuring Academic Growth in PVAAS PVAAS does NOT compare the percentages of students at various PSSA performance levels to measure growth!

37. Measuring Academic Growth in PVAAS PVAAS does NOT compare the percentages of students at various PSSA performance levels to measure growth!

38. Measuring Academic Growth in PVAAS Students don’t all start at the same place academically

39. Key Ideas for PVAAS Growth Measures No individual student growth measures! PVAAS Growth provides a measure of the growth of a teacher’s group of students Students serve as their own control

40. MEASURING GROWTH FOR TEACHER SPECIFIC REPORTING Learning More About PVAAS Teacher Reporting

41. Teacher Verified Roster Per Subject/Grade/Course

42. Why Would a Student NOT Be Included in the Analyses? Did not take the PA state assessment Took the PASA (alternative assessment) Did not have a sufficient testing history in prior years Is a first year ELL student Is a Foreign Exchange student Teacher’s Instructional Responsibility for a student does NOT meet minimum threshold  < 10%, then not included in the value-added analyses

43. Did all students take the test? Per Subject/Grade/Course

44. Did all students take the test? 2 students did not take the test - not included Did Not Take Test

45. % Instructional Responsibility 100% 50% 100% 5%

46. % Instructional Responsibility Any student <10% NOT Included 100% 5% 50% 100% 5%

47. Are there at least 11 different students? Actual N Count ≥ 11 100% 50% 100% YES Actual N=22

48. Instructional Responsibility Active N Count Must ≥ 6 100% 50% 100%

49. Instructional Responsibility Is there an Active N Count ≥ 6? 100% 50% 100% Actual N = 22 Active N = 19 students 100% = 1 Active YES

50. PA Data Teacher’s Group of Students All Prior Data Estimated Prior Achievement of Teacher’s Group of Students Measuring Growth: Students’ Starting Point

51. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Measuring Growth: Next Round of Testing

52. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Measuring Growth: Next Round of Testing

53. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Measuring Growth: Next Round of Testing

54. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Measuring Growth: Next Round of Testing

55. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Using the % Instructional Responsibility, students are weighted accordingly in the value-added analysis. Concept of Measuring Growth

56. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Using the % Instructional Responsibility, students are weighted accordingly in the value-added analysis. Concept of Measuring Growth

57. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Using the % Instructional Responsibility, students are weighted accordingly in the value-added analysis. Concept of Measuring Growth

58. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Using the % Instructional Responsibility, students are weighted accordingly in the value-added analysis. Concept of Measuring Growth

59. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Using the % Instructional Responsibility, students are weighted accordingly in the value-added analysis. Concept of Measuring Growth

60. Prior Achievement Teacher’s Group of Students Higher Achievement Lower Achievement Higher Achievement Lower Achievement Current Achievement Teacher’s Group of Students Using the % Instructional Responsibility, students are weighted accordingly in the value-added analysis. Concept of Measuring Growth

61. What is the PVAAS Growth Index? The Growth Measure is a measure of growth for a teacher’s group of students’ growth in a tested grade/subject/course The Growth Index is a value based on the relationship between the Growth Measure and its corresponding Standard Error If the Standard Error is not accounted for, users might get a skewed picture of the academic growth for a teacher’s group of students The PVAAS Growth Index is calculated by dividing the Growth Measure by the corresponding Standard Error. The PVAAS Average Growth Index provides a common measure across subjects, grades, courses, and teachers!

62. All Reporting: Same Color Legends

63. TYPES OF REPORTS FOR TEACHERS Learning More about PVAAS Teacher Reporting

64. Types of Teacher Specific Reporting Single Year, Multiple Grades/Subjects/Courses – Value-Added Summary – Diagnostic Summary Multiple Years, Single Grade/Subject/Course – Teacher Value-Added Report – Diagnostic Report Single Year, Single Subject/Grade/Course – Custom Diagnostic

65. % Instructional Responsibility % Student + Teacher Enrollment % Full/Partial Instruction Weighting of students is included in a teacher’s Value Added report

66. % Instructional Responsibility NOT Included in Diagnostic Reporting A diagnostic report does NOT reflect: – If there is more than one teacher – If there is less than 100% Student + Teacher Enrollment Students are not weighted

67. Teacher Value Added Summary

68. Diagnostic Summary

69. Teacher Value Added Report

70. Diagnostic Report

71. Custom Diagnostic Report

72. EFFECTIVE USE AND SUPPORT FOR TEACHER SPECIFIC REPORTING Learning More About PVAAS Teacher Reporting

73. Copyright © 2013, SAS Institute Inc. All rights reserved. Purpose of Teacher Specific Reporting: To inform teaching practice in order to guide replication, changes, or adjustments for continual professional growth.

74. Remember….. PVAAS is one measure that provides information about the academic growth of a teacher’s students Designed to help teachers reflect on their professional practice May be a new type of information for a teacher

75. Teacher Specific Reporting 101 Educate Communicate Collaborate Challenge Security Support Feedback Teachers Will Need Support

76. If seeds are thrown on unprepared ground, left to their own devices to survive without water or care, how positive will the results be for that season? If teacher reports are released to teachers without training, understanding, coaching, support and validation, how effective will the reports be to inform teacher practices? Teacher reports are much like planting a garden

77. Guiding Questions for Teachers How can you use the information to define areas of strength for replication over the next year? How can you use the information to define focus areas for self- improvement over the next year?

78. TIMELINE FOR PVAAS TEACHER SPECIFIC REPORTING Learning More About PVAAS Teacher Reporting

79. Counts Towards the FIRST 3 Year Rolling Average! This Year!

80. SY13-14 + SY14-15 + SY15-16= First PVAAS 3 Year Rolling Average in SY15-16

81. SY14-15 + SY15-16 + SY16-17 = Next PVAAS 3 Year Rolling Average in SY16-17

82. RESOURCES AND SUPPORTS Learning More About PVAAS Teacher Reporting

83. Become VERY Familiar with Concept of Growth TWO Virtual Learning Modules Introduction to Measuring Academic Growth (12-13 min) Concept of Growth: Connections to Value- Added Reporting (30 min)

84. Virtual Learning Modules: PILOT Teacher Specific Reporting Free, online professional development Access through PVAAS website 2 NEW Modules – Teacher Reports (45 min) – Admin Reports (15 min) Available since November 19 th

85. Misconceptions Addressed Common Misunderstandings Available on PVAAS login page https://pvaas.sas.com https://pvaas.sas.com

86. PVAAS Help Menus 33 PVAAS Custom Help Menus Including NEW help menus for: Teacher Reports Admin Reports

87. PVAAS Login Page

88. PDE PVAAS Webpage http://www.education.state.pa.us/pvaas

89. Intermediate Unit PVAAS Contacts

90. Questions? PVAAS Materials or Statewide Implementation pdepvaas@iu13.org 717-606-1911 PVAAS Report Web Site https://pvaas.sas.com

91. www.pde.state.pa.us PENNSYLVANIA Value Added Assessment System ______________________________________________________________________ PVAAS 333 Market Street Harrisburg, PA 17126