Measuring Principal Effectiveness Overview for Central Office Administrators 08/13/14 - Draft

Multiple Measures Principal Effectiveness Ratings as Defined in Act 82 For the overall principal effectiveness rating, we know that… 50% will comprise the observation/practice piece (Framework for Leadership). 15% will be derived from building level data (School Performance Profile – SPP). 15% will be determined by the relative strength of conversation regarding the connectedness between the average teacher observation/practice rating and teacher-level measures. 20% will incorporate Elective Data / SLOs for principals. (note: for 2014-14, elective data is optional) This applies for professional employees and temporary professional employees serving as principals, assistant principals, and CTC Directors. 2

Principal Effectiveness System in Act 82 of 2012 Building Level Data/School Performance Profile Indicators of Academic Achievement Indicators of Closing the Achievement Gap, All Students Indicators of Closing the Achievement Gap, Historically Underperforming Students Academic Growth PVAAS Other Academic Indicators Extra Credit for Advanced Achievement Observation/ Practice Framework for Leadership Domains Strategic/Cultural Leadership Systems Leadership Leadership for Learning Professional and Community Leadership Language within HB 1901 (pages 42 -44) For professional employees and temporary professional employees serving as principals, assistant principals, and CTC Directors the following shall apply: Correlation Data/Relationship Based on Teacher Level Measures Elective Data/SLOs District Designed Measures and Examinations Nationally Recognized Standardized Tests Industry Certification Examinations Student Projects Pursuant to Local Requirements Student Portfolios Pursuant to Local Requirements 3 June 26, 2014

Multiple Measures Non Teaching Professional Employee Ratings as Defined in Act 82 For the overall principal effectiveness rating, we know that… 80% will comprise the observation/practice piece (Framework for Leadership). 20% will be derived from building level data (School Performance Profile – SPP). This applies for professional employees and temporary professional employees who supervise, but are NOT serving as principals, assistant principals, and CTC Directors. Special Education and Curriculum Directors who supervise are examples of employees who fall within this category. 4

Non Teaching Professional Employee Effectiveness System in Act 82 of 2012 Observation and Practice Planning and Preparation Educational Environment Delivery of Service Professional Development Student Performance/School Performance Profile (SPP) June 26, 2014

Rating Tools Useful for Determining a Principal / School Leader’s Rating A rating tool (PA 82-2) has been developed that incorporates all aspects of the Principal Effectiveness System such that a final rating for a principal can be determined. Final ratings and all supporting evidence/documentation for individuals will be kept at the local level. Only aggregate ratings information will be submitted to PDE. The rating tool for Non Teaching Professional Employees (PA 82-3) has also been developed. The rating tool reflects a similar design as the one developed for teacher evaluation (PA 82-1). Show only PA 82-2 (for principals / school leaders) The goal is to show participants how data is entered into the rating tool and the resulting final rating form. Providing this information at the beginning of the presentation address the “compliance” questions related to what “forms” need to be completed. Prior to continuing presentation: Check for understanding on the use of the Rating Tool. 6

Framework for Leadership 7

Framework for Leadership Domains The Framework for Leadership establishes a set of four leadership domains: Domain 1: Strategic/Cultural Leadership Domain 2: Systems Leadership Domain 3: Leadership for Learning Domain 4: Professional and Community Leadership The Framework for Leadership contains specific components (with corresponding descriptors) to be included in each of the four domains. 8

Domain 1: Strategic/Cultural Leadership With Components Domain Descriptor: Principals/school leaders systemically and collaboratively develop a positive culture to promote student growth and staff development. They articulate and model a clear vision of the school’s culture that involves students, families, and staff. Components Included in Domain: Creates an Organizational Vision, Mission, and Strategic Goals Uses Data for Informed Decision Making Builds a Collaborative and Empowering Work Environment Leads Change Efforts for Continuous Improvement Celebrates Accomplishments and Acknowledges Failures Selection of the components in this domain are intended to be strategic in nature. These components help to establish the climate/culture of the school. Here we are looking at the context of the school. Principals are critical in establishing the climate of the building. Uses of Data  Does the principal understand and use data? Collaborative and Empowering Work Environment  Principals play a critical role in retaining teachers. Principals promote collegiality in the development of building level goals. 9

Domain 2: Systems Leadership With Components Domain Descriptor: Principals/school leaders ensure that there are processes and systems in place for budgeting, staffing, problem solving, communicating expectations and scheduling that result in organizing the work routines in the building. They must manage efficiently, effectively and safely to foster student achievement. Components Included in Domain: Leverages Human and Financial Resources Ensures a High Quality, High Performing Staff Complies with Federal, State, and LEA Mandates Establishes and Implements Expectations for Students and Staff Communicates Effectively and Strategically Manages Conflict Constructively Ensures School Safety Selection of the components in this domain are intended to be tactical in nature, which look at a school from a systems perspective. Federal Mandates  Schools receiving School Improvement Grants (SIG) need to be sure those requirements are included here. This is the managerial piece as well. It focuses on: Communication Compliance Conflict Management 10

Domain 3: Leadership for Learning With Components Domain Descriptor: Principals/school leaders ensure that a Standards Aligned System is in place to address the linkage of curriculum, instruction, assessment, data on student learning and teacher effectiveness based on research and best practices.. Components Included in Domain: Leads School Improvement Initiatives Aligns Curricula, Instruction, and Assessments Implements High Quality Instruction Sets High Expectations for All Students Maximizes Instructional Time Federal Mandates  Schools receiving School Improvement Grants (SIG) need to be sure those requirements are included as part of their school improvement initiatives as principals are evaluated. Student achievement is at the core of this Domain. Principals must establish high expectations, facilitate quality instruction, which are the key components of student performance. 11

Domain 4: Professional and Community Leadership With Components Domain Descriptor: Principals/school leaders promote the success of all students, the positive interactions among building stakeholders and the professional growth of staff by acting with integrity, fairness and in an ethical manner.. Components Included in Domain: Maximizes Professional Responsibilities Through Parent Involvement and Community Engagement Shows Professionalism Supports Professional Growth How does the principal work with stakeholders? This Domain provides an excellent opportunity for principals to bring evidence to the table, such as 360 degree surveys and other indicators of their interactions with multiple constituents. Planning is taking place to offer professional development within SAS. Also professional development within the PIL program will be directly aligned to Principal Effectiveness. 12

Essential Questions to ask Yourself: What are you currently doing to develop an effective system for improvement with administrators? How are you, the supervising administrator, gathering data to add value in the process and conversation? What are all the possibilities to enhance effectiveness? How and in what way would be most efficient? How and in what way would be most effective?

Chart out your current process used for implementing the Framework for Leadership, Correlation Data and SPP portions of the system?

What are the possibilities? Evidence Collection Beginning of the Year Meeting Coaching Conversation Survey Data (Staff, student, parent, community) Perceptions of School Teacher Performance Discussions End of Year Meeting Communications Review Co-Planning Sessions Site Visit Middle of the Year Meeting Self Evaluations One-Way Feedback Session Data Analysis Discussion Online Discussion Session Email conversations

Place the possibilities on the chart below for you and your district and then discuss in small group? Efficient Effective

Chart out your ideal process for the 2015-2016 school year?

Process for Principal Effectiveness Assistance to Implement the Framework for Leadership The steps outlined below are intended to incorporate current LEA principal evaluation processes and enrich them with the Framework for Leadership. In addition, an LEAs comprehensive plan can be blended with the Framework for Leadership to attain principal effectiveness. The process identified is provided to spur some thoughts as to the individual discussions that occur between a supervising administrator and principal/school leader. The actual process/discussions that occur should be determined locally. It is not mandatory to use this process, nor should it be viewed as a “checklist” to be followed. It is provided solely as a resource. No additional notes provided. 18

Process for Principal Effectiveness Assistance to Implement the Framework for Leadership (continued) Steps for Supervising Administrators (those conducting evaluations)… Review your current LEA evaluation process and identify ways to incorporate the Framework for Leadership into your existing system. Explore possible Framework for Leadership components that will be of focus for the school year. 19

Process for Principal Effectiveness Assistance to Implement the Framework for Leadership (continued) Steps for Supervising Administrators (those conducting evaluations)… The principal/school leader (person being evaluated) reviews the Framework for Leadership with you to determine agreed upon components based upon data and previously identified professional development needs. A recommendation is to include a minimum of two components per domain. This forms the basis of the evaluation. For discussion in your table groups… What criteria will you use to determine which components in each of the domains should be included in the evaluation of a principal / school leader? What types of data / evidence could be used to help in determining the criteria used? Have participants work in table groups to discuss… When debriefing, the following points should be flushed out during the conversations: Determine District (LEA) components  Are there components that all principals/school leaders within the district/LEA should have included in their evaluations? Determine specific principal/school leader components  Which components would apply to an individual principal / school leader to promote their professional growth? Principal / School Leader defined components  Which components would the principal/school leader like to include as part of his/her evaluation? Recommendation: At a minimum, two components per domain should be included. 20

Process for Principal Effectiveness Assistance to Implement the Framework for Leadership (continued) Map out an agreed upon observation schedule and types of data/evidence to be collected throughout the year by you and the principal/school leader for each agreed upon component. For discussion in your table groups… In what ways might you gather the appropriate data to help ascertain the level of effectiveness for each agreed upon component? Step #4  It is important to remember that the superintendent / supervising administrator has a key role in gathering data throughout the school year to support a domain rating through the preponderance of evidence collected for each domain. Step #5  Use data / evidence to make any adjustments to the specific components that are a focus of the year. The key here is not to “water down” the components to meet a proficiency level with the domain; rather, a key question to ask would be ask what supports can be provided to be successful. 21

Process for Principal Effectiveness Assistance to Implement the Framework for Leadership (continued) Meet midway through the year to discuss progress in collecting evidence relative to the agreed upon components (and other components upon which evidence has been collected). Provide written and/or oral feedback. If needed, based on data and feedback, make any necessary adjustments. Types of Evidence and Possible Guiding Questions/Connectedness documents are available to help frame the conversations. The principal/school leader completes an End of Year (EOY) self-assessment by highlighting areas within the defined performance levels of the Framework for Leadership based upon evidence that support these performance levels. You should also complete an independent EOY assessment based upon evidence you collected relative to the principal’s/school leader’s performance. Step #7  Domain ratings should be based on the superintendent’s / supervising administrators professional assessment of the evidence/data gathered throughout the year. The evidence collected should be based upon the impact or results. For example, meeting agendas show evidence of conducting meetings, not necessarily the results obtained based upon the action items established during the meetings. 22

Process for Principal Effectiveness Assistance to Implement the Framework for Leadership (continued) Conduct an End of Year (EOY) meeting with the principal/school leader to compare your assessment with the principal/school leader’s self-assessment to determine the final evaluation ratings for each of the domains (areas of disagreement should be resolved by a review of the evidence). Note: A domain rating is not intended to be an average of the component ratings for that domain; rather, it is based upon the preponderance of evidence presented for that domain. Enter final domain ratings into the Principal Effectiveness Rating Tool. Opportunities exist to use the results of the EOY meeting to inform possible Principal SLOs for the following year. No additional notes provided. 23

Building Level Data 15% will be derived from building level data (School Performance Profile – SPP). School Performance Profile (SPP) Scores can be found on the SPP Website http://paschoolperformance.org/ 24

Correlation Data This does NOT apply to supervisors who are classified as “Non-Teaching Professional Employees” (individuals who are not classified as principals, assistant principals, or CTC directors as defined in Act 82). 25

Multiple Measures Principal Effectiveness Ratings as Defined in Act 82 For the overall principal effectiveness rating, we know that… 50% will comprise the observation/practice piece (Framework for Leadership). 15% will be derived from building level data (School Performance Profile – SPP). 15% will be determined by the relative strength of conversation regarding the connectedness between average teacher observation/practice ratings and teacher-level measures. 20% will incorporate Elective Data / SLOs for principals. (note: for 2014-15, elective data is optional) 26

Correlation Data Based Upon Teacher Level Measures It was recommended that we adopt a process that focuses on the conversation between the supervising administrator and the principal / school leader based on the principal’s knowledge, understanding and intended use of the evidence / data presented. The conversation between the supervising administrator and the principal / school leader is focused on the possible level of connectedness between Teacher Level Measures and Observation and Practice (Framework for Teaching) ratings. Teacher Level Measures “shall include, but not be limited to any combination of one or more of the following data” Building Level Data / SPP Teacher Specific Data (PVAAS, etc.) Elective Data / Principal SLOs 27

Correlation Data Based Upon Teacher Level Measures The rating for correlation data is based on a qualitative assessment by the supervising administrator of the level of understanding (0, 1, 2, or 3) of the quantitative analysis conducted by the principal / school leader. To aid the supervising administrator in assigning a correlation rating (0, 1, 2, or 3), a Performance Level Descriptor Chart has been developed. Note: Regulation §19.2 Principal/School Leader Effectiveness Rating Tool of Title 22, Chapter 19 Educator Effectiveness Rating Tool states that, “”The Department will provide direction, guidance and templates for LEAs to use in applying the Correlation Data Performance Level Descriptors in Table H and validating the Correlation Rating for a Principal/School Leader.” 28

Correlation Data Performance Level Descriptor Chart The conversation will be measured based upon the principal /school leader’s understanding of these aspects: Degree of understanding of evidence presented regarding the relationship between teacher-level measures and teacher observation and practice ratings. Quality of explanation provided for observed relationships between teacher-level measures and teacher observation and practice ratings. Plans for how the data will be used to support school and LEA goals. The Department has provided guidance for LEAs to use in applying the Correlation Data Performance Level Descriptors in Table H (within the regulations) and validating the Correlation Rating for a Principal/School Leader. Table H within the regulations define Degree, Quality, and Plans relative to the conversations that occur between the supervising administrator and principal/school leader. 29

Possible Levels of Connectedness Between Average Teacher Observation/Practice Ratings and Teacher Level Measures Data (SPP) This slide shows the possible levels of connectedness between average teacher observation/practice ratings and teacher level measures (SPP). Stress to participants this is one type of connectedness that can be considered when looking at possible connectedness. Similar connectedness charts exist for other teacher level measures (PVAAS and Elective Data) Exploration of connectedness using Observation/Practice Ratings for all teachers in the building, as well as subsets of teachers, should be included in the analysis to uncover possible trends/patterns. In circumstances where a building does not have an SPP score, substitute another teacher level measure (e.g. NOCTI Scores for part time CTCs). Similar connectedness charts exist for other teacher level measures (PVAAS and Elective Data) 30

Guidance for Determining Correlation A Process for Exploring Connectedness Principal / School Leader explores possible levels of connectedness between teacher-level measures and teacher observation/practice ratings for the entire school and subsets of teachers within the schools (e.g. math teachers, teachers with less than two years of experience, etc.). For discussion in your table groups… What types of questions can you pose to the principal / school leader to further clarify possible levels of connectedness? What types of questions can you pose to help the principal; / school leader define root causes and plausible explanations for possible levels of connectedness? How can the Performance Level Descriptor Chart aid in your determination of a 0, 1, 2, or 3 for the correlation score? 31

Elective Data / SLOs 32

Multiple Measures Principal Effectiveness Ratings as Defined in Act 82 For the overall principal effectiveness rating, we know that… 50% will comprise the observation/practice piece (Framework for Leadership). 15% will be derived from building level data (School Performance Profile – SPP). 15% will be determined by the relative strength of conversation regarding the connectedness between average teacher observation/practice ratings and teacher-level measures. 20% will incorporate Elective Data / SLOs for principals. (note: for 2014-15, elective data is optional) 33

Elective Data / SLOs Purpose and Approach Goal Setting: The intention is that principal Elective Data/SLOs align with LEA and/or schools goals. Elective data/SLOs are designed to serve several purposes: 1. To provide educators with an opportunity to actively participate in their own evaluation 2. To increase student achievement 3. To improve educator effectiveness 4. To foster collaboration among colleagues 5. To align the work of individual educators with LEA and school goals 34

Elective Data / SLOs Template with Guiding Questions Components Administrator Responses Student Learning Objective (SLO) State your measurable student academic SLO. Does the content support attainment of PA Core Standards, international, national or industry standards? Explain why this particular objective was chosen and how it aligns with school and/or LEA goals? Data and Evidence Describe the data and evidence used to create and measure your SLO. Identify specific data sources used in the data analysis process. What is the baseline data used for current student performance levels including student focus populations (ELL, special education, truancy, attendance, out of school suspension, free and reduced lunch, gifted, race/ethnicity, etc.)? 35

Elective Data / SLOs Template with Guiding Questions Components Administrator Responses Student Population Identify the student population(s) selected for this SLO. What is the total number and school percentage of students selected for this SLO? What is the grade level? Does this student population(s) represent the majority of the school and/or does it represent a student focus group (ELL, special education, truancy, attendance, out of school suspension, free and reduced lunch, gifted, race/ethnicity, etc.)? 36

Elective Data / SLOs Template with Guiding Questions Components Administrator Responses Action Plan and Timeframe Describe the action plan and timeframe in reference to implementation, analysis of data, and reporting for this SLO. The action plan should identify the data-driven strategies that will be used to successfully implement this SLO, as well as the approaches to reporting of results. The timeframe is a school year unless there is a compelling reason for a shorter or longer interval. If other than a year, please state the interval and provide the rationale for the change in the timeframe. 37

Elective Data / SLOs Template with Guiding Questions Components Administrator Responses Performance Indicators Describe the expected results for students included in this SLO. What are the expectations for all students and/or focus groups? If a focus student group(s) is identified, what are the expectations for these students? Performance Level Measures Describe the performance measures to be used to determine student progress. How will you measure student progress? What are the performance measures or assessments used to provide on-going assessments of students’ progress toward the goal? Do the performance measures/assessments meet the criteria established by the LEA, if applicable? 38

Elective Data Template with Guiding Questions Components Administrator Responses Principal Expectations Describe what criteria will be used to determine the levels of Distinguished, Proficient, Needs Improvement, and Failing. This needs to be agreed upon by both the supervising administrator and the principal/school leader Framework for Leadership Describe your leadership role in facilitating the attainment of this SLO by referencing appropriate components within the four Domains of the Framework for Leadership. The Four Domains of the Framework for Leadership are: Strategic / Cultural Leadership Systems Leadership Leadership for Learning Professional and Community Leadership 39

For Discussion… After reviewing the Elective Data / SLO Template with Guiding Questions document, discuss the following in your table groups… How is the Elective Data / SLO process similar to what you are currently doing relative to goal setting for a principal / school leader? Where are the differences? What modifications (if any) are required of your current goal setting process to incorporate the Elective Data / SLO process? 40

Final Thoughts… During this session we explored Principal Effectiveness as a system and how each of the parts are not only are critical for evaluating principals/school leaders, but are also essential for creating a foundation of leadership effectiveness. As we return to the Rating Tool, any additional questions remain regarding the various part of principal effectiveness?

The mission of the Pennsylvania Department of Education is to lead and serve the educational community, to enable each individual to grow into an inspired, productive, fulfilled lifelong learner. No additional notes provided. 42 42 42