DEVELOPING DEEPER AND MORE MEANINGFUL EVALUATION SYSTEMS Lisa Hood, Center for the Study of Education Policy Funded by a grant through the McCormick Foundation

Teacher Evaluation in the Early Grades: Developing a PreK-3 Support Services for Danielson Framework for Teaching

Danielson Early Supports Project Staff Lisa Hood, Deb Kasperski, Erika Hunt, Pam Rosa, Website

District Level – Teacher Evaluation in the Early Grades  Danielson Framework for Teaching has been validated for Grades 4 and above, yet many districts were planning to use the Danielson FfT for early childhood teachers (PreK-grade 3)  Is this a valid and reliable tool for PreK-3 teachers?  What tiered assessments are appropriate for PreK-3 classrooms?

Developing Early Learning Supports for Danielson Framework for Teaching (FfT)  Fall 2012:Review and adaptation by PreK-3 administrators, teachers, and stakeholders  Critical Attributes and Examples  Winter 2012: NBCT’s developed examples and coded PreK-3 videos using adapted Framework  Winter 2013: Training developed for pilot principal and teachers  March 15, 2013: Feedback from early childhood stakeholders  April 26, 2013: Second round of feedback from early childhood stakeholders  June 2013: National feedback of framework supports solicited at NAEYC conference, San Francisco  Fall 2013: Pilot of framework supports and training

Piloting the 2013 FfT with Early Learning Supports  Pilot will occur in seven selected schools throughout the region (Northern, Central, and Southern Illinois) in fall  Prior to pilot, selected principals and teachers will undergo a three day pilot training that will utilize three inter-connected modules.

Module 1: Understanding and using the 2013 Danielson FfT in the context of early learning (PreK-3 rd grade)  Present the need for piloting the 2013 FfT Early Learning supports to enhance early learning teaching practice growth opportunities  Identify the structure/priorities of the 2013 FfT Domains/Components and Critical Attributes  Understand shifts in professional practice for different levels of performance and the link to the 2013 FfT Framework Components/Critical Attributes  Explore and connect to Early Learning Grade Band examples at four different levels of practice

Module 2: Applying the 2013 Danielson FfT instrument through observation-based evidence collection in early learning teaching settings  Understand observation-based data collection within the Plan-Teach-Reflect-Apply (PTRA) Professional Improvement Cycle for supporting and improving teaching practice  Develop/enhance observation process skills for collecting, sorting, and scoring evidence focused upon early learning Classroom Environment and Instruction  Determine Master Scores for early learning Classroom Environment and Instruction teaching practices using 2013 FfT Components and Critical Attributes

Module 3: Applying the 2013 Danielson Framework for Teaching instrument through conversation-based evidence collection in early learning teaching settings  Understand data collection and artifacts of practice and student learning within Professional Improvement Cycle  Enhance skills focused upon early learning Planning and Preparation practices, review of Classroom Environment and Instruction, and Responsibilities  Determine Master Ratings for Planning/Preparation and Responsibilities using FfT Components and Critical Attributes and protocols to confirm inter-rater reliability  Extend knowledge of professional learning-focused coaching language, conversations, and protocols of practice to support planning and reflection conversations

New Proposals  February 2013: Submitted a proposal to validate the 2013 Danielson Framework for Teaching for PreK-3 rd grade teachers  April 2013: Submitting a proposal to validate the training and professional development modules for principals and teachers associated with the early learning supports for the 2013 Danielson FfT. 7/25/11

Purpose of Proposed Validation Study  To ensure that FfT provides accurate and reliable data that PreK-3 rd grade teachers and their supervisors can use to identify strengths and weaknesses  To help teachers and supervisors use the data from the FfT to identify appropriate professional development mechanisms to promote teachers’ growth.  To utilize data from the early learning Framework’s observation along with other with student growth data so that teachers who are observed to be “Unsatisfactory, Needs Improvement, Proficient, or Excellent” have appropriate student growth that correlates with the observation ratings

Other Benefits of Proposed Validation Study  Construct a student/parent survey  Explore other student growth measures  Increase principals’ understanding of different teaching pedagogies related to early childhood education

Purpose of Proposed Professional Development Project  Develop and validate professional development training and supplemental materials to support Danielson FfT for PreK-3 rd grade teachers.  To use the data from validation study to identify appropriate professional development mechanisms to promote teachers’ growth and principals’ understanding of early childhood education (PreK-grade 3).  Training geared to principals, other persons evaluating teachers, the PreK-3 rd grade teachers

Group Activity  Question One: What do evaluators of PreK-grade 3 teachers need to know about early learning (PreK-grade 3) classrooms?  Question Two: What do principals need to know in general about doing effective evaluations that will grow teacher’s practice?

Building Evaluation Capacity Within P-12 Principal Programs and Partners

Approval of new Principal Endorsement Programs – a beginning not an end  Annual reports required by Illinois Educator Licensure Board  program data (e.g., number of faculty, number of graduates, etc.)  formative data that is internal to the programs and varies by institution  Recommendations by P-20 Council Committee on Teacher and Leadership Effectiveness for Principal Preparation Performance Indicators\  Tied to new P-20 Data Longitudinal System

Principal Preparation Program Performance Indicators Principal Performance Indicators Recommended by P-20 Council Teacher and Leadership Effectiveness Committee (2011) 1.Percentage of graduates that pass the required state principal endorsement certification exam Note: This is a competency based assessment that will be administered to all candidates 2. Percentage of program graduates that obtain principal or assistant principal positions within 1, 3 and 5 years of graduation 3. Performance evaluations of program graduates using the four category rating system once placed as principals and assistant principals in Illinois Note: This data will be available after implementation of the new statewide requirements in the school year. 4. Data and indicators of student growth within 1, 3 and 5 years of placement as an assistant principal or principal Note: According to Illinois requirements, this measure will be incorporated into performance evaluation ratings of Illinois principals and assistant principals. 5. Results of learning climate surveys in schools where principals and assistant principals are placed within 1, 3 and 5 years of graduation Note: A state standardized learning climate survey will be developed 6. Percentage of program graduates retained as principals, assistant principals, superintendents or district leadership roles within 1, 3 and 5 years of placement

Principal Preparation Program Continuous Improvement Process Project  Project Staff  Lisa Hood, Project Director  Erika Hunt, Project Staff  Alicia Haller, Evaluation Coach  Joe Pacha, Evaluation Coach  Brad White, Illinois Education Research Council, Team Consultant  Project Partners:  University of Illinois at Urbana-Champaign  Southern Illinois University at Edwardsville  North Central College  Loyola University  New Leaders for New Schools

Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process Inputs: (What to invest) Faculty Time Money Research Materials Technology Partners Activities: (What we do) Review Principal Preparation Programs (Quality Program Assessments and Feedback. Support university personnel in their continuous improvement processes (Process Improvements) Help faculty implement best practices for improving programs (Knowledge Building) Participation: (Who we reach) Principal Preparation Program Faculty members and ISBE University Partners, Faculty, and ISBE Faculty Members and ISBE Outputs:Outcomes-Impact Short Term: (What the short term results are) Improved candidate selection Improved courses and programs Improved internship Improved graduates of programs Medium Term: (What the medium term results are) Candidates prepared through quality coursework Candidates ready to perform internship assessments successfully Candidates prepared to lead schools in improving learning for all students Long Term: (What the ultimate impacts are) Improved learning for ALL students Improved teacher practices Improved school leadership

20 Evaluation: 1. Focus – 2. Collect Data – 3. Analyze and Interpret – 4. Report – Repeat the Cycle Focus 1.Program Requirements 2. Internship Requirements 3. Internship Assessments 4.Coursework Requirements 5.Staffing Requirements 6.Candidate Selection Collect Data 1.Data Collected for Program Improvement 2.Analysis of the Program Improvement Data 3.Program Changes Based on Data Analysis 4.Summative INTERNSHIP Assessment Data: (Number of candidates; listing of each and all assessments with the scoring of each candidate on the assessment and number of times each candidate needed to complete the assessment successfully; length of the internship for each candidate) 5.Listing of Courses and Course Syllabi 6.Number of candidates and Faculty (FTE) 7.Number of Candidate Applications 8.Number of Candidates Selected 9.Number of Candidate Completers 10.Graduate’s School Data Analysis over time Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process Analyze the results 1.Analysis and planning for program improvement based on data 1, 2, &3. 2.Analysis and implementation for improvement for Internship and candidate success based on data 4, 5, 6, 7, 8 and 9. 3.Analysis and planning for improvements based on data 10. Report 1.Program Improvement Plan 2.Internship Improvement Plan 3.Candidate Selection Improvement Plan 4.Coursework and Staffing Improvement Plan

 Implementation of a Program Theory Evaluation (PTE) model to identify how policies, practices, and processes have changed as a result of the process of redesigning;  Creation of an evaluation plan of Principal Preparation Programs;  Alignment of current data and creation of needed data sources to continuously inform evaluation plan process;  Coaching, technical assistance, and tools to inform the evaluation process  Collaboration with the ISBE to provide input to the annual reporting process and requirements for Principal Preparation Programs. Project Objectives

Project Activities  Programs will receive technical assistance through evaluation coaches to help build internal capacity  Programs will be provided with:  Tools to inform the evaluation process (e.g., Self-Assessment Rubric, Sample Data Instruments)  Technical assistance (e.g., Coaching and Program Team Meetings)  Meeting facilitation  Timeline planning  Data analysis  Reporting framework

Two Project Phases  Phase One: Initial Assessment Phase  Phase Two: Continuous Improvement Phase

Phase One: Initial Assessment Phase  Beginning March 2013, evaluation coaches will convene program faculty and partners.  Step One: Completion of a self-assessment process developed by project staff  Step Two: Use self-assessment instrument to capture baseline information on:  partnerships with districts and other community members  content and pedagogy  assessments  internship  Step Three: Share results with program team and develop action plan

 Evaluation coaches and program faculty/partners will begin testing the program’s theory of change and whether it is having the intended effects on principal candidates and new principals.  Assessment of the program’s “continuous improvement evaluation plan” will be conducted to determine appropriate level of outcomes  Center staff and evaluation coaches, in partnership with program faculty and partners, will document the process and any on-going program modifications made. Phase Two: Continuous Improvement Phase

Next Steps  Interest with funding a multi-phase statewide evaluation of principal preparation programs  Proposed development of a framework for a statewide evaluation of principal preparation  Advisory committee to assist with development of Evaluation Framework  Statewide forum to collect input into draft conceptual framework  Development of an Request for Proposal (RFP) to select an external evaluator for the statewide evaluation

Group Activity  Question One: What are the highest priority issues to keep in mind when developing a statewide evaluation framework for principal preparation?  Question Two: What do you foresee will be the benefits and challenges to doing a statewide evaluation of principal preparation programs?

Questions?

Thank you!  Please remember to complete an evaluation form before you leave.  Evaluation form includes questions on challenges and successes of the new P-12 Principal Endorsement programs.