H ARNESSING D ATA for E DUCATIONAL L EADERSHIP  UCEA’s Webinar Series on Educational Leadership is sponsored by the Wallace Foundation A UCEA webinar featuring panelists and presenters Julia Ballenger, Mark Gooden, Cori Groth and Susan Korach. Tuesday, May 17, 11:00 AM – 12:15 PM CST

UCEA Panelists  M ARK G OODEN – Associate Professor, Director of the UT-Austin Principalship Program, Department of Educational Leadership, University of Texas at Austin  J ULIA B ALLENGER – Professor, Stephen F. Austin State University  C ORI G ROTH – Associate Director, Utah Education Policy Center, University of Utah  S USAN K ORACH – Assistant Professor, Department of Educational Leadership and Policy Studies, University of Denver

NCEELPP Program Evaluation Progress  Evaluation Pathway for Preparation Programs.  constructed using evaluation research- validated through several studies  In Pathway, there are two categories of program input, three categories of initial graduate outcomes, and two categories of expected school outcomes.  This model serves as a blueprint for evaluation planning-helps programs measure and track program attributes and outcomes

Preconditions: Program Participants Prior Experiences  Application (Rubric)  Statement of Purpose (why leadership, why now, why you?)  Resume  Letters of Recommendation  GRE, GPA, Parents Education, Race/Ethnicity  Assessment Center (Rubric)  Leadership Interview  Teacher Obs/role play  Data Presentation  Leadership Identity Problem Framing Continua

Program Quality Features: Leadership Program Experiences  SLPPS Program Feature Survey  SLPPS Learning Outcomes  SLPPS Internship Outcomes  Exit Interviews 2008 & 2009 Cohorts

 Graduate Edition (SLPPS-G) —The purpose of SLPPS-G is to elicit feedback from graduates on their leadership preparation experiences, learning outcomes, and career intentions.  Program Features Edition ( SLPPS-P ) — This instrument enables the educational leadership preparation program to systematically document its core program features.  Practicing Principal Edition ( SLPPS-PPE )— The purpose of SLPPS-PPE is to document leadership practices and school improvement and organizational indicators from the perspective of program graduates who are working as school principals.  Teacher Edition ( SLPPS-T ) — The purpose of SLPPS-T is to to document leadership practices and school improvement and organizational indicators in the schools where program graduates work from the perspective of teachers. SLPPS Suite of Surveys



Graduate Perceptions of Program– SLPPS Program Graduate  2007 college-wide administration of SLPPS Graduate Edition – 3 different programs  Evaluation of Program Features  Evaluation of Leadership Learning Outcomes  Learned to lead vision and ethics  Learned to leading learning  Learned to lead organizational learning  Learned management and operations  Learned to lead parent and community involvement

Analysis  All programs were rated highly on Instructional Leadership  District Collaborative and District Provider programs were rated more highly in the Leadership for Learning Outcomes - Lead Learning and Learning for School Improvement  All programs were rated lower in the Leadership for Learning Outcome - Management  District Collaborative and District Provider Programs were rated more highly in Program Features – Program Coherence

Process  Program faculty reviewed survey results together  Clear consensus that N8 and N9 had program features that were more positively perceived than N6  Discussion about specific program differences

Conclusions and Action  Revision of N6 to reflect course integration through field-based projects  Increased emphasis on management issues through the internship experiences  This year will be the first administration of SLPPS with new N6 – internal evaluations indicate significant change

Implications  The results from the examination of the patterns within the responses of graduates from all three program types reveal a relationship between program design and graduate perceptions of their principal preparation programs.  The presence of district collaborations, project-based learning, and integrated content led to higher ratings by program graduates.  Year to year tracking of this data illuminates issues in program delivery

The Teacher Survey of Leadership Preparation and Practice (TSLPP)  The TSLPP, formerly the UCEA/LTEL-SIG Teacher Survey, aligns with the SLPPS and is designed to compile teachers’ assessments of:  the principal’s leadership practices,  their school improvement practices and recent accomplishments,  organizational contexts in the schools, and  their own demographic and educational experiences.

The Teacher Survey of Leadership Preparation and Practice (TSLPP)  This survey adds teacher voice to a chain of measures that bridges principal preparation to practice and school improvement.  The TSLPP was originally fielded and tested in the Stanford Study of exemplary leadership preparation (Darling- Hammond, Meyerson, La Pointe, & Orr, 2009).

The Teacher Survey of Leadership Preparation and Practice (TSLPP)  A subsequent study corroborated that the scales have strong reliability (Cronbach’s α was greater than.7 for all items) and are positively related to whether the principal had completed an exemplary program (Orr & Orphanos, 2007).  The TSLPP was recently piloted by three university preparation programs, and teacher reactions to the survey were collected at one site through observations and interviews.  Each pilot consisted of teachers at 5-7 schools of different levels with principals who had graduated from the principal preparation programs.

References  Darling-Hammond, L., Meyerson, D., La Pointe, M. M., & Orr, M. T. (2009). Preparing principals for a changing world. San Francisco, CA: Jossey-Bass.  Orr, M. T., & Orphanos, S. (2007). Learning Leadership Matters: The influence of innovative leadership preparation on teachers’ experiences. Paper presented at the American Educational Research Association, Chicago, Ill.

Stephen F. Austin State University: TSLPP Pilot Results  East Texas Regional Institution  Purposive sample consisted of 102 teachers and seven principals in three rural school districts.  Principals had served on their campuses for at least three years.

TSLLP: Four Constructs  Principal Instructional Leadership  Principal Collaboration and Shared Decision Making  Teacher Policy Making  Teacher Collaboration

Findings

Conclusions  Orr (2009) stated, “What the principal does can have a huge effect on what the teacher does and the quality of learning experiences for students” (p.1)  However, in this current pilot study, the researchers found that the context within which principals worked varied across multiple factors in the environment.

Conclusions  The contextual factors in this study that impacted principals’ impact on school improvement, despite what they learned in their principal preparation program were:  Small rural school setting  Top-down influence of school superintendent  Experience level of teachers in the school  School resources available to support learning in the school, and  Lack of autonomy to be the instructional leader on their campus.

Conclusions  Context is pivotal in understanding the situation that principals encounter and experience.  Teacher survey results’ ratings were considerably low on areas related to shared decision making, teacher empowerment, and teacher influence.

Recommendations  In exploring answers to the research question of the impact of the preparation program on school improvement, the use of this survey might be more effective as part of comprehensive case studies of the schools using mixed methods of data collection than as a single measure.

Discussion  M ARK G OODEN  J ULIA B ALLENGER  C ORI G ROTH  S USAN K ORACH

 Closing Remarks