THE RIPPLE EFFECT OF PRINCIPAL BEHAVIOR: Improving Teacher Instructional Practices through Principal-Teacher Interactions Kim Banta & Brennon Sapp A Dissertation Defense presented to the University of Louisville in partial fulfillment of the requirements of the Degree of Doctor of Education

To discover how a specific set of principal-teacher interactions affect: Teacher Instructional Practices Student Performance Frequency & Focus of Teacher Conversations Page 11 Goal of the Study Key Constructs

Research Questions Page 11 RQ-1 How will the treatment of principal-teacher interactions affect teachers’ instructional practices? RQ-2 How will changes in teachers’ instructional practices, initiated by the set of principal-teacher interactions, affect student performance? RQ-3 How will changes in principal-teacher interactions affect the frequency and focus of teacher conversations with principals, students, and other teachers?

Conceptual Framework Page 8-11, Figure 1

Organization of Methodology Research QuestionResearch DesignMeasuresAnalysis 1 Teacher Instructional Practices Pretest/Post test Quality Instruction Rubric (4 Domains & Overall) ANOVA 2Student Performance Single Cross-Sectional Interrupted Time Series Student Grades & Discipline Referrals Linear Regression 3 Freq & Focus of Teacher Conversations Pre-Mid-Post test Teacher & Student Surveys Chi Square

Teacher Instructional Practices RQ-1 Research QuestionResearch DesignMeasuresAnalysis 1 Teacher Instructional Practices Pretest/Post test Quality Instruction Rubric (4 Domains & Overall) ANOVA Methodology RQ 1 - How will the treatment of principal-teacher interactions affect teachers’ instructional practices?

PrePost tp-value Effect size (SD) (Cohen’s d) TEACHER-COMPLETED Planning & Preparation * (0.48)(0.38) Learning Environment * (0.46)(0.36) Instruction – (0.50)(0.48) Assessment – (0.62)(0.48) Overall * (0.51)(0.42) PRINCIPAL-COMPLETED Planning & Preparation – (0.78)(0.770) Learning Environment – (0.72)(0.70) Instruction < * (0.64)(0.60) Assessment < * (0.76)(0.60) Overall < * (0.73)(0.67) Table 15 * Indicates a small effect size ( ). (Cohen, 1988) Table 15 Page 85

Teacher Instructional Practices (Change in instructional practices) Teachers and principals differed in where they perceived improvement. According to teachers, instructional practices improved in two domains – Planning & Preparation and Learning Environment. According to principals, instructional practices improved in two domains – Instruction and Assessment It is more difficult for principals to observe Planning & Preparation. Teachers have closer personal knowledge of Planning & Preparation and Learning Environment. Pages 110

TeacherPrincipal tp-value Effect size (SD) (Cohen’s d) PRETEST Planning & Preparation *** (0.48)(0.78) Learning Environment < *** (0.46)(0.72) Instruction < *** (0.50)(0.64) Assessment < *** (0.62)(0.76) Overall < *** (0.51)(0.73) POSTEST Planning & Preparation < *** (0.38)(0.770) Learning Environment < *** (0.36)(0.70) Instruction < *** (0.48)(0.60) Assessment < *** (0.48)(0.60) Overall < *** (0.42)(0.67) Table 16 Page 86

Teacher Instructional Practices (Differences in the Ratings of Instructional Practices) Principals’ ratings of instructional practices were significantly different than teacher’s ratings of instructional practices in each domain. Principals’ ratings of instructional practices were lower than the teachers instructional practices ratings in each domain. Page & 112

Principals’ ratings of instructional practices are hypothesized to be more valid and reliable than teachers ratings. – Extensive procedures were used throughout the study to increase the reliability and validity of principal ratings – (see chapter three). – Principals are trained to be observers of instruction and therefore see changes in instruction that the teacher may not signify as improvement. (Fullan, 2005b) – According to Ross, 1985, Schacter & Thum 2004, found that teachers over-rated their quality of instructional practices on effort. A review of the literature revealed that within other professions the validity of self evaluations vary depending on the actual quality of the individual performing the self-evaluation. (Dunning et al.,2003; Kruger & Dunning, 1999; and Yariv, 2009) Page & 112 Teacher Instructional Practices (Differences in the Ratings of Instructional Practices)

Grouping Teachers into Performance Levels

p- value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) PRETEST-HIGH PERFORMING TEACHERS PRETEST-MEDIUM PERFORMING TEACHERS PRETEST-LOW PERFORMING TEACHERS Planning & Preparation – ** < *** (0.56)(0.61) (0.51)(0.47)(0.35)(0.45) Learning Environment – ** < *** (0.46)(0.64) (0.44)(0.35)(0.50)(0.49) Instruction – *** < *** (0.53)(0.55) (0.23)(0.42)(0.45) Assessment – *** < *** (0.71)(0.66)(0.61)(0.48)(0.54)(0.44) Overall – *** < *** (0.50)(0.57)(0.47)(0.31)(0.38)(0.41) POSTTEST-HIGH PERFORMING TEACHERS POSTTEST-MEDIUM PERFORMING TEACHERS POSTTEST-LOW PERFORMING TEACHERS Planning & Preparation – *** < *** (0.40)(0.380) (0.33) (0.39)(0.40) Learning Environment – < *** < *** (0.37)(0.33) (0.31)(0.21)(0.42)(0.56) Instruction – ** < *** (0.51)(0.35) (0.23)(0.51)(0.47) Assessment – *** < *** (0.38)(0.19)(0.52)(0.40) (0.53)(0.50) Overall – < *** < *** (0.39)(0.31)(0.25)(0.14)(0.40)(0.37) Table 17 Page 90

p- value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) PRETEST-HIGH PERFORMING TEACHERS PRETEST-MEDIUM PERFORMING TEACHERS PRETEST-LOW PERFORMING TEACHERS Planning & Preparation – ** < *** (0.56)(0.61) (0.51)(0.47)(0.35)(0.45) Learning Environment – ** < *** (0.46)(0.64) (0.44)(0.35)(0.50)(0.49) Instruction – *** < *** (0.53)(0.55) (0.23)(0.42)(0.45) Assessment – *** < *** (0.71)(0.66)(0.61)(0.48)(0.54)(0.44) Overall – *** < *** (0.50)(0.57)(0.47)(0.31)(0.38)(0.41) POSTTEST-HIGH PERFORMING TEACHERS POSTTEST-MEDIUM PERFORMING TEACHERS POSTTEST-LOW PERFORMING TEACHERS Planning & Preparation – *** < *** (0.40)(0.380) (0.33) (0.39)(0.40) Learning Environment – < *** < *** (0.37)(0.33) (0.31)(0.21)(0.42)(0.56) Instruction – ** < *** (0.51)(0.35) (0.23)(0.51)(0.47) Assessment – *** < *** (0.38)(0.19)(0.52)(0.40) (0.53)(0.50) Overall – < *** < *** (0.39)(0.31)(0.25)(0.14)(0.40)(0.37) Table 17 Page 90

p- value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) PRETEST-HIGH PERFORMING TEACHERS PRETEST-MEDIUM PERFORMING TEACHERS PRETEST-LOW PERFORMING TEACHERS Planning & Preparation – ** < *** (0.56)(0.61) (0.51)(0.47)(0.35)(0.45) Learning Environment – ** < *** (0.46)(0.64) (0.44)(0.35)(0.50)(0.49) Instruction – *** < *** (0.53)(0.55) (0.23)(0.42)(0.45) Assessment – *** < *** (0.71)(0.66)(0.61)(0.48)(0.54)(0.44) Overall – *** < *** (0.50)(0.57)(0.47)(0.31)(0.38)(0.41) POSTTEST-HIGH PERFORMING TEACHERS POSTTEST-MEDIUM PERFORMING TEACHERS POSTTEST-LOW PERFORMING TEACHERS Planning & Preparation – *** < *** (0.40)(0.380) (0.33) (0.39)(0.40) Learning Environment – < *** < *** (0.37)(0.33) (0.31)(0.21)(0.42)(0.56) Instruction – ** < *** (0.51)(0.35) (0.23)(0.51)(0.47) Assessment – *** < *** (0.38)(0.19)(0.52)(0.40) (0.53)(0.50) Overall – < *** < *** (0.39)(0.31)(0.25)(0.14)(0.40)(0.37) Table 17 Page 90

p- value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) PRETEST-HIGH PERFORMING TEACHERS PRETEST-MEDIUM PERFORMING TEACHERS PRETEST-LOW PERFORMING TEACHERS Planning & Preparation – ** < *** (0.56)(0.61) (0.51)(0.47)(0.35)(0.45) Learning Environment – ** < *** (0.46)(0.64) (0.44)(0.35)(0.50)(0.49) Instruction – *** < *** (0.53)(0.55) (0.23)(0.42)(0.45) Assessment – *** < *** (0.71)(0.66)(0.61)(0.48)(0.54)(0.44) Overall – *** < *** (0.50)(0.57)(0.47)(0.31)(0.38)(0.41) POSTTEST-HIGH PERFORMING TEACHERS POSTTEST-MEDIUM PERFORMING TEACHERS POSTTEST-LOW PERFORMING TEACHERS Planning & Preparation – *** < *** (0.40)(0.380) (0.33) (0.39)(0.40) Learning Environment – < *** < *** (0.37)(0.33) (0.31)(0.21)(0.42)(0.56) Instruction – ** < *** (0.51)(0.35) (0.23)(0.51)(0.47) Assessment – *** < *** (0.38)(0.19)(0.52)(0.40) (0.53)(0.50) Overall – < *** < *** (0.39)(0.31)(0.25)(0.14)(0.40)(0.37) Table 17 Page 90

p- value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) p-value Effect size (Cohen’s d) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) Teacher (SD) Principal (SD) PRETEST-HIGH PERFORMING TEACHERS PRETEST-MEDIUM PERFORMING TEACHERS PRETEST-LOW PERFORMING TEACHERS Planning & Preparation – ** < *** (0.56)(0.61) (0.51)(0.47)(0.35)(0.45) Learning Environment – ** < *** (0.46)(0.64) (0.44)(0.35)(0.50)(0.49) Instruction – *** < *** (0.53)(0.55) (0.23)(0.42)(0.45) Assessment – *** < *** (0.71)(0.66)(0.61)(0.48)(0.54)(0.44) Overall – *** < *** (0.50)(0.57)(0.47)(0.31)(0.38)(0.41) POSTTEST-HIGH PERFORMING TEACHERS POSTTEST-MEDIUM PERFORMING TEACHERS POSTTEST-LOW PERFORMING TEACHERS Planning & Preparation – *** < *** (0.40)(0.380) (0.33) (0.39)(0.40) Learning Environment – < *** < *** (0.37)(0.33) (0.31)(0.21)(0.42)(0.56) Instruction – ** < *** (0.51)(0.35) (0.23)(0.51)(0.47) Assessment – *** < *** (0.38)(0.19)(0.52)(0.40) (0.53)(0.50) Overall – < *** < *** (0.39)(0.31)(0.25)(0.14)(0.40)(0.37) Table 17 Page 90

High, Medium, and Low Performing Teachers (Validity of Ratings) High performing teachers rated their instructional practices equivalent to principal ratings. Medium performing teachers rated their instructional practices higher than the principals by.3 to.4 of a performance level. Low performing teachers rated their instructional practices higher than the principals by a full performance level. Low performing and medium performing teachers rated the quality of their instructional practices equivalent to high performing teachers ratings. The principals did not. Page & 112

PretestPosttest tp-value Effect size (SD) (Cohen’s d) TEACHER-COMPLETED Planning & Preparation – (0.56)(0.40) Learning Environment ** (0.46)(0.37) Instruction – (0.53)(0.51) Assessment – (0.71)(0.53) Overall * (0.50)(0.39) PRINCIPAL-COMPLETED Planning & Preparation – (0.61)(0.380) Learning Environment * (0.64)(0.33) Instruction *** (0.55)(0.35) Assessment * (0.66)(0.50) Overall ** (0.57)(0.31) High Performing Teachers (Change in the Quality of Instructional Practices) Table 18 Page 91

PretestPosttest tp-value Effect size (SD) (Cohen’s d) TEACHER-COMPLETED Planning & Preparation – (0.51)(0.33) Learning Environment – (0.44)(0.31) Instruction – (0.55)(0.35) Assessment – (0.61)(0.38) Overall – (0.47)(0.25) PRINCIPAL-COMPLETED Planning & Preparation – (0.47)(0.33) Learning Environment – (0.35)(0.21) Instruction ** (0.23) Assessment – (0.48)(0.19) Overall – (0.31)(0.14) Medium Performing Teachers (Change in the Quality of Instructional Practices) Table 19 Page 93

PretestPosttest tp-value Effect size (SD) (Cohen’s d) TEACHER-COMPLETED Planning & Preparation ** (0.35)(0.39) Learning Environment – (0.50)(0.42) Instruction – (0.42)(0.51) Assessment – (0.54)(0.52) Overall ** (0.38)(0.40) PRINCIPAL-COMPLETED Planning & Preparation – (0.45)(0.40) Learning Environment – (0.49)(0.56) Instruction ** (0.45)(0.47) Assessment *** (0.44)(0.40) Overall * (0.41)(0.37) Low Performing Teachers (Change in the Quality of Instructional Practices) Table 20 Page 94

High, Medium, and Low Performing Teachers (Change in instructional practices) High Performing teachers improved according to teacher self-ratings (.2 * ) and principal ratings (.29 ** ). Medium performing teachers perceived no change in the quality of their instructional practices and principals perceived essentially no change. Low performing teachers improved according to teacher self-ratings (.2 ** ) and principals (.19 * ) Page

Student Performance RQ-2 Research QuestionResearch DesignMeasuresAnalysis 2Student Performance Single Cross-Sectional Interrupted Time Series Student Grades & Discipline Referrals Linear Regression RQ 2 -How will changes in teachers’ instructional practices, initiated by the set of principal-teacher interactions, affect student performance?

Figure 6 Page 97

d Figure 7 Page 100

Figure 8 Page 101

Figure 9 Page 102

Classroom Grade Distributions and Discipline Referrals Improved Percentage of As were higher than expected. Percentage of Ds were higher than expected Percentage of Fs were lower than expected. Discipline referrals were lower than expected. – Mainly due to decreases in aggressive discipline and male discipline. – Freshman and senior discipline were impacted more than other grades. Page 116

Conceptual Framework Page 8-11, Figure 1

Student Performance Indicators for High, Medium and Low Performing Teachers According to QIR ratings, high performing teachers, had the highest quality of instructional practices and improved them the most over the course of the year. According to QIR ratings, medium performing teachers fell in the middle of the spectrum of teacher quality and did not improve. According to QIR ratings, low performing teachers had the lowest quality of instructional practices according to the QIR and improved similarly to the high performing teachers. Page

Student Performance Indicators for High, Medium and Low Performing Teachers If the overall quality of instructional practices were the main reason for improved grade distributions and discipline referrals then, High Performing teachers would have the best grade distributions and lowest discipline referral number. Medium Performing teachers would have the next best grade distributions and next lowest discipline referrals. Low Performing teachers would have the worst grade distributions and the highest discipline referrals. But according to data analysis, the classroom grade distributions and discipline referrals for high, medium and low performing teachers were equivalent. Table 25 & Page

Frequency & Focus of Teacher Conversations Research QuestionResearch DesignMeasuresAnalysis 3 Freq & Focus of Teacher Conversations Pre-Mid-Post test Teacher & Student Surveys Chi Square RQ 3-How will changes in principal-teacher interactions affect the frequency and focus of teacher conversations with principals, students, and other teachers?

Frequency and Focus of Teacher Conversations QuestionResponse Spring 07 1 Spring 08 1 Spring 09 1 χ2 (df=2) 07/08 χ2 (df=2) 08/09 Principal-Teacher Conversations How often do you discuss curriculum issues with a principal? Weekly or Daily Monthly Never or Annually How often do you discuss discipline issues with a principal? Weekly or Daily Monthly * 5.35 Never or Annually How often do you discuss teaching strategies with a principal? Weekly or Daily 8103 Monthly Never or Annually Table 26 & Page 122

QuestionResponse Spring 07 1 Spring 08 1 Spring 09 1 χ2 (df=2) 07/08 χ2 (df=2) 08/09 Teacher-Teacher Conversations How many times per day do you speak to another teacher? 8 or more times or *** 0.03 None or One1300 How often do you discuss curriculum issues with other teachers? Daily or Weekly Monthly *** 6.69 * Never or Annually1443 How often do you discuss discipline issues with other teachers? Daily or Weekly Monthly *** 2.76 Never or Annually3343 Frequency and Focus of Teacher Conversations Table 26 & Page 124

Frequency and Focus of Teacher Conversations According to teacher surveys, the frequency of principal-teacher conversations improved, but the focus remained unchanged. According to teacher surveys, the frequency and focus of teacher-teacher conversations improved during the pilot year and maintained in the year of full implementation. According to student surveys, the frequency and focus of teacher-student conversations remain unchanged. Pages & 122

Findings Teacher instructional practices improved according analysis of QIR data. Student performance increased according to the analysis of student grade distributions and discipline. Freq & Focus of some teacher conversations changed according to analysis of teacher and student surveys. Pages 109

Implications Principal Visits and Collaboration with Teachers Rubric Based Assessment of Instructional Practices Working with Teachers of Differing Qualities of Instructional Practices Page

Unintended Outcomes Exiting Teachers Principal-Student Relationships Principal-Parent Discussions Increased Job Satisfaction for the Principals Page

Recommendations for Future Research Further research on particular treatment needed for teachers at various levels of performance How principal interactions in the classroom could strengthen and support the walk-through model currently used by many schools and districts Research on this treatment in other settings (generalizability) Individual effects of each of the four interventions used in this study Page 133

This Study’s Resolutions to Central Dilemmas of Nearly all Principals How can I find time to get into classrooms? How do I engage teachers in job related conversations about instructional practices? How do I get teachers to look at performance data of their students? How can I increase principal job satisfaction? How can I reduce discipline referrals? How can I decrease failure rates (improve student grades) while increasing the quality of instructional practices? How can I know the actual quality of instructional practices? Table 28 Page 135

Thank You Brennon SappKim Banta