1. National Summit for Principal Supervisors
Building Capacity through the Pilot Florida Implementation Network’s Leadership Cohort
National Summit for Principal Supervisors
May 19, 2017

2. Currently, what are your district’s biggest academic challenges?
Do Now: With a partner you are not already familiar with, discuss the following:
Currently, what are your district’s biggest academic challenges?
What role(s) do school leaders play in addressing those challenges?
How are you supporting leaders to do this as effectively as possible?
Split room and assign one of these questions. Share out all responses

3. The Pilot Florida Implementation Network
Agenda
The Pilot Florida Implementation Network
The Leadership Cohort Approach
District Application: Broward County
Questions and Discussion
Time: 20 minutes
Since 2015, the Pilot Florida Implementation Network, a collaboration between six Florida counties, has worked to develop the capacity of its central-office and school leaders to support teaching aligned to the Florida Standards. During this session, TNTP, the hub organization for the Pilot Florida Implementation Network, will discuss the efforts that these districts have undertaken to build knowledge and skill among principal supervisors and other leaders. In particular, TNTP will discuss the network's Leadership Cohort, a group of district and school-based leaders from each district which convenes regularly to observe classroom practice using a common instructional vision and share strategies for teacher and leader development.

4. Great Teaching Is at the Core of What We Do
TNTP's mission is to end the injustice of educational inequality by providing excellent teachers to the students who need them most and by advancing policies and practices that ensure effective teaching in every classroom.
Great Teaching Is at the Core of What We Do
Ben
For those of you who don’t know TNTP, we’re a national nonprofit focused on ending educational injustice in schools, districts, states and across the country.
Our work falls largely into three big buckets:
Rigorous Academics
Talented People
Supportive Environments
Rigorous Academics
Talented People
Supportive Environments
Are students studying challenging, engaging and relevant content?
Are educators in the right roles with the right skills to help students thrive?
Are policies, systems and communities supporting great schools for all?

5. Teaching and Learning Aligned to the Florida Standards
The Pilot Florida Implementation Network consists of five districts working together to address the challenges of effectively implementing the Florida Standards—with TNTP as the “hub.”
Clear definitions of great teaching and learning
Aligned instructional materials
Leaders who can support the implementation of those materials
Teaching and Learning Aligned to the Florida Standards
PFIN represents roughly 20% of the state’s students, 21% of the state’s non-white students and 26% of the Florida’s D and F schools.

6. Recent research has shown that though states adopted more rigorous standards five years ago, many assignments still lack alignment to those standards.
Key Findings:
Overall, only about 5% of assignments fell into the high range on the assignment analysis rubric
Only 38% of assignments were aligned with a grade-appropriate standard
Only 55% of assignments were connected with a text
Only 16% of assignments required students to use a text for citing evidence a support for a position or claim
Only 4% of all assignments reviewed pushed student thinking to higher levels
Checking In: Do Classroom Assignments Reflect today’s Higher Standards? The Education Trust, September 2015, p.4

7. We conducted diagnostics in Florida that seek to answer one key question.
To what extent are current practices at the classroom, school and district levels supporting or hindering efforts to ensure rigorous, standards-aligned instruction for all students?

8. Why do we need a network focused on the implementation of the FL Standards to begin with? Four key findings from our initial diagnostics: 1
Florida teachers are not yet adjusting their instructional practices to meet the demands of the Florida Standards. 2
Teacher and leader confidence in their ability to teach the Florida Standards doesn’t match up with reality observed at schools. 3
Districts lack a clear definition of excellent instruction and a strategy to achieve it. 4
Teacher instructional materials, evaluations and professional development opportunities are not well-aligned, and do not help teachers raise the rigor of their instruction.
Questions:
Do these findings surprise you?
Where else do you see districts struggling with these findings?

9. Performance on the ACT Reading Test by Textual Element
A large scale study of ACT results showed that isolated comprehension skills did not separate strong readers from weak ones.
Performance on the ACT Reading Test by Textual Element
ACT was surprised by the results, so they next separated out the questions by different types of comprehension skills. They looked at different kinds of questions- about main idea, drawing conclusions, etc. thinking that maybe students struggled with one skill or type of question more than others.
Again they found there was no statistical significance in the data. Students were able to answer the different kinds of questions in the same ways.
Other notes:
Similarly, question types by textual element—such as main idea or author’s purpose-- did not differentiate students’ performance.
Similarly, a student who is scoring a 16 is getting about 40% of the questions on main idea, supporting details, word meanings, generalizations and conclusions. Which begs the question– why are these individual standards or skills the driver for literacy instruction. If the student performs poorly on the main idea question, he/she is very likely to perform poorly on the details and conclusion questions. So what is tripping the students up?
This research, highlighting the correlation of how students performing on question types, far precedes the ACT study. I was just reading a study from 1974 that illustrated the exact same idea.
Source: Reading Between the Lines: What the ACT Reveals About College Readiness in Reading

10. Performance on the ACT Reading Test by Degree of Text Complexity
Instead, what separated students was their ability to comprehend complex text.
Performance on the ACT Reading Test by Degree of Text Complexity
Finally, they looked at the text itself instead of the what students were trying to do with the texts.
And here you can see a separation.
Text complexity is what makes a different. Students who struggled had a hard time answering questions of any type on complex texts.
Pause– What take aways do you have about this study? What does this mean for the instruction in our schools?
We need students to have practice with complex texts and exposure to and ability to understand them.
What is separating kids is their ability to understand the text- not the strategies and skills.
Other notes:
The complexity of the text! Until the college and career ready benchmark (the score of 21) kids are basically guessing– they are flat-lining on complex text. But as students’ ability to comprehend the complex text improves, they are getting more answers correct.
What does this mean? That we can’t just put increasingly complex texts in front of our students, but rather, we also need to move from instruction that focuses on question types or individual standards (revisit the 2 preceding slides), to alternate means for getting kids into complex texts. But we’ll get there with the other Core Actions and indicators.
Source: Reading Between the Lines: What the ACT Reveals About College Readiness in Reading

11. High knowledge of baseball Low knowledge of baseball
Research also shows that students’ knowledge of a topic is a far more powerful predictor of their comprehension.
Recht & Leslie (1988) Compared Reading Comprehension for Four Categories of Students
High reading ability
High knowledge of baseball
Low knowledge of baseball
Low reading ability
Researchers Recht and Leslie (1998) explored the role of background knowledge on reading comprehension by comparing the relative impact of reading ability to the impact of knowledge of a topic. They had students read a passage about baseball and then tested students comprehension of the passage.
Had 7th and 8th grade students read a short text about baseball and tested them for comprehension using:
Verbal retelling
Reenactment with figurines
Verbal summary
Rating ideas from the story in terms of importance.
“Effect of Prior Knowledge on Good and Poor Readers‘ Memory of Text” from Journal of Educational Psychology 1988, Vol. SO, No. 1, 16-20
In the Recht & Leslie study, students in these 4 groups read a passage about baseball and then took a test of their comprehension.
(Ask participants to predict what percent of questions on a reading comprehension test each category of student would get right.)
Technical Notes for study:
“The high-ability /high-knowledge cell had 10 boys and 6 girls; the high ability/low-knowledge cell had 3 boys and 13 girls; the low-ability/
high-knowledge cell had 12 boys and 4 girls; and the low-ability/low knowledge cell had 7 boys and 9 girls.”
High reading ability was defined as 70th percentile or higher on a standardized test of reading. Low reading ability was defined as 30th percentile or lower.
High knowledge of baseball was defined as 70th percentile or higher on a test of baseball knowledge. Low baseball ability was defined as 30th percentile or lower.
Source: Recht, D. & Leslie, L. “Effect of Prior Knowledge on Good and Poor Readers Memory of Text.” (1998) Journal of Educational Psychology, Vol. 80, No. 1, 16-20

12. The standards therefore call for us to move away from isolated, skills-based instruction and instead put a high-quality complex text at the center of lessons.
before
now
Starting with a high-quality, complex text, sequenced to build knowledge, and using the standards in service of understanding the deeper meaning of that text.
Starting with a standard and picking a text to “teach” to that standard.
Regular practice with complex text and its academic language.
Reading, writing, and speaking grounded in evidence from text, both literary and informational.
Building knowledge through content rich nonfiction.

13. International Math assessments show a more complex picture than typically discussed. On the TIMSS, which focuses on “hard skills,” American students fare relatively well.
(click to reveal 8th grade)

14. However compared to PISA results, which focus on “soft skills” like application and problem solving, US students are below average.
Say: Relative to the other countries who take this assessment – the US is significantly below the OECD average, with over 30 countries with significantly higher scores.

15. before now Answer-getting “Mathematizing” Focus Coherence Rigor
Therefore the new standards demand paradigm shift away from “answer-getting” towards instruction which helps students flexibly and creatively use mathematics.
before
now
Answer-getting
“Mathematizing”
Focus
Focus strongly where the standards focus.
Coherence
Think across grades, and link to major topics within grades. 
Rigor
In major topics pursue: conceptual understanding, procedural skill and fluency, and applications with equal intensity.  
Effective instruction requires students to use mathematics, not just get an answer, as well as do the right mathematics
Change paradigm away from “answer-getting”
Our experience has taught us that without this shift, we have a tendency to proceduralize
Combination of both content and practice.

16. TIMSS “Toggle Items” reveal whether students understand the mathematical concept or if they have only learned procedures. 9 4 13
𝟑 𝟒 𝟏 𝟑
𝟏𝟑 𝟏𝟐
Item #1 12
𝟏 𝟐 + 𝟏 𝟑 + 𝟏 𝟒 =
Lead facilitator: A closer investigation of items on national and international assessments can lead to an understanding of why we do not continue to see growth on assessments and continue to lag behind international peers. Phil Daro calls these items “toggle items” - items that reveal whether students may understand the concept or have only learned procedures.
<reveal example with butterfly method>
Item #2

17. Despite the perception that teachers had made the instructional shifts required of the new standards, few lessons reflected those changes.
Teachers are making attempts to implement instructional strategies to align to Common Core, but lack a clear vision of what aligned practice looks like in action, so their attempts fall short. This is a strong foundation from which to build. Teachers are willing to try new things and are eager to help students meet the expectations of the standards. At one school we saw three classrooms teaching the same basic lesson, but each was executed very differently, which is evidence of the lack of a clear vision for what instruction should look like.
n = 303 lessons

18. Most students meet expectations for their assignment, but the lack of rigor in tasks means students aren’t meeting the expectation of the standards.
n = 1,350 student work samples collected

19. What may be leading to these perceptions?
And yet, many school leaders and educators think that classroom practices are aligned with the shifts necessary to teach the FL Standards.
70%
of school leaders* are confident in their ability to lead teachers to shift instruction
70%
of teachers** believe their school is prepared to shift to the new standards
What may be leading to these perceptions?
This confusion plays out when we ask teachers and school leaders whether they feel like they’ve “got it.” Most school leaders and teachers believe they’ve made the changes they need to—however, the kinds of activities we just practiced at the beginning of this session are not in all classrooms. Only 32% of the teachers we observed when we conducted diagnostics were demonstrating even some of the shifts and only 27% of students met the expectations of the standards in their assignments.
Teachers and school leaders express confidence in their understanding of the standards, but there is limited evidence that their knowledge is translating to changes in practice.
Note: Data are averages across districts
*n= 972 total administrator survey respondents; **n= 16,970 total teacher survey respondents

20. Recent research is telling us more and more about the effects of having more aligned, higher quality materials.
“…if all schools could be persuaded to switch to one of the top quartile textbooks, student achievement would rise overall by roughly .127 student-level standard deviations or an average of 3.6 percentile points. Although it might sound small, such a boost in the average teacher’s effectiveness would be larger than the improvement the typical teacher experiences in their first three years on the job, as they are just learning to teach.” -Thomas J. Kane, Harvard School of Education (2016)
1500 teachers
150 principals
Feb – Mar 2015

21. Furthermore, students learn primarily through interactions with teachers and instructional materials. Strong materials affect student learning.
“…choice of instructional materials can have an impact as large as or larger than the impact of teacher quality.” “…improving teacher quality is challenging, expensive, and time-consuming, making better choices among available instructional materials should be relatively easy, inexpensive, and quick.”
Chingos and Whitehurst, “Choosing Blindly: Instructional Materials, Teacher Effectiveness and the Common Core,” Brown Center on Education Policy at Brookings, (April 10, 2012),

22. We discovered that our network districts have begun to introduce materials that have a greater degree of alignment with standards, but the implantation of those materials do not address the shifts in practice.
The selection and implementation of high-quality materials is a huge factor in ensuring that students are engaging with appropriately complex text. And Pasco has implemented standards-aligned materials across grades K-12.
However, the challenge comes in implementation. Though we see a majority of students have the right content in their hands.
The Common Core standards will only have a chance of raising student achievement if they are implemented with high-quality materials...
The Brown Center on Education Policy, Choosing Blindly
Choosing Blindly, Chingos & Whitehurst, The Brown Center on Education Policy at Brookings:

23. Recognizing this context, we worked with our districts to set big goals for the Pilot Florida Implementation Network
Overall: Students and teachers in the implementation network districts are more often engaged in teaching and learning that is aligned with the expectations of the Florida Standards.
Central OFFICE KNOWLEDGE
Key leaders in core departments are knowledgeable about the standards and shifts and are clear how they support the vision of excellent instruction. The foundation is laid for goals-based management.
DEFNITION OF TIER I INSTRUCTION
Each of the 6 districts has a vision of excellent instruction (that is aligned with the standards and shifts).
ALIGNED
MATERIALS
Districts have an accurate assessment of the alignment of their materials K-12 ELA and Math.
INSTRUCTIONAL LEADERSHIP
All network districts have a strategy in place for how principal managers support schools with the implementation of the vision of excellent instruction.

24. Individualized District Work TNTP’s Role as the “Hub”
To meet our goals as a network, we are using a combination of shared supports and individualized supports for each district.
Network Goals
Leadership Cohort
Curriculum Cohorts (K-5, & 6-12 ELA, K-5 Math)
Goals for the network
Pathways to Adoption
Shared Work
Brevard
Broward
Duval
Highlands
Pasco
Individualized District Work
TNTP’s Role as the “Hub”
Network Communication
Progress Monitoring
Convenings
Knowledge Management

25. Our theory of action is guiding our work toward one big goal: ensuring more teaching and learning that prepares students for college and career, as defined by the Florida Standards. The next phase of our work will focus on reviewing, selecting, and implementing instructional materials.
These five districts have all agreed that there are four key things that they would like to work on together:
Districts need a clear definition of what great, FL Standards-aligned instruction is.
Districts need instructional materials that are aligned to the standards (many are not)
Leaders, including principals, need to know what to look for when they are in classrooms
The data and feedback from this work should inform continuous improvement

26. The Pilot Florida Implementation Network
Agenda
The Pilot Florida Implementation Network
The Leadership Cohort Approach
District Application: Broward County
Questions and Discussion
Time: 15 minutes

27. We’re attempting to move from this...
New Teacher Mentors
Teacher Support & Development Team
Human
Resources
Schools & Operations
Academics
Principals
Curriculum Specialists
Assessment Specialists
DISTRICT TEACHER
Assistant Principals
Specialized Student Services
Data, Systems & Strategy
External Evaluators
Instructional Coaches
Teacher
Leaders
Ben
Through interviews and focus groups, we gained greater insight into the maze of development activities and the people teachers engage with along the way.
We heard that there are many central office employees focused on helping teachers, but they span different departments and perform different functions, so working consistently as a team can be a challenge.
We also heard from teachers that often, the people employed to support their development may not actually be on the same page about their development goals.
Broadly speaking, teachers described a system that lacks any real vision or strategy, where few seem to agree on what “teacher improvement” means or what “good teaching” looks like.

28. A common vision for excellent instruction Curriculum
…to this.
A common vision for excellent instruction
Curriculum
set the floor for instructional excellence with high quality materials.
Assessment
Monitors the expectations for teaching and learning.
Professional learning supports teachers to meet those expectations and administrators to be instructional leaders.
Ben
What we want is a system where that vision is driving the work of supporting teachers and classrooms in a coherent way.

29. Culture and Community Builder
Of all district- and school-level responsibilities, the Florida Network focuses most on developing leaders’ ability to provide instructional leadership.
Instructional Leader
Talent Manager
Culture and Community Builder
Operation Expert
1:00 – 1:05 Addison
In your roles as instructional leaders, you are critical to ensuring all classrooms in your building are engaging in the rigorous instruction required by the Florida Standards.

30. Leadership Cohort Objectives
Deepen our knowledge and understanding of the Florida Standards and instructional shifts through observing instruction in action.
Learn from each other in order to develop strategies for improving the alignment of instruction within their own districts.
Where appropriate, provide feedback to the host district on any specific strategies, processes and materials it uses to support teachers, leaders and students.

31. Five Components of the Leadership Cohort’s Approach
Collaborating Cohort
Consistent cohort of collaborating instructional leaders with decision-responsibility
Focus on Content
Deep focus on academic content through the ELA and Mathematics standards and shifts
Shared Observation Tools
Observations utilize common tools and immediate debriefs based on a shared instructional vision rooted in the standards 
Focus on Materials
Observations and learning sessions discuss adoption and implementation of aligned instructional materials
Progress Monitoring
Observations allow for the collection and monitoring of classroom-level data to guide implementation of academic initiatives 1 2 3 4 5

32. Academic Central Office Schools
Building a cohort of participants that can collaborate across district functions is essential to the Leadership Cohort model. Each district has built a team of participants across these three functions
Academic
Central Office
Schools
Assistant Superintendent of Curriculum and Instruction ELA Director Math Director ELA Specialist Math Specialist Chief Academic Officer
Assistant Superintendent of Professional Development
Assistant Superintendent of Leadership Development
Chief of Data and Accountability
Teacher Evaluation Manager
Principal Manager
Head of Elementary Schools
Head of Secondary Schools
Principal
Assistant Principal
Teacher Leader

33. The Leadership Cohort has utilized the common tool of the Instructional Practice Guides, which focus on the shifts in the Florida Standards.
Instructional Practice Guide
Daily lessons or over the course of the year
Instructional Shifts
Core Actions
Key practices (numbered sections)
Indicators
Observable (lettered details under each Core Action
5 minutes
SAY: Student Achievement Partners, a national nonprofit founded by several authors of the Common Core standards, has been creating resources and materials for teachers, coaches, schools, and districts to use in making the shift to college and career ready instruction in alignment with the new standards.
We’re going to use a tool today, called the Instructional Practice Guide, that is designed to support teachers, coaches and administrators in identifying how the shifts manifest in classroom practice and what supports may be necessary to improve or refine instruction. The tool is not intended to be evaluative; it is intended to be used as a coaching resource to support teacher development and instruction.
WALK through how the IPG is organized to help orient participants to the structure and set up. Be sure to note that the particular IPG we’re working with today can be used in observations, in reviewing lesson plans, and as a way to review student work. Point out that the tool is derived from the Instructional Shifts. State that there are three Core Actions with a series of indicators under each Core Action. Let them know that there are ELA guides for each grade band and math guides as well– are available for free to download.
GIVE participants a few minutes to independently review the IPG, have them focus specifically on the Core Actions and Indicators as this is what will be new to them.

34. Looking at student work using the EQUiP protocol developed by ACHIEVE provides another perspective on how teaching and learning reflects the expectations of the letter and spirit of the standards.

35. The schedule of Leadership Cohort allows for focused learning sessions as well as classroom observations and immediate debriefs that target the content of those sessions.

36. Across multiple Leadership Cohort meetings, we attempted to build a coherent progression of learning.
Brevard
Aug 31 – Sep 1
Complex text and high quality questions
Building/deepening understanding of Core Actions 1 & 2, and the actual changes in practice required by teachers.
Pasco
Oct 13-14
High quality questions and student engagement in complex text
Analyzing the paradigm shift required to ensure all students are accessing grade level standards and are meaningfully engaged in the work of the lesson.
Highlands
Nov - TBD
Building content knowledge effectively through literacy
Building an awareness that deep content knowledge is an essential foundation for reading comprehension
Spring 2017
Math
What does it mean to be college and career ready in math?
What experiences do our students need to be ready?
How can teachers capitalize on strong instructional materials?

37. Stakeholder Investment
All of our work is informed by the idea that instruction and student learning require aligned systems and practitioners with the knowledge and skill to implement effectively. No one policy is a silver bullet.
Curriculum
Do teachers have access to and use standards-aligned materials and resources that provide a pathway to achieve the instructional vision?
Stakeholder Capacity
Do stakeholders have the capacity to realize the shared vision and implement the strategy to achieve it?
Student Support Is there a data-driven system of student support that is effectively implemented to ensure that all students thrive?
Accountability
Is there a system to monitor progress and hold educators accountable for improving student learning?
Vision & Expectations Is there a shared vision of great teaching and learning?
Assessment
Do educators have access to and use high quality assessments that provide timely, accurate data and enable educators to adjust instruction, allocate resources and evaluate learning?
Stakeholder Investment
Are stakeholders invested in the shared vision and the strategy to achieve it?
Say: Imagine we had a world with perfect standards, perfect instructional materials, end of year assessments that measured what we believed matter most, and even had a strong educator evaluation system. <pause> Creating the student environment that our kids need still relies on strong instruction and instructional culture that supports strong instruction. If we don’t address our tendencies to over-proceduralize mathematics and uncover the systems that maintain these tendencies, we will not see progress even when we adopt strong standards and instructional materials. Likewise, we must work to better understand how these components interact to better support a coherent vision for the environments we want to provide more students in mathematics.

38. Leadership Cohort Tools
Artifact Review: Consider the following questions as you examine tools and resources from the Leadership Cohort.
Leadership Cohort Tools
Math and ELA Instructional Practice Guides (Doc 1 and 2)
EQUiP Student Work Protocol (Doc 3)
Example Observation Schedule (Doc 4)
Text complexity rubric (Doc 5)
Mathematics Task Narratives (Doc 6 and 7)
Questions for Review
What content does your district’s leadership development efforts target?
How does the structure of observations compare to learning walks in your district?
How are you incorporating student work into your current efforts?
What structures are in place for leaders to collaborate in your district?
10 minutes

39. The Pilot Florida Implementation Network
Agenda
The Pilot Florida Implementation Network
The Leadership Cohort Approach
District Application: Broward County
Questions and Discussion
Time: 10 minutes

40. Application of Leadership Cohort: Broward
Description of the Broward cadre structure
Goals Broward has for developing school leaders
Areas of supports provided by Cadres
How Broward has adapted and used the leadership cohort content with leaders
What impact Broward has seen

41. The Pilot Florida Implementation Network
Agenda
The Pilot Florida Implementation Network
The Leadership Cohort Approach
District Application: Broward County
Questions and Discussion
5 minutes

42. Discussion
What goals do you have for the school and central office leadership in your district?
How does the content focus of the Leadership Cohort compare to leadership capacity building efforts in your districts?
What aspects of the Leadership Cohort model could you adopt as a part of your principal development efforts.

43. Learn more about our work
Learn more about the Leadership Cohort and the tools we use at our portal:

44. facebook.com/thenewteacherproject
Connect with us.
tntp.org
facebook.com/thenewteacherproject
twitter.com/tntp
linkedin.com/company/tntp