1. RE-THINKING HOW SCHOOLS IMPROVE: A Team Dialogue Model for Data-Based Instructional Decision Making Dr. Michael E. Hickey mehickey@towson.edu Dr. Ronald S. Thomas rathomas@towson.edu Center for Leadership in Education at Towson University CCSSO Education Leaders Conference September 12, 2007

2. 2 The Big Picture In today’s session, we are going to: 1.Re-think our understanding of how schools improve—moving from the dysfunction of the old model to the requirements for what a “new model” might look like. 2.Focus on a “new model” for improving performance that enables content, vertical, or departmental teams to use data more effectively for classroom instructional improvement and increased student learning

3. Part 1: What are we trying to do and why?

4. 4 “Every organization is perfectly designed to get the results it achieves.” --W. Edwards Deming

5. 5 Think about how long you have been engaged in the school improvement process. Has the school gotten better each year? Has the performance of each student improved as a result of each year he/she spends in the school? If your answer to one or both questions is no, what will it take to change it to yes?

6. 6 What are data? Data are observations, facts, or numbers which, when collected, organized and analyzed, become information and, when used productively in context, become knowledge.

7. 7 The DRIP Syndrome

8. 8 Being Data Rich Your school may suffer from You may need ways to organize the data.

9. 9 Sources of Student Achievement Data External assessment data Benchmark or course-wide assessment data Individual teacher assessment data --Supovitz and Klein (2003)

10. 10 Data-driven schools and school districts use data for two major purposes: Accountability (to prove) Instructional decision making (to improve)

11. 11 The Hierarchy of Data for Accountability Purposes External (State & National) Assessments System Benchmark Assessments Common School or Course Assessments Classroom Assessments of Student Work

12. 12 The Hierarchy of Data for Instructional Decision Making Classroom Assessments of Student Work Common School or Course Assessments System Benchmark Assessments External (State & National Assessments)

13. 13 Think about it... Do you have a school improvement plan? Or a school accountability plan? A SIP ? Or a SAP? Have a three minute conversation with someone sitting near you about what you think most schools currently have.

14. 14 The Old Model The School Improvement Team, a Data Committee, or one person analyzes data, using primarily state test data. These data are mined for every possible nuance.

15. 15 The Old Model The data are presented at a faculty, SIT, or department meeting, and faculty members brainstorm ideas for what to do to increase student performance.

16. 16 The Old Model Faculty or team members “average opinions” and put forth the solutions that are acceptable to the largest majority of people.

17. 17 The Old Model This results in school-wide or department-wide initiatives that may or may not be implemented. Data expert Mike Schmoker has estimated that about 10% of what is planned in SIPs actually is implemented at a high level of quality.

18. 18 Results of the Old Model

19. 19 Why is the old model not working anymore?

20. 20 Why? Wrong Data We have been using the wrong data. State test data are: Way too general Instructionally insensitive – not designed for instructional improvement

21. 21 Why? Wrong Time The data come at the wrong time. State test data are: Out of date when they arrive For students we no longer have The results of the changes that are implemented will not be known for a year.

22. 22 Why? Wrong Team The SIT, a full department, or a Data Committee is the wrong team to do the analysis. Membership is too diverse (often including parents) Meets too infrequently Not connected to immediate classroom needs

23. 23 Why? Wrong Plan The initiatives that are put in place are: Too global to address the diversity of students Aimed at performance increases of groups on average Looking for the “silver bullet” that will have a schoolwide impact

24. 24 We need a new model. Real time Specific to each grade and subject Addresses individual students’ needs Results in instructional improvements that will actually occur at a high level of quality Can be re-directed frequently Has meaning for teachers (seen by teachers as a worthwhile use of their time) THREE MINUTE CONVERSATION: How do the data conversations in schools that you know of rate against these criteria?

25. 25 What should that new model look like? “School improvement is most surely and thoroughly achieved when teachers engage in frequent, continuous, and increasingly concrete and precise talk about teaching practice... adequate to the complexities of teaching, capable of distinguishing one practice and its virtue from another.” --Judith Warren Little

26. 26 In other words... A Classroom-Focused Improvement Process (CFIP)

27. 27 Education After Standards

28. 28 The Classroom-Focused Improvement Process is the work that professional learning communities do. A professional learning community is not an organizational structure. It is a WAY OF DOING BUSINESS.

29. 29 CFIP: A WAY TO MOVE SCHOOLS From To Focus on teaching Emphasis on what was taught Coverage of content Curriculum planned in isolation Infrequent summative assessments Focus on average scores Focus on learning Fixation on what students learned Demonstration of proficiency Shared knowledge of essential curriculum Frequent common formative assessments Monitoring individual proficiency on every essential skill

30. 30 CFIP: A WAY TO MOVE SCHOOLS From To Remediation One opportunity to demonstrate learning Isolation Each teacher assigning priority to different learning standards Privatization of practice Focus on inputs Intervention Multiple opportunities Collaboration Teams determining priority of learning standards Sharing of practice Focus on results

31. 31 Fundamental Concepts of Collaborative Learning Communities Teachers establish a common, concise set of essential curricular standards and teach to them on a roughly common schedule. Teachers meet regularly as a team for purposes of talking in “... concrete and precise terms” about instruction with a concentration on “thoughtful, explicit examination of practices and their consequences.” Teachers make frequent use of common assessments. Continued on next slide

32. 32 “These elements, so rarely emphasized in school... improvement plans, deserve our attention more than anything else we do in the name of school improvement.” --Mike Schmoker (2006)

33. 33 Our Goal in the Data Dialogues: Frequent, continuous, and increasingly concrete and precise dialogue by school teams, informed by data

34. 34 IS IT WORTH THE EFFORT? Take a look at the following results. Then you tell us.

35. 35

36. 36

37. 37

38. 38

39. 39 Grasonville Elementary School Maryland School Assessment - Reading

40. 40 Grasonville Elementary School Maryland School Assessment - Mathematics

41. 41 THE CLASSROOM-FOCUSED IMPROVEMENT PROCESS (CFIP): A Team Data Dialogue Protocol Part 2: Components of THE NEW MODEL

42. 42 What are the right teams to conduct data dialogues?  Grade-level  Vertical  Content

43. 43 When is the right time to conduct data dialogues? At a minimum, devote at least one hour to data dialogues every two weeks. According to several studies, schools that realized the greatest results from a shift to a data culture scheduled data dialogues at least once a week.

44. 44 Frequency of Data Dialogues Source: Stanford University, Stanford Research Institute, Education Week, January 24, 2004

45. 45 What are the right data to use in the data dialogues? Triangulate three types of data: External Assessment Data Course-wide Benchmark Assessment Data Classroom Assessment Data --Supovitz & Klein (2003)

46. 46 THE GPS ANALOGY

47. 47 Conclusions are specific to students in the class. Conclusions are used to plan upcoming daily instruction. The plans are implemented. What is the right plan where the results of the data dialogues should be used?

48. 48 What is the right way to use the results of the data dialogues?  Conclusions are used to identify enrichments and interventions for the students in the class.  Conclusions are used to plan upcoming daily instruction.

49. 49 The new process needs to be built on: 1. Dialogue 2. Protocols 3. Triangulation of Data

50. 50 Why True Dialogue? “In dialogue, a group accesses a larger ‘pool of common meaning,’ which cannot be accessed individually. People are no longer primarily in opposition, rather they are participating in generating this pool of common meaning…. We are not trying to win in a dialogue. We all win if we are doing it right.” - Senge, The Fifth Discipline (2006)

51. 51 Team Learning Team learning is the process of aligning and developing the capacities of a team to create the results its members truly desire. The discipline of team learning starts with “dialogue,” the capacity of members of a team to suspend assumptions and enter into a genuine “thinking together.” It also involves learning how to recognize the patterns of interaction in teams that undermine learning. --Peter Senge (2006)

52. 52 What Is a Data Protocol? A protocol consists of guidelines for dialogue – which everyone understands and has agreed to – that permit a certain kind of conversation to occur, often a kind of conversation which people are not in the habit of having. Protocols build the skills and culture necessary for collaborative work. Protocols often allow groups to build trust by doing substantive work together.

53. 53 Using a Data Protocol Protocols can help us to navigate difficult and uncomfortable conversations by:  Making it safe to ask challenging questions  Making the most of scarce time  Providing an opportunity for all to be involved  Resulting in an analysis that will lead to positive action

54. 54 Using a Data Protocol The point is not to do the protocol well, but to have team dialogue that is: In-depth Insightful Concrete Precise

55. 55 The Big Six of Data Analysis 1. Begin with a question. 2. Understand the data source. 3. Look for the big picture. 4. Look for patterns in the data. 5. Identify and act on the implications of the patterns for your students. 6. Identify and act on the implications of the patterns for your instruction.

56. 56 CFIP DATA DIALOGUE PROTOCOL FORMATS One-page overview of the model, page 15 CFIP model with reflection questions, pages 17-18 CFIP model worksheets, pages 19-22 Reflection Guide to Instructional Changes, pages 23-24 Examples of CFIP model as completed by school teams, pages 25-38 Take a few minutes to preview these pages.

57. 57 SIX-STEP PROCESS - TEAM DATA DIALOGUE PROTOCOL: MOVING FROM DATA TO INCREASED STUDENT LEARNING DATA SOURCE(S): __________________________________________________________________________ Step 1: Identify the questions to answer in the data dialogue. Step 2: Build assessment literacy. Define terms (if needed). Step 3: Identify the “big picture” conclusions from the data. Step 4: Identify the patterns of class strengths and weaknesses (using more than one data source, if possible). STUDENT STRENGTHSSTUDENT WEAKNESSES Step 5: Drill down in the data to individual students. Identify and implement needed enrichments and interventions. STUDENTS WHO EXCELLED ENRICHMENTS TO BE PUT IN PLACE STUDENTS NEEDING FURTHER WORK INTERVENTIONS TO BE PUT IN PLACE Step 6: Reflect on the reasons for student performance. Identify and implement needed instructional changes for the next unit.

58. 58 CFIP Step 1: When analyzing data, begin with a question.  All data analyses should be designed to answer a question.  Unless there is an important question to answer, there is no need for a data analysis.

59. 59 CFIP Step 2: Understand the data source Build ASSESSMENT LITERACY with questions like these:  What assessment is being described in this data report? What were the characteristics of the assessment?  Who participated in the assessment? Who did not? Why?  Why was the assessment given? When?  What do the terms in the data report mean? Be sure you have clear and complete answers to these questions before you proceed any further.

60. 60 CFIP Step 3: Look for the “big picture” views in the data. Identify:  What do we “see” in the data?  What “pops out” at us from the data?  What questions do the data raise?

61. 61 CFIP Step 4A: Look for data patterns in a single data source.  What do you see over and over again in the data?  What are the students’ strengths? What knowledge and skills do the students have?  What are their weaknesses? What knowledge and skills do the students lack?

62. 62 CFIP Step 4B: Identify Patterns of Class Strengths and Weaknesses from Multiple Data Sources. TRIANGULATION In what ways are the results similar among data sources? For example, how do benchmark test results compare with ongoing classroom assessment data? In what ways do the results among data sources differ? Why might these differences occur?

63. 63 Power When Multiple Types of Data Are Used Reduces the anxiety and the mistakes of relying on a single measure as the only definition of student success Provides more frequent evidence on which to act Develops and sustains a culture of inquiry in the school based on data

64. 64 CFIP Step 5: Drill Down to Individual Students. Identify and Implement Needed Enrichments and Interventions.  What are the implications for enrichments and interventions from what you learn from the data?  Which students need enrichments and interventions?  What should enrichments and interventions focus on?

65. 65 CFIP Step 6: Reflect on the reasons for student performance -- What in our teaching might be preventing all students from being successful? To what extent did we implement research-based instructional practices as we:  Planned instruction?  Introduced instruction?  Taught the unit?  Brought closure to instruction?  Assessed formatively?

66. 66 CFIP Step 6: Reflect on the reasons for student performance. Identify and implement instructional changes in the next unit. How will we change instruction in our next unit? Content focus Pacing Teaching methods Assignments

67. 67 CFIP Step 6: Reflect on the reasons for student performance. Identify and implement instructional changes in the next unit.  When will we review the data again to determine the success of the enrichments, interventions, and instructional changes?  What do the data not tell us?  What questions about student achievement do we still need to answer?  How will we attempt to answer these questions?

68. 68 The Next Steps 1.Unless teams emerge from the data analysis process with a clear plan of action for their classroom, they have wasted their time. 2.Implement the plan of interventions, enrichments, and changes in instruction. 3.Collect the next set of data.

69. 69 Where does a school go from here in becoming more data-driven? The DriversThe Barriers DISCUSSION: What drivers and barriers would you see schools facing in implementing the CFIP model?

70. 70 Typical School Improvement Plan (SIP) Classroom Focused Improvement Process (CFIP) Process established at district level Process designed at team level Linear and prescriptiveNon-linear/non-prescriptive Annual strategic planShort-cycle operational plan Impact: total schoolImpact: students in class SIT developsClassroom-level team develops Purpose: meet AYPPurpose: adjust practice Results determined end of year Results determined when unit is taught

71. 71 So what about the School Improvement Team? The School Improvement Team (SIT) as typically constituted is designed to do exactly what its name implies: IMPROVE THE SCHOOL. It is not designed to improve teaching and learning at the classroom level. That is the focus of the content or grade-level team or the department.

72. 72 Core Functions of the SIT Keep the vision alive. Develop and monitor school-wide plan for meeting state accountability standards. Build a data-driven culture. Establish priority focus on instruction. Provide a safe and supportive environment for all students. Connect school with parents and stakeholders. Provide needed resources.

73. 73 Caveats about CFIP It is a paradigm shift from traditional lesson planning format. It is not easy, especially at first. Follow the steps faithfully until they become second nature. The CFIP is a guide until you make the process your own. Expect mistakes and imprecision in the data. The results are worth the effort.

74. 74 Coming together is a beginning, staying together is progress, and working together is success. - Henry Ford