1. Deena Abu-Lughod 1 Children First Intensive Collaborative Data Analysis, ARIS, Network Data Celebration ESO Network 14 Eastwood Manor, May 21, 2009 Facilitator: Deena Abu-Lughod, SAF Adapted from Nancy Love, Presentation to SAFs, April 2, 2009

2. Deena Abu-Lughod 2 Provisional Agenda 8:30-8:45Welcome! Announcements (share fair) 8:45-10:15 Cause-and-Effect/Verifying Causes with Data Protocol 10:15-10:30 Break 10:30-10:45 Network 14 ELA Highlights Celebration 10:50-11:05 Planning for ARIS Parent Link 11:05-11:30A Word from our Sponsor 11:30 – 11:35Evaluation 11:35 –12:35 Study Groups 12:35 – 1:30Lunch 1:30-2:30 Afternoon Consultations: Progress Report Modeler

3. Deena Abu-Lughod 3

4. 4 Learning Intention Build capacity to move into and sustain system- level change by learning to use two collaborative data tools to move the level of analysis from the target population to the system:  Cause and Effect Analysis  Verifying Causes Tree

5. Deena Abu-Lughod 5 5 Core Process Phase II: Move the Students Phase I: Identify Students and Targets Phase III: Move the System

6. Deena Abu-Lughod 6 Analyze systems that produced conditions of learning Design and implement change strategy Evaluate and revise based on interim progress measures A More Detailed Look at the Inquiry Process: Phase III

7. Deena Abu-Lughod 7 Conditions of Student Learning Curriculum (What is taught) Instruction / Teacher Preparation (How it is taught? How well it is taught? Who is teaching?) Additional considerations: >Assessment (How learning is assessed and used to inform instruction?) >Equity (Teacher assignments, student groupings, access to rigor) >Critical supports (collaboration, leadership, extra help systems, technology, policies, parent and community engagement, PD)

8. Deena Abu-Lughod 8 What really matters? Alignment Rigor Assessment Professional Development Critical Supports

9. Deena Abu-Lughod 9 Taught Curriculum Written Curriculum Assessed Curriculum STANDARDS Curriculum Alignment Matters Adapted from Fenwick English. From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

10. Deena Abu-Lughod 10 Rigor Matters: Two Gr. 7 Writing Assignments Essay on Anne Frank Your essay will consist of an opening paragraph which introduces the title, author and general background of the novel. Your thesis will state specifically what Anne's overall personality is, and what general psychological and intellectual changes she exhibits over the course of the book. You might organize your essay by grouping psychological and intellectual changes OR you might choose 3 or 4 characteristics (like friendliness, patience, optimism, self doubt) and show how she changes in this area. All About Me (fill in the blanks): My best friend… A car I want… A chore I hate… My heartthrob…

11. Deena Abu-Lughod 11 Teaching Matters Tools we have used: 1)Lesson observation with the differentiated instruction rubric 2)Low inference transcripts 3)Teacher reflections 4)Tuning Protocol 5)Teacher Data Initiative

12. Deena Abu-Lughod 12 Assessment Matters 250 research studies from several countries establish that improving formative assessments raises achievement. Few initiatives in education have had such a strong body of evidence to support a claim to raise standards. – Paul Black et al., 2004, p. 9

13. Deena Abu-Lughod 13 Professional Development Matters Little Impact: short, episodic, and disconnected from practice has little impact. High Impact: Well-designed programs offering extended PD (49 hours on average over 6 to 12 months) Professional Learning in the Learning Profession A Status Report on Teacher Development in the United States and Abroad Linda Darling- Hammond & Nikole Richardson 2009 (www.nsdc.org)

14. Deena Abu-Lughod 14 Critical Supports Matter Factors that support expanding opportunities to learn: >Collaborative culture and structures >Leadership >Extra help for students who need it >Effective uses of technology >Policies that align with learning >Parent and community engagement

15. Deena Abu-Lughod 15 ” “ Powerful Words Research has found that faculty in successful schools always question existing instructional practice and do not blame lack of student achievement on external causes.…The “source of the problem” in ordinary schools is always someone else: the students, the parents/caretakers, the school board, and so on. — Carl Glickman, 2002, pp. 4, 6

16. Deena Abu-Lughod 16 Review the “Conditions of Learning” slide (#7). Give a one-minute elevator speech about it. Add any additional research you know. Partner and Talk

17. Deena Abu-Lughod 17 Data-Driven Dialogue Agenda (full) Agenda: Meeting 1 (3-4 hours) Group roles and collaborative norms (15-20 minutes) Share findings from the research (60-120 minutes) Frame questions for school-level data collection (60 minutes) Develop a school-level data collection plan (30 minutes) Agenda: Meeting 2 (2-3 hours) Review group roles and collaborative norms (15-20 minutes) Engage with Data-Driven Dialogue to examine the school-level data and verify causes (60-120 minutes) Plan for engaging stakeholders and review school culture (30 minutes)

18. Deena Abu-Lughod 18 Cause-and-Effect Analysis and Verification 1: Generate Possible Causes 2: Dialogue about Causes 3: Identify Causes for Further Verification 4: Frame Research Questions Based on Causes 5. Study the Research 6. Frame Questions for Local Data Collection 7. Analyze Local Data and Verify Causes 8. Plan for Engaging Stakeholders and Review School Culture

19. Deena Abu-Lughod 19 Verify Causes Tree Directions 1.Enlarge the graphic organizer, leaving space in all boxes for writing 2.Write the student-learning problem in the box on top 3.Generate possible explanations for the problem in the first row of boxes, organized by the categories provided 4.Gather research to verify your causes 5.Record your research findings on the Verify Causes Tree 6.Modify, eliminate, or add to your list of possible causes 7.Collect and analyze local data to investigate the causes in your own setting that have been verified through research 8.Record local data findings on Verify Causes Tree 9.Record causes that are verified through research and local data in the Verified Causes boxes in the Tree Adapted from P. G. Preuss, Root Cause Analysis: School Leader’s Guide to Using Data to Dissolve Problems, Larchmont, NY: Eye on Education. From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

20. Deena Abu-Lughod 20 Shifts Toward Verifying Causes Quick fix and knee-jerk reactions to data Analysis of root causes using local data (surveys, observations, interviews from diverse voices, enrollment data) and research Attributing causes to circumstances outside of the school’s control Looking for causes in beliefs, practices, and policies

21. Deena Abu-Lughod 21 Verify Causes Tree: Placemat Activity Adapted from Paul G. Preuss, Root Cause Analysis: School Leader’s Guide to Using Data to Dissolve Problems, Larchmont, NY: Eye on Education, 2003. Used with permission. From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

22. Deena Abu-Lughod 22 Background on Practice School Grades K-6 urban school The school did not meet adequately yearly progress and has been placed on the SINI list 35% of students were proficient in Language Arts in 2007 A traditional mathematics textbook program has been offered, and little professional development has been provided in mathematics In Grade 6, two levels of mathematics are offered: regular mathematics uses the Scott Foresman textbook; high-ability offers the Merrill pre-algebra program

23. Deena Abu-Lughod 23 Student-Learning Problem Statement Some sixth-grade students have a problem with mathematics achievement. Weak areas include Patterns, Relations, & Algebra and problem-solving. 34% were proficient and advanced in mathematics on the state test in 2008. The weakest strand in multiple-choice on state test in was Patterns, Relations, & Algebra (51% correct) 52% are below basic on the 2008 district assessment; 70% of students scored a 1 out of 4 on the open-response section These performance gaps were noted on the state test: A 40 percentage point gap between White and Hispanic students A 34 percentage point gap between Non-LEP and LEP students A 46 percentage point gap between Non-SPED and SPED A 24 percentage point gap between Non-Low Income and Low Income

24. Deena Abu-Lughod 24 Verify Causes Tree Directions 1: Generate Possible Causes Enlarge the graphic organizer, leaving space in all boxes for writing Write the student-learning problem in the top box (“Sixth grade mathematics problem-solving; achievement gap between White and African American students”) Generate possible explanations for the problem on large Post-its (or go through the Cause Card deck). Place them in the first row, organized by the categories provided Adapted from P. G. Preuss, Root Cause Analysis: School Leader’s Guide to Using Data to Dissolve Problems, Larchmont, NY: Eye on Education. From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

25. Deena Abu-Lughod 25 Examples of Possible Causes The curriculum is not aligned with standards We are not differentiating instruction to reach students who aren’t learning from traditional approaches We are not using formative assessments to make adjustments in our teaching Some of us don’t feel comfortable with mathematics problem-solving approaches; we didn’t learn that way

26. Deena Abu-Lughod 26 Examples of Possible Causes From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

27. Deena Abu-Lughod 27 2: Dialogue About Causes Dialogue about the causes you have generated. Which causes… >Are within our control to act on? >Reflect cultural competence? (Respect for diversity) >Can have a great impact on solving the student-learning problem? >Can be verified with additional data and/or research? >Can be addressed given our resources and time constraints? >Do we want to investigate further? Would acting on any of these causes do any harm to any student or group of students? Equity Lens

28. Deena Abu-Lughod 28 3: Identify Causes for Further Verification – Spend-a-Buck 1. You have four quarters (dots) to spend 2. You can spend your quarters however you like (three on one item, one on another; two on two items; one on four items) except you cannot spend all four quarters on one item 3. Spend your quarters on the items within the categories, not on the categories 4. Circle the items with the most “money” Adapted from Spencer Kagan with permission.

29. Deena Abu-Lughod 29 4: Frame Research Questions Based on Causes (validating questions) Questions to determine whether research validates a possible cause: >Does tracking contribute to achievement gaps between White and African American students? >Is class size a factor in student achievement? >Does inclusion hurt regular education students? >Does incorporation of literacy strategies in the content areas improve students’ academic performance in content areas?

30. Deena Abu-Lughod 30 4: Frame Research Questions Based on Causes (discovering questions) Questions to elicit what we can learn from research about a possible cause: >What instructional strategies help students become better writers? >What kind of preparation helps teachers to be successful in teaching science? >What kinds of grouping practices help students be successful? >What are the effects of tracking in the short run? In the long run? >What instructional practices help students acquire academic vocabulary? >Are there literacy strategies that are particularly effective with African American boys?

31. Deena Abu-Lughod 31 Verify Causes Table: Research (Course Handout pp. 209, 211) Research Question Research Source Research Findings From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved. Task: Based on the possible causes you prioritized, generate three research questions for your own student-learning problem

32. Deena Abu-Lughod 32 Activity 13: Selected Research Sources Course Handout, P. 213 Add your own From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

33. Deena Abu-Lughod 33 Reflect Reflect on this task with your school team: >What do you foresee as the challenges? >Brainstorm solutions and approaches >Share one challenge and its possible solution Write at least one question on a 3x5 card that you would like to have addressed

34. Deena Abu-Lughod 34 Verify Causes with Research and Local Data Generate Possible Causes Dialogue about Causes Identify Causes for Further Verification Frame Research Questions Based on Causes 5. Study the Research 6. Frame Questions for Local Data Collection 7. Analyze Local Data and Verify Causes 8. Plan for Engaging Stakeholders and Review School Culture

35. Deena Abu-Lughod 35 Data-Driven Dialogue Goals Goals Share research findings that inform our understanding of the causes contributing to the student-learning problem Frame questions that guide the collection of local data related to the verified causes Develop a local data collection plan Engage with Data-Driven Dialogue as we share and examine the local data

36. Deena Abu-Lughod 36 Verify Causes Tree Directions (Course Handouts, p. 209) Adapted from P. G. Preuss, Root Cause Analysis: School Leader’s Guide to Using Data to Dissolve Problems, Larchmont, NY: Eye on Education. From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved. 1.Enlarge the graphic organizer, leaving space in all boxes for writing 2.Write the student-learning problem in the box on top 3.Generate possible explanations for the problem in the first row of boxes, organized by the categories provided 4.Gather research to verify your causes 5.Record your research findings on the Verify Causes Tree 6.Modify, eliminate, or add to your list of possible causes 7.Collect and analyze local data to investigate the causes in your own setting that have been verified through research 8.Record local data findings on Verify Causes Tree 9.Record causes that are verified through research and local data in the Verified Causes boxes in the Tree

37. Deena Abu-Lughod 37 Sample Verify Causes Table: Research (Course Handouts, p. 225) Research QuestionResearch SourceResearch Findings What kind of instruction reaches students who aren’t achieving in mathematics? EDThoughts, pp. 2-3 Course Handouts, pp. 234-235) Is tracking a factor in student achievement? How does it impact low-track students? High- track students? EDThoughts, pp. 4-5 Course Handout, pp. 236-237 Does teacher preparation impact student learning? EDThoughts, pp. 26-27 (Course Handouts, pp. 240-241) What strategies contribute to closing achievement gaps in mathematics? “Achievement Gap,” pp. 586, 590-591 (Course Handouts, pp. 242, 246- 247) From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

38. Deena Abu-Lughod 38 Study the Research Jigsaw, Part I: Divide the Reading 1.EDThoughts, pp. 2-3 (instruction for equity) (Course Handouts, pp. 234-235) 2.EDThoughts, pp. 4-5 (tracking) (Course Handouts, pp. 236-237) 3.EDThoughts, pp. 26-27 (teacher preparedness) (Course Handouts, pp. 240-241) 4.“Achievement Gap,” pp. 586, 590-591 (strategies) (Course Handouts, pp. 242, 246-247)

39. Deena Abu-Lughod 39 Study the Research Jigsaw 1. Read through your research and underline key points 2. Meet in expert groups (optional) 3. Summarize your findings on the Sample Verify Causes Table: Research (optional) 4. Prepare to teach your research to your team 5. Teach your research to your home team members 6. Record your findings on the Verify Causes Tree in the appropriate column in the Research Findings row 7. Modify, eliminate from, or add to your list of possible causes

40. Deena Abu-Lughod 40 Verify Causes Tree: Example of Revising Causes After Research From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

41. Deena Abu-Lughod 41 Frame Questions for School-Level Data Collection: “to what extent” questions To what extent: >are we implementing our curriculum? >are we using higher-order questioning? >do our teachers feel prepared to teach ______? >are we explicitly teaching academic vocabulary? >are students writing across the curriculum?

42. Deena Abu-Lughod 42 Framing Questions: Correlational Questions These questions focus on gathering data that provide evidence of whether there is a relationship between two factors. Who is enrolled in our high-track and low-track Math courses? How are students in different tracks performing on our state assessments? (enrollment and achievement) Do students in lower tracks receive a different type of instruction than students in high tracks? (tracking and instruction) How much time are teachers spending teaching Math? Are students in classrooms where more time is spent achieving better? (time spent and achievement) Do students who are in the reading-in-the-content-area program perform better on exams in other content areas? (What variables would you correlate?) How much time are teachers spending teaching academic vocabulary? Are students in classrooms where more time is spent teaching vocabulary achieving better? (What variables would your correlate?)

43. Deena Abu-Lughod 43 From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved. Develop the School-level Data Collection Plan (Handout, p. 232)

44. Deena Abu-Lughod 44 Verify Causes Table: Local Data (Handout H14.1) Question for Local Data Collection Local Data Sources/Tools Findings Are diverse voices represented here? Is this doable? Will the data answer our questions? Equity Lens From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

45. Deena Abu-Lughod 45 Develop the Local Data Collection Plan Based on the research that you have studied, generate three questions to investigate with local data (use Nevazoh possible causes) Record them on your Verify Causes Table: Local Data template Fill in what kind of data sources you will use to investigate your questions

46. Deena Abu-Lughod 46 Data-Driven Dialogue Adapted from B. Wellman and L. Lipton, Data-Driven Dialogue: A Facilitator’s Guide to Collaborative Inquiry, Sherman, CT: MiraVia LLC, 2004. From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

47. Deena Abu-Lughod 47 Sample Verify Causes Table: Local Data: What do you predict? Question for Local Data Collection Local Data Sources/ToolsFindings (Predicted) Who is enrolling in our mathematics advanced courses? Regular courses? How many classes are offered at each level? To what extent are students receiving high-quality instruction? To what extent does this differ in advanced vs. regular mathematics courses? How are students achieving on the state assessment disaggregated by course? How prepared do teachers feel to teach mathematics? From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

48. Deena Abu-Lughod 48 Analyze Local Data Jigsaw, Part 1 (Data Example Handout, pp. 248-252) Data Example 14.1: Student Interview Data (use Key Concepts/Key Words) Data Example 14.2: Classroom Observation Data (use Key Concepts/Key Words) Data Example 14.3: Teacher Survey Data (use Stoplight Highlighting) Data Example 14.4: Enrollment and Achievement Data (use bar graph)

49. Deena Abu-Lughod 49 Analyze Local Data Jigsaw, Part 2 Pair up with someone who has the same data set as you do. Go visual with your piece of data and make observations Present your visual and findings to your home team

50. Deena Abu-Lughod 50 Complete Verify Causes Table: Local Data (Course Handouts, p. 227) Question for Local Data Collection Local Data Sources/ToolsFindings (Actual) Who is enrolling in our mathematics advanced courses? Regular courses? How many classes are offered at each level? Enrollment data, Data Example 14.4 and graph To what extent are students receiving high-quality instruction? To what extent does this differ in advanced vs. regular mathematics courses? Classroom observation, Data Example 14.2 Student interviews, Data Example 14.1 How are students achieving on the state assessment disaggregated by course? Disaggregated enrollment and achievement data, Data Example 14.4 and graph How prepared do teachers feel to teach mathematics? Survey data, Data Example 14.3 From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

51. Deena Abu-Lughod 51 Analyze Local Data and Verify Causes (continued) Record local data findings on the Verify Causes Tree Go to Phase 4: Infer/Question: What conclusions are you drawing now about the possible causes based on research and local data? Record causes that are verified through research and local data in the Verified Causes boxes on the Tree. From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

52. Deena Abu-Lughod 52 Verify Causes Tree: Example of Revising Causes After Research From N. Love, K. Stiles, S. Mundry, and K. DiRanna, The Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry, Thousand Oaks, CA: Corwin Press, 2008. All rights reserved.

53. Deena Abu-Lughod 53 Reflect What have been your most important lessons learned from this simulation? What do you want to share with others? How?

54. Deena Abu-Lughod 54 Essential Question How can we scale up the work of inquiry teams? Bring more students into the sphere of success? I discovered that I…. I intend to…..

55. Deena Abu-Lughod 55 “ Powerful Words ” I wonder how many children’s lives we would save if we educators shared what we knew with each other. — Roland Barth

56. Deena Abu-Lughod 56 Network ELA Data: Predictions How many of our schools had overall gains that exceeded the City average of 11%? What percent of students made 1 year progress? In what grade were Proficiency Rate gains the highest? What was the average Proficiency Rate gain for the Level 1+2 students? What was the average Proficiency Rate gain for the Level 3+4 students? Was there a difference in the Proficiency Rate gains of Level 1+2 students with IEPs and Level 1+2 General Education students?

57. Deena Abu-Lughod 57 Small increases in proficiency translate into large increases in probability of graduating 8 th Grade Test Score (Average of ELA & Math) 4-Year Regents Graduation Rate Source: 2004 Graduation Cohort (“Class of 2008”) from the 2007/08 Progress Reports Guiding Principle Performance and Progress

58. Deena Abu-Lughod 58 Acuity Predictive Correlations In June, most schools will administer the Acuity Predictive assessments. How reliable are these assessments in predicting the outcomes on the NYS test? The correlation of the Grade 8 ELA Proficiency Rates with the Fall Acuity Predictive was.759. Very high.

59. Deena Abu-Lughod 59 Grade 8: Scatterplot of Acuity with Prof. Rate