Children First Intensive Mathematics Best Practices Connecting Assessment, Inquiry and the Quality Review December 9, 2009 ESA CFN 6 (Bob Cohen) and ESO Network 19 (Elvira Barone) Co-Facilitators: Deena Abu-Lughod, SATIF; Frederica Capshaw, Instructional Support Specialist; Karen Ames, Achievement Coach Contact us at Dabulug@schools.nyc.gov or FCapsha@schools.nyc.gov Electronic copies available at http://dabulughod.wikispaces.com

Agenda 8:00 Breakfast 8:30 Welcome, Introductions, Agenda, Rationale and Outcomes 9:00 Prediction and Analysis – Data tools and a new lens on School-Specific 2009 NYS 3-8 Mathematics Data to evaluate the effectiveness of the implemented curriculum (QR Statement 1.1) 10:00 Break 10:15 Lesson Planning and Research Lessons 11:30 Mathematics Resources, Lois Sharzer 12:15 Lunch 1:00 Breakout Groups: (A) Elementary - (B) M/S and H/S-Donna Davis, National Consultant, Glencoe 2:30 Questions, Debrief, Evaluation

Rationale The NYCDOE Progress Report uses NYS assessments to tell part of the story about how students performed and progressed in mathematics. Item-by-item comparisons of a school’s students relative to the city’s students, and State benchmarks and “distinguishing” items identify strengths in a school’s teaching and its implemented curriculum. Today, we will explore how teacher teams can use an inquiry approach to analyze assessment data, student work, and teacher work, to adjust curriculum, instruction, and assessments to improve learning outcomes. (QS 4.2)

What is Collaborative Inquiry? Collaborative inquiry is a sustained process of investigation and action that empowers teachers to improve student achievement and close the achievement gap. Collaborative inquiry is: Focused on student outcomes, using a systematic, data-informed approach. Conducted by teams of teachers with a focus on small groups of students, paying close attention to those who are struggling while supporting the learning of all students. Designed to develop and deepen rigorous, research-based instructional strategies and frameworks. Currently, citywide, 990 of 3900 teacher teams (25%) are focusing on Mathematics.

Outcomes What are we going to know at the end of the day? How to analyze item data in relation to your school’s implemented curriculum (QS 1.1) How to analyze summative data to create a portrait of mastery and provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions (QS 2.2) How to use these items to evaluate and adjust instructional practices and to monitor progress (QS 5.2, 5.3)

Guide to Resources Item Analyses and Test Items: Available on DOE web for 2007-2009, for evaluating and aligning curriculum and monitoring progress. http://schools.nyc.gov/Academics/Mathematics/EducatorResources/Item+Analyses.htm Curriculum Resources, pacing guides, strand trace, etc: Available on DOE web. http://schools.nyc.gov/Academics/Mathematics/EducatorResources/default.htm Trend Maps and State benchmarks: Available on ARIS Connect, for identifying power standards, and for identifying items that separate the 4s from the 3s, and the 3s from the 2s. https://www.arisnyc.org/connect/node/388417 ItemData: Available in your school’s private community on ARIS Connect for viewing individual responses; available by past and current classroom (July and September versions). Web resources: Available on ARIS Connect, for instructional and assessment support. https://www.arisnyc.org/connect/node/369071 Lesson study toolkit: http://www.lessonresearch.net/nsf_toolkit.html Acuity ITAs, Predictives and Customized assessments: From home computer, https://nyc- acuity.mcgraw-hill.com/index.jsp To Schedule Professional Development: http://schools.nyc.gov/Accountability/ResourcesforEducators/PeriodicAssessments/Periodi c+Assessment+Professional+Development.htm

Activity #1: Item Analysis Predicting your outcomes Think about the 2009 NYS assessment results in a grade you know well (you should be sitting at that table). Look at the trend map for that grade, the Network averages by item relative to NYC, and the items themselves. On which items do you think your students did well, relative to other students in the Network or City? On which items did they struggle? Commit yourself to paper. On Worksheet #1, write down at least 3 strengths and 3 weaknesses and justify your predictions.

Trends Maps The Trends Maps help you identify the “Power Standards”: those performance indicators that are most frequently tested on the State exams. They show which items tested which performance indicator over the last few years and the percent of questions used to test each strand. Let’s look at the Trend Map for Grade 8 together. Be aware that although some indicators are not tested every year, they may involve necessary skills.

Gr 8 2009 Trend Map (from BOCES)

Priorities: NYS Math 2008 Distribution by Strand Gr 3 Gr 4 Gr 5 Gr 6 Gr 7 Gr 8 Number Sense 52% 44% 40% 32% 13% Algebra 16% 15% 20% 47% Geometry 26% 17% Measure ment 19% 18% 14% Statistics 10% 12% 23% 0% Note that this is the actual 2008 strand distribution. You also have NYSED’s Approximate Percentage of Questions Assessing Each Strand document in your resource packet.

2009 Item Analysis from Educator Resources http://schools.nyc.gov/Academics/Mathematics/EducatorResources/Item+Analyses.htm

2009 Items (from DOE Educator Resources) http://schools.nyc.gov/Academics/Mathematics/EducatorResources/Item+Analyses.htm

Activity #1, cont.: Check your predictions Examine your school-specific item analysis chart . How accurate were your predictions about your school’s strengths and weaknesses? What might explain the variance on the items? What are the implications? Share out. Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

Comparing ourselves with Benchmarks How do you know which questions are strategic (matter the most)? How do you know which strands and indicators represent a strength or a weakness for your school? How do you know whether a question was difficult just for your own students or for all students? Comparing our own scores with the State Benchmarks helps us answer these questions. Then we must ask: To work strategically, what questions should we focus on? The ones that were difficult for all students or just for some? How should we deal with infrequently tested indicators?

What are the State Benchmarks? The Benchmarks will tell you what percent of Level 2, Level 3 and Level 4 students answered each item correctly. This is a more useful indicator than the p-value provided in the Item Analysis. The benchmarks help you identify which questions distinguish the Level 4 students from the Level 3 students and the Level 3 students from the Level 2 students. Understanding these differences may help you select what to work on if you want to push 2s to 3s, and keep 3s and 4s from slipping.

Let’s understand how to read the files Item 5 distinguished the Level 3 from Level 2 students Item 6 distinguished the Level 4 from Level 3 students On average, there was a 34 percentage point difference between 4s and 3s, and 25 points between 3s and 2s.

Clarification: ARIS Taxonomy You saw these codes in the ItemData files. Focus on the 1st, 3rd and last characters to see the grade, strand, and indicator number.

Geometry: Gr 8 Distinguishing Questions In Grade 8, X% of the questions deal with geometry. In this extract of “distinguishing questions”, what do you observe about geometry?

Algebra: Gr 8 Distinguishing Questions In Grade 8, X% of the questions deal with algebra. In this extract of “distinguishing questions”, what do you observe about algebra?

Activity #2: Research Lesson Modeling The following question represents a power standard, and one that distinguishes 4s from 3s. Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

Activity #2: Research Lesson Modeling Use Worksheet #2 to record the following (individually): Sequence of understandings a student must have in order to solve the problem (use the strand trace, if desired) Tasks and experiences that help students develop these understandings Solve the task and record your solution. Indicate additional ideas about how other students might solve the item, anticipating common errors. If the tasks spark any new thoughts about student understanding and how it develops, modify your concept map as needed. Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

Problem Solving Strategies (p. 43 Van De Walle) 1. Draw a picture, act it out, use a model. 2. Look for a pattern. 3. Guess and check. (Make an attempt; reflection can lead to a better idea.) 4. Make a table or chart. (Often for problems involving ratios or measurements.) 5. Try a simpler form of the problem. (Simplify quantities so the resulting task is easier to understand and analyze.) 6. Make an organized list. (Show the number of possibilities and verify that all possible outcomes have been included.) 7. Write an equation. (Convert the story into numbers or symbols) Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

Four Step Problem Solving Template As you plan, consider the value of the 4-Step Problem Solving Template (adapted from Bob Gyles, Math Think Tank): Note: Let’s see if anyone identified at least 5 correctly. I have 3 copies of a book that we can give out as a prize.

Activity #3: Small Group Research Lesson At your table, select ONE question that is both important and that represents a weakness at one or more of your schools. Use your copy of Worksheet #2 to record the following: Sequence of understandings a student must have in order to solve the problem (use the strand trace, if desired) Tasks and experiences that help students develop these understandings Solve the task and record your solutions and ideas about student solutions. If the tasks spark any new thoughts about student understanding and how it develops, modify your concept map as needed.

Activity #3: Small Group, Cont. Using Worksheet 3, select a few of the tasks/activities you have identified, and indicate them and the associated questions, the anticipated student responses, the teacher supports, and the points of evaluation. How can this type of planning produce better outcomes in class? What else would you add or want to do?

Activity #4: Monitoring Progress Group A: Assume you have developed and delivered the lesson you just drafted to the students currently in that grade. How can you use the item or items from the 2009 State exam (and/or exams from previous years) to know whether this year’s students on that grade are performing better than last year’s students did? Group B: Assume you have delivered the lesson you just drafted to the same students who had trouble with that indicator last year (eg, a target population). How can you use the item or items from the 2009 State exam to know whether these students have now mastered the skill?

Both Collaborative Inquiry and Lesson Study Emphasize collaboration around a common focus Link directly to planning and classroom practice Focus on a specific topic or problem related to goals for student learning Use change strategies or research lessons as learning tools, rather than as products Involve teachers’ use of existing curriculum and learning activities to improve them Focus on the lesson or instruction, rather than the teacher Involve team members who contribute equally Involve live observation of students in the act of learning Develop a common understanding by teachers of the context of student work.

Inquiry Cycle Lesson Study Cycle Set Goals Plan the lesson using research Teach, observe, debrief Revise and reteach Reflect and share results

Lesson Study and Toolkit

Important Resources for our Schools Special Thanks to: Lois Sharzer Lois Sharzer and Associates

Teacher Actions Reference Sheet Before: Activate prior knowledge: begin with a simple version of the task, brainstorm solutions, estimate or use mental computation. Be sure the problem is understood. Establish clear expectations. During: Let go! Once students understand what the problem is asking, hold yourself back from stepping in when students struggle. Listen actively to understand the student’s approach to the problem. Ask: What ideas have you tried so far? Can you tell me more about…? Why did you …? Provide appropriate hints. Provide worthwhile extensions. After: Encourage student-to-student dialogue; request explanations to accompany all answers; call on students for their ideas, encouraging those who are shy; encourage students to ask questions; be certain your students also understand what you understand. Listen actively without evaluation. Summarize main ideas and identify future problems.

Afternoon: Breakout Groups Donna Davis, National Staff Developer, Glencoe Deep Dive into Lesson Study Text-based Discussion: What’s Sophisticated about Elementary Mathematics? Tech Tools: An Exploration of Web Resources Core Standards

Learning Intentions: How did we do? How did we analyze data? (QS 1.1) How did that analysis of summative data help us create a portrait of mastery? How did we think we could provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions? (QS 2.2) How did we consider using these items to evaluate and adjust instructional practices and to monitor progress? (QS 5.2, 5.2) Please complete the Evaluation form and leave on your table. We analyzed school-level item data in relation to the city average on each item to see where our school’s implemented curriculum was stronger or weaker relative to the city, and where there were items that overall, separated the 4s from the 3s and 3s from the 2s. We discussed what it would look like if we were to provide meaningful and actionable feedback on the effectiveness of classroom level, curricular and instructional decisions.