School and District Supports for Interim Assessment: Lessons from Two US School Districts INRP Seminar The Strategic Challenges Behind the Implementation of Accountability in Education Policies Many factors influence how teachers access, manage, interpret and act on data. This morning, Leslie Nabors Olah reported on how teachers in two school districts in the United States analyzed, interpreted and used data from interim assessments in their teaching. This presentation examines strategies that these school districts and their schools used to facilitate data-driven decision-making and to support use of interim assessments in elementary mathematics. Abington, PA: Small district (7 elementary schools). Has used Everyday Math (EDM) since 1991. District-created, with teacher input, practice assessments given every 4-6 weeks. Teachers may review scores with math coaches, but were not made public. Interim assessment is viewed as “practice test” for teacher’s use as part of their regular instruction. Rather scores on end-of-unit tests are used by principals and district staff to identify larger instructional issues at teacher and/or school level. Philadelphia, PA: Large, urban district (177 elementary schools). Has used Everyday Math (EDM) since 2003. Benchmarks developed by external organization with district. Given every six weeks. Scores posted on SchoolNet Information Management System. May 26, 2009 Margaret E. Goertz Consortium for Policy Research in Education University of Pennsylvania

Conditions Supporting Data Use Alignment of assessment, standards and curriculum Expectations for use “User-friendly” information management systems Professional support Time Leadership We identified six conditions that helped teachers learn from and make use of assessment data. Alignment of assessment, standards and curriculum Expectations for use User-friendly information management systems Professional Support Time Leadership

Alignment of Assessment, Standards and Curriculum District-wide curriculum in mathematics aligned to state standards. Common textbooks across schools aligned to curriculum. Interim assessments aligned to content of district curriculum and textbooks for each instructional period. FIRST, Alignment of assessment with content standards and district curriculum ensured that data generated from the assessments was relevant to what teachers had been teaching in the classroom. Both districts enacted district-wide curriculum in mathematics aligned to state standards Adopted common textbooks across schools (Everyday Mathematics in elementary grades) Developed instructional timelines linked to units in textbook (what we call “pacing guides”) In Urban District, details skills, topics or content to be taught during each instructional week linked to specific lessons in text, state stds and District’s core curriculum In Suburban District, identified number of lessons in each EDM unit, number of days it should take to cover content and expected date of completion Finally, districts aligned interim assessment tasks with content of district curriculum and textbooks for each instructional period (generally every four to five weeks).

Expectations for Use Instructional Assessments as “teaching tools” Provide remediation or enrichment for students Reflect on their instruction Communicating expectations Dedicated “re-teaching” time in instructional schedule Structure of data management systems Formats of analysis reports, principal meetings and postings on District website (urban district) SECOND, Both districts created and communicated expectations for data use at all levels of the system The primary expectation was that interim assessments should be used INSTRUCTIONALLY, as “teaching tools” Expected that teachers would use the results of the interim assessments for remediation and enrichment of instruction to students, as well as to reflect on their instructional practices. Districts communicated their expectations through (as I will discuss): dedicated “re-teaching time” in instructional schedule structure of their data management systems, and in urban district, through formats of reports, principal meetings and direct dissemination on the District websites. School Principals accepted and reinforced District’s expectations by expecting their teachers to analyze interim assessment data and use it to inform their instruction, and for teachers to discuss and share common problems and instructional solutions.

Expectations for Use Organizational Principals will create time for teachers to collaborate on data analysis and share instructional practices. Principals (in urban district) will collaborate on how to raise student achievement. Districts also had a set of ORGANIZATIONAL expectations for teachers and schools. Districts expected teachers to meet (particularly in grade groups) to examine data, exchange strategies and share instructional practices; In urban district, gave elementary teachers in same grade common planning time. Principals shared this expectation and expected teachers to use their common planning time for these activities. Urban district also expected principals to collaborate on how to raise student achievement through monthly meetings of principals in given region of district. Districts also looked to principals (in urban district) and curricular staff (in suburban district) to provide leadership to enable teachers to use benchmarks as intended

“User-friendly” Information Management Systems Data and reporting systems that: permitted timely and easy access to student performance information; linked to standards and/or curriculum; and permitted easy analysis of assessment information. THIRD, Both districts developed data and reporting systems that permitted timely and easy access to student performance information linked to standards and/or curriculum and that allowed teachers to easily analyze the assessment information. The IMS system in the urban district had test results available within 24 hours of assessment. System in urban district enabled teachers and principals to generate reports for every class with: Student item analysis: [Student responses by item, including wrong answer selected by student (if relevant)] [% of items that individual students scored correctly] Class item analysis:[% of students scoring correctly on each item] In both districts, Content and skill analysis:[ number of items that students answered correctly by state standard]. And as Leslie reported earlier today, teachers conducted all three types of analyses

“User-friendly” Information Management System: Urban District Teachers are required to complete Data Analysis Protocols that: Identify weakest skills/concepts for class for assessment period Report how they will group or regroup students based on information in item analysis reports Indicate what changes in teaching strategies (and resources) are indicated by analysis IMS also provides links to information about how to re-teach a particular standard and practice worksheets for students. To assist in their analysis of data, teachers in urban district were required to complete and submit to their principal a Data Analysis Protocol that: Identified weakest skills/concepts for class for assessment period Reported how they would group or regroup students based on information in item analysis reports; and Indicate what changes in teaching strategies (and resources) were indicated by analysis Principals had to complete similar forms on their schools and submit these to their regional supervisors. IMS also provides links to information about how to re-teach a particular standard and practice worksheets for students, but few of the teachers in our study accessed these.

Professional Support Curriculum District workshops Curriculum specialists, mathematics coaches and consultants Use of data management systems Turnkey training Principals and school-based technology teachers Analysis of data Principals (urban district) School-based coaches District curriculum specialists (suburban district) FOURTH, Both districts provided professional support in curriculum, the use of the data management systems, analysis of assessment data, and, to differing extent, instructional approaches. On content (generally learning how to use new instructional materials, like EDM text) On data systems Urban district: turn-key training; usually technology teacher and principal trained in how to access and use IMS at district level; they in turn trained their teachers Ex: of one principal who periodically asked teachers to do small exercise using SchoolNet so they used it. Teachers in both districts were comfortable with using data system On analysis of data Principals (urban district) School-based coaches District curriculum specialists (suburban district)

Professional Support Acting on data analysis Little, if any, formal professional development Primarily role of curricular coaches (both districts) and, to a lesser extent, principals in urban district Teachers turn to each other for help Role of coaches Locate information on different ways of teaching math or introducing math skills for teachers Help teachers with data analysis Do demonstration lessons Suburban district: provide remediation and enrichment to students Acting on data analysis Little if any formal professional development on the “what to do next” area Primarily role of curricular coaches (both districts) and, to a lesser extent, principals in urban district Teachers reported turning to each other, usually informally, for help Role of coaches Locate information on different ways of teaching math or introducing math skills for teachers Help teachers with data analysis Do demonstration lessons Suburban district: provide remediation and enrichment to students Differential levels of support, however. Limited released time of classroom teacher in urban district Full-time curriculum specialist in suburban district Teachers in suburban district also had access to district math coach and district mathematics curriculum supervisor

Time Discuss assessment results and instructional techniques Grade group meetings; common planning time Re-teach content and skills to students Dedicated time in instructional schedule Participate in professional development Scheduled time during school day FIFTH, The districts scheduled dedicated time for teachers to discuss assessment results and instructional techniques, to re-teach content and skills to students and to participate in professional development. Analysis time Teachers expected to do initial analysis on their own time; Expectation that teachers would use grade group meetings or common planning time to discuss benchmark results and follow-up actions, but teachers reported that this time was taken up with other topics Tended to talk informally. Used more structured interaction around students with learning problems Reteach skills: Reteaching week in urban district; dedicated time when teachers expected to plan and execute reteaching, remediation, and enrichment activities In suburban district, time between practice test and end-of-unit test PD Time: Both districts had contractual time for PD during the school day. Urban district was ½ day each Friday; suburban district was eight (8) two hour sessions after school and one faculty mtg a month Issue was competing demands on this time, particularly on district mandated topics or to do school improvement planning. Some urban schools did use these ½ days to hold grade group meetings and some suburban schools used monthly faculty meetings for groups of teachers to discuss data.

Leadership Modeling data analysis Monitoring teachers’ use of data Conducting their own analyses Monitoring teachers’ use of data Reviewing and providing feedback Creating time for teacher collaboration Providing direct support to teachers Modeling instruction FINALLY, school leadership and a culture of data use were the most critical factors in supporting teachers’ use of data. School leaders (principals and/or school-based curriculum specialists) reinforced expectations for and supported data use by: modeling data analysis by conducting their own analyses In urban schools, principals looked at assessment results directly to: ID struggling students; ID areas of weak skills across classes in a grade level; and/or to ID a teacher whose class was falling behind others in a grade level to determine how to support that teacher. monitoring teachers’ use of data in all of our urban school sample, principals reported reviewing data with grade groups Some principals also reviewed and provided feedback on teachers’ Data Analysis Protocols; might compare teachers’ analyses of student data with their own; Some principals reviewed lesson plans for the re-teaching week for evidence of data analysis and related reteaching strategies In suburban district, district math coach and ECS would review practice test results with each other and then with teachers; principals would examine end-of-unit tests to identify more programmatic needs. creating time for teacher collaboration, insuring that dedicated time (usually contracted PD time) is used to support data analysis. and providing direct support to teachers through modeling instruction in the classroom; providing feedback to teachers on their instruction; help teachers find new ways of teaching (from IMS system, other sources)

Accountability Accountability for data use versus accountability for student results Formative assessment must have low stakes, but teachers must be held accountable for its use Slippery slope between internal and external use of assessment data CODA—the press of accountability`