1 Using the 2009 NECAP Reports February 1-5, 2010

2 Welcome and Introductions Susan Smith MEA/NECAP Coordinator Maine Department of Education Tim Crockett Senior Vice President Measured Progress

3 Welcome and Introductions Wanda Monthey Team Leader Susan Smith MEA/NECAP Coordinator Dan Hupp Assessment Director Michele Mailhot Math Specialist Patsy Dunton ELA Specialist Leeann Larsen ELA Specialist Rachelle Tome NCLB Title I Director, Accountability & AYP Sue Fossett PAAP Coordinator Department of Education Staff

4 Welcome and Introductions NECAP Service Center

5 Purpose of the Workshop Review the different types of NECAP reports Review Maine NECAP baseline data Conduct a demonstration of the Analysis & Reporting System

6 Involvement of Local Educators Test Item Review Committees Bias and Sensitivity Review Committees

7 NECAP Grade Level Expectations NECAP test measures the NECAP Grade Level Expectations (GLEs) GLEs were adopted by the legislature as Maine’s Federal, State, and Local Accountability Standards GLEs became part of the Maine Learning Results: Parameters for Essential Instruction GLEs are located at GLEs should be fully incorporated into each school’s curriculum to ensure that students have an opportunity to learn the content assessed on the NECAP

NECAP Scores are Baseline Data Maine adopted NECAP content standards (GLEs), cut scores and achievement standards. This means that NECAP results are Maine’s baseline data that begin a new trend line. Although both the MEA and NECAP programs use an 80 point scale to report scores, they are different tests that measure different content standards and have different cut scores between achievement levels so the scores can NOT be compared.

9 Why MEA and NECAP Scores Can’t be Compared Mathematics GradeMEA % Proficient NECAP % Proficient % Difference MEA Scaled Scores NECAP Scaled Scores Point Difference

10 Why MEA and NECAP Scores Can’t be Compared Reading GradeMEA % Proficient NECAP % Proficient % Difference MEA Scaled Scores NECAP Scaled Scores Point Difference

11 Basics of NECAP Test Design Fall test – previous year’s GLEs Reading and Mathematics – grades 3-8 Writing – grades 5 and 8 (2009 was a Pilot test) Variety of Item Types Reading: multiple choice and constructed response Mathematics: multiple choice, short answer (one point), short answer (two points), and constructed response Writing: multiple choice, constructed response, and extended response Common and Matrix Items

12 Basics of NECAP Test Design Reading – Grades Common points Mathematics – Grades 3 & 4 65 Common points Mathematics – Grades Common points Writing – Grades 5 & 8 34 Common points No results this year because of Pilot testing

13 Workshop Materials

14 Types of NECAP Reports Public Reports Results Report School and District level Summary Report School/District/State level Password Restricted Reports Released Items Summary Data School and District level Achievement Level Summary School and District level

15 Types of NECAP Reports Confidential Reports Student Report Information for Parents/Guardians Item Analysis Report School level by student Longitudinal Data Student level, across test administrations Student Level Data Files Excel/csv files by grade on district and school confidential site

16 The Family Educational Rights and Privacy Act (FERPA) Access to individual student results is restricted to:  the student  the student’s parents/guardians  authorized school personnel Superintendents and principals are responsible for maintaining the privacy and security of all student records. Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties. FERPA website:

17 Student Report

18 Looking at the Student Report This part of the report provides the individual student’s achievement level and scaled score

19 Looking at the Student Report

20 Looking at the Student Report This part of the report provides a comparison between the performance of this student and his/her school, district, and state

21 Looking at the Student Report

22 Looking at the Student Report This part of the report gives specific information about the student’s performance in content area subcategories

23 Looking at the Student Report

24 Teaching Year vs. Testing Year Rationale Inclusion of students in “Teaching Year” reports

25 Looking at a School-Level Results Report Schools can view reports for Testing Year ( )

26 Looking at a School-Level Result Report Or for Teaching Year ( )

27 District and School Results Report

School-Level Results Report – Grade Level Summary 28

School-Level Results Report – Grade Level Summary 29

School-Level Results Report – Grade Level Summary 30

31 School-Level Results Report – Grade Level Summary

32 School-Level Results Report – Content Area Results

33 Looking at the Results Report – Content Area Results Three years of data are typically shown on this report. As Maine schools participate in NECAP in coming years, more data will be added. After three years, a Cumulative Total will be shown.

34 Looking at the Results Report – Content Area Results

35 Looking at the Results Report – Content Area Results Total Possible Points includes both common and matrix items (not field-test). Total Possible Points also represents the test’s balance of representation. Mathematics

36 Looking at the Results Report – Content Area Results Please note: The Total Possible Points column is organized differently on the Reading Results Report 106 possible points are represented here – they are sorted by “Type of Text” The same 106 possible points are represented here – they are sorted by “Level of Comprehension”

Looking at the Results Report – Content Area Results Total Possible Points includes both common and matrix items (not field-test). Reading

38 Looking at the Results Report – Disaggregated Results

39 Looking at the Results Report – Disaggregated Results Important Note: Disaggregated results are not reported for sub- groups of less than 10

40 Looking at the Results Report – Disaggregated Results Because this is a small school, and so many of the sub-groups are smaller than 10, this part of the report is not as useful. But we can still look at district and state disaggregated results. Does this data match what we know about the district’s program?

41 Summary Report

42 NECAP District and School Student-Level Data Files Contain: All demographic information for each student that was registered through MEDMS/Infinite Campus The scaled score, achievement level, and subcategory scores earned by each student in all content areas tested NECAP files also contain: Performance on released items Student questionnaire responses

43 Item Analysis Report

44 Item Analysis Report

45 Looking at the Item Analysis Report

46 Looking at the Item Analysis Report This part of the report gives specific information about the released items and student performance on individual items

47 Looking at the Item Analysis Report

48 Looking at the Item Analysis Report This part of the report represents all of the items used to compute student scores - points are displayed by subcategory. Each student’s Scaled Score and Achievement Level are shown.

49 Guides to Using the 2009 NECAP Reports NECAP Analysis and Reporting System User and Training Manual Companion PowerPoint presentation Grade Level Expectations NECAP Accommodations Guide Released Items documents Preparing Students for NECAP: Tips for Teachers to Share with Students Supporting Materials and Resources

50 How Do I Find Public Assessment Reports? Go to… necap/results

51 Big Picture: MEA and NECAP Data

52 The MEA has consistently shown that female students outperform male students statewide in reading at every grade level. Gender Gap

53 MEA Performance Analysis Gender Differences in Reading Performance Chart Statistics Based on School Year Student Performance

54 MEA Gender Gap The gap ranged from percentage points. The NECAP is a different test measuring different standards - what pattern might you expect to see in NECAP data?

55 NECAP Performance Analysis Gender Differences in Reading Performance Chart Statistics Based on School Year Student Performance

56 NECAP Gender Gap The gap still exists and ranges from 7 – 14 percentage points. If this data is similar to your school/district data, what strategies have you employed or plan to employ to address this gender gap?

57 For instance, both the MEA and NECAP reading tests have been/are designed to include both narrative and informational texts equally. Does your reading curriculum reflect that balance? Gender Gap

58 If you organized the released items in the interactive reporting tool by GLE and chose gender as a filter, what would you predict about gender differences based on type of text? Note: Reading GLE codes R4, 5, and 6 refer to literary text and R7 and 8 refer to informational text. Gender Gap

59 Years ago, the MEA showed a considerable gender gap in mathematics, with male students outperforming female students statewide at every grade level. Gender Gap

60 Closing this gender gap in mathematics became a priority. The MEA showed over time that the gap had narrowed or disappeared. Gender Gap

61 MEA Performance Analysis Gender Differences in Mathematics Performance Chart Statistics Based on School Year Student Performance

62 The gap ranges from only 0 to 4 percentage points. What pattern might you expect to see in NECAP data? MEA Gender Gap

63 NECAP Performance Analysis Gender Differences in Mathematics Performance Chart Statistics Based on School Year Student Performance

64 The gap is even narrower for this year, ranging from 0 to 3 percentage points. Is this pattern similar to your school/district data? NECAP Gender Gap

65 The percent of students with IEPs who met or exceeded standards in the MEA ranged from 27 – 40 in reading and from 14 – 42 in math depending on grade level. IEP Data – MEA

66 MEA Performance Analysis Students with IEPs Chart Statistics Based on School Year Student Performance

67 IEP Data – NECAP The percent of students with IEPs who scored as proficient or proficient with distinction in the NECAP ranged from 23 – 38 in reading and from 17 – 33 in mathematics, depending on grade level.

68 NECAP Performance Analysis Students with IEPs Chart Statistics Based on School Year Student Performance

69 IEP Data What percentage of students who scored at proficient or proficient with distinction in reading and in mathematics might you expect to find in your school/district data? If the percent is below (or above) state averages, what strategies have you employed or plan to employ to increase performance?

70 SES Data - MEA The percent of students classified as economically disadvantaged who met or exceeded standards in the MEA ranged from 53 – 66 in reading and from in mathematics depending on grade level.

71 MEA Performance Analysis Students Identified as SES Chart Statistics Based on School Year Student Performance

72 What pattern might you expect to see in NECAP data? SES Data

73 NECAP Performance Analysis Students Identified as SES Chart Statistics Based on School Year Student Performance

74 SES Data - NECAP The percent of students classified as economically disadvantaged who scored as proficient and proficient with distinction in the NECAP ranged from 56 – 62 in reading and from 45 – 50 in mathematics, depending on grade level.

75 SES Data What percentage of students who scored at proficient and proficient with distinction in reading and in mathematics might you expect to find in your school/district data? If the percent is below (or above) state averages, what strategies have you employed or plan to employ to increase performance?

76 MDOE content specialists are planning webinar presentations for teachers that will assist them in using the content-specific NECAP data contained in the interactive portion of the reporting tool and the released items to inform their teaching and assessing of NECAP GLEs. Working with NECAP Data

77 Coming Attractions Administration window of the MEA science test for grades 5 and 8: May 10 – 21. MEA test administration materials: ninfo.htm There will be a test administration webinar on April 27 from 9-10:30 AM that will also be posted on the web.