1. 1 Using the 2009 NECAP Reports February 1-5, 2010

2. 2 Welcome and Introductions Susan Smith MEA/NECAP Coordinator Maine Department of Education Tim Crockett Senior Vice President Measured Progress

3. 3 Welcome and Introductions Wanda Monthey Team Leader 624-6831 wanda.monthey@maine.gov Susan Smith MEA/NECAP Coordinator 624-6775 susan.smith@maine.gov Dan Hupp Assessment Director 624-6827 dan.hupp@maine.gov Michele Mailhot Math Specialist 624-6829 michele.mailhot@maine.gov Patsy Dunton ELA Specialist 624-6625 patsy.dunton@maine.gov Leeann Larsen ELA Specialist 624-6628 leeann.larsen@maine.gov Rachelle Tome NCLB Title I Director, Accountability & AYP 624-6705 rachelle.tome@maine.gov Sue Fossett PAAP Coordinator 624-6774 susan.fossett@maine.gov Department of Education Staff

4. 4 Welcome and Introductions NECAP Service Center 1-877-632-7774

5. 5 Purpose of the Workshop Review the different types of NECAP reports Review Maine NECAP baseline data Conduct a demonstration of the Analysis & Reporting System

6. 6 Involvement of Local Educators Test Item Review Committees Bias and Sensitivity Review Committees

7. 7 NECAP Grade Level Expectations NECAP test measures the NECAP Grade Level Expectations (GLEs) GLEs were adopted by the legislature as Maine’s Federal, State, and Local Accountability Standards GLEs became part of the Maine Learning Results: Parameters for Essential Instruction GLEs are located at http://www.maine.gov/education/necap/standards.html http://www.maine.gov/education/necap/standards.html GLEs should be fully incorporated into each school’s curriculum to ensure that students have an opportunity to learn the content assessed on the NECAP

8. 8 2009-10 NECAP Scores are Baseline Data Maine adopted NECAP content standards (GLEs), cut scores and achievement standards. This means that 2009-10 NECAP results are Maine’s baseline data that begin a new trend line. Although both the MEA and NECAP programs use an 80 point scale to report scores, they are different tests that measure different content standards and have different cut scores between achievement levels so the scores can NOT be compared.

9. 9 Why MEA and NECAP Scores Can’t be Compared Mathematics GradeMEA % Proficient NECAP % Proficient % Difference MEA Scaled Scores NECAP Scaled Scores Point Difference 36962-7348342-6 46662-4446443-3 56564547543-4 65463+9643 --- 75860+2745742-3 85359+6843842

10. 10 Why MEA and NECAP Scores Can’t be Compared Reading GradeMEA % Proficient NECAP % Proficient % Difference MEA Scaled Scores NECAP Scaled Scores Point Difference 36573+8345346+1 47167-4446444-2 56772+5546 --- 67069647645-2 77968-11751745-6 87169-2850846-4

11. 11 Basics of NECAP Test Design Fall test – previous year’s GLEs Reading and Mathematics – grades 3-8 Writing – grades 5 and 8 (2009 was a Pilot test) Variety of Item Types Reading: multiple choice and constructed response Mathematics: multiple choice, short answer (one point), short answer (two points), and constructed response Writing: multiple choice, constructed response, and extended response Common and Matrix Items

12. 12 Basics of NECAP Test Design Reading – Grades 3-8 52 Common points Mathematics – Grades 3 & 4 65 Common points Mathematics – Grades 5-8 66 Common points Writing – Grades 5 & 8 34 Common points No results this year because of Pilot testing

13. 13 Workshop Materials

14. 14 Types of NECAP Reports Public Reports Results Report School and District level Summary Report School/District/State level Password Restricted Reports Released Items Summary Data School and District level Achievement Level Summary School and District level

15. 15 Types of NECAP Reports Confidential Reports Student Report Information for Parents/Guardians Item Analysis Report School level by student Longitudinal Data Student level, across test administrations Student Level Data Files Excel/csv files by grade on district and school confidential site

16. 16 The Family Educational Rights and Privacy Act (FERPA) Access to individual student results is restricted to:  the student  the student’s parents/guardians  authorized school personnel Superintendents and principals are responsible for maintaining the privacy and security of all student records. Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties. FERPA website: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html

17. 17 Student Report

18. 18 Looking at the Student Report This part of the report provides the individual student’s achievement level and scaled score

19. 19 Looking at the Student Report

20. 20 Looking at the Student Report This part of the report provides a comparison between the performance of this student and his/her school, district, and state

21. 21 Looking at the Student Report

22. 22 Looking at the Student Report This part of the report gives specific information about the student’s performance in content area subcategories

23. 23 Looking at the Student Report

24. 24 Teaching Year vs. Testing Year Rationale Inclusion of students in “Teaching Year” reports

25. 25 Looking at a School-Level Results Report Schools can view reports for Testing Year (2009-10)

26. 26 Looking at a School-Level Result Report Or for Teaching Year (2008-09)

27. 27 District and School Results Report

28. School-Level Results Report – Grade Level Summary 28

29. School-Level Results Report – Grade Level Summary 29

30. School-Level Results Report – Grade Level Summary 30

31. 31 School-Level Results Report – Grade Level Summary

32. 32 School-Level Results Report – Content Area Results

33. 33 Looking at the Results Report – Content Area Results Three years of data are typically shown on this report. As Maine schools participate in NECAP in coming years, more data will be added. After three years, a Cumulative Total will be shown.

34. 34 Looking at the Results Report – Content Area Results

35. 35 Looking at the Results Report – Content Area Results Total Possible Points includes both common and matrix items (not field-test). Total Possible Points also represents the test’s balance of representation. Mathematics

36. 36 Looking at the Results Report – Content Area Results Please note: The Total Possible Points column is organized differently on the Reading Results Report 106 possible points are represented here – they are sorted by “Type of Text” The same 106 possible points are represented here – they are sorted by “Level of Comprehension”

37. Looking at the Results Report – Content Area Results Total Possible Points includes both common and matrix items (not field-test). Reading

38. 38 Looking at the Results Report – Disaggregated Results

39. 39 Looking at the Results Report – Disaggregated Results Important Note: Disaggregated results are not reported for sub- groups of less than 10

40. 40 Looking at the Results Report – Disaggregated Results Because this is a small school, and so many of the sub-groups are smaller than 10, this part of the report is not as useful. But we can still look at district and state disaggregated results. Does this data match what we know about the district’s program?

41. 41 Summary Report

42. 42 NECAP District and School Student-Level Data Files Contain: All demographic information for each student that was registered through MEDMS/Infinite Campus The scaled score, achievement level, and subcategory scores earned by each student in all content areas tested NECAP files also contain: Performance on released items Student questionnaire responses

43. 43 Item Analysis Report

44. 44 Item Analysis Report

45. 45 Looking at the Item Analysis Report

46. 46 Looking at the Item Analysis Report This part of the report gives specific information about the released items and student performance on individual items

47. 47 Looking at the Item Analysis Report

48. 48 Looking at the Item Analysis Report This part of the report represents all of the items used to compute student scores - points are displayed by subcategory. Each student’s Scaled Score and Achievement Level are shown.

49. 49 Guides to Using the 2009 NECAP Reports NECAP Analysis and Reporting System User and Training Manual Companion PowerPoint presentation Grade Level Expectations NECAP Accommodations Guide Released Items documents Preparing Students for NECAP: Tips for Teachers to Share with Students www.maine.gov/education/necap Supporting Materials and Resources

50. 50 How Do I Find Public Assessment Reports? Go to… www.maine.gov/education/ necap/results

51. 51 Big Picture: MEA and NECAP Data

52. 52 The MEA has consistently shown that female students outperform male students statewide in reading at every grade level. Gender Gap

53. 53 MEA 08-09 Performance Analysis Gender Differences in Reading Performance Chart Statistics Based on 2008-09 School Year Student Performance

54. 54 MEA 08-09 Gender Gap The gap ranged from 6 -11 percentage points. The NECAP is a different test measuring different standards - what pattern might you expect to see in NECAP data?

55. 55 NECAP 09-10 Performance Analysis Gender Differences in Reading Performance Chart Statistics Based on 2009-10 School Year Student Performance

56. 56 NECAP 09-10 Gender Gap The gap still exists and ranges from 7 – 14 percentage points. If this data is similar to your school/district data, what strategies have you employed or plan to employ to address this gender gap?

57. 57 For instance, both the MEA and NECAP reading tests have been/are designed to include both narrative and informational texts equally. Does your reading curriculum reflect that balance? Gender Gap

58. 58 If you organized the released items in the interactive reporting tool by GLE and chose gender as a filter, what would you predict about gender differences based on type of text? Note: Reading GLE codes R4, 5, and 6 refer to literary text and R7 and 8 refer to informational text. Gender Gap

59. 59 Years ago, the MEA showed a considerable gender gap in mathematics, with male students outperforming female students statewide at every grade level. Gender Gap

60. 60 Closing this gender gap in mathematics became a priority. The MEA showed over time that the gap had narrowed or disappeared. Gender Gap

61. 61 MEA 08-09 Performance Analysis Gender Differences in Mathematics Performance Chart Statistics Based on 2008-09 School Year Student Performance

62. 62 The gap ranges from only 0 to 4 percentage points. What pattern might you expect to see in NECAP data? MEA 08-09 Gender Gap

63. 63 NECAP 09-10 Performance Analysis Gender Differences in Mathematics Performance Chart Statistics Based on 2009-10 School Year Student Performance

64. 64 The gap is even narrower for this year, ranging from 0 to 3 percentage points. Is this pattern similar to your school/district data? NECAP 09-10 Gender Gap

65. 65 The percent of students with IEPs who met or exceeded standards in the MEA ranged from 27 – 40 in reading and from 14 – 42 in math depending on grade level. IEP Data – 08-09 MEA

66. 66 MEA 08-09 Performance Analysis Students with IEPs Chart Statistics Based on 2008-09 School Year Student Performance

67. 67 IEP Data – 09-10 NECAP The percent of students with IEPs who scored as proficient or proficient with distinction in the NECAP ranged from 23 – 38 in reading and from 17 – 33 in mathematics, depending on grade level.

68. 68 NECAP 09-10 Performance Analysis Students with IEPs Chart Statistics Based on 2009-10 School Year Student Performance

69. 69 IEP Data What percentage of students who scored at proficient or proficient with distinction in reading and in mathematics might you expect to find in your school/district data? If the percent is below (or above) state averages, what strategies have you employed or plan to employ to increase performance?

70. 70 SES Data - MEA The percent of students classified as economically disadvantaged who met or exceeded standards in the 08-09 MEA ranged from 53 – 66 in reading and from 36 - 58 in mathematics depending on grade level.

71. 71 MEA 08-09 Performance Analysis Students Identified as SES Chart Statistics Based on 2008-09 School Year Student Performance

72. 72 What pattern might you expect to see in NECAP data? SES Data

73. 73 NECAP 09-10 Performance Analysis Students Identified as SES Chart Statistics Based on 2009-10 School Year Student Performance

74. 74 SES Data - NECAP The percent of students classified as economically disadvantaged who scored as proficient and proficient with distinction in the 09-10 NECAP ranged from 56 – 62 in reading and from 45 – 50 in mathematics, depending on grade level.

75. 75 SES Data What percentage of students who scored at proficient and proficient with distinction in reading and in mathematics might you expect to find in your school/district data? If the percent is below (or above) state averages, what strategies have you employed or plan to employ to increase performance?

76. 76 MDOE content specialists are planning webinar presentations for teachers that will assist them in using the content-specific NECAP data contained in the interactive portion of the reporting tool and the released items to inform their teaching and assessing of NECAP GLEs. Working with NECAP Data

77. 77 Coming Attractions Administration window of the MEA science test for grades 5 and 8: May 10 – 21. MEA test administration materials: http://www.maine.gov/education/mea/admi ninfo.htm There will be a test administration webinar on April 27 from 9-10:30 AM that will also be posted on the web.