11 Using the 2010 NECAP Reports February/March, 2011

22 Welcome and Introductions Susan Smith MEA/NECAP Coordinator Maine Department of Education Harold Stephens NECAP Program Director Measured Progress

33 Welcome and Introductions Wanda Monthey Team Leader Susan Smith MEA/NECAP Coordinator Dan Hupp Assessment Director Michele Mailhot Math Specialist Patsy Dunton ELA Specialist Lee Anne Larsen ELA Specialist Rachelle Tome NCLB Title I Director, Accountability & AYP Sue Fossett PAAP Coordinator Department of Education Staff

44 Welcome and Introductions NECAP Service Center

55 Purpose of the Workshop Review the different types of NECAP reports Conduct a demonstration of the Analysis & Reporting System Review Maine NECAP data

66 Involvement of Local Educators Test Item Review Committees Bias and Sensitivity Review Committees

77 NECAP Grade Level Expectations If students do not have the opportunity to learn the content of the NECAP Grade Level Expectations (GLEs), they will be at a disadvantage when taking the NECAP test. This means all students, including special education students.

88 NECAP Grade Level Expectations Remember that Every item on the NECAP test is specifically written to measure a NECAP GLE GLEs were adopted by the legislature as Maine’s Federal, State, and Local Accountability Standards and became part of the Maine Learning Results: Parameters for Essential Instruction

99 NECAP Grade Level Expectations GLEs are located at ards.html ards.html GLEs should be fully incorporated into each school’s curriculum to ensure that students have an opportunity to learn the content assessed on the NECAP

10 NECAP Grade Level Expectations “We are focusing on Common Core State Standards now. Why should we pay attention to NECAP GLEs?” The NECAP test will be administered 3 more times in Maine schools: Oct. ‘11, ‘12 and ‘13. The assessment to measure the CCSSs does not yet exist and will not be administered until

11 Basics of NECAP Test Design Fall test – previous year’s GLEs Reading and Mathematics – grades 3-8 Writing – grades 5 and 8 Variety of Item Types Reading: multiple choice and constructed response Mathematics: multiple choice, short answer (one point), short answer (two points), and constructed response Writing: multiple choice, constructed response, and extended response Common and Matrix Items

12 Basics of NECAP Test Design Reading – Grades Common points Mathematics – Grades 3 & 4 65 Common points Mathematics – Grades Common points Writing – Grades 5 & 8 34 Common points

13 Workshop Materials

14 Types of NECAP Reports Public Reports Results Report School and District level Summary Report School/District/State level Password Restricted Reports Released Items Summary Data School and District level Achievement Level Summary School and District level

15 Types of NECAP Reports Confidential Reports Student Report Information for Parents/Guardians Item Analysis Report School level by student Longitudinal Data Student level, across test administrations Student Level Data Files Excel/csv files by grade on district and school confidential site

16 The Family Educational Rights and Privacy Act (FERPA) Access to individual student results is restricted to:  the student  the student’s parents/guardians  authorized school personnel Superintendents and principals are responsible for maintaining the privacy and security of all student records. Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties. FERPA website:

17 Student Report

18 Looking at the Student Report

19 Looking at the Student Report This part of the report provides the individual student’s achievement level and scaled score

20 Looking at the Student Report This part of the report provides a comparison between the performance of this student and his/her school, district, and state

21 Looking at the Student Report This part of the report gives specific information about the student’s performance in content area subcategories

22 Students at Proficient Level

23 Teaching Year vs. Testing Year Rationale Inclusion of students in “Teaching Year” reports

24 Looking at a School-Level Results Report Schools can view reports for Testing Year ( )

25 Looking at a School-Level Result Report Or for Teaching Year ( )

26 District and School Results Report

27 School-Level Results Report – Grade Level Summary 27

28 School-Level Results Report – Grade Level Summary 28

29 School-Level Results Report – Grade Level Summary

30 School-Level Results Report – Content Area Results

31 Looking at the Results Report – Content Area Results Three years of data are typically shown on this report. This year, Maine school reports will include data for 2 years. After three years, a Cumulative Total will be shown.

32 School-Level Results Report – Content Area Results

33 Looking at the Results Report – Content Area Results Total Possible Points includes both common and matrix items (not field-test). Total Possible Points also represents the test’s balance of representation. Mathematics

34 Looking at the Results Report – Content Area Results Please note: The Total Possible Points column is organized differently on the Reading Results Report 106 possible points are represented here – they are sorted by “Type of Text” The same 106 possible points are represented here – they are sorted by “Level of Comprehension”

35 Looking at the Results Report – Content Area Results Total Possible Points includes both common and matrix items (not field-test). Reading

36 Looking at the Results Report – Content Area Results Total Possible Points are all the points included in the test by item type. Writing

37 Looking at the Results Report – Disaggregated Results

38 Looking at the Results Report – Disaggregated Results Important Note: Disaggregated results are not reported for sub- groups of less than 10

39 Looking at the Results Report – Disaggregated Results Because this is a small school, and so many of the sub-groups are smaller than 10, this part of the report is not as useful. But we can still look at district and state disaggregated results. Does this data match what we know about the district’s program?

40 Summary Report

41 NECAP District and School Student-Level Data Files Contain: All demographic information for each student that was registered through MEDMS/Infinite Campus State Edition The scaled score, achievement level, and subcategory scores earned by each student in all content areas tested NECAP files also contain: Performance on released items Student questionnaire responses

42 Item Analysis Report

43 Item Analysis Report

44 Looking at the Item Analysis Report

45 Looking at the Item Analysis Report This part of the report gives specific information about the released items and student performance on individual items

46 Looking at the Item Analysis Report

47 Looking at the Item Analysis Report This part of the report represents all of the items used to compute student scores - points are displayed by subcategory. Each student’s Scaled Score and Achievement Level are shown.

48 Includes ALL Maine assessment reports using one password: NECAP, MEA, MHSA, PAAP and AYP Student Level CSV Files Interactive Capability Construct item-level reports by sub-group Construct achievement level graphs View longitudinal data by student (including PAAP) View item-level response percents/averages Account management: Set up accounts to view data on groups of students Set up teacher accounts so they can view pertinent data MeCAS Analysis and Reporting System

49 Guide to Using the 2010 NECAP Reports NECAP Analysis and Reporting System User and Training Manual Companion PowerPoint presentation Grade Level Expectations NECAP Accommodations Guide Released Items documents Supporting Materials and Resources

50 2 Years of NECAP Data

51 2 Years of NECAP Data Mathematics Grade 2009 NECAP % Proficient or Above 2010 NECAP % Proficient or Above % Change 2009 NECAP Scaled Scores 2010 NECAP Scaled Scores Point Change

52 2 Years of NECAP Data Reading Grade 2009 NECAP % Proficient or Above 2010 NECAP % Proficient or Above % Change 2009 NECAP Scaled Scores 2010 NECAP Scaled Scores Point Change

53 Maine tests have consistently shown that female students outperform male students statewide in reading at every grade level. Sub Group Data - Gender

54 NECAP Performance Analysis Gender Differences in Reading Performance Chart Statistics Based on School Year Student Performance

55 Gender Gap - Reading If your school/district data is similar to this state data, what strategies have you employed or plan to employ to address this gender gap?

56 For instance, the NECAP GLEs address both narrative and informational texts, so NECAP reading tests are designed to include both types equally. Does your reading curriculum reflect that balance? Gender Gap - Reading

57 If you organized the released items in the interactive reporting tool by GLE and chose gender as a filter, what might you predict about gender differences based on type of text? Note: Reading GLE codes R4, 5, and 6 refer to literary text and R7 and 8 refer to informational text. Gender Gap - Reading

58 Years ago, the MEA showed a considerable gender gap in mathematics, with male students outperforming female students statewide at every grade level, but closing that gap became a priority and it was closed over time. Gender Gap - Mathematics

59 NECAP Performance Analysis Gender Differences in Mathematics Performance Chart Statistics Based on School Year Student Performance

60 What percent of students with disabilities score in the proficient or proficient with distinction levels statewide? Sub Group Data - SPED

61 NECAP Performance Analysis Students with IEPs Chart Statistics Based on School Year Student Performance

62 IEP Data What percentage of students scored at proficient or proficient with distinction in reading and mathematics in your school/district data? Are all of your SPED staff familiar with the NECAP GLEs? Are they included in the same NECAP professional development as classroom teachers?

63 Sub Group Data - SES What percent of students identified as low socio-economic status score in the proficient or proficient with distinction levels statewide?

64 NECAP Performance Analysis Students Identified as SES Chart Statistics Based on School Year Student Performance

65 SES Data What percentage of students who scored at proficient and proficient with distinction in reading and in mathematics might you expect to find in your school/district data? If the percent is below (or above) state averages, what strategies have you employed or plan to employ to increase performance?

66 MDOE content specialists have held webinar presentations on instructional strategies to assist teachers in implementing the NECAP GLEs in their classrooms, available at: necap/presentations.html They will continue to present and post NECAP-related webinars on subjects such as DOK, incorporating CR-type questions in daily instruction and assessment, etc. NECAP Implementation

67 For more information regarding content- related webinars and other standards-related work, teachers may join the ELA or Mathematics Listserves through links at: m. m The MEA/NECAP Listserve link is at: info.html Listserves

68 Coming Attractions AYP reports released on March 9 th AYP Performance - teaching year data AYP Participation – testing year data AYP 2011 targets: Reading – 75% Mathematics – 70% ADA – 93%

69 Coming Attractions Administration window of the MEA science test for grades 5 and 8: May 9 – 20. MEA test administration materials: ninfo.htm There will be a test administration webinar on April 27 th from 9:30 -10:30 AM that will also be posted on the web.

70 How Do I Find Public Assessment Reports? Go to… necap/results