1 Guide to Using the 2006 NECAP & NH-Alternate Reports: Companion PowerPoint Presentation February 2007 New Hampshire Alternate Assessment New Hampshire Statewide Assessment

2 Introductions NHDoE Updates Background NH-Alternate Assessment Reporting Short break NECAP Reporting Agenda

3 New Hampshire Alternate Assessment Welcome & Introductions Tim Kurtz Director of Assessment NH Department of Education Gaye Fedorchak Alternate Assessment Coordinator NH Department of Education

4 Measured Progress Service Center: Measured Progress Offices: New Hampshire Alternate Assessment Welcome & Introductions Harold Stephens NECAP Program Director Amanda Breitmaier NH Alt. Assessment Program Manager Amanda Smith NECAP Program Manager – Grade 11 Mellicent Friddell NECAP Program Assistant Josh Evans NECAP Program Manager – Grades 3-8 Tina Haley NECAP Program Assistant Elliot Scharff NECAP Program Manager – Science Jennifer Varney NECAP Program Assistant

5 New Hampshire Alternate Assessment NHDoE Contacts Tim Kurtz Director of Curriculum and Assessment Phone: (603) Gaye Fedorchak Alternate Assessment Coordinator Phone: (603) Susan Morgan - ELL Access Specialist Phone: (603) Carol Angowski Assessment Program Specialist Phone: (603) Linda Stimson - ELA Coordinator Phone: (603) Rich Andrusiak - Mathematics Coordinator Phone: (603) Jan McLaughlin - Science Coordinator Phone: (603) Ken Relihan - Social Studies Coordinator Phone: (603) Deb Fleurant - Bias and Sensitivity and Title I Phone: (603)

6 Assessment NECAP Piloted Reading, Math, and Writing in November No student, school, or district scores will be produced First operational test – October 2007 NH-Alt Grade 10 portfolios in Reading, Mathematics, and Writing In addition to above, Grade 11 portfolios in science Accountability Schools – AYP based on Graduation rate only Districts – AYP based on May 2006 NHEIAP and Alt Update: High School Assessment

7 Assessment NECAP Pilot Science May 14-24, 2007 Two Sessions – about 90 minutes each No student, school, or district scores will be produced First operational test (three sessions) – May 2008 Workshops during the week of April 30 – May 4 NH-Alt Grade 11 Science Portfolios will be added in school year Accountability None at this time, but NCLB reauthorization proposals include it starting in school year Update: Science Assessment

8 i.4.see Workbench Hub for district submitted student level data Many new build-it checks on submitted data i.4.see Dashboard Check summative student demographic files that affect AYP Ability to change incorrect data Notice will go to Superintendents, Special Education Directors, i.4.see coordinators, and Principals Hope to go live on Monday, February 12, 2007 Deadline on changes has not been set yet Follow the Child Assistance Center Phase I – April/May Phase 2 – early summer Update: Student Demographic & Assessment Data New

9 Any questions about Updates? Updates

10 New Hampshire Alternate Assessment Purpose of the Workshop To assist schools and districts in using the variety of NH Statewide Assessment reports and support material in conjunction with the release of results To provide school and district personnel an opportunity to ask questions regarding the reports and support materials

11 Development of Grade Level Expectations (NECAP) Development of Alternate Achievement Standards Linked to Grade Level Expectations (NH-Alt) Test Item Review Committees (NECAP) Bias and Sensitivity Review Committees (NECAP) Classroom Teacher Judgment Data (NECAP) Standard Setting Panelists (NECAP and NH-Alt) Technical Advisory Committee (NECAP and NH-Alt) NH-Alt Advisory Task Force (NH-Alt) Local Educator Involvement New Hampshire Alternate Assessment

12 When: NECAP: Oct 2006 NH-Alternate: School year What: Reading and Mathematics – NECAP: Grades 3-8 (tests previous year’s GLEs) NH-Alt : Grades 2-7, 10(tested during instruction last year) Writing – NECAP:Grades 5 & 8(tests previous year’s GLEs) NH-Alt:Grades 4 & 7(tested during instruction last year) Test Basics New Hampshire Alternate Assessment

13 NECAP Variety of Item Types –Reading: multiple choice and constructed response –Mathematics: multiple choice, short answer 1 point, short answer 2 points, and constructed response –Writing: multiple choice, constructed response, and extended response Common and Matrix Items –Common items used to generate student scores –Matrix items used for equating and field testing Basics of Test Design New Hampshire Alternate Assessment

14 NH Alternate Portfolio Assessment: What is an “Item” in NH-Alt? –In each content area, two standards-based targeted skills are identified that address student learning needs –Baseline data are collected for each targeted skill –Work samples are gathered over the course of the year to show how the student is progressing on these two targeted skills –The collection of work samples and data presented in the portfolio are scored as a single item –Progress data and student work samples are collected across the school year (Sept – April) on each targeted skill as the student is learning the skill. Reading: 2 Targeted Skills Mathematics: 2 Targeted Skills, Writing: 2 Targeted Skills Basics of Test Design New Hampshire Alternate Assessment

15 NH Alternate Portfolio Assessment Targeted skills are represented by a variety of work samples collected over the course of the year. –Collections include multiple samples of actual student work, which typically show the student solving problems in many ways, i.e. multiple choice responses, short answer responses, and longer responses that look similar to constructed response problems. Responses often involve use of manipulatives, communication systems, or other instructional technology. –Authentic classroom constructed response and short answer tasks provide the best methods for allowing students with severe disabilities to show their work in the way they are able to produce it. –Provides richer evidence of access to standards-based content and student progress. Basics of Test Design New Hampshire Alternate Assessment

16 Norm-Referenced: What is the student’s status or rank in a designated group? Criterion-Referenced (Standards-Based): Has the student learned the skills or achieved the standard? Both NECAP and NH Alternate Assessments are Criterion-Referenced Tests. Types of Educational Tests New Hampshire Alternate Assessment

17 Questions about the purpose of the workshop or NECAP or NH-Alternate background? Questions and Answers New Hampshire Alternate Assessment

18 New Hampshire Alternate Assessment  Student Report (Confidential) with Information for Parents and Report Interpretation Guide  Student Roster Report (Confidential) School and District levels by student  District Student Level Data Files (Confidential)  Disaggregated Results by Content Area: (Public) School, District, & State reported separately for each grade  State Summary Reports (Public)  NH-Alt Students included in NECAP reports as “NT- Approved” Basics of Test Design

19 New Hampshire Alternate Assessment

20

21

22

23 New Hampshire Alternate Assessment Names of Achievement Levels are the same as in NECAP - Level 4: Proficient with Distinction Level 3: Proficient Level 2: Partially Proficient Level 1: Substantially Below Proficient But - describe performance of grade-linked standards-based skills reduced in depth, breadth, and complexity for the student. NH-Alt Achievement Levels

24 NH-Alt: Two Scoring Dimensions New Hampshire Alternate Assessment 1. The Performance Dimension – Most heavily weighted in score and includes: Student Progress (from start of year baseline) Connections and Access to the General Curriculum (Student work samples must show linkage to grade-level content at reduced level of depth, breadth, complexity)

25 NH-Alt: Two Scoring Dimensions New Hampshire Alternate Assessment 2. The Program Dimension - Less heavily weighted in scoring, and includes: Generalized Performance (Student uses skill across different settings and situations) Self-Determination (Student attempts to self direct and monitor work) Supports (Does level of assistance provided support growth of independence and match student need?)

26 Sample Portfolio Score Calculation New Hampshire Alternate Assessment Scoring Sub-Dimension: Base Points (Range 1-4 pts.) Weight Given Weighted Sub-Score Student Progress4 ptsX 416 Access to Curriculum2.5 ptsX 410 Skill Generalization2 ptsX 36 Self-Determination4 ptsX 14 Supports4 ptsX 14 Calculate Content Area Raw Score: = 40

27 Raw Score To Achievement Level Conversion Chart New Hampshire Alternate Assessment If - Total Raw Score for the content area is: Then - Achievement Level for the content area is: 47-52Level 4: Proficient with Distinction 38-46Level 3: Proficient 29-37Level 2: Partially Proficient 13-28Level 1: Substantially Below Proficient

28 Questions about NH-Alternate Reporting? Questions and Answers New Hampshire Alternate Assessment

29 Student Report Item Analysis Report School/District Results Report School/District Summary Report District Student Level Data Files NECAP: Types of Reports

30 The Family Educational Rights and Privacy Act (FERPA) Access to individual student results is restricted to: the student the student’s parents/guardians authorized school personnel Superintendents and principals are responsible for maintaining the privacy and security of all student records. Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties. FERPA website: Family Educational Rights and Privacy Act

31 National Council on Measurement in Education

32 Student Report Two copies mailed to the school (February 7) color copy for parents/guardians black and white copy for local use Dual sided: Parent Letter & Student Report Spanish version of Parent Letter and Student Report shell available for NECAP (available at:

33

34

35

36

37 Questions about the Student Report? Questions and Answers

38 Rationale Inclusion of students in “Teaching Year” reports Teaching School vs. Testing School

39 Teaching School vs. Testing School Schools Can View Reports for Testing Year ( )

40 Teaching School vs. Testing School Or Teaching Year ( )

41 Testing Year Teaching Year

42

43 New this year Please note: You must always log in through this page to view confidential reports and data

44

45

46

47 Item Analysis Report Available at the school level via password protected website (User ID and passwords sent to Principals and Superintendents in January) One report for each content area/grade level combination Issued in both Testing Year and Teaching Year formats Use in conjunction with Released Items documents

48

49

50

51 Released Item Documents

52 Released Item Documents New information provided in the Released Item Support Materials - documents include: For all content areas, providing the multiple-choice items and the GLE code and text that each item is linked to For math, indicating which items were included in session 1 of the math test by adding a “no calculator” symbol above the items For reading, listing the passage name and type of text For writing, listing passage name and mode of writing

53 Questions about the Item Analysis Report or the Released Item documents? Questions and Answers

54 Available on-line (January 30) at Grade Level Report One for each grade in each school and district Issued in both testing year and teaching year formats Summary Reports One for each school or district Issued in both testing year and teaching year formats School/District Results

55

56 Page 1:Title Page Page 2:Grade Level Summary Report Page 3:Reading Results Page 4:Disaggregated Reading Results Page 5:Mathematics Results Page 6:Disaggregated Mathematics Results Page 7:Writing Results (Grades 5 and 8 only) Page 8:Disaggregated Writing Results (ditto) Grade Level Results Report

57

58 Questions about the Grade Level Summary Report page? Questions and Answers

59 Pages 3, 5, and 7 of the NECAP School/District Results Report Provides current and historical summative information about achievement in tested content areas Provides information about achievement in subtopics of tested content areas Content Area Results

60 Content Area Results - Historical Data Data is included for two years and is combined in the Cumulative Total row In the Cumulative Total row for the achievement level columns: The N columns are summed The % columns are found by dividing the total number of students in the level by the total number of students tested The Mean Scaled Score column is a weighted average

61

62 Content Area Results – Subscore Results Results based on both common matrix equating items Many Reading items are included twice – once under Type of Text, and once under Level of Comprehension Standard Error Bars are included to help determine if differences between scores are statistically significant

63

64 Comparisons of NECAP Scores Across Years School and District-Level Scaled Scores and Achievement Levels Student-Level Scaled Scores and Achievement Levels Content Area Subscores

65 Number of Scaled Score Points Denoting Minimally Statistically Significant Difference for Average Group Results* GradeSubject Number of Students Tested in Group (Class, School etc.) Reading Mathematics Reading Mathematics Reading Mathematics Writing Reading Mathematics Reading Mathematics Reading Mathematics Writing *Standard error of the mean difference with assumption n1=n2 and s1=s2

66 Percentage Difference in Student Achievement Level Classification Denoting Minimally Statistically Significant Differences for Group Results* Percentages of Students in Achievement Level(s) Number of Students Tested in Group (Class, School etc.) *0.05 level of statistical significance

67 Questions About the Content Area Results Page? Questions and Answers

68 Pages 4, 6, and 8 of the NECAP School/District Results Report Provides information about achievement for different groups in a school/district Testing year reports have detailed information about all students enrolled in October 2006 Teaching year reports include only those students with NECAP scores from Oct Detailed information for districts and the state may be found in the district and state reports. CSV files containing all this data will be posted to the NHDoE website in mid February. Disaggregated Content Area Results

69

70 Questions about the Disaggregated Content Area Results page? Questions and Answers

71 Separate report for each school and district Provides summative information for each grade level within the school or district Provides summative achievement level data for whole school or district Provided in both teaching year and testing year formats CSV files containing all this data will be posted to the NHDoE website in mid February. School/District Summary Report

72 Identifying the Report The first line designates the year the test was administered The second line is the name of the report The third line differentiates between “teaching year” and “testing year”

73

74 Questions about School/District Summary Reports? Questions and Answers

75 Student Level Data Files CSV files available at the district level via password protected website (User ID and passwords sent to Superintendents in January) All demographic information for each student that was provided by the state Provided in both teaching year and testing year formats The scaled score, achievement level, and subscores earned by each student in all content areas tested Also contain: Performance on released items Student questionnaire responses Optional reports fields data

76

77 Questions about Student Level Data Files? Questions and Answers

78 Supporting Materials Materials available now Guide to Using the 2006 NECAP Reports 2006 Reporting Workshop Ppt Grade Level Expectations Released Item documents GLE Support Materials Materials available soon Roster of school and district results (teaching year and testing year formats) (csv) Disaggregated Data for school & districts by grade level (teaching year and testing year formats) (csv) Student work samples without scores or annotations (pdf) Release Item distracter stats (pdf) 2006 Technical Manual