1. NeSA-W Grade 11 Online Pilot Assessment Nebraska Department of Education Statewide Assessment Pat Roschewski – pat.roschewski@nebraska.govpat.roschewski@nebraska.gov Edward Foy – edward.foy@nebraska.govedward.foy@nebraska.gov Support Personnel Julie Barger – ESU #16, Ogallala, NE Doug Pierson – Avenue Scholars Foundation, Omaha, NE 4/25/20151

2. The NeSA-W/DRC Partnership Working together to improve teaching and learning for Nebraska students. 4/25/20152

3. 3 Why How What

4. The Nebraska Department of Education and Nebraska educators continue to direct the process of statewide assessment. 4/25/20153

5. Nebraska Educators continue to be involved in these aspects of NeSA-W: Prompt Creation Field Testing Prompt Selection Rangefinding Appeals Continued training and communication 4/25/20155

6. Recent Involvement The Rangefinding Process –Involved 10 Nebraska educators from a variety of educational backgrounds –Two DRC reps facilitated the work –Analytically scored, discussed, and came to consensus on 60 papers from the field testing of the pilot test prompt –DRC will use the papers selected and scored by this group to establish the anchors and train the raters prior to the scoring of the pilot assessment. 4/25/20156

7. Grade 11 Online Pilot Assessment 1.Grade 11 will remain a persuasive essay. 2.The 11 th grade NeSA-W pilot will only be available online. 3.Students in the 11 th grade will not be required to take the NeSA-W in 2011. 4.The results will be analytically scored. 4/25/20157

8. Grade 11 Online Pilot Assessment 5.Districts may elect to have their 11 th graders participate in the online pilot in 2011. 6.No advance notice/registration is required. 7.Districts will only need to schedule a download or update of software prior to administering the pilot. 4/25/20158

9. Grade 11 Online Pilot Assessment 8.No results will be returned from the pilot. 9.Printing options are available.  Both the practice and pilot test may be printed in 2011.  The operational test in 2012 and beyond will also be printable. 4/25/20159

10. Grade 11 Online Pilot Assessment 10.Changes have been made to the 11 th grade rubric.  The rubric was changed to an analytic rubric in the fall of 2010 from the previous holistic rubric.  The content of the new rubric has been tightened, and although most of the previous criteria remain, they have been redistributed into fewer domains. 4/25/201510

11. Grade 11 Online Pilot Assessment  The domains of the rubric will be weighted as follows: –Content/Ideas – 35% –Organization– 25% –Word Choice/Voice/Tone – 20% –Sentence Fluency/Conventions - 20% 4/25/201511

12. Grade 11 Online Pilot Assessment  The rubric is available on the Assessment page of the NDE website. http://www.education.ne.gov/Assessment/index. html 4/25/201512

13. Grade 11 Online Pilot Assessment 11. Changes to the testing window dates and recommended testing time have been made.  The testing window for the 11 th grade online pilot will be from January 31 to February 18.  The testing window for the 11 th grade online pilot will open one week later than the testing window for the paper/pencil 4th and 8 th grade writing assessment.  This will give districts the opportunity to install and/or update software for the online pilot. 4/25/201513

14. Grade 11 Online Pilot Assessment  The recommended testing time is 90 minutes to occur on one day.  The test is not timed, but 90 minutes is recommended by NDE as the amount of time to schedule for the assessment.  If a student needs more than 90 minutes, that is allowable, the same as for NeSA-R. 4/25/201514

15. Grade 11 Online Pilot Assessment 12. Pre-writing may be done on paper furnished by the district. Students may choose to pre-write and produce a rough draft on paper before entering it on the computer, or they may choose to work entirely on the computer. 4/25/201515

16. Grade 11 Online Pilot Assessment 13. All editing tools including thesaurus, dictionary, cut/paste, and spell check are to be available for editing the final copy before submission. 14. An online practice test for writing will be available on January 24 th. You can access this assessment through the CAL software. 4/25/201516

17. Grade 11 Online Pilot Assessment  Graphic organizers, story starters, and story frames are not to be provided to students as part of the state testing program.  In the testing room, visual aids and clues should be removed or covered and remain covered throughout the administration of the writing assessment. Additional Information 4/25/201517

18. Grade 11 Online Pilot Assessment  Teachers are NOT to provide editing assistance to students on the NeSA-W.  No NeSA-W 11 th grade pilot test will be provided in Spanish. Additional Information 4/25/201518

19. Grade 11 Online Pilot Assessment Formula for each rater’s scoring: R= 100[0.35D 1 +0.25D 2 +0.20D 3 +0.20D 4 -1] 3 Two readers’ scores combined will create a scale score of 0 to 200. Cut scores will be set based on the actual assessment. Additional Information 4/25/201519

20. Questions? Nebraska Department of Education Statewide Assessment Pat Roschewski – pat.roschewski@nebraska.govpat.roschewski@nebraska.gov Edward Foy – edward.foy@nebraska.govedward.foy@nebraska.gov 4/25/201519

21. The New Rubric A Closer Look 4/25/201520

22. The New Rubric Changes were made to the rubric for a number of reasons.  A revision of the Language Arts standards occurred in 2008-2009.  An external study determined that the new standards and the writing assessment rubric did not match, necessitating a review/revision of the rubric. 4/25/201522

23. The New Rubric Changes were made to the rubric for a number of reasons.  Influence favoring analytical assessment was expressed in an effort to provide more specific data to data users.  The use of a computer based assessment reflects a “real world” application of writing. 4/25/201523

24. 4/25/201524

25. The New Rubric Specific changes to the rubric include:  Domains--New combinations within the domains have been added, however the traits and the criteria within the domains remain similar.  Revised wording throughout the rubric  Consistency in wording across score points  Domains hold specific weight when scores are calculated  Number of score points changed from 10 to 4 4/25/201525

26. 4/25/201526

27. Holistic vs. Analytical Scoring Holistic scoring involves one score being given that represents the overall quality of the paper against a set of criteria, i.e. the six traits and their criteria. 4/25/201527

28. 4/25/201528

29. Holistic vs. Analytical Scoring Analytic scoring involves a score being given for each domain and its criteria. These individual weighted scores are combined to calculate the final score. 4/25/201529

30. 4/25/201530

31. Holistic vs. Analytical Scoring Formula for each rater’s scoring: R= 100[0.35D 1 +0.25D 2 +0.20D 3 +0.20D 4 -1] 3 Two readers’s scores combined will create a scale score of 0 to 200. 4/25/201531

32. The New Rubric  The new rubric is available on the Assessment page of the NDE website. http://www.education.ne.gov/Assessment/index. html 4/25/201532

33. The Process of Analytical Scoring The New Statewide Process 4/25/201532

34. The Analytical Scoring Process The Process  First read: focus on Ideas/Content and Organization--assign a score to each domain of 1, 2, 3, or 4.  Then review paper: focus on Word Choice/Voice and Sentence Fluency/Conventions-- assign a score to each domain of 1, 2, 3, or 4. 4/25/201534

35. The Analytical Scoring Process The Process  Each paper is scored by two readers.  Agreement: Exact matches or adjacent scores  A Third Read by the table leader is required for each domain if scores are not in agreement 4/25/201535

36. The Analytical Scoring Process Rules for obtaining the final score for a student after the third read: 1.If the third score is an exact match to one of the original scores, the two matching scores are used. 2.If the third score is adjacent to one of the original scores but not the other, the third score and the score adjacent to it are used. 3.If the the third score is adjacent to both scores, the third score is used twice. 4/25/201536

37. The Analytical Scoring Process Possible impact of analytical scoring:  Information: Each student report will include four sub scores and one scale score. These reports are shared with students and parents.  Trends: Over time, data from this assessment can indicate domain strength or weakness at a classroom, school, and/or district level. 4/25/201537

38. Scoring Practice 4/25/201537

39. The school board has received funds to support one current or one new activity/program. Think about a current or new activity/program you would support. It could be artistic, athletic, dramatic, educational, or social. Write an essay persuading the school board that yours is a good choice. Give specific reasons to support your recommendation. 4/25/201537 Prompt

40. Further Questions?? Nebraska Department of Education Statewide Assessment Pat Roschewski pat.roschewski@nebraska.gov Edward Foy edward.foy@nebraska.gov pat.roschewski@nebraska.govedward.foy@nebraska.gov 4/25/201540