1. Los Angeles Unified School District Local District E Instructional Support Services Standards-Based Education Performance Assignment Training Session for Analytic Scoring 2003-04

2. PERFORMANCE ASSIGNMENT ELEMENTARY AND MIDDLE SCHOOL LOGISTICS Data Results Changes Testing Schedule Rater Reliability

3. PERFORMANCE ASSIGNMENT DATA RESULTS In grades 2-9, scores increased 2% (total students tested = 276,319) 40% scored proficient or above in English (107,497) 42% scored proficient or above in Spanish (3,181) 29% improved their performance level (53,608)

4. PERFORMANCE ASSIGNMENT DATA RESULTS Responses from Student and Teacher Questionnaire validate the correlation between students’ opportunity to learn and their Performance Assignment scores. Positive relationship between higher frequency of quality reading and writing experiences and higher scores. Students who have less practice with reading and writing score lower.

5. PERFORMANCE ASSIGNMENT CHANGES FOR 2003-04 Elementary Grades Performance Assessment aligns with OC Third periodic assessment Middle School Third periodic assessment SBACs & Literacy Coaches work together Analytic scoring replaces holistic scoring

6. PERFORMANCE ASSIGNMENT TESTING SCHEDULE Elementary Third Periodic Assessment Middle School Third Periodic Assessment Single Track: February 23-27 Three Track: A March 29 - April 3 B & CJanuary 26-30 Four Track: A & BMarch 1-5 C & DJanuary 26-30

7. PERFORMANCE ASSIGNMENT RATER RELIABILITY Target Percentage 2001-0250-60% 2002-0355-65% 2003-0460-70%

8. PERFORMANCE ASSIGNMENT RATER RELIABILITY – SCHOOLS 70-79% RATER AGREEMENT Allesandro Coop. Learning Magnet (grade 4) Alta Loma (grade 2) Arlington Heights (grade 4) Atwater (grade 3) Buchanan (grade 3) Bushnell Way (grade 4) Clifford (grade 4)

9. PERFORMANCE ASSIGNMENT RATER RELIABILITY – SCHOOLS 70-79% RATER AGREEMENT Dayton Heights (grade 2) Fletcher Drive (grades 2 and 3) Garvanza (grade 3) Grant (grade 4) Hobart (grade 3) Ivanhoe (grade 4) Micheltorena (grade 4)

10. PERFORMANCE ASSIGNMENT RATER RELIABILITY – SCHOOLS 70-79% RATER AGREEMENT Monte Vista (grade 5) Pio Pico (grade 3) Ramona (grades 2 and 3) San Pascual (grade 3) Selma (grade 4) Wilshire Crest (grade 3) Wilton Place (grade 3)

11. PERFORMANCE ASSIGNMENT RATER RELIABILITY – SCHOOLS 80-89% RATER AGREEMENT Atwater (grade 2) Franklin (grade 3) Gardner (grade 5) Grant (grade 3) Selma (grade 3) Wilton Place (grade 5)

12. PERFORMANCE ASSIGNMENT RATER RELIABILITY – SCHOOLS 90-100% RATER AGREEMENT Aragon (grade 2) Arroyo Seco Museum Sci. Magnet (grade 5) Cheremoya (grade 4) Dorris Place (grade 4) Eagle Rock G/HA Elem. Magnet (grade 3)

13. PERFORMANCE ASSIGNMENT RATER RELIABILITY – SCHOOLS 90-100% RATER AGREEMENT Micheltorena (grade 3) Pio Pico (grade 2) Queen Anne (grades 4 and 5) Selma (grade 2) Rater Reliability Data from: Use of Periodic Assessments to Promote Literacy Achievement

14. PERFORMANCE ASSIGNMENT RATER RELIABILITY SECONDARY SCHOOLS Eagle Rock (gd. 8)74% Pio Pico(gd. 7)58% Irving (gd. 8)57% Burbank (gd. 6)54% Burroughs (gd. 8)51% LeConte(gd. 6)51% King (gd. 6)50% Mount Vernon (gd. 7/8)47%

15. PERFORMANCE ASSIGNMENT Analytic Scoring for Elementary

16. SCORING Holistic Scoring Procedure which yields a single score based on an overall impression of students performance

17. SCORING Analytic Scoring Procedure which yields multiple scores based on two or more separate scales for evaluating performance on selected dimensions or traits; each trait receives a separate score.

18. SCORING looks at the entire writing piece analyzes separate a spects of student writing a form of assessment that looks at writing both use rubrics both can be indicators for teacher’s next steps captures an overall picture of student’s writing gives one score a single scale is often more efficient writing is read through for each score scores are given for each analytic category uses multiple scales Provides more detailed info for improving instruc- tion and communicating with students ANALYTICAL HOLISTIC

19. SCORING Analytic Scoring for Unit 3 Applications (genre) = 45% Writing Strategies (traits) = 45% WOEL Writing and Oral English Language (Conventions) = 10%

20. PREPARING FOR A SCORING TRAINING SESSION Scoring Handbook (Part I) Major Responsibilities: Determine who will attend Schedule training session Find appropriate location Prepare training packets

21. PREPARING FOR A SCORING TRAINING SESSION Scoring Handbook (Part I) Materials Training packet (grade level) Prompt and rubric Set of anchor and training papers Scoring Handbook Assessment form for recording student scores (Literacy Coach)

22. PREPARING FOR A SCORING TRAINING SESSION Scoring Handbook (Part I) Materials Training Paper Score Sheet Rater Agreement Summary Sheet Calculator Training Papers Key

23. PREPARING FOR A SCORING TRAINING SESSION Scoring Handbook (Part I) Time 1.5 to 2.5 hours

24. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Open Court Writing Unit 1 Unit 1 Scoring Materials Grade 3

25. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 1 Read the task (prompt) carefully… Think about the stories you have read in the Friendship Unit and what it is like to be a friend. Write a story about your best friend and tell about a special time or memorable event that you have shared as friends. Be sure to use concrete sensory details to describe where you were. Describe what was special to you and your friend and tell why this special time was something you will always remember.

26. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 2 Review the rubric for Applications… Working with a partner, carefully read aloud the description of each score point in the rubric. Start with level 3/Proficient. 5 minutes

27. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 3 Read and discuss anchor paper focusing only on Applications Anchor papers represent the lowest performance level In packet, there are anchor papers for 2, 3, and 4

28. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 3 Focusing only on Applications… Read Anchor 3 paper aloud. Briefly discuss the qualities of the paper, referring to each component of the rubric. Describe why it received the score it did. Repeat for Achors 2 and 4. Discuss distinctions in levels referring to the rubric as needed. Read Anchor 3 paper aloud. Briefly discuss the qualities of the paper, referring to each component of the rubric. Describe why it received the score it did. Repeat for Achors 2 and 4. Discuss distinctions in levels referring to the rubric as needed. 10 minutes

29. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 4 Repeat the same process from Steps 2 and 3 focusing only on the Writing Strategies category. Step 2 – review the rubric Step 3 – read and discuss anchor papers 10 minutes

30. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 5 Step 5 Repeat the same process from Steps 2 and 3 focusing only on the WOEL (Conventions) category. Repeat the same process from Steps 2 and 3 focusing only on the WOEL (Conventions) category. Step 2 – review the rubric Step 3 – read and discuss anchor papers 5 minutes

31. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6 Step 6 Score training papers Training papers already have assigned scores by expert raters Goal is to obtain at least 70% agreement

32. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers A. Score Training Paper A Read Training Paper A. Compare each response to the set of anchor papers and rubric criteria for each category. Use the Training Paper Score Sheet (at the bottom of your packet) to record scores. 5 minutes

33. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers A. Score Training Paper A You should have 3 scores for paper A. Using the Rater Agreement Summary Sheet, record the frequency of each score (Applications, Writing Strategies, Conventions). In each category, circle the highest number. 5 minutes

34. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers A. Score Training Paper A Overall Score = 2

35. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers B. Score Training Paper B (use same procedures as before) (use same procedures as before) Read Training Paper B. Compare each response to the set of anchor papers and rubric criteria for each category. Use the Training Paper Score Sheet (at the bottom of your packet) to record scores. 5 minutes

36. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers B. Score Training Paper B (use same procedures as before) (use same procedures as before) Read Training Paper B. Compare each response to the set of anchor papers and rubric criteria for each category. Use the Training Paper Score Sheet (at the bottom of your packet) to record scores. 5 minutes

37. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers B. Score Training Paper B Overall Score = 3

38. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers C. Score Training Paper C & D Score training papers C and D following the same procedures used to score training papers A and B. 5 minutes

39. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers C. Score Training Paper C & D Training Paper C – Score=3 Training Paper D – Score=2

40. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 6: Score Training Papers D. Check rater agreement Using the Rater Agreement Summary Sheet, and the sample page WS-4 in your Scoring Handbook, follow the directions to calculate rater agreement. 10 minutes

41. CONDUCTING A SCORING TRAINING SESSION Scoring Handbook (Part II) Step 7: Student Writing Samples Score student writing samples and record on the Classroom Assessment Summary Form provided by the Literacy Coach.