1. 2008 ASSESSMENT & ACCOUNTABILITY CONFERENCES Training for ELPA and MI-Access Scoring and Administration 1

2. PURPOSE To help coordinators who train assessment administrators better understand scoring procedures for MI- Access and the ELPA To provide information on resources available to aid understanding And, ultimately, to…  increase the reliability and validity of assessment results through better understanding, and  ensure that students’ efforts are accurately scored, reported, and counted 2

3. MI-Access Assessments Participation — for students who have, or function as if they have, severe cognitive impairment Supported Independence — for students who have, or function as if they have, moderate cognitive impairment Functional Independence — for students who have, or function as if they have, mild cognitive impairment Three Assessments 3

4. Content Areas Assessed T he MI-Access assessments cover three content areas:  English language arts (ELA),  mathematics, and  science The assessments reflect Michigan’s GLCEs, HSCEs, and/or Benchmarks, but they have been extended — or reduced in depth, breadth, and complexity — so they are appropriate for the student populations being assessed 4

5. MI-Access Participation & Supported Independence Assessments The Participation and Supported Independence (P/SI) assessments use TWO item formats:  Activity-based observation — items are presented to students during familiar classroom activities or routines, which provide a performance context for assessing specific EGLCEs, EHSCEs, and/or EBs  Selected response — students are read an item stem, or question, and asked to select the correct response 5

6. Activity-Based Observation Item ACTIVITY: The student will correctly indicate the difference between whole and part when engaged in a familiar food preparation activity, such as assembling ingredients for a snack. For example, the student could be shown a whole cup of milk and part of a cup of milk and then be asked, “Which one is a whole cup?” SCORING FOCUS: Differentiating between a whole object and part of an object Sample Participation Mathematics Item (Grades 6-8) 6

7. Selected-Response Item Sample Participation Science Item (Grade 8) 7 P items have 2 picture answer choices SI items have 3 picture answer choices

8. Picture Card Presentation Styles 8 Picture 1Picture 2 Picture 1 Show firstReverse and show again Participation Selected-Response Item with 2 Answer Choices Both picture cards must be presented at the same time in one order, then presented again with the positions reversed ELA activity-based observation items that use words paired with pictures also have specific presentation styles (see Appendix C in the manual)

9. P/SI Assessment Administrators Administered by TWO people—a Primary Assessment Administrator (PAA) and a Shadow Assessment Administrator (SAA) The PAA and SAA work together prior to the assessment to determine the student’s anticipated response mode. The PAA and SAA observe the student being assessed and simultaneously and independently score the student using a standardized scoring rubric (regardless of item format) 9

10. P/SI Scoring Rubrics The scoring rubrics…  are based on the student responding correctly, and  take into consideration the amount of assistance required to engage the student in the item The scoring rubrics contain both…  score points, and  condition codes (which add meaning to zeroes) 10

11. P/SI Scoring Rubrics MI-Access P/SI Scoring Rubrics P Score Point/ Condition Code SI Score Point/ Condition Code Term 32 Responds correctly with no assessment administrator assistance 21 Responds correctly after assessment administrator provides verbal/physical cues 1Not allowed in SI Responds correctly after assessment administrator provides modeling, short of hand-over-hand assistance AAIncorrect BBResists/Refuses CC Assessment administrator provides step-by-step directions and/or hand-over-hand assistance 11

12. P/SI Scoring Rubrics Online Learning Program www.mi-access.info Explains the P/SI rubrics in detail Shows AAs using them to score students on sample items Is being updated (still says “pilot”) 12

13. MI-Access Coordinator & Assessment Administrator Manual The manual includes…  an entire section explaining how the P/SI assessments are designed and should be administered,  flow charts showing how to apply the scoring rubrics, and  score point and condition code definitions and examples 13

14. P/SI Item Analyses 14 Compares how AAs scored released items Can be used at district/building level to discuss issues of consistency Produced only when 10 or more students in the same grade take the same test

15. Functional Independence For Functional Independence, a scoring rubric is used only for ELA Expressing Ideas Students are asked to respond to an open-ended prompt:  Tell about what you like to do outside when the weather is cold. Be sure to include details and examples in your response. Students can write, draw, or use a combination to express their response Students can also dictate their response if their disability prevents them from writing or drawing 15

16. ELA Expressing Ideas Rubric Student responses are evaluated by external hand-scorers using a 4-point rubric that measures  topic focus,  organization, and  use of language and visual conventions 4 is the highest score a student can receive The rubric includes condition codes and comment codes 16

17. ELA Expressing Ideas Rubric Condition codes add meaning to a score point of zero  A = off topic  B = illegible  C = written in a language other than English  D = blank/refused to respond Comment codes elaborate on each score point  Lacks clear focus on the prompt topic  Shows limited development with insufficient details and/or examples  Lacks clear organization of ideas and/or arrangement of figures  Contains errors in language/visual conventions that interfere with understanding 17

18. FI ELA Item Analyses 18 Shows how many students received each score point or condition code Shows how many student received each comment code

19. RESOURCES MI-Access P/SI Scoring Rubrics Online Learning Program  www.mi-access.info under the “Online Learning” tab www.mi-access.info 2008/2009 MI-Access Coordinator and Assessment Administrator Manual  www.mi.gov/mi-access under “Resources” www.mi.gov/mi-access Sample Assessment Booklets and Released Item Booklets  www.mi-access.info under the “Functional Independence” and “Participation and Supported Independence” tabs www.mi-access.info 19

20. RESOURCES Extended GLCEs, HSCEs, and Benchmarks  www.mi.gov/mi-access under the “Participation and Supported Independence” and “Functional Independence” sections www.mi.gov/mi-access Assessment Plans  www.mi.gov/mi-access under the “Participation and Supported Independence,” “Functional Independence,” and “IEP Team Information” sectionswww.mi.gov/mi-access IEP Team Tools  www.mi.gov/mi-access under the IEP Team Information” section www.mi.gov/mi-access 20

21. CONTACTS MI-Access Linda Headley lheadley@cablespeed.com (517)339-3940 Deb Rakas debrakas@comcast.net (517)699-3017 21

22. ELPA: Speaking Scoring Item formats at all levels:  Sentence Repeat  Short Conversation  Guided Discussion  Storytelling  Story Retell 22

23. ELPA: Speaking Scoring Sentence Repeat Student hears and reads sentence before repeating it One point maximum score Standard focus is on S.6 Demonstrate comprehensible pronunciation and intonation for clarity in oral communication 23

24. ELPA: Speaking Scoring Sentence Repeat Proctor listens for fluency and smoothness of speech, proper pronunciation, and appropriate intonation 24

25. ELPA: Speaking Scoring Short Conversation Student hears and reads prompt and responds to question(s) Questions are as open-ended as possible, but still guide student toward focused answer 25

26. ELPA: Speaking Scoring Short Conversation Two points maximum score Standard focus is on S.2 Engage in conversations for personal expression and enjoyment 26

27. ELPA: Speaking Scoring Short Conversation Proctor listens for fluency and accuracy, but with an emphasis on the student’s ability to use appropriate vocabulary to express personal opinions, ideas, or points of view 27

28. ELPA: Speaking Scoring Guided Discussion Item format is a two-part set of questions with short prompt to lead into discussion Student hears and reads prompt and responds to first question 28

29. ELPA: Speaking Scoring Guided Discussion Student then hears and reads follow-up question and responds Two points maximum score for each prompt; total four points maximum Proctor scores each portion individually 29

30. ELPA: Speaking Scoring Guided Discussion Standard focus is on S.4 Use English to interact in the classroom Benchmark S.4.2.f Participate in guided discussions 30

31. ELPA: Speaking Scoring Guided Discussion Proctor listens for conversational flow and exchange of information; also for the student’s ability to focus the discussion on specific details or examples after the second prompt 31

32. ELPA: Speaking Scoring Story Telling Student sees sequence of three graphics and creates oral story or narrative about pictures Sequence of graphics is marked First, Next, and Last 32

33. ELPA: Speaking Scoring Story Telling Four points maximum score Standard focus is on S.7 Present information, concepts, and ideas to an audience of listeners on a variety of topics 33

34. ELPA: Speaking Scoring Story Telling Proctor listens for complex sentence structures, varied and precise vocabulary, and logical presentation of ideas with appropriate transitions 34

35. ELPA: Speaking Scoring Story Retell Student hears and reads narrative or content-related text Student responds by telling back to the proctor as much as possible, with allowance for added details 35

36. ELPA: Speaking Scoring Story Retell Student bases response on comprehensible English input appropriate to grade level Four points maximum score 36

37. ELPA: Speaking Scoring Story Retell Standard focus is on S.7 Present information, concepts, and ideas to an audience of listeners on a variety of topics 37

38. ELPA: Speaking Scoring Story Retell Proctor listens for authentic speech patterns of student, including use of grammatical constructions, descriptive vocabulary, and transitional phrases to retell recognizable story with fluency of speech 38

39. ELPA: Speaking Scoring Story Retell Speaking DVD available for training purposes Contact OEAA for copies of DVD 39

40. CONTACTS ELPA Phil Chase Department Specialist, Assessment of English Language Learners chasep2@michigan.gov (517) 335-3967 40