1. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 1 January 2011 California Alternate Performance Assessment (CAPA) Train-the-Trainer Workshop: Training CAPA Examiners Webcast will begin at 1:00 PM

2. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 2 January 2011 California Alternate Performance Assessment (CAPA) Train-the-Trainer Workshop: Training CAPA Examiners

3. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 3 January 2011 Objectives Participants will train examiners to accurately and consistently  Prepare for the CAPA  Administer the CAPA  Score the CAPA  Return CAPA answer documents Handout (H) Page 1

4. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 4 January 2011 Agenda Introductions Training California content standards and CAPA blueprints Preparing Administering the CAPA Scoring the CAPA After-test procedures Q & A H 1

5. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 5 January 2011 Training CAPA Examiners Required ANNUALLY Train before testing window opens Tips :  Active learning Practice Discuss Problem-solve  Repetition Quiz (Q) Question #1 H 1

6. Quiz Question 1 How often should CAPA examiners be trained? a. Annually b. Every other year c. Once d. When they think they need it 2011 STAR CAPA Train-the-Trainer Workshop || Slide 6 January 2011

7. Quiz Question 1 How often should CAPA examiners be trained? a. Annually 2011 STAR CAPA Train-the-Trainer Workshop || Slide 7 January 2011

8. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 8 January 2011 Standards provide the basis of instruction and testing In 2006 the State Board of Education adopted new blueprints for CAPA CAPA Level I blueprints: link to grades K–1 of California content standards and include two additional grade 2 science content standards. Levels II–V blueprints: link to grade-level (grades 2–11) California content standards CAPA blueprints can be found on the CDE Web site at: http://www.cde.ca.gov/ta/tg/sr/capablueprints.asp H 2 California Content Standards and CAPA Blueprints

9. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 9 January 2011 California Content Standards and CAPA Blueprints Sample of grade 5 English–language Arts (ELA) Strand: Reading/Word Analysis Standard: 6RW1.1: Word Recognition: Read aloud narrative and expository text fluently and accurately, and with appropriate pacing, intonation, and expression. Alternate Achievement Standard: Read a simple four- to five-word sentence composed of high-frequency words. H 2

10. Preparing for CAPA Identify students Select students for second rating Select examiners Select observers Put information on non–Pre-ID answer documents Prepare manipulatives and stimulus cards 2011 STAR CAPA Train-the-Trainer Workshop || Slide 10 January 2011 H 3−5

11. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 11 January 2011 Preparing: Are Correct Students Identified for CAPA? Has significant cognitive disabilities Has individualized education program (IEP) Is served under Individuals with Disabilities Education Act (IDEA) Meets eligibility guidelines  CAPA Examiner’s Manual — Appendix A  Assigned correct CAPA level H 3

12. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 12 January 2011 All students take ELA and mathematics. Only students in grades 5, 8, and 10 take science. Preparing: CAPA Levels LevelGrades Age Ranges for Ungraded Programs I2─117─16 II2─37─8 III4─59─10 IV6─811─13 V9─1114─16 H 3

13. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 13 January 2011 Students who repeat grade 11 for multiple years continue to take the CAPA Level I or Level V until:  Enrolled in grade 12 (or)  Turn 22 years of age and are no longer receiving compensatory education Preparing: CAPA Levels H 3

14. Preparing: Students in Ungraded Classrooms To determine a student’s grade (for ungraded programs), subtract 5 from student’s age as of December 2 Formula: (age – 5 = grade) Example: Chris is 14 years old. Subtract 5 from his age (14) to calculate his grade for CAPA testing (9). Chris would take Level V. (14 – 5 = 9) 2011 STAR CAPA Train-the-Trainer Workshop || Slide 14 January 2011 H 3

15. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 15 January 2011 Preparing: Do Your Examiners and Observers Have the Correct Qualifications? Licensed or certificated Experienced working with students to be tested with CAPA Trained either by attending a “live” session or watching the DVD or online video segments (www.mavricmedia.com/startest) Signed security affidavit H 3

16. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 16 January 2011 Preparing: Purpose of Observers Increase objectivity Data used to document inter-rater reliability Provide credibility/confidence to scores H 3

17. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 17 January 2011 Preparing: Double Rating Choose students randomly at each school/site  10% per CAPA level per school/site  At least one student at each level tested  Selected students should be double rated in all content areas tested Observe at same time as examiner scores Observers must score independently of examiner Q 2 H 3

18. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 18 January 2011 Observers must mark sections 1 - Name, etc. 3 - Date of Birth 5 - Name 6 - Gender 7a - CAPA Level 7b - Grade 11 - Student ID 12 - SSID Preparing To Administer the CAPA

19. Quiz Question 2 How many students should have a second rater/observer complete an answer document for an elementary school (K-6) with 47 CAPA students (1 Level I, 26 Level II, and 20 Level III)? a. 47 b. 3 c. 5 d. 6 2011 STAR CAPA Train-the-Trainer Workshop || Slide 19 January 2011

20. Quiz Question 2 How many students should have a second rater/observer complete an answer document for an elementary school (K-6) with 47 CAPA students (1 Level I, 26 Level II, and 20 Level III)? d. 6 2011 STAR CAPA Train-the-Trainer Workshop || Slide 20 January 2011

21. Preparing Answer Documents: CAPA Level 2011 STAR CAPA Train-the-Trainer Workshop || Slide 21 January 2011 Mark correct level Not counted if out of level H 4

22. Preparing: CAPA Answer Documents Race and Ethnicity 2011 STAR CAPA Train-the-Trainer Workshop || Slide 22 January 2011 Sections 8 and 9, “Ethnicity/Race” Make sure that you mark only one circle in Section 8. Make sure you mark at least one circle in Section 9. H 4 Q 3

23. Preparing: CAPA Answer Circles Levels II–V Red-line #5 answer circles so examiners do not mark option 5. If anything other than red pen used, the marks could be picked up by the scoring machine  If no other answers are marked, the test could be considered to be for Level I 2011 STAR CAPA Train-the-Trainer Workshop || Slide 23 January 2011 H 4

24. Quiz Question 3 What should an examiner mark on a non–pre-identified answer document for Pu, who identifies herself as Chinese, in sections 8 and 9?18 a. Nothing in 8, nothing in 9 b. Yes in 8, White in 9 c. Yes in 8, Chinese in 9 d. No in 8, Chinese in 9 2011 STAR CAPA Train-the-Trainer Workshop || Slide 24 January 2011

25. Quiz Question 3 What should an examiner mark on a non–pre-identified answer document for Pu, who identifies herself as Chinese, in sections 8 and 9?18 d. No in 8, Chinese in 9 2011 STAR CAPA Train-the-Trainer Workshop || Slide 25 January 2011

26. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 26 January 2011 Preparing Materials Read Examiner’s Manual Gather manipulatives for correct version Cut stimulus cards and have them available in the correct order  To obtain stimulus cards online, contact STAR TAC Have an answer document for every student If no Pre-ID, demographics must be hand-marked Prepare adaptations  For students with additional disabilities  Must be same as in classroom H 4

27. BREAK 2011 STAR CAPA Train-the-Trainer Workshop || Slide 27 January 2011

28. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 28 January 2011 Administering the CAPA Task preparation  Set up manipulatives and/or stimulus cards  Must be in exact order specified (left to right and/or top to bottom) Cue/directions  Must be read exactly as printed H 6

29. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 29 January 2011 Administering CAPA Level I Up to 5 cues given Up to 3 prompts given If no response after verbal/gestural prompt, then modeled prompt only If no response after modeled prompt, then hand- over-hand Student must answer independently after hand- over-hand or any prompt Q 4, 5, 6 H 6, 7

30. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 30 January 2011 Cue/Directions Level I 1.Give cue/directions  Wait minimum of 5 seconds for verbal response; Wait minimum of 7 seconds for physical response  If student responds (correct or not), STOP. 2.Repeat cue/directions ONCE  Wait minimum of 5 seconds for verbal response; Wait minimum of 7 seconds for physical response  If student responds (correct or not), STOP. H 6

31. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 31 January 2011 Cue/Directions: Level I Prompts 3. Give a verbal/gestural prompt, repeat cue/directions  Wait  If student responds (correct or not), STOP. 4. Give a modeled prompt, reset objects, repeat cue/directions  Wait  If student responds (correct or not) respond, STOP. 5. Use hand-over-hand prompt, reset objects, repeat cue/directions  Wait  If student responds (correct or not) or does not respond, STOP. H 6 Q 6

32. Video: CAPA Level I Scoring 2011 STAR CAPA Train-the-Trainer Workshop || Slide 32 January 2011 [explaining Level I video]

33. Quiz Question 4 What is the maximum number of cues for each task that an examiner can give to a student testing at CAPA Level I? a. 5 b. 2 c. As many as the student needs to get correct d. As many as the student needs to respond 2011 STAR CAPA Train-the-Trainer Workshop || Slide 33 January 2011

34. Quiz Question 4 What is the maximum number of cues for each task that an examiner can give to a student testing at CAPA Level I? a. 5 2011 STAR CAPA Train-the-Trainer Workshop || Slide 34 January 2011

35. Quiz Question 5 What is the maximum number of prompts for each task that an examiner can give to a student testing at CAPA Level I?22 a. As many as the student needs to respond b. 2 c. 3 d. 5 2011 STAR CAPA Train-the-Trainer Workshop || Slide 35 January 2011

36. Quiz Question 5 What is the maximum number of prompts for each task that an examiner can give to a student testing at CAPA Level I? c. 3 2011 STAR CAPA Train-the-Trainer Workshop || Slide 36 January 2011

37. Quiz Question 6 When should an examiner use a hand- over-hand prompt to administer CAPA Level I? a. If the student does not respond to the first cue b. If the student does not respond to the second cue c. If the student does not respond to a gestural prompt d. If the student does not respond to a modeled prompt 2011 STAR CAPA Train-the-Trainer Workshop || Slide 37 January 2011

38. Quiz Question 6 When should an examiner use a hand- over-hand prompt to administer CAPA Level I? d. If the student does not respond to a modeled prompt 2011 STAR CAPA Train-the-Trainer Workshop || Slide 38 January 2011

39. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 39 January 2011 Practice Use the chart on page 7 of the handout. Work with a partner and explain to one another how to administer CAPA Level I. H 6

40. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 40 January 2011 Administering CAPA Levels II–V Give cue Repeat cue only once, if necessary NO prompts are given Q 7 H 8

41. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 41 January 2011 Cue/Directions Levels II–V Cue/direction  Wait minimum 5 seconds for verbal response; Wait minimum 7 seconds for physical response  If student responds (correct or not), STOP. Repeat cue/direction  Wait minimum 5 seconds for verbal response; Wait minimum 7 seconds for physical response  STOP Go to next task in question H 9

42. Video: CAPA Level II Scoring 2011 STAR CAPA Train-the-Trainer Workshop || Slide 42 January 2011 [explaining levels II-V video]

43. Quiz Question 7 What is the maximum number of cues for each question within a task that an examiner can give to a student testing at CAPA Levels II–V? a. 1 b. 2 c. As many as the student needs to get it correct d. As many as the student needs to respond 2011 STAR CAPA Train-the-Trainer Workshop || Slide 43 January 2011

44. Quiz Question 7 What is the maximum number of cues for each question within a task that an examiner can give to a student testing at CAPA Levels II–V? b. 2 2011 STAR CAPA Train-the-Trainer Workshop || Slide 44 January 2011

45. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 45 January 2011 Practice Use the chart on page 8 of the handout. Work with a partner and explain to one another how to administer CAPA Levels II–V. What are the differences between administering CAPA Level I and CAPA Levels II–V? H 8

46. 2010 STAR CAPA Train-the-Trainer Workshop || Slide 46 January 2011 Scoring: Rubrics Scoring points Level I : Based on prompting required for student to complete task  Score 1–5 Levels II–V: Based on number of correct responses  Score 1–4  Red-line score of 5 on answer document H 9

47. 2010 STAR CAPA Train-the-Trainer Workshop || Slide 47 January 2011 Scoring CAPA Level I If wrong answer given, score 1 Correct answer with first or second cue, score 5 Correct with verbal/gestural prompt, score 4 Correct with modeled prompt, score 3 Correct with hand-over-hand prompt, score 2 Orients to task, score 1 No response at any time, score NR Q 8, 9 H 9

48. Quiz Question 8 If the student does not respond after a hand-over-hand prompt and does not orient to the task, what is the score? a. 5 b. 2 c. 1 d. NR 2011 STAR CAPA Train-the-Trainer Workshop || Slide 48 January 2011

49. Quiz Question 8 If the student does not respond after a hand-over-hand prompt and does not orient to the task, what is the score? d. NR 2011 STAR CAPA Train-the-Trainer Workshop || Slide 49 January 2011

50. Quiz Question 9 If the student gives the correct answer after a hand-over-hand prompt, what is the score? a. 3 b. 2 c. 1 d. NR 2011 STAR CAPA Train-the-Trainer Workshop || Slide 50 January 2011

51. Quiz Question 9 If the student gives the correct answer after a hand-over-hand prompt, what is the score? b. 2 2011 STAR CAPA Train-the-Trainer Workshop || Slide 51 January 2011

52. Video: CAPA Level I Science 2011 STAR CAPA Train-the-Trainer Workshop || Slide 52 January 2011 [video]

53. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 53 January 2011 Practice Scoring CAPA Level I See trainer model with Lucerio video Using the video clip, Jasper, ELA (apple and shoe)  Plan a practice session on scoring  Give directions for the practice to a partner  Give feedback on how the partner scored Jasper H 9−11

54. Video: CAPA Level I English 2011 STAR CAPA Train-the-Trainer Workshop || Slide 54 January 2011 [video]

55. Scoring Levels II–V Rubric unique to task  Provided with each task Based on number of questions within the task (cue/directions) Example: ELA, Level III: Read a given story; ask the 4 questions about the story.  Scoring Rubric: (4) Completes task with 100% accuracy (3) Partially completes task (2–3 correct) (2) Minimally completes task (1 correct) (1) Attempts task 2011 STAR CAPA Train-the-Trainer Workshop || Slide 55 January 2011 H 12

56. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 56 January 2011 Practice CAPA Levels III and V See trainer model with mathematics, Level III, Steven video Using the video clip, science, Level V, Luis  Plan a practice session on scoring  Give directions for the practice to a partner  Give feedback to how the partner scored Luis H 12−14

57. Video: CAPA Level III Math 2011 STAR CAPA Train-the-Trainer Workshop || Slide 57 January 2011 [video]

58. Video – CAPA Level V Science 2011 STAR CAPA Train-the-Trainer Workshop || Slide 58 January 2011 [video]

59. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 59 January 2011 After Test Procedures: Answer Document Section A2 Special Conditions AThe student was absent for regular and makeup testing. EThe student was not tested because of a significant medical emergency. FThe student became ill while taking the test, had to leave the room, and was unable to return and complete the test. LThe student enrolled after first day of testing and was given this test. MThe student was enrolled on first day of testing but moved before this test was administered or completed. PThe student was not tested by parent/guardian request. TStudent enrolled on the first day of testing and was not given this test because a staff member verified that the student had taken the test at a previous school. H 15

60. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 60 January 2011 After-test Procedures Section A4—English Learner Test Variations  A: The student was given directions in a language other than English. Complete name, sign, date, and mark “Examiner” or “Observer” to attest training received Q 10 H 15

61. Quiz Question 10 Who signs a completed student answer document? a. Examiner b. Examiner and/or Observer c. Observer d. Site Coordinator 2011 STAR CAPA Train-the-Trainer Workshop || Slide 61 January 2011

62. Quiz Question 10 Who signs a completed student answer document? b. Examiner and/or Observer 2011 STAR CAPA Train-the-Trainer Workshop || Slide 62 January 2011

63. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 63 January 2011 Questions and Answers

64. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 64 January 2011 Summary Training Standards Preparation  Selecting CAPA students  Selecting CAPA examiners  Observers  Answer documents  Materials Administration Scoring After-test procedures Complete evaluation

65. 2011 STAR CAPA Train-the-Trainer Workshop || Slide 65 January 2011 For more information go to: http://www.cde.ca.gov/ta/tg/sr/capa.asp http://www.startest.org/capa.html To access online stimulus cards, contact STAR TAC: 800-955-2954 or star @ets.org