1 Illinois Alternate Assessment Directors Conference July 31, 2008

2 Misc. Assessment Update Vacancy 2% flexibility Accommodations Guidelines

3 IAA Agenda Overview Where weve been What weve learned Adjustments Scoring Validation Calendar Resources

4 IAA Life Cycle to Date: Fall 2006 Writing Pilot February 2007 Portfolios R, M, S Spring 2007 Writing Operational Fall 2007 Pilot R, M, S Spring 2008 Operational for R, M, S, W 2009 adds Gr. 3 Writing to complete operational test

5 ISBE IAA 2008 Survey May, 2008, an electronic survey was sent out to teachers and coordinators using contact information from the Pearson Assessment Network and Scoring System. 345 Teachers & Coordinators participated in the survey

6 Excellent Work, Coordinators!! 92% of teachers knew in advance when the testing window would fall. 94% of teachers reported the IAA coordinator had the online scoring set up and ready for scoring. 95% of teachers said they knew who to contact if there was a problem.

7 Time required for administration: SubjectLess than 15 minutes minutes 11 Reading tasks 28%58% 12 Math tasks 31%55% 8 Science tasks 37%52% 6 Writing tasks 40%46%

8 Concern #1: 51% of teachers attended a training session This raises questions regarding the use of the trainer of trainers model. The accuracy of test administration and scoring may have been impacted.

9 Concern #2: Comparability to regular assessments in terms of content coverage. In states I have worked with, there tend to be more alternate assessment items than (ISBE) has developed. (peer reviewer)

10 Response to the Two Concerns: Potential for Refinement

IAA Training, Part 1: More training sessions will be provided in 2009: Northern Illinois including Chicago January 26-30, 2009 Central and Southern Illinois February 2-6, 2009 Watch the ISBE website for signup opportunities in your area of the state in Fall, 2008.

Training, (Possible) Part 2: To further ensure precise administration and scoring of the tasks, especially for those teachers who have not been trained: Rubric language could be embedded in the teacher administration instructions.

13 Built-In Task Characteristics: Plus: Task-specific texts, symbols, and artwork embedded in the task. The General Prompt embedded in the task (in the event it is needed). All tasks would follow the same rubric template for teacher administration instructions. Flexibility in presentation of tasks and how the student responds would be maintained.

14 Sample Science Task: Primary Task: a dog and a cat a cat and a plant three bushes Which picture shows only mammals? General Prompt: Rabbits are mammals.

15 Sample Draft Teacher Instructions: Level 4: Show the student the primary task and say: Which picture shows only mammals? Present the answer options in order and say a dog and a cat, a cat and a plant, three bushes. The student receives a score of 4 for a correct response. If the student does not respond or responds incorrectly, you may repeat the question exactly as it appears one time only. If the student again chooses an incorrect answer or does not respond, move to Level 3. Level 3: Show the student the general prompt and say: Rabbits are mammals. Re-present the primary task and say: Which picture shows only mammals? Re-present the answer options in order and say a dog and a cat, a cat and a plant, three bushes. The student receives a score of 3 for a correct response. If no response or incorrect response, move to Level 2. Level 2: Indicate the specific prompt and say: Cats and dogs are mammals. Re-present the primary task and say: Which picture shows only mammals? Indicate the choices in order and say: a dog and a cat, a cat and a plant, three bushes. The student receives a score of 2 for a correct response. If no response or incorrect response, move to Level 1. Level 1: If no response or an incorrect response after providing level 2 support, the student receives a score of 1. Correct Answer: a dog and a cat

16 Build a Better IAA by: Extending content coverage requires increasing the quantity of items.

IAA Operational Test ReadingMathScienceWriting 9 tasks* 2 FT Total: tasks* 2 FT Total: 12 6 tasks* 2 FT Total: 8 5 tasks* 1 FT Total: 6 * Required tasks covering identified priorities. FT – field test task (pilot) for future tests

18 Sample Content Coverage Plan: Each Standard assessed with: At least 1 task for each objective 2 tasks for each identified high priority objective (2 priority objectives at each subjects grade level). 4 objectives always assessed (tasks assessed annually as anchor items)

19 Using the previous content coverage plan, the 2009 Operational Test would look like: ReadingMathScienceWriting 17 tasks* 4 FT Total: tasks* 4 FT Total: tasks* 4 FT Total: 23 5 tasks 1 FT Total: 6 * Required tasks covering all objectives in each subject. FT – field test task (pilot) for future tests

20 Some Things Never Change Assessment frameworks remain the same. IAA Participation Guidelines remain the same. Use of mode of communication specific to the individual student remains the same. Options in types of student responses remains the same. Score submission online remains the same. Number of Writing tasks remains the same. Content coverage is expanded, not changed, in Reading, Math, and Science. Training will be provided on administration of the tasks.

21 Potential Results? Refinements should: Enhance content coverage. Improve likelihood that the IAA will pass peer review. Avoid rejection of the IAA by the federal agency.

22 August 4-5, 2008 To enhance the quality and quantity of the IAA performance tasks: Item Writers will be working on Reading Math Science Number of Writing tasks is sufficient at this time.

23 IAA Scoring Validation Has Not Changed Recruiting more Coordinators to serve as: Second scorers Super raters If interested, contact Pearl at 217/ or

24 IAA 2009 Proposed Calendar Jan & Feb. 2-6, 2009 IAA training sessions throughout the state Feb. 17, 2009 Tentative date for delivery of IAA test materials to schools/districts February, 2009 Online scoring system IDs & passwords distributed March 9-27, 2009 – Regular test window

25 IAA Online Resources Implementation Manual Online Scoring User Guides Frameworks Scoring Rubric RCDTS codes Accommodations Guidelines

26 Pearson Contact Information Pearsons Assessment Network Pearson Customer Support 800/ State Code 814 Pearson Online Scoring Questions 888/

27 ISBE Contact Information Pearl Schneider, IAA Coordinator Phone: 217/