Administering the ACCESS for ELLs® Listening, Reading, and Writing Tests In this training module participants will receive a comprehensive orientation to the organization of the ACCESS for ELLs® test booklet and administration procedures for the Listening, Reading, and Writing components. Participants will walk through test administration logistics by discussing sample group test session schedules. They will also be introduced to the materials, such as test administrator scripts and practice test items for each of the components from different theme folders in the three tiers (A, B, and C). In viewing sample test items, participants will begin to understand the tasks students encounter on the ACCESS for ELLs® test. Emily Evans, Center for Applied Linguistics January 2007 New Jersey Department of Education Developed by the Center for Applied Linguistics

Training Objectives To understand the organization of the ACCESS for ELLs® test booklet To understand administration requirements of the Listening, Reading and Writing components of the ACCESS for ELLs®

Outline General Information Test Administration Overview Listening Test Reading Test Writing Test

General Information All test sessions must take place within a state’s testing window. NJ’s testing window is April 16th – June 1st, 2007 Tests are secure and you must follow state test security regulations. The Test Administration Manual contains guidance on test accommodations for students with disabilities. The Test Administration (TA) Manual contains detailed information on Test Administration. This slide contains a few general points. The list of acceptable accommodations for students with disabilities can be found in the TA Manual. Your school must administer the test within your state’s testing window. Security is very important so make sure to follow those regulations carefully. Specific information about security can be found in the Test Security Training presentation.

Test Administration Overview Listening, Reading and Writing Administered in groups of up to 20 students Centrally scored by MetriTech, not by the Test Administrator Each grade level cluster and each tier must have separate group sessions The administrator’s scripts arrive with the test booklets Scripts are different for each test form Listening, Reading and Writing tests have scripted instructions in the Test Administrators Script which parallels the test booklet. Make sure to follow the scripts exactly. When scheduling test sessions, be sure to schedule separate sessions for each test form. For example, 35A is a different test than 35B. The scripts and the test forms will be different, so you must administer them in separate sessions. Do not mix tiers or grade levels in a testing session. Scripts are specific to the test form so they are not the same for every Tier (Tiers refer to A B or C) Note: There is a separate PowerPoint presentation explaining the administration procedures for the Speaking portion of the ACCESS for ELLs® test.

Test Booklet Organization Test booklet order: 1) Listening 2) Reading 3) Writing Each test will begin with practice items or a model Tests are organized by theme folders – A series of questions about one topic with graphic and/or text support Each theme folder for Listening and Reading contains 3-5 items (test questions) Each theme folder for Writing contains 1-4 items per theme folder in Tier A for all grade levels and form 1-2B. The theme folders for the rest of the forms contain one item per theme folder. There are practice questions or a model at the beginning of each test. Questions in the Listening and Reading sections are organized into theme folders which contain sets of 3-5 questions on one topic/theme with graphic and/or text support. On the Writing test, more questions are given for Tier A for all grade levels and form 1-2B to provide extra support for the writing.

Test Administration Times Listening and Reading are administered together in one group session Listening Administration: 20-25 minutes Break in between Listening and Reading sections: 5 minutes Reading Administration: 35-40 minutes Logistics: 15 minutes (approximately) (including break in between sections) Total: 75 minutes (approximately) Writing is administered in a separate group session Writing Administration: 60 minutes 15-20 students per group session The listening and reading sections are administered in one session. Together they will take approximately 75 minutes when logistical considerations are factored in such as taking attendance or distributing and collecting test materials. Use your own judgment with the break in between the Listening and Reading sections. Note also that the length of the oral passages gets longer with increasing proficiency; thus, the Tier C Listening Test takes longer to administer than the Tier A Listening Test. The writing section is administered in a separate group session and will also take approximately 75 minutes when logistical considerations are factored in. If you think it is appropriate or necessary to schedule both sessions listening/reading and writing on the same day, make sure that an adequate break is provided in between sessions. Generally schools administer the L/R and Writing tests on separate days. An example of how to schedule a group of 90 students is given on the next slide.

Scheduling Guidelines Example Assume there are 90 students in the same grade level cluster with 30 students per Tier tested 15 at a time You can test all students in both Listening/Reading and Writing in a total of 12 sessions over 4 days This example assumes there is a Single Test Administrator in the school Number of test takers per time block This slide shows an example of how test sessions could be scheduled for a group of 90 students that has 30 students per tier. This is just an example to give you an idea of how scheduling might take place. There are many other options depending on how many students and administrators your school has. The testing schedule is determined by the School Testing Coordinator. In this example, Tier A, B and C are each split in half and tested on separate days. The first group of 45 students (15 from Tier A, 15 from Tier B, and 15 from Tier C) are tested on Monday and Tuesday. The second group of 45 students are tested on Wednesday and Thursday. Again, two tiers (ex tier A and tier B) can not be tested at the same time. Similarly 35A and 68A can not be tested in the same session. The scripts are different for each tier and for each grade level. The example on this slide assumes there is a single Test Administrator in the school, so there would be no simultaneous sessions. If the school had multiple test administrators, simultaneous sessions (in different rooms) could be scheduled depending on the specific situation.

How to read the Script Read all of the text in Bold aloud example: “Part A: Classroom Objects” The text which is not bolded serves as a guideline and should not be read aloud example: “Pause”, “… make sure all students are in the right place.” This page is an excerpt from the beginning of the Test Administrator’s Script that you will receive.

How to read the Script (con’t) The test item is in Blue and should be read aloud

Listening Test Overview Format: Multiple choice Time: 20-25 minutes Scoring: Machine scored (by MetriTech) Weight: 30% of comprehension score 15% of composite score 6-7 thematic folders with at least one from each standard (Math, Science, Language Arts, Social Studies, Social Instructional) Test items read aloud only one time The listening test includes questions from all English Language Proficiency (ELP) standards: Math (MA), Science (SC), Language Arts (LA), Social Studies (SS), and Social and Instructional Language (SI). Only listening and reading factor into the Comprehension score. Listening is worth 30% of the comprehension score and Reading is worth 70%. The composite score includes all domains (Listening, Reading, Writing and Speaking). More information on the weights of each test can be found in the WIDA Background PowerPoint presentation. You cannot read the answer choices out loud. Under normal circumstances, you shall read each item only once. The exception is if there is an interruption while you are reading an item, such as an announcement over the loudspeaker, a fire drill, or a loud noise outside an open window. In that case, the item may be repeated one time only, for a total of two complete readings of the item.

Listening Test Overview (con’t) Keep the test going at a steady pace Follow pause times given in the script After an item is read, give students up to 25 seconds to mark their answers Follow the Test Administration Script exactly Give students up to 25 seconds to mark their answers or until the last student is finished, whichever comes first. Do not pause longer that 25 seconds. The D2L course has sound files where you can listen to sample items being read. This will help you understand the pacing of the test better.

Listening Test: Introduction The beginning of the listening test starts with a scripted introduction. Read the bolded text, but not the plain text

Listening Test: Practice Problems Occur at the beginning of the test, after the introduction, and are scripted Must be administered Help students understand the structure of the test

Listening Test: Practice Problems This is an example of how a practice problem is structured.

Listening Test: Practice Problems Script: First the boy and girl play catch, and then they ride their bikes. Find the set of pictures that shows this in the right order. PAUSE. This is the first question from the practice section with the picture the student sees.

Listening Item Example: Science 1-2 A What you see on the screen is an example of a theme folder. This theme folder has three questions. Notice there is a theme graphic at the beginning of the folder of a classroom with growing plants which sets the context for the three questions. This grounds the question, but may not be necessary to answer all of the questions. In the example on this slide all of the response options are graphics. The test specifications dictate whether the options are text and/or graphics. In some cases, of course, the response options will contain text instead of or in addition to graphics. This folder is an example from Tier A, so it covers proficiency levels 1, 2, and 3 (P1, P2, P3).

Listening Example Levels 1 and 2 - Scripts Script: A seed is small. Find the small seed. Script: One day the seed will grow into something large, round, and heavy. Find what the seed grows into. These are two questions from the previous theme folder. The script is listed above the question so you can see how the script relates to what the student is seeing. On the actual test, students would not see the script. These questions are at a proficiency level 1 and 2 for grades 1-2.

Reading Test Overview Format: Multiple choice Time: 35-40 minutes Scoring: Machine scored (by MetriTech) Weight: 70% of comprehension score 35% of composite score 6-7 thematic folders with at least one from each standard Follow the Test Administration Script exactly This is the reading test overview. The reading test should be given following the listening test. The reading test may begin with some practice problems which are scripted. During the reading test you should follow the script. Answer choices may not be read out loud. The reading test includes questions from all English Language Proficiency (ELP) standards: Math (MA), Science (SC), Language Arts (LA), Social Studies (SS), and Social and Instructional Language (SI).

Reading Test Scripting Requirements Begins with an Introduction

Reading Practice Problems Practice problems follow the introduction and must be administered. In some cases, the correct answer to a practice problem will be filled in.

Sample Items Within a Reading Test Only on forms 1-2A and 1-2B 1-2A occur after every folder and are scripted (the answer is filled in together as a group) 1-2B occur after some folders and are not scripted Purpose: Extra assistance for the student The exception: The listening test has one folder that has to do with patterns which includes a sample item. Otherwise, sample items are only on form 1-2A and 1-2B reading. On the 1-2A test sample problems occur after every folder as a group check-in and they are mandatory. The whole class fills in the correct answer together and the question is scripted.

Individual and Group Check-ins Intended to help guide students though parts of the test and are indicated with a stop sign Whole-group check-ins – Check in with all of the students at the same time and explain the next part of the test. Occurs after every section in the 1-2A Reading Test. Individual check-ins – Students raise their hand when they arrive at a stop sign and you check to see the student has completed the section. Invite them to respond to any skipped questions. Halfway through 1-2B, 1-2C, 3-5A, 3-5B, 6-8A, 9-12A At the end of every tier and grade level The reading section has check-ins. Whole group check-ins only occur on the 1-2A Reading test. As you walk around the room and monitor the students’ work, check to make sure that each question has been answered. Do not check whether responses are correct or not. If any responses are missing, point them out to the students and invite them to try again. When students have finished, check their work. Again, if any responses are missing, point them out to the students and invite them to try again.

Reading Item Example: Math 6-8 B This is an example of a graphics and text stimulus for a Theme Folder entitled Multicultural Club. Each Theme Folder will have graphic and/or text support. This is for grade level cluster 6-8 Tier B in the math content area.

Reading Item Example: Math 6-8 B These are two examples of questions based on the previous graphics and text stimulus for the Theme Folder.

Writing Test Overview Format: Student constructed responses Time: Up to 1 hour A break can be given halfway through the test if necessary Scoring: Rater scored (by MetriTech) Weight: 35% of composite score The following standards are covered: Social Instructional (SI), Math (MA), Science (SC) On Tier B and C: Integrated Task (IT) - combines Language Arts (LA), Social Studies (SS), and Social Instructional (SI) Writing tasks are modeled to scaffold the student Follow the Test Administration Script exactly In this section students produce a sample of their writing. Students are given prompts and sometimes the writing the student needs to do is modeled. Tier A only has three tasks SI, MA, and SC. Tiers B and C have an additional Integrated Task (IT) The writing section is 35% of the overall composite score but does not factor into the comprehension composite score. The writing section is administered in a separate group session and will also take approximately 75 minutes when logistical considerations are factored in.

Writing test organization Tier A 3 folders covering Social Instructional (SI), Math (MA), Science (SC) standards. About 15 minutes for each section. *Form 1-2A only has SI standards and there are 4 folders. Tier B and C 3 folders (about 10 minutes each) covering SI, MA and SC standards. *Form 1-2B doesn’t cover the SC standard. Fourth folder is an IT standard combining LA, SS, and SI standards (student is given about 30 minutes). A folder may contain several questions, but they will be on the same topic. This is why it is called a folder.

Individual and Group Check-ins Whole-group check-ins – used to guide students through the model in Tier A for all grade levels and in all Tiers for grades 1-2 Individual check-ins – Students raise their hand when they arrive at a stop sign which occurs at the end of every writing question. Check to make sure that they have completed the writing task. After 10 minutes, 20 minutes, and 30 minutes, circulate in the room to monitor students’ progress. If necessary, you can prompt students who are lagging behind by saying, “Make sure you save enough time for the other parts.”

Writing Item Example: IT 6-8 C This is the theme folder for this integrated writing task for 6-8 tier C. Remember an IT task combines Language Arts (LA), Social Studies (SS), and Social Instructional (SI) standards. The student is given background text and graphics to stimulate writing in the LA, SS and SI standards. Part 1 gives some questions for the student to think about, and Part 2 models the structure of the writing and helps the student plan their composition. Part 3 reiterates the structure of the composition and gives the writing assignment. In the test development process, the writing questions are designed to elicit responses in line with the writing rubric.

Writing Performance Description Writing Rubric Level Writing Performance Description 6 Reaching A variety of sentence lengths of varying linguistic complexity in a single tightly organized paragraph or in well-organized extended text; tight cohesion and organization Consistent use of just the right word in just the right place; precise vocabulary usage in general, specific, or technical language Has reached comparability to that of English proficient peers functioning at the “proficient” level in state-wide assessments 5 Bridging A variety of sentence lengths of varying linguistic complexity in a single organized paragraph or in extended text; cohesion and organization Usage of technical language related to the content area; evident facility with needed vocabulary Approaching comparability to that of English proficient peers; errors don’t impede comprehensibility 4 Expanding A variety of sentence lengths of varying linguistic complexity; emerging cohesion used to provide detail and clarity Usage of specific and some technical language related to the content area; lack of needed vocabulary may be occasionally evident Generally comprehensible at all times, errors don’t impede the overall meaning; such errors may reflect first language interference This is the rubric used for scoring writing items. *** The descriptions above pertain to the original text the student has produced, as opposed to the copied or adapted text, unless indicated. Generally, of the three features, give the first feature more weight than the other two because it is a more comprehensive characteristic of writing than the other two. Do not penalize the mechanical errors such as spelling as long as the comprehension is not impeded. Usually, more than two of the three features should fall into the description of one level. If the three components are spread over more than three consecutive levels, choose the middle level at your discretion.

Writing Rubric (con’t) Level Writing Performance Description 3 Developing Simple and expanded sentences that show emerging complexity used to provide detail Usage of general and some specific language related to the content area; lack of needed vocabulary may be evident Generally comprehensible when writing in sentences; comprehensibility may from time to time be impeded by errors when attempting to produce more complex text 2 Beginning Phrases and short sentences; varying amount of text may be copied or adapted; some attempt at organization may be evidenced Usage of general language related to the content area; lack of vocabulary may be evident Generally comprehensible when text is adapted from model or source text, or when original text is limited to simple text; comprehensibility may be often impeded by errors 1 Entering Single words, set phrases, or chunks of simple language; varying amounts of text may be copied or adapted; adapted text contains original language Usage of highest frequency vocabulary from school setting and content areas Generally comprehensible when text is copied or adapted from model or source text; comprehensibility may be significantly impeded in original text *** The descriptions above pertain to the original text the student has produced, as opposed to the copied or adapted text, unless indicated. Generally, of the three features, give the first feature more weight than the other two because it is a more comprehensive characteristic of writing than the other two. Do not penalize the mechanical errors such as spelling as long as the comprehension is not impeded. Usually, more than two of the three features should fall into the description of one level. If the three components are spread over more than three consecutive levels, choose the middle level at your discretion.

Scoring an IT Writing item Rater scored by MetriTech using a rubric The preparing and planning sections are not scored. Planning sections not scored by MetriTech Make sure the students are spending enough time on the actual composition because that is the only thing scored by MetriTech.

Example of Modeled Writing Item Math 6-8Tier A Look at the sentence in the black box. Point to it in your test booklet. Scan the room and make sure all students are in the right place. It says, "For a math assignment, Andrei needs to find the area of a triangle.” Look at the diagram. The triangle is labeled with the Base and the Height. The formula for the area of a triangle is given as Area equals Base times Height times one half. Point to picture of triangle and formula in the book as you read it. Look at the next black box. It says “Andrei answered two questions about the math assignment.” What do the first question and answer say? Allow time for student response, or say: It says, “What is the name of the shape in the picture? Triangle.” What do the second question and answer say? It says, “What do you need to do to find the area of the shape? I need to multiply the base times the height times one half.” The script is on the right hand side of the slide to give you an idea of what it will look like.

Example of Modeled Writing Item (con’t) Math 6-8Tier A Look at the last black box. It says, “The pictures show the steps to follow to find the area. Andrei explained each step in writing.” What did Andrei write for step 1? Allow time for student response, or say: First, I have to measure the length of the base. The base is six inches long. We can’t read everything Andrei wrote for step 2, only the beginning. What do you think he wrote for step 2? Allow time for student response, or say: Maybe he wrote something like, “Then I have to measure the height of the triangle. The height is four inches.” Step 3 shows us the formula again, and the math Andrei did to find the area of the triangle. On the lines we see the beginning of his answer. How did he begin? He began, “Now I have to use the formula to find the area of the triangle. I multiplied…” Go to the top of the next page. It says, “Now it’s your turn to write!” Point to this in your test booklet. Scan the room and make sure all students are in the right place.

Example of Writing Item Task Now look at the directions in the black box. They say, “Now you have to find the area of the rectangle.” Find number 1 and look at the directions. You need to answer the questions in the box. You can use Andrei’s writing as a model for your own. You can begin in a moment, but first find number 2. The pictures show you how to find the area of a rectangle. On the lines you need to write at least 1 sentence to explain each step in the process. Again, remember that you can use Andrei’s writing as a model for your own. Before you begin, are there any questions? Answer any questions. Now you can begin writing on this page. You can have up to 15 minutes to do this part. Remember to stop when you get to the stop sign. Monitor the students’ work. When students have finished, remind them to check their work. If a student finishes early and wants to go back to complete Part A or Part B, he or she may do so. This part is over when all the students have finished or 15 minutes have passed, whichever comes first. (If 15 minutes have passed and some students are still working, say: If you are not finished, take a few more seconds to finish up this part. Pause 15 seconds.) This is the question that follows the model on the previous slide. The ending of the test has the following script: “End the testing session by saying: This is the end of the Writing test. Please close your test booklets and I will come around and collect them. “

When the Test Administration is Complete Collect test materials This happens after the Listening/Reading sections AND the Writing section Follow test security policies After the test is collected you must follow the test security policies for submitting the materials. More information on test security can be found in the Test Security Training PowerPoint presentation.

Questions or Comments? For more information, please contact the WIDA Hotline: 1-866-276-7735 or www.wida.us/helpform World Class Instructional Design and Assessment, www.wida.us Center for Applied Linguistics, www.cal.org Metritech, Inc., www.metritech.com