Assessment Policy Updates and Reminders Welcome to the Assessment Policy Updates and Reminders tutorial. This presentation is the next step for adult education test administrators after they have completed the first assessment tutorial, Adult Learner Assessment Policies and Procedures. Program administrators, who are responsible for overseeing assessment practices in their programs and other adult education personnel who are involved in assessments in different ways may find this presentation useful in their roles (e.g., data entry, maintaining assessment records, and providing assessment feedback to students). Technical College System of Georgia Office of Adult Education September 6, 2017

Objectives Review the most important components of quality test administration Provide a reinforcement of policies and procedures Give practical guidance for common assessment issues The objectives of this tutorial is to review the most important components of quality test administration that can sometimes become lax over time. We will also provide some reminders of key assessment policies and procedures. Practical guidance for common assessment issues will be included throughout the presentation.

Overview of Resources OAE Adult Learner Assessment Policies and Procedures Manual Updated each fiscal year Required by our federal grant Test Publishers Training sessions, test manuals, student diagnostics, and technical assistance Local Assessment Training Local Assessment Experts Before we get started, let’s review the resources are that available to you as you develop your knowledge of the assessment policies and overall best practices for giving assessments. The official policies and procedures manual that is posted with this presentation should be your first stop when you have a question about the rules around giving assessments. This document is updated each year. The contents of the manual are required as a part of our federal grant funding. The Assessment Policy Manual does not try to duplicate test administration manuals. Test Publishers provide these materials for each of their assessments, such as test manuals, training videos, scoring guides, student diagnostics, and technical assistance. Local programs have the responsibility for coordinating or providing the training that employees need to administer assessments. Systems should be in place to ensure that training is provided to new hires and that annual training is given to everyone who administers a particular assessment. Most local programs have seasoned employees who have been administering assessments for years. These colleagues can be some of the best resources related to local testing procedures and what to do when issues arise.

Setting the Testing Environment The testing environment should be clean, comfortable, and free of distractions. Gather all materials and set up the testing room. Post signs on doors or near the testing area. In order to ensure that students are able to perform their best on assessments, programs must consistently provide a clean, comfortable, and quiet testing environment. It is imperative that students are not distracted by loud noises, classroom instruction, or other students. Student assessments should be conducted in a space that has been set aside for testing. Examinees should have enough space between themselves and other examinees. Make sure you have everything you will need for the testing session – testing materials, sign-in sheet, timers, calculators, etc. The doors going into the room should be closed during testing and notices posted outside to remind others to remain as quiet as possible in or near the testing area.

Setting the Testing Environment Pre-testing and Post-testing environments must meet the same standards. Testing students in an active instructional classroom in not acceptable. Examine the assessment practices in your classroom or site (for both pre and post-testing) and work with program leadership to make changes, if needed. Adult Education programs usually do really well with the testing environment for pre-tests, but then they often undervalue the importance of the testing environment for post-tests. This can lead to making compromises for the sake of convenience. A program’s pre-testing and post-testing environments must meet the same standards. Testing students in an active classroom, even if other students are engaged only in independent study, is not acceptable. Students are often moving around, asking questions of the teacher, or doing other activities such as signing in, getting work folders, or retrieving materials from shelves or cabinets. This is not a testing environment that is conducive to successful outcomes. If you have only one classroom space available, try to schedule assessments before or after your normal class hours a couple of times a week to create a quiet testing environment. If your program has had difficulty meeting measurable skill gains goals, one factor could be the program’s post-testing environment. Take the time to examine testing in your classroom or at your site to see if it meets the standard of a good testing environment. Be sure to check both pre-testing and post-testing experiences. Bring any concerns to the appropriate person and work with program leadership to make needed changes.

Testing Environment Check-up How would you rate the pre-testing environment in your classroom or site? How would you rate the post-testing environment in your classroom or site? Rating Scale Excellent Good OK Fair Poor Take a few minutes to complete these questions on your participant worksheet. What changes would improve the pre- and post-testing environments in your classroom or site?

Administering Assessments Proctored by a trained test administrator Follow Test Publishers’ guidelines Always time an assessment according to testing guidelines Review all requests for Special Accommodations before the session TABE Online must be used for both pre and post testing One of the most important components of good testing is to always make sure that a test is proctored by a trained test administrator who follows the test publishers’ guidelines. Training helps ensure that all assessments given within the adult education program are given in the same way and under the same conditions. Most assessments have time limits that must be enforced. Disregarding the time limits of an assessment makes that test invalid. Kitchen timers are an inexpensive and easy way to keep up with testing times, especially if you are assessing a group of students who are taking tests with different time limits. Individuals with disabilities may be eligible for testing accommodations. In order to be considered, they must disclose their disability to the designated program staff member, request accommodation(s), and submit documentation of their disability. All requests for Special Accommodations should be reviewed before the testing session begins. Starting in FY18, Georgia has a statewide contract with TABE Online. Local programs that administer TABE should administer the Online version for both pre- and post-testing. Exceptions to this (such as a correctional setting) must be approved by OAE through each program’s assigned GPS Coordinator.

Documentation and Accommodations Students must submit documentation of disability, such as: Doctor’s Report Certified Diagnostic Assessment Individual Education Plan (IEP) Other clinical records Accommodations may include, but are not limited to: Flexible scheduling and setting Flexible presentation and responding Assistive devices Each assessment provides guidance on acceptable accommodations. Since we’ve brought up accommodations, some examples of acceptable forms of documentation include: a doctor’s report, a certified diagnostic assessment from someone such as a psychologist or other professional, an Individual Education Plan (IEP) that was utilized during their K-12 education, or other clinical records. Some acceptable accommodations include: flexible scheduling and setting, flexible presentation, flexible responding, and assistive devices. Each of these accommodations are described in more detail on pages 9 and 10 of the assessment policy manual and include guidance regarding acceptable accommodations.

Post-testing Timeframes Assessment Timeframe BEST Literacy After 60 hours of instruction BEST Plus 2.0 TABE 9&10 After 40 hours of instruction (ABE) After 30 hours of instruction (ASE) TABE CLAS-E Here’s a reminder about post-testing timeframes. BEST Literacy, BEST Plus, and TABE CLAS–E post-tests may be administered after 60 hours of instruction. TABE may be administered to ABE students after 40 hours of instruction and ASE students after 30 hours of instruction. Assessments that are administered before reaching the minimum numbers of hours, are blocked from being entered into GALIS. Starting in FY18, all assessments have a validity period of 270 calendar days. That is an increase over the prior 180-day rule. This change makes assessments administered in the early part of the fiscal year valid through the end of the year, thus making the process easier when students return after a long absence. Starting in FY18, all assessments have a validity period of 270 calendar days. That’s an increase from the prior 180-day rule.

Post-Testing Guidance from GPS Coordinators After meeting minimums, programs can be somewhat flexible with post-testing times to meet student needs. Assessment helps students know where they stand and what skills they have yet to master. GPS Coordinators look to see if post-tests were given within 20 hours of meeting the minimum. Seek technical assistance if you have a group of students who accumulate hours very quickly. Our team gets a lot of questions about post-testing. While OAE has set the minimum number of hours needed for a student to be post-tested, that doesn’t mean that the student needs to be tested the minute they meet the minimum. Local programs should work to balance post-testing with class instruction and student learning needs. Post-testing is a retention tool. It lets students know where they stand and the skills they have yet to master. If a student is post-tested only once per year, he/she misses out on valuable feedback. Conversely, if a student is post-tested too frequently, he/she becomes weary of the process and also becomes too familiar with the assessment. When the GPS Coordinators conduct their monthly data reviews of programs, they check if any students have an excessive number of hours without a post-test. Personnel in the field have the impression that one hour over the minimum will be viewed as excessive. It is not. A reasonable expectation for post-testing is to administer it within 20 hours of meeting the minimum. If your program is structured in a way that students are earning 15-20 hours of attendance or more per week, the program administrator should communicate with his/her GPS Coordinator for technical assistance about how these post-testing situations should be managed.

Test Administration Check-up How would you rate the quality of the timing of assessments in your classroom or site? How would you rate your understanding of what to do if a student requests an assessment accommodation? Rating Scale Excellent Good OK Fair Poor Take a few minutes to complete these questions on your participant worksheet. What questions do you have about testing accommodations?

Scoring Assessments Follow the Test Publisher’s Procedures Maintain the integrity of test results at all time Try to distance yourself from your students’ assessments For TABE, use TABE Online or SCOREZE answer sheets (scanned) Trade testing duties with another instructor (BEST Plus) Have another test administrator score the written responses or at least check your scoring (BEST Literacy and TABE CLAS-E) When scoring assessments, it’s important to follow established procedures. It’s only when the same procedures are followed from person to person that we are able to have consistent test validity. Anyone who is involved in student assessments must maintain the highest level of integrity at all times – from the test administrator to the person who enters the scores into GALIS. Try to put some distance between the tests that you administer and the scoring of those assessments. For example, use TABE Online or SCOREZE TABE answer sheets, instead of hand scoring, which puts more distance between the test administrator and the test results. Orally-administered assessments can become subjective if the test administrator is not diligent. One example is BEST Plus. Following the scoring rubric and asking the questions word-for-word, without deviation is key. If your program has enough personnel who give BEST Plus, trade off with another test administrator: you test his students; he tests yours. It can be very hard not to “fill in” the gaps and award higher scores than a student deserves when you have listened to that student in your class for an extended period of time. Another example where familiarity can creep in are the written responses on BEST Literacy and TABE CLAS-E. Try to arrange for someone else to score your students’ responses. If this is impractical, find a second experienced scorer to review the assessments that you have scored to check for consistency with the scoring rubric.

Unethical Behavior in Student Assessment Changing student responses Explaining test items to students before, during, or after testing Testing before the student reaches the minimum number of hours Destroying or withholding assessments that aren’t the results you want Meeting performance measures can be stressful. We’ve all seen stories in the news where teachers violated assessment rules or assisted students in order to increase student test scores. This type of behavior is unethical and it violates the trust that the public has placed with professional educators. Changing student responses; explaining test items to students before, during or after testing; testing students before they reach the minimum number of hours; and destroying or withholding assessments because they aren’t the results you want are all examples of unethical behavior in student assessment. All student testing must be conducted in a fair and ethical manner. All student testing must be conducted in a fair and ethical manner.

Scoring and Ethics Check-up How would you rate the accuracy and impartiality of the tests administered in your classroom or site? Rating Scale Excellent Good OK Fair Poor If you ever witnessed unethical behavior in assessment administration, what would you do? What kind of guidance has your program provided about how to report testing violations? Take a few minutes to complete these questions on your participant worksheet.

Maintenance of Assessment Records Keep all student assessment records in the Student Permanent Record No assessment records in the Student Work Folder Ensure that student recognition efforts still maintain confidentiality Securely maintaining assessment records is an important part of your assessment program. Hard copies of all assessment records should be maintained in the student permanent record. The student permanent record is the file or folder that is kept in locked storage and is only accessible by designated program staff. Many programs also utilize a student work folder as a place to keep a student’s learning plan and student work. These are usually kept in an open area and are accessible to instructors and students. A student work folder is not the proper place to store student assessment records because they are not secure. All personnel are responsible for maintaining the confidentiality of individual student records. Publicizing an individual student’s test results is not allowed. Student testing achievements are often recognized by local programs. If your program does this, make sure that students able to be recognized for making gains while keeping their individual test results confidential.

Required Assessment Records Documentation BEST Literacy Test book and scoring sheet BEST Plus 2.0 Score report TABE 9&10 (online) Score report, (Locator report made available upon request) (print-based) Locator answer sheet*, test answer sheet and score report TABE CLAS-E Locator answer sheet, answer book, writing folio, score report Each assessment has different documentation that must be maintained in the student permanent record. If the documents in this list seem unfamiliar, please consult the test publisher materials or your local expert test administrator for assistance. The required assessment documentation for BEST Literacy is the test book and the scoring sheet. For BEST Plus, it’s a printed copy of the score report. The required documentation for TABE Online is a score report (a Locator report does not have to be printed unless requested by OAE staff). TABE Print-based requires a copy of the TABE Locator answer sheet, the test answer sheet, and the score report. For TABE Print-based, either the 7&8 Locator or 9&10 Locator are acceptable. For TABE CLAS-E, the program should maintain a copy of the Locator answer sheet, the answer book, writing folio, and the score report. *Both TABE Locator 7&8 and 9&10 are acceptable.

Assessments in GALIS Starting in FY18, student assessment results must be entered into GALIS within 10 business days. GALIS must be a reflection of actual student activity; Ex. Test administration date should be the actual administration date. Record all assessments in GALIS, unless there is a reason that is explicitly allowed by the assessment policy manual. Record all assessment scores in the student record and GALIS whether they go up, go down, or remain the same. Record all subject area scores administered (TABE). Starting in FY18, student assessment results must be entered into GALIS within 10 business days. OAE acknowledges that this is a short timeframe, but the overall goal is to improve data quality and the timeliness of student test reports. When entering student data into GALIS, the main thing to remember is that it must be a reflection of actual student activity. For example, the date of the assessment should match the date that is entered. All assessment results must be entered unless there is a reason that is explicitly allowed by the assessment policy manual. It doesn’t matter if scores go up, go down, or remain the same. This includes all TABE subtests. Since educational gain under WIOA can come from any subject area (not just the lowest), there is no reason not to enter all subject area scores on TABE.

Test Security – Testing Materials Keep all testing materials in locked storage. Don’t put testing materials in Student Work Folders. Don’t leave testing materials sitting out, including the teacher’s desk. All paper-based assessment materials should be kept in locked storage – test books, answer sheets, scoring guides, test manuals, training materials, etc. The program should always know the quantities of test materials on hand and keep up with ordering additional materials, so that there are no shortages. Securing testing materials doesn’t only involve locked storage. It also involves making good decisions every day, like not placing assessment materials in student work folders (in order to remind them to take or finish an assessment). It’s better to keep the materials in locked storage and place a note in the student work folder, such as a sticky note or a colored piece of paper with a pre-printed reminder like, “It’s TABE time!” When you have completed a testing session, return all materials to locked storage. Leaving materials out (for example, on the teacher’s desk) is not conducive with good test security practices.

Test Security – Proctoring Take the task of proctoring assessments seriously. Watch for cheating or the use of devices, such as electronic dictionaries or phones. Refrain from assisting students during a test. Don’t review missed problems with students after the assessment. It’s easy to forget that giving assessments is a serious business. They determine a student’s placement in the program, they provide feedback on skills progression, they can determine if a student is recommended to take the GED Test or not, but ultimately, assessments are used to demonstrate performance outcomes to our funders. Each administration of an assessment should be proctored by a trained test administrator who actively monitors testers and watches for cheating or the use of devices, such as electronic dictionaries or phones. A good proctor must refrain from assisting students during assessments, even if a student says, “the answer is not here.” A good proctor is prepared to respond in these situations with a, “just do your best” or “pick the best one.” After the assessments have been scored, you may want to give students feedback out the specific items they missed. This is not allowed. You can use a diagnostic tool to identify the types of questions that were missed, but showing students the correct answers to specific test questions or even how to work those specific questions (math), is a violation of test security.

Test Security Check-up How would you rate the maintenance of assessment records in your classroom or site? How would you rate the test security in your classroom or site? Rating Scale Excellent Good OK Fair Poor Take a few minutes to complete these questions on your participant worksheet. What changes would improve record-keeping or test security in your classroom or site?

Annual Training Required for all test administrators Assessment Training Ideas: Local testing procedures, test publishers’ training materials, test security protocols, testing issues and solutions to those issues BEST Plus: Scoring Refresher training (offered by OAE), testing observations and peer critiques BEST Literacy: Local scoring calibration activity Assessment training is required annually for all test administrators. After testing personnel have completed initial assessment training, some programs wonder what can be done as an additional training. First, test administrators should train with people who give the same test. As a group, they can conduct a review of local test procedures to ensure that they are adequate and that everyone is on the same page. The group can use the test publisher’s training materials, like workbooks and videos, to refresh. You can examine your local test security protocols. For example: What are your rules about cell phones during testing? What are the consequences for breaking the rule? The group could also have a discussion about testing issues and possible solutions. OAE offers the BEST Plus Scoring Refresher training twice a year, and additional training sessions can be requested for large groups. The Refresher should be completed a minimum of six months after initial BEST Plus training. With BEST Plus, you can also observe test administrations and provide constructive critiques to your fellow testers as a local assessment training activity. For BEST Literacy, one of the recommended activities is to conduct a local scoring calibration activity, especially with student note writing, to ensure that all scorers are following the instructions and rubrics the same. The BEST Literacy Test Manual gives extensive details about how to conduct this activity.

Office of Adult Education Instructional Services Director Bobby Creech bcreech@tcsg.edu 404-679-1641 Central/Southeast Georgia Grant Program Support Coordinator Felicia Phelps fphelps@tcsg.edu 404-679-5829 Metro Atlanta/Northeast Georgia Charita Boles cboles@tcsg.edu 404-679-1628 Metro Atlanta/Northwest Georgia Leatricia Williams lwilliams@tcsg.edu 404-679-5234 South/Southwest Georgia Steve Pearce spearce@tcsg.edu 404-679-0511 Instructional Services Technical Coordinator Kimberlee Bryant kbryant@tcsg.edu 404-248-1060 GALIS Helpdesk Data Management Assistant Davida Shelton datacenter@tcsg.edu 404-327-6907 Thank you for completing the Assessment Policy Updates and Reminders tutorial. Be sure to complete the Participant Worksheet and turn it into your supervisor.