BEST Plus and BEST Literacy Directors’ Training Feb. 14, 2013Columbia, SC.

Slides:



Advertisements
Similar presentations
Ed-D 420 Inclusion of Exceptional Learners. CAT time Learner-Centered - Learner-centered techniques focus on strategies and approaches to improve learning.
Advertisements

IDEA® English Language Proficiency Tests (IPT)
EdTPA World Languages NYSAFLT Rochester Regional March 8, 2014 Dr. Nancy Barnett Empire State College MAT Program
1 CASAS Overview Symposium on Issues and Challenges in Assessment and Accountability for Adult English Language Learners May 16, 2003 Washington DC Linda.
South Carolina Adult Education ASSESSMENT POLICY UPDATE Directors’ Training – 08/22/13.
The REEP Pre and Post Writing Assessment (RWA)
Presented by Denise Sibley Laura Jean Kerr Mississippi Assessment Center Research and Curriculum Unit.
1 The New Adaptive Version of the Basic English Skills Test Oral Interview Dorry M. Kenyon Funded by OVAE Contract: ED-00-CO-0130 The BEST Plus.
Introduction to the Child & Adolescent Needs and Strengths Assessment (CANS) Our Community. Our Kids. Dr. Gary Buff, Ed.D. President and COO.
CATEMA™ Career and Technology Education Management Application CATEMA™ Training Workshop Jefferson College Tech Prep Director: Sarah Bright.
Grants Pass High School Four Year Plan Tracking System Celena Shouse-Bland.
Comprehensive Assessment System Webinar #6 December 14, 2011.
Diploma of Project Management Course Outline NSW Course Number Qualification Code BSB51407.
EDUCATOR CERTIFICATION UPDATE Michigan Association of School Personnel Administrators Conference December 3, 2010 Flora L. Jenkins, Director Office of.
Weaving Pathways: Interculturalism and Language
ACCESS for ELLs and Alternate ACCESS for ELLs
[Presentation by] [Date] [INSERT DISTRICT LOGO]. Table of Contents Big Picture: How Does It All Connect? Keystone Exams: The 33% Option Keystone Exams:
Assessment Policy. Reporting Student Data in AERIS All student data must be entered into AERIS by the 15 th and approved by the 22 nd of each month for.
P.E.R.T. Postsecondary Education Readiness Test. TODAY’S PERT HANDOUT Evaluation Services copied only the pages that pertained to the actual administration.
U.S.A. Learns A Web site to teach English to adult learners Leslie Petty Project IDEAL Support Center A Web site to teach English to adult.
The Five New Multi-State Assessment Systems Under Development April 1, 2012 These illustrations have been approved by the leadership of each Consortium.
1 DEVELOPING ASSESSMENT TOOLS FOR ESL Liz Davidson & Nadia Casarotto CMM General Studies and Further Education.
1 Literacy and Numeracy Gains Webinar February 3, :00 am - 11:00 am.
Information on New Regents Examinations for SCDN Presentation September 19, 2007 Steven Katz, Director Candace Shyer, Bureau Chief Office of Standards,
Technical College System of Georgia Office of Adult Education As required by the National Reporting System for Adult Education.
STAR Pointers IES Team Staff Development Specialists, E2CCB Kim Oakes Dana Serure Justine Stephan.
Professional Development by Johns Hopkins School of Education, Center for Technology in Education Supporting Individual Children Administering the Kindergarten.
CAMBRIDGE CERTIFICATE IN TEACHING ENGLISH TO SPEAKERS OF OTHER LANGUAGES CELTA.
McLendon and Polis1 An Administrator’s Guide to Assessment: A Menu of Assessment Options for MAERS and Instructional Guidance.
TOM TORLAKSON State Superintendent of Public Instruction National Center and State Collaborative California Activities Kristen Brown, Ph.D. Common Core.
Technical College System of Georgia Office of Adult Education As required by the National Reporting System for Adult Education.
College and Career Readiness: Postsecondary Education Readiness Test (P.E.R.T.) Overview and 2013 Updates Florida Department of Education 1.
Diagnostics Mathematics Assessments: Main Ideas  Now typically assess the knowledge and skill on the subsets of the 10 standards specified by the National.
Learn Academy Orientation – Chapter 3 This Session… 1.Instructor Training 2.Certification Vouchers 3.Continuing Education 4.Alumni 5.On Site Support.
TWS Aids for Student Teachers & Interns Overview of TWS.
P.E.R.T ADMINISTRATION PROCEDURES AND POLICIES.
NRS and Data Collection Part 2. NRS and Data Collection Part 1 and 2  Why? Make sure all programs understand what needs to be collected and definitions.
The ArtWorx Museum Training Program for Volunteers The heart of a volunteer is not measured in size, but by the depth of the commitment to make a difference.
English Language Learner Assessments New Testing Coordinator Workshop Yutzil Becker SDDOE Jonathan Nesladek SDDOE.
Evaluating Assessment for the NRS Evaluating Assessments for the NRS American Institutes for Research February 2005.
Holistic Rating Training Requirements Texas Education Agency Student Assessment Division ©2010 TEXAS EDUCATION AGENCY. ALL RIGHTS RESERVED.
NRS JEOPARDY! The Adult Education Community’s Favorite Quiz Show.
Alternate Assessment Attainment Task Overview Part 2.3 Click here to download the Administration Guide (Required for completion of this training)
MIGRANT LITERACY NET GUIDE FOR TEACHERS: USING TUTORIALS November
2011–2012 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division.
Adult Education Assessment Policy Effective July 1 st, 2011.
P.E.R.T. Paula Wicker, Testing Manager Tonya Mass, Program Specialist October 22, P.E.R.T Administration Procedures and Policies.
Appropriate Testing Administration
Testing Liaison Basic Training. Who can be a Testing Liaison? ONE RULE: INSTRUCTORS AND INSTRUCTIONAL AIDES CANNOT BE TESTING LIAISONS OR PROCTORS Typically,
CALIFORNIA DEPARTMENT OF EDUCATION Jack O’Connell, State Superintendent of Public Instruction California English Language Development Test (CELDT) Program.
1 CASAS Assessment Valid and Reliable Measures for ABE, ASE and ESL Programs to Meet Students' Needs and Federal Reporting Requirements East Coast Summit.
 WIDA MODEL: Grades 1-12 Measure of Developing English Language.
Ready, Set, Start! Using CASAS in Your Even Start Program Ready, Set, Start! Using CASAS in Your Even Start Program Presenter: Martha Gustafson Date: June.
Presentation by Katya Arpon Marandino Irish May 2016.
V: Maryland’s High School Assessments (HSAs) & the Bridge Plan for Academic Validation Overview.
WestEd.org Washington Private Schools RtI Conference Follow- up Webinar October 16, 2012 Silvia DeRuvo Pam McCabe WestEd Center for Prevention and Early.
Assessment Services CELDT Nuts & Bolts Training.
AAPPL Assessment Follow Up June What is AAPPL Measure? The ACTFL Assessment of Performance toward Proficiency in Languages (AAPPL) is a performance-
Presented by Nguyen Kieu Oanh (M.A)
Online Testing System Assessment Viewing Application (AVA)
Administration Procedures and Policies
BEST Plus and BEST Literacy
Introduction to the WIDA Consortium
ACCESS for ELLs Score Reports
Alternate Assessment Updates
Maryland Online IEP System Instructional Series - PD Activity #5
Hawaii TAC Meeting WIDA Assessments
Jaeliza Morales CUR/516 Dr. Mary Poe
Presentation transcript:

BEST Plus and BEST Literacy Directors’ Training Feb. 14, 2013Columbia, SC

What are BEST Plus and BEST Literacy? BEST Plus and BEST Literacy Overview

Who can administer BEST Plus? BEST Plus Test Administrators must complete and pass a six-hour training, led by a certified BEST trainer. They should then administer 10 practice tests and observe one experienced Test Administrator before giving a “real” test. Anyone who has not been approved as a test administrator by CAL (Center for Applied Linguistics) should not assess students using BEST Plus.

After BP initial training, the Director and the prospective Test Administrator will receive:

Who can administer BEST Plus? Newly approved BEST Plus Test Administrators must complete a Scoring Refresher training six weeks to three months after their initial training and show competency in scoring all three areas of the test. Thereafter, Test Administrators must attend a Scoring Refresher training annually and demonstrate scoring competency.

After each Refresher training, the Director and the approved Test Administrator will receive:

CAL recommends that all Test Administrators (BEST Plus and BEST Literacy) have a strong command of the English language and be familiar with working with second language learners. Who can administer BEST Literacy? BEST Literacy may be administered with no formal training after the Test Administrator has reviewed and studied the BEST Literacy Test manual.

Now to the “Nuts and Bolts” of BEST Plus

Overview 1.What is BEST Plus? 2.What does BEST Plus look like? 3.What is the content of BEST Plus? 4.How is BEST Plus scored? 5.What do we know about the reliability and validity of BEST Plus? 6.How is BEST Plus used?

What is BEST Plus? BEST Plus is... –a performance-based assessment (an individually administered face-to-face oral interview)... –that assesses functional oral language skills (interpersonal communication) of adult English language learners... –and is designed with current assessment needs in mind.

Objectives for BEST Plus development Respond to the assessment and accountability needs of adult ESL programs. –Produce a test that is short and practical. –Assess learner language for a variety of purposes and stakeholders. –Increase accuracy in measuring oral language proficiency. –Provide multiple forms for pretesting and post testing.

BEST Plus uses current measurement techniques to improve assessment. Measurement is based on Item Response Theory (IRT). All questions assess listening and speaking ability on the same underlying measurement scale. Multiple scores are awarded for each question. IRT ensures that results of pretests and post tests are comparable.

BEST Plus provides a large item pool. The item pool is large enough that examinees are administered different sets of questions with each assessment. –The ability to memorize test questions is minimized. –BEST Plus can be used for pretesting and post testing. (No “forms”) Currently, the item pool contains 256 questions

What does BEST Plus look like? (Computer-adaptive version) Test items for the test administrator to ask appear on the computer screen. If a test item requires a picture, examinees view the picture on the computer screen. Test administrators enter scores directly into the computer.

Picture Cue “Tell me about this picture.” Beginning level test question Advanced level test question

What is the content of BEST Plus? PersonalOccupationalPublic Personal identification Health Family/Parenting Consumerism Housing Recreation/ Entertainment Getting a job On the job Civics Community services Transportation/ Directions Weather/Seasons Education Content: 13 specific domains nested in three general domains These topics are representative of the ones covered in the ESL Scope and Sequence of Skills. ESL Scope and Sequence of Skills

How is BEST Plus scored? The test administrator uses the BEST Plus Scoring Rubric to guide scoring decisions. The examinee’s response to each item is scored on three aspects of language : –Listening Comprehension –Language Complexity –Communication

Three aspects of language Listening Comprehension: How well did the examinee understand the setup and question? Language Complexity: How did the examinee organize and elaborate the response? Communication: How clearly did the examinee communicate meaning?

Path through the computer- adaptive version Following fixed warm-up questions, examinees are asked questions drawn from several thematically based folders. The test administrator scores each question after the examinee responds. After each set of scores is entered, the computer updates its estimate of the examinee’s ability and chooses folders and questions as appropriate. The test gives the minimum number of items to ascertain an examinee’s proficiency level. Users can immediately receive a complete score report.

What do we know about the reliability and validity of BEST Plus? Rigorous development procedures –Feasibility study ( ) –Initial development ( ) –Pilot, small-scale field test, initial reliability study (2001) –Revisions ( ) –Pilot, full-scale field test, reliability study, standard- setting study (2002) –Finalization of training materials, ancillary materials, further refinements ( ) –Field test for test update ( )

Full involvement of stakeholders Oversight from U.S. Department of Education, Office of Vocational and Adult Education (OVAE) Technical Working Group (TWG), comprised of researchers, state directors, and local program directors and practitioners Item writers, comprised of experienced adult ESL teaching professionals Instructors and students in the field

Full-scale field test Participants: –Eight states (DE, FL, IL, MA, MD, OR, PA, VA) and the District of Columbia –23 programs –40 test administrators –2,420 examinees

Reliability study 32 adult ESL students Two testing rooms (A, B) –Test Administrator (project staff member) –Observer/Co-Scorer (project staff member) –Observer/Co-Scorer (novice scorer) Each student was tested in Room A or B, then immediately retested in the other room.

Test/Retest reliability Between two test administrations (different rooms).89

Average interrater agreement Within one administration (same room) Room A (3 raters) Room B (3 raters).98.97

Ability estimation After each question, the computer program underlying BEST Plus estimates the examinee’s ability based on scores awarded on the current and all previous questions. With each estimation, the accuracy of the measurement increases. Goal: To level off in estimation with an acceptable level of accuracy.

Convergence

The BEST Plus score report Information includes –BEST Plus scale score –Student Performance Level (SPL) –Diagnostic information

NRS Level DescriptorScaled Score Beginning ESL Literacy 400 and below Low Beginning ESL High Beginning ESL Low Intermediate ESL High Intermediate ESL Advanced ESL BEST Plus and NRS Functioning Levels

How is BEST Plus used? BEST Plus allows programs to assess the everyday listening and speaking skills of English language learners for a variety of purposes: –To determine placement into a program. –To track progress. –To provide feedback to learners and teachers to improve instruction. –To evaluate program effectiveness. –To report outcomes (e.g., pretesting and post testing to meet NRS requirements).

BEST Plus Score Management The Score Management Software program provides users with the ability to manage data contained within the BEST Plus Scores Database. Features include –Importing and exporting student data. –Backing up the scores database. –Producing a complete score report for the print-based version of the test. –Purchasing more test administrations. For detailed information on this topic, please refer to the “Score Management Software User Guide” located in the BP Test Administrator Guide

BEST Plus: Print Based Version May be administered without a computer; useful for sites without computer access or in the event of a computer malfunction. However, a computer is necessary to determine a scaled score for an individual student. To get a scaled score for the print based version, the test administrator must access the “Print-based scoring” tab on the BEST Plus testing CD. They must then input scores from each test item to get a scaled score.

Path through the print- based version The test administrator administers and scores the locator test (fixed warm-up items and two high-end discriminators). Examinees answer questions from the appropriate level test based on the chart in the examinee test booklet. The test administrator totals the raw scores and finds the approximate SPL range. The raw scores are entered into the computer via the BEST Plus Score Management Software to obtain a complete score report.

Test –Three forms –Within each form: locator test + three level tests Materials –Picture cue book –Examinee test booklet (scripts and score sheet combined) What does BEST Plus look like? (Print-based version)

Print-based version

BEST Literacy BEST Literacy assesses a student’s English reading and writing skills. Life skills based. BEST Literacy may be administered to a group or to an individual. Again, no formal training is required to administer this test; Test Administrator must study the BEST Literacy training manual to learn to score this test. Also aligns with the ESL Scope and Sequence of Skills

BEST Literacy Comes in three parallel forms (B,C and D). Should (generally) only be administered to students with oral proficiency (on BEST Plus) of Level 3 or higher. Takes 60 minutes to complete. Must be hand-scored.

Why use BEST Literacy with your ESL students? Use to assess reading and writing instruction in the ESL classroom. Use as a method to show reading and writing progress in ESL students. Use with students who test at Level 7 or higher on BEST Plus to retain them as ESL students. Aligns with instruction in the ESL classroom better than TABE.

Students should be pre-tested on BEST Plus (and BEST Literacy, if necessary/desired) PRIOR to attending 12 hours of class. Students must attend a MINIMUM of 60 hours before post-testing (different than the 40 hour requirement in ABE) Pre- and post-test must use the same assessment (either BEST Plus OR BEST Literacy) Pre- and Post-Testing

ESL Student Assessment Progression 1 BEST Plus If Level 7 or above (OR as an alternative assessment), then….. 2 BEST Literacy If Level 7 or above, then….. 3 TABE Student now classified as ABE student.

BEST Assessments and LACES Choose or enter a student and then click “Assessments” 2. Click “Add New”

BEST Assessments and LACES To input BEST Literacy Scores: 1. Click BEST Literacy 2. Select a form (B,C, or D) 3. Click (not applicable) When the Continue button becomes active, click Continue

BEST Assessments and LACES 1.Input scores and date 2. Click save 1. 2.

Practical Information You must purchase tests directly from CAL (see sample order form). These tests are added to your program’s “virtual bank.” Approved Test Administrators are given a unique username and password to access the “bank.” Tests are activated in groups of 20. Tests are connected to the computer on which they are activated. BEST Plus tests cost approximately $1.50/test (depending on quantity ordered). Print-based tests are the same price and use one “banked” test (the same as a Computer Adaptive test).

Practical Information, cont’d To order BEST Literacy tests, you also must order directly from CAL (see sample order form). Each test includes a consumable test booklet and answer form. Tests cost about $2.50/test.

Online Orders For expedited and credit card orders, go to:

Still have questions? Contact your Regional ESL Consultant: Region 1 Monica Rodriguez Region 2 Karen Hodges Region 3 Rebecca Carter Region 4 Jan Camp Region 5 Wendy Griffin

For more information Call: BEST (2378) Write: Center for Applied Linguistics Attn: BEST Plus th Street, NW Washington, DC Visit the Web site:

Questions/ Comments?

Resources for this presentation Center for Applied Linguistics ( SC Adult Education Assessment Policy Manual BEST Plus Test Administrators Guide