ABE Performance Measures, Accountability and Assessment Brad Hasskamp| Adult Secondary Credential and Education Policy Specialist Jodi Versaw| Adult Basic Education Program Quality Specialist

The 6 Accountability Measures Brad

6 Measures for WIOA Success Measurable skill gain Employment at second quarter after exit* Employment at fourth quarter after exit* Median earnings at second quarter after exit* Credential earned (and Entering Postsecondary or Employment)* Employer engagement** Aggregated performance outcomes analyzed statewide, by consortium and by 11 groups and by age, gender, race. *These measures will apply for ABE programs starting in 2020. **This measure is not yet full defined nor being measured for ABE programs.

Getting MSG (Measurable Skill Gain) Heather

Measurable Skill Gain Indicator The Federal Definition: The percentage of participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment and who are achieving measurable skill gains, defined as academic, technical, occupational, or other forms of progress, towards such a credential or employment.

4 Types of Measurable Skill Gain for ABE Secondary Diploma/ Equivalent Secondary or Post-Secondary Transcript Educational Functioning Level Gain Pre-Post Test Completion of Carnegie Units Program Exit + Entry into Postsecondary Education Progress toward Milestones Passing Technical / Occupational Knowledge Based Exam Used in Title II

Counting Measurable Skill Gains (MSG) Evaluation Learning Community Training December 7-8, 2011 Counting Measurable Skill Gains (MSG) Multiple Ways to Count MSG (Only one type of gain will be counted for each participant) 1. Earn higher scores on approved pre- and post-tests to advance to a higher level 2. Earn a secondary credential (high school or equivalency diploma) by June 30 3. Enter postsecondary education (only after participant exits ABE) by June 30 4. Participants in “adult high school” can complete Adult Basic Education (ABE) Level 5 by earning enough Carnegie Units or credits to move to 11th- or 12th-grade status* (*Not yet applicable for ABE in Minnesota)

National Reporting System Targets for Measurable Skill Gain NRS Educational Functioning Level and Core Indicator Goals MN FY 2017 (2016-17) FY 2019 (2018-19) FY 2020 (2019-20) % Completing Each Level/Goal % Earning Measurable Skill Gain Target MN Actual U.S. Actual ALL ABE LEVELS N/A* 43 44 ABE 1: ABE Beginning Literacy 78% 59 48 61 ABE 2: ABE Beginning Basic Education 67% 46 45 ABE 3: ABE Intermediate Low 50% 39 42 ABE 4: ABE Intermediate High 34% 36 40 41 ABE 5: Low Adult Secondary 26% 52 ABE 6: High Adult Secondary 28 36* 42* ALL ESL LEVELS ESL 1: ESL Beginning Literacy 48% ESL 2: ESL Beginning Low 58% 54 50 56 ESL 3: ESL Beginning High 51% 47 51 ESL 4: ESL Intermediate Low 39% 38 ESL 5: ESL Intermediate High 35% 34 37 ESL 6: ESL Advanced 16% 23 24 25 26

Important Notes about the Accountability Measures The U.S. Department of Education has set overall ABE-level and ESL-level targets. There are now targets for ABE Level 6. The U.S. Department of Education is setting higher targets for states. Minnesota is expected to be at the median or higher in all ABE and ESL levels.

What questions do you have about MSG, the targets and your ABE program measuring it? Measurable Skill Gain Secondary Diploma/ Equivalent Secondary or Post-Secondary Transcript Educational Functioning Level Gain Pre-Post Test Completion of Carnegie Units Program Exit + Entry into Postsecondary Education Progress toward Milestones Passing Technical / Occupational Knowledge Based Exam

Program Improvement

Program Improvement Take Note! Results from program year 18-19 will be used for a Program Improvement process starting Fall 2019.

Program Improvement Q&A Q: What is Program Improvement? A: A technical assistance process for ABE programs that fall below set benchmarks on key accountability measures. Q: Why Program Improvement? A: To work towards consistent implementation of accountability practices state- wide; to ensure compliance with federal accountability standards. Program Improvement Q&A

Program Improvement Q&A Q: What accountability measures will Program Improvement be based on for PY18-19 data? Measurable Skill Gains (Level Gains and Diplomas) for ABE students Measurable Skill Gains (Level Gains and Diplomas) for ESL students Post-Testing Rates for students with 40+ contact hours Program Improvement Q&A

Program Improvement flags ABE consortia will be flagged for possible participation in Program Improvement if: Measurable Skill Gains (Level Gains and Diplomas) for ABE students falls more than 30% below NRS target Measurable Skill Gains (Level Gains and Diplomas) for ESL students falls more than 30% below NRS target Post-testing rate for students who have 40+ hours falls below 60%

Program Improvement Q&A Q: How will we know if our consortium is flagged for Program Improvement? Pay attention to “Level Gains and Post-Test Rates” report in SiD. MNABE Report Card Report card for 17/18 will be produced this fall – NO Program Improvement associated with this report card! Report card for 18/19 will be produced late summer/early fall 2019 Program Improvement Q&A

Report card preview – Measurable Skill Gains

Report card preview – Post-Testing Rates

Program Improvement cohort participation From all ABE consortia flagged for possible participation in Program Improvement, a group of ABE consortia will be selected for participation in the Program Improvement Cohort. Factors that will be taken into consideration include: Total number of flagged consortia Participant numbers Unique circumstances

Program Improvement Cohort The Program Improvement Cohort group will participate in (details TBD): Full-day workshop in Fall 2019 Site visit by state ABE staff Improvement planning Technical assistance from state ABE staff

Program Improvement Next Steps Watch for 17-18 Report Card coming out this fall!

Assessments: The how and when of the new tests Brad

Collecting your questions What questions do you have about ABE assessments, especially the new TABE 11- 12 and CASAS GOALS? (Jot down your questions at your table.)

What’s Happening? Date Approved ABE Level Tests Approved ESL Level Tests July 2017- June 2018 TABE 9-10 CASAS Life and Work BEST Plus July 2018 – June 2019 TABE 11-12 CASAS GOALS July 2019- June 2020 Programs can use old tests but may be able to transition to new tests July 2020 – June 2021 Programs can use old tests but may need to transition to new tests

Awkward Test Transitions New tests are coming and your program needs to switch! 2018-19: All ABE programs need to buy and start implementing TABE 11/12 and/or CASAS GOALS for ABE level students As of July 2019 CASAS Life and Work will be valid for ESL students but not ABE-level students. Publishers submitting ESL tests for approval this fall to U.S. Department of Education.

ABE Spring Application Workshop TABE 11-12 Approved! The U.S. Department of Education’s Office of Career, Technical and Adult Education (OCTAE) has approved TABE Forms 11 and 12 for the NRS and for local ABE programs in Minnesota. Computer and paper versions 3 Subject Tests: Reading, Math, Language 5 Level Tests in each subject: L, E, M, D, A Each test has only 1 option (no survey) TABE 9 and 10 will be dropped from the approved list starting July 1, 2019 Brad

TABE 11-12 Test Times Subject Revised Maximum Allowable Test Time (levels E, M, D, A) Math 75 minutes (2 parts in levels M, D, A timed separately) Reading 120 minutes (2 parts timed separately) Language 60 minutes

ABE Spring Application Workshop From TABE 9-10 to 11-12 TABE 9 & 10 are not considered the same as 11 & 12 for data purposes, meaning that if a student has a pretest in 9 or 10, the student: Cannot be post-tested with an 11 or 12. Does not need to wait 40 contact hours before taking a 11 and 12 as a new pre-test to determine a student’s level. Brad

Scoring TABE 11-12 Not all correct answers are +1 point. Some two- part answers have different scores and some items are not worth any points. Hand scoring will be more difficult than with TABE 9-10, but you may be able to scan official test answer sheets for electronic scoring. It would not be surprising to see students drop levels as the take their first TABE 11-12 since the levels are different and more challenging with content standards alignment. Some students may need TABE Level L tests now. The TABE Level L test is intended to be a consumable book.

Buying TABE 11/12 Paper Test Books $156/25 test books $86.50/25 locators Paper Answer Sheets $24/25 (test) $22.50/25 (locator) Computer Testing $2.17-3.40/test Administration $32.50/11 or 12 Test Directions $25/Test Administration Manual $30.50/Scoring Guide www.tabetest.com

CASAS New GOALS Reading Test Approved! ABE Spring Application Workshop CASAS New GOALS Reading Test Approved! The U.S. Department of Education approved the new CASAS GOALS Reading test for ABE levels (not ESL) in the NRS. Math test still in development. Same test being submitted for ESL Levels this fall. Brad

Buying CASAS GOALS Reading Paper Test Books $100/25 test books or locators $190/5 test books each form Paper Answer Sheets $63/100 (test) Computer Testing $1.75-2.85/test Administration $40/Test Administration Manual www.casas.org

CASAS and TABE Look Each group will get a copy of new and old CASAS and TABE. Discuss: What is different between the tests? What is similar? What issues with the new tests might your students struggle with? What might you need to change locally as you implement the new tests?

New Assessment Policy See the new Assessment Policy at www.mnabe.org/abe-law-policy/mn-abe-policies that includes: TABE 11 and 12 CASAS Goals is added to the approved list for ABE 1-6 in Reading New Educational Functioning Level Descriptors for ABE levels 1-6 Please note that the revised Educational Functioning Level Descriptors for ESL levels 1- 6 are not included since there are no approved tests for the new ESL descriptors yet!

SW ABE as Central (Computer) Test Purchaser The Assessment Training ABE Support Network Provider (SouthWest ABE) is coordinating central purchasing of computer-based tests (TABE 11 & 12 and CASAS GOALS) with local ABE programs This allows programs to get the best bulk purchasing rate. Local programs will need to pay for the tests Purchases made potentially annually or semi-annually, upon demand (Computer-based test voucher purchasing information will be distributed this fall. Look for an email from Cherie later this fall.)

Assessment Professional Development Professional development on tests is offered at: Summer Institute Regionals Support Services Conference (November 8-9, 2018) Sessions Offered: Test Certification (required for all staff working with testing) will provide an overview of the allowable tests Critical Issues in Assessment will dive into the new tests, computer-based testing and other topics

Answering your Questions What additional questions did you have about ABE assessments, especially the new TABE 11-12 and CASAS GOALS?

Table Discussion: Test Transition Planning with Local ABE Programs Start thinking now about when you will transition testing: When will you purchase tests? Which ones will you purchase? Which tests will you purchase for ABE students (TABE 11/12 or CASAS GOALS)? Will you purchase paper and/or computer tests? How will you train yourself and your staff on the new tests? Which staff need test certification and which staff just need new test info? How will your program implement the new tests? With current students that already have a pretest? With new students? How will you adjust your program for the longer testing times?