Innovation and Value in Basic Skills and ESL: Got Noncredit?

Innovation and Value in Basic Skills and ESL: Got Noncredit? Host: Wheeler North, Basic Skills/Noncredit Committee Chair Karen Dennis, Santa Ana College Janet Fulks, Noncredit Ad Hoc Task Force Candace Lynch-Thompson, School of Continuing Education

Over 70% of our students needing basic skills Information from a two-year pilot measuring student success in ESL and basic skills with a variety of instructional methods (e.g. self-paced, open entry/open exit, managed enrollment). Examined faculty norming (with primarily adjuncts), assessment using multiple measures, and hours of study. Do some basic skills and ESL courses belong as noncredit rather than credit? Basic skills units are limited to 30 and this credit is not transferable or degree applicable. Over 70% of our students needing basic skills Would students benefit by teaching initial basic skills and ESL courses in noncredit? Summary of description

Assessment Levels of Incoming Students Janet – Less than one third of our students arrive ready to take college level credit courses. Most students need remediation in at least one area. Beginning in Fall these students will be paying $46 unit for which most units are not applicable to a degree or transferable.

Total Enrollment (2006-2007 headcount) Credit Noncredit * Basic Skills & ESL Credit Noncredit * 326,478 393,004 * supervised tutoring This is a revealing number that confirms many students enter the community college system through Noncredit This total is from a report that appears in Chapter 1 of your handbook (hold up page) and is printed again in the Noncredit chapter for your reference NOTE: The asterisk represents some of the colleges that use the Noncredit option to provide “Supervised Tutoring” on campus

Course Alignment Prior to TRANSFER Discipline Credit Noncredit Likely bridge noncredit to credit Math Four levels CB 21 A, B, C, D Six levels CB 21 A, B, C, D, E, F Levels C & D English Seven levels CB 21 A, B, C, D, E, F, G Level B or C Reading Five levels CB 21 A, B, C, D, E Level A or B ESL 6 levels ESL Reading CB 21 8 levels ESL Integrated CB 21 A,B,C,D,E, F, G, H Includes vocational and Cultural skills Most noncredit end 2 levels prior to English 1 A at Level B 6 levels ESL Writing CB 21 6 levels ESL Speaking & Listening CB 21 Comparison of levels of each basic skills & ESL level. Noncredit starts lower and generally has more levels. But since CB 21 they are aligned with credit courses by SLOs. Student Success Conference 2009

ALL AT NO COST to the STUDENT What is Noncredit? Fact: Noncredit serves over 350,000 FTES in our system and represents about half of the basic skills work in the CCCs. Fact: Students are significantly more diverse, represent students with greatest need and least likely to succeed Link to data Fact: Noncredit offers flexible schedules, increased contact hours, self paced learning ALL AT NO COST to the STUDENT Karen

Traditional Educational Expectations and Accountability Percent Successful Percent Successful Karen - Lee Schulman's concept of traditional education – Time is the constant success is the variable A B C D F Grades for a specified semester Time is the independent variable Success is dependent on the time

Adult Ed & Noncredit Education Time Karen - Noncredit education success is the constant – time is the variable Percent Successful Success is the independent variable and time is dependent on success

Why Accountability? Enhanced Noncredit Funding Noncredit gets funded less per FTES than credit ($4,565) SB 361 increased noncredit funding from $2745per FTES to $3232 per FTES Applies to students enrolled in a sequence of courses leading to career development or college preparation (CDCP certificates) Candace - A change came about with the request to equalize funding for noncredit and credit – with it cam required accountablility

The Puzzle of Noncredit Accountability Current statewide data Only 2.3 – 5.1% of noncredit students transition to credit All noncredit courses without grades report zero success. Wage data is incomplete because of SSN#s CDCP data is incomplete or programs are undefined. This is not the noncredit story? Candace - But there were problems – the numbers were incongruous with individual institutional data and understanding.

This is not really the noncredit story. And we have data to prove it! 2007-08 SCE Award Data from MIS SCE Actual Data Award hours Program Type Counts Unknown (Top code 99) 9 192 – to fewer than 288 Business and Management 27 288 – to fewer than 480 Family and Consumer Sciences Health 20 960 or more Interdisciplinary Studies 214 Total 279 Program Type Counts Administrative Assistant 62 Management 10 Early Childhood Education 21 Pharmacy Technician 50 High School Diploma 322 Total 465 Candace - SCE looked at the CCCCO published data and found that their own actual data was far higher – imagine this for credit coursework? Data Link

Credit students start in Noncredit Statewide – 1 of every 4 AA/AS degree-earners started in Noncredit Source: Leslie Smith, 2006. Noncredit: The Education Gateway. City College of San Francisco Candace - Yet CCSF had a study indicating that 1 of every 4 AA/AS degree earners had some noncredit course in their past. This is more evidence that the step into credit often begins with taking a class in Noncredit. Many of our students do not realize the potential they have to earn a college degree or certificate. They may have had very little encouragement in school or from family to pursue higher education. They NEVER thought of themselves as being a “college graduate”. Being a student in a Noncredit program (such as basic skills, adult high school or ESL) will give them the advantages of having counseling and advisement services. Noncredit instructors can also have a great influence on the academic path a Noncredit student may take.

CDCP Wage Reporting Candace - One area that was a problem compared to credit was the wage data – which showed HUGE gains in credit but real problems in noncredit. The main problem was that SSN numbers were not collected.

Data Collection Strategies SCE’s “You Count!” Campaign Collecting more SSN’s DREAM team efforts Program improvement Tracking student progress Benefits of Banner Assessment scores Enrollment trends Certificates earned Candace - SCE developed a strategy to get this data and found it useful in many areas.

Accountability Background & Rationale How progress has been measured The interplay of proof of progress and funding The role of CB21 coding Enhanced funding for noncredit Career Development and College Preparation (CDCP) courses The need for progress indicators in noncredit Review and Evaluate current mandated noncredit metrics – further meetings and discussion to take place in November Pilot project to allow the use of noncredit progress indicators – Fall pilot to begin this semester and another group beginning in Spring Janet - Historically, progress in community college courses has been measured by successful completion of a course and movement into the next course in a sequence. While this is the method for measuring progress in credit course sequences, applicability of this method to noncredit has been less than successful due to the fact that evaluative symbols (grades) are not collected for noncredit courses and, until recently, little attention has been paid to accurately coding noncredit course sequences for the purposes of MIS data reporting. As funding of community colleges has come under increased stress in the last decade, accountability reporting of student progress has become a requirement for the receipt of any increase in state funding. In 2004, the Community College System received additional funding contingent on such accountability reporting (AB1417). Today, this report is known as Accountability Reporting for the Community Colleges (ARCC). Under SB361, the apportionment level for some noncredit courses was increased. The requirement for receipt of this increase was the establishment of course sequences for specific noncredit areas. Due to the problems listed above, increased student progress in noncredit has not been possible to demonstrate. As a result, future funding to raise noncredit apportionment cannot be justified and, in fact, many noncredit programs are now at risk because of their inability to demonstrate student progress. At many colleges, significant portions of money provided under the Basic Skills Initiative (BSI) were used to support students enrolled in noncredit courses. BSI funding is also at risk of being cut (further) without data to demonstrate that the funding has supported greater student progress. The level of a course in a sequence is recorded in Course Basic (CB) data element 21. CB21 designates the level of a mathematics or English/ESL course relative to the first course accepted for transfer. Because CB21 was originally created with reporting of credit course sequences in mind, the limited number of levels established resulted in the coding of almost all noncredit courses at a single level. While colleges established course sequences to qualify for enhanced noncredit funding, little has been done to track student progress in moving through and completing sequences. While the work done on CB21 helps with such reporting, progress through a sequence is not necessarily movement from one level to another (e.g. moving through different ESL skill courses at the same level); in addition, courses outside of mathematics and English/ESL, namely CTE, are not coded for CB21 and must be tracked by other means. To address this problem, in 2009-2010, faculty representatives from all around the state met to review the existing framework for using CB21. Discipline faculty groups designed a revised framework for the disciplines of English reading, English writing, mathematics, and the four areas of English as a Second Language - reading, writing, listening, and communication skills, in most cases adding additional levels to accommodate noncredit courses. This framework was provided to the field, and discipline faculty at each college revised their CB21 coding based on the framework for all courses in a sequence. To address the lack of evaluative symbols in MIS data reporting for noncredit courses, the Academic Senate has pulled together a task force to develop progress indicators that can be used for noncredit courses. The pilot project you are embarking on is a part of that effort.

Academic Senate Resolution 13.04 S10 Improve Noncredit Accountability Reporting through Progress Indicators • Task force of primarily noncredit faculty and administrators representing all noncredit areas and other representative. Janet - A resolution was developed and the taskforce spent 2 years looking at data

Pilot Project Goals Develop a set of working progress indicators to use in the pilot project Establish clear communication between institution MIS reporting and noncredit programs Collect a pilot set of accountability data based on these indicators Evaluate the ability for noncredit programs to work with these indicators Evaluate the effectiveness of these indicators for use as accountability requirements Karen - They developed clear goals to address the identified problems

Grades and Title 5 55021: not required for noncredit 55023: currently accepted symbols Currently some noncredit classes are graded in order to qualify for federal funding. But the CCCCO only accepts UG for noncredit. Every grade submitted by a faculty member is changed to UG and all student success data reads as zero. Pilot Progress Indicators Pass (P) Satisfactory Progress (SP) No Pass (NP) A – B – C – D - F Karen - Title 5 does not prohibit the use of grades in noncredit; however, they are not required (section 55021) (a) The governing board of each community college district shall establish a uniform grading policy for all colleges within the district. The policy shall be based on sound academic principles and be consistent with the requirements of this chapter. (b) The grading policy shall require that all work in all degree-applicable and nondegree-applicable credit courses shall be graded in accordance with a grading system adopted by the governing board consistent with section 55023. (c) The grading policy may provide for award of grades in noncredit courses, including courses which are part of a high school diploma program or may be accepted for high school credit by a high school. Title 5 section 55023 – currently accepted evaluative symbols and definitions – no definition for ABCDF or P/NP

Timeline & Guidelines Pilot Project Work is risk free Data will not be shared with anyone else. Data will be available to individual institutions about their own college Information will be analyzed as aggregate anonymous data outside of the institutions First Cohort - Fall 2010 Collection of First Cohort data - February 2011, July 2011, February 2012 Participant Evaluation of Usability of the Indicators - Fall 2011, Spring 2012 Data can still be submitted Karen - 13 colleges have signed up for the pilot; 5 beginning Fall 2010 and 8 beginning Spring 2011 Programs participating include: 13- ESL, 6 CTE, ^ Older Adults, 4 Parenting, 7 Basic Skills, 6 DSPS, 5 GED/HSD, 2 Immigrant programs, 2 Health and Safety 3 Programs signed up to use grades, 6 to use P and NP only and 8 to use P/SP/NP (of course some are using a combination of grades and P/NP or P/SP/NP)

Measuring Learning Gains Multiple Measures Scoring by Rob Jenkins for Santa Ana College School of Continuing Education Factors to consider include : tests scores school experience speaking and writing ability. These are examples of how a faculty member might establish the criteria for assigning one of the progress indicators; emphasize that a teacher probably uses multiple measures Mira Costa College uses EL Civics assessments to measure completion of course-level SLOs (student learning outcomes); the School of Continuing Education for the San Diego Community College District uses a writing/portfolio approach to document learning and attainment of learning objectives Many agencies use CASAS, TABE, or other standardized assessments that show progress and learning gains Colleges with high school diploma programs already collect grades for courses in the program Could ask participants about their current practice What progress indicators do you currently use? What do you do with the progress indicators? Are they collected in local MIS systems? What do the data elements of such indicators look like? What are the issues in collecting and recording such data? If you collect progress indicators, you can create crosswalks with the indicators in this pilot

Example of Mt Sac. Rubric for Instructors Progress Indicator Level Courses Skills Courses P completed all necessary requirements (written & oral evaluation and SLO or Measurable Assessment), good attendance and participation, scored 70% or higher on the final exam. priority attendance, good participation, ability at level, passed SLO or Measurable Assessment NP poor attendance and participation, didn’t complete necessary requirements, scored below 70% on the final exam. poor attendance and participation, ability below level, didn’t pass SLO or Measurable Assessment. SP Added class too late to make decision of P / NP Added class to late to make decision of P / NP Karen – multiple measures; what is involved in a passing grade SAC firmly believes that a face to face -norming session was necessary for every person using the grades.

Noncredit Grading (Progress Indicators) Pilot Final Survey Please provide your demographic information. Answer Options Response Percent Response Count Name: (Mostly Faculty) 99.5% 186 College: Over 14 Institutions (some responding for groups of faculty e.g. Mira Costa 1 response 25 faculty) 100.0% 187 answered question Janet - After 2 years of data a survey was sent out to practitioners from over 14 noncredit institutions

Noncredit Grading (Progress Indicators) Pilot Final Survey Q2 Noncredit Grading (Progress Indicators) Pilot Final Survey Q2. What was your assessment and grading practice in your noncredit classes prior to your participation in this pilot? Answer Options Response Percent Response Count I assessed and submitted grades (progress indicators) before this pilot 39.1% 66 I assessed in my class but did not need to submit grades (progress indicators) 43.2% 73 I did not assess or grade my students 8.9% 15 Other (Please specify below) Other: 19 answered question 169 skipped question 26 Janet - 82% indicated that they had always assessed students but not everyone reported the assessments – so this was not new

Noncredit Grading (Progress Indicators) Pilot Final Survey Q3 Noncredit Grading (Progress Indicators) Pilot Final Survey Q3. How practical was the assignment of P/SP/NP in measuring learning progress for your student population? Answer Options Response Percent Response Count Very practical 45.5% 76 Somewhat practical 33.5% 56 Neutral 10.2% 17 Somewhat impractical 7.2% 12 Very impractical 3.6% 6 Please explain your choice: 72 answered question 167 Janet - The majority found that using the indicators in the pilot was practical – those with problems turned out to be faculty at colleges with limited training. Those that were neutral felt they always did it but were unsure of the benefit of reporting.

Noncredit Grading (Progress Indicators) Pilot Final Survey Q8 Noncredit Grading (Progress Indicators) Pilot Final Survey Q8. Would you support an ASCCC resolution to implement progress indicator reporting for noncredit areas with the caveat that some areas (e.g. older adults, parenting) may need more time to adequately explore and implement what indicators work best? Answer Options Response Percent Response Count Yes 72.5% 116 No 8.8% 14 I would support it if (please add comment) 18.8% 30 answered question 160 Janet - 91.3% of the respondents support the resolution for using indicators. Explain the yes..if – these became the details of the resolutions

Statewide ARCC Data 2008-2011 ARCC DATA Statewide Rates Indicator 2008 2009 2010 2011 Change since inception Student Progress & Achievement 51.2% 51.8% 52.3% 53.6% 2.4%   Completed 30 or more units 70.4% 71.2% 72.4% 72.8% Fall to Fall persistence 68.3% 69.2% 68.7% 67.6% 0.7% Vocational Ed Course Completion 78.2% 77.7% 77.6% 77.0% 1.2% Basic Skills Course Completion 60.5% 61.5% 61.4% 0.9% Basic Skills Course Improvement 50.0% 53.8% 58.6% 8.6% ESL Course Improvement 44.7% 50.1% 50.2% 54.6% 9.9% Janet - These data indicate that since we have published data – it has gotten better in most circumstances (sometimes due to better data methods or communication CB 21). But these do not include noncredit.

Cost over time for SP Open Entry Open Exit Costs less over the years. What have we learned about noncredit? How much time is required for Success? Cost over time for SP Open Entry Open Exit Costs less over the years. Karen - We learned that success requires a specific amount of time per course Candace comment on credit versus noncredit. Lower level should be noncredit.

Cost over time for SP Open Entry Open Exit Costs less over the years. Typical class: 30 students, 17 weeks, 10 hrs per week   Total Cost Total Hours %P P_Cost P_Hrs %SP SP MEDIANHRS SP_Hrs %NP NP _Hrs 3 terms ESL-410 Beginning ESL 1 30 $31,397 5100 7.44 $14,192 2305.2 9.36 $11,743 1907.4 12.9 $ 5,149 836.4 1 Term 5.58 $11,177 1815.6 12.1 $13,469 2187.9 12.3 $ 6,719 1091 Karen - We learned that open entry/exit is cheaper – the SPs become P’s in less time. Candace – P and SP = success

Santa Ana College School of Continuing Education ESL Program 2011SPN (19 weeks) CB21 Title Total Students Median Hours FTES %P %P Hrs P Median Hrs %SP %SP Hrs SP Median Hrs %NP %NP Hrs NP Median Hrs G Beginning ESL 1 1,894 45.0 230.8 17.4 32.7 129.0 43.6 45.4 48.0 39.0 21.9 20.0 F Beginning ESL 2 1,739 59.5 247.7 24.3 40.8 120.0 34.4 57.5 43.0 24.8 24.0 E Beginning ESL 3 1,303 65.0 178.9 25.9 48.2 138.0 26.9 34.9 80.5 47.2 17.0 21.0 D Intermediate ESL 1 1,040 60.0 135.9 26.8 48.3 131.0 30.7 31.4 57.0 42.5 20.3 22.8 C Intermediate ESL 2 941 72.8 160.4 24.5 36.9 136.5 24.2 23.8 77.0 51.2 39.3 44.0 B Intermediate ESL 3 652 73.5 90.3 39.4 68.9 135.0 22.9 22.4 55.0 37.7 8.6 14.0

Santa Ana College School of Continuing Education ESL Program 2011SPN Beginning ESL 3 P SP NP Students (%) 25.9 26.9 47.2 Hours (%) 48.2 34.9 17.0 Due to the at-risk situation of non-credit students, almost one half of them are either not graded or the instructor didn’t have enough elements to assess progress. However, those students consume only 17% of the hours.

Santa Ana College School of Continuing Education ESL Program 2011SPN (19 weeks) Vs. Three Terms (2010Fall, 2011 Spring and 2011 Fall) 51 weeks of instruction and skipping summer (5 weeks) Beginning ESL 3 P SP NP Students 1 Term (%) 25.9 26.9 47.2 Students 3 Terms(%) 30.6 26.6 42.8 Hours 1 Term (%) 48.2 34.9 17.0 Hours 3 Terms (%) 55.6 30.8 13.6 The metrics improve when scope is increased from a single term to multiple terms.

Santa Ana College School of Continuing Education ESL Program 2011SPN (19 weeks) Vs. Three Terms (2010Fall, 2011 Spring and 2011 Fall) 51 weeks of instruction and skipping summer (5 weeks) Typical class: 30 students, 17 weeks, 10 hrs per week 5100 attendance hours = 9.71 FTES 9.71 x $ 3,232 = $ 31,397 Beginning ESL 3 P SP NP Students (out of 30) 9.18 7.98 12.84 Cost (out of $31,397) $ 17,456 $ 9,670 $ 4,270

Noncredit is successful & efficient We need more time to finalize the info on Parenting, OA and DSPS ESL Pass rate 64-80% (lowest ESL less & middle more successful, highest less successful) HS good Pass rate Parenting high pass Number of hours necessary to Pass & SP ESL CTE Cost for success and non-success Cost over three semesters Candace

Considering Accountability and the Resolutions Healthy accountability should: Address higher level learning outcomes Report on authentic student proficiencies Indicate potential interventions and improvement Target improved practice not just reporting Noncredit has piloted and examined training fro progress indicators and the results of student data. Fully support resolutions Urgency due to the link of funding and reporting Candace – relates to resolutions

Questions What are the factors in basic skills classes that represent barriers for students at your college? Do you see anything here that would benefit your institution? In addition to innovative and flexible scheduling, what other ideas do you have?

MiraCosta Noncredit ESL Data 2008 - 2009 Term I Persistence % Promotion % Morning Classes 80% (10% Perfect Attendance) 43% Evening Classes 80% (7% Perfect Attendance) 47% Term II 79% (9% Perfect Attendance) 50% 79% (7% Perfect Attendance) 56% Term III 81% (8% Perfect Attendance) 30% 76% (5% Perfect Attendance) 54% Term IV 78% (10% Perfect Attendance) 63% 74% (8% Perfect Attendance) 46% Sylvia

Brain Anatomy overlaid with Kolb’s Learning Cycle Zull p 18

Active Learning: Engages all of the Brain Epilepsy Foundation of America http://www.epilepsyfoundation.org/about/science/functions.cfm Note that as we go down the pyramid, we are engaging additional areas of the brain, creating deeper learning.

OVERVIEW and Background 1. Issues with Accountability in Higher Education 2. Mandated reporting– ARCC, CDCP and Basic Skills legislated reports – some does not include noncredit due to lack of success and progress indicators 3. CDCP certificates – current state of noncredit and CDCP 4. BSI and the attempt to capture progress CB 21 a. Noncredit and credit aligned b. Progress only by subsequent enrollment c. inability to count success 5. In noncredit progress all indicators and grades turned in are converted to UG ungraded at the state level See Background document and Noncredit Accountability Documents for more information Examining success and different methodologies was not possible in noncredit with no way to indicate progress other than enrolling in later courses – which was not used.

Persistence Indicators For instance the data showed that noncredit students did not persist and did not move to credit Is this the noncredit story? Link to Mira Costa Data