Institutional Effectiveness & Student Success Scorecard Update on Institutional Effectiveness Process & Report on Student Success Scorecard Prepared by the Office Of Institutional Research May 2014 Presented to the Board of Trustees by: Dr. Hannah Lawler, Dean, Institutional Research Teresita Rodriguez, Vice President, Enrollment Development

SMC Office of Institutional Research Institutional Effectiveness Definition Institutional Effectiveness is the systematic and continuous process of measuring the extent to which a college achieves its mission, as expressed through the goals and strategic objectives developed in an educational master plan 2 Dashboards Institutional Priorities Innovative & Responsive Academic Supportive Learning Stable Fiscal Sustainable Physical Supportive Collegial

SMC Office of Institutional Research Institutional Effectiveness Cycle/Process 3 Step 1: Organize Data Develop IE key indicators (KI) Select KIs for Institutional Priorities Dashboard and draft targets Identify KIs needing further inquiry Step 2: Dig into Data Conduct follow-up studies Step 3: Develop Action Plan Identify areas for intervention and develop strategies for improvement Step 4: Act Implement strategies Step 5: Assess Action Plan Assess effectiveness of implemented strategies Annually: Key Indicators will be updated with the most recent annual data and a report of the progress of the Institutional Effectiveness process will be reported to the Board of Trustees

SMC Office of Institutional Research Update for Year 3 ( ) Addition of 8 new key indicators Changes in methodology for 3 key indicators Elimination of 2 key indicators Findings from Student Equity Interviews Impact of Student Behaviors Study 4

SMC Office of Institutional Research Institution-Set Standards v. Target Goals 5 Institution-set standards: standards reflecting satisfaction performance of student learning and achievement. Target goals: a measurable outcome expressed either as a quantifiable value (for example, a target of 75%) or a trend (for example, year-over-year decrease), when achieved, will meaningfully move the needle on institutional effectiveness by

SMC Office of Institutional Research Institutional Priorities - Standards KI Institution- set std. Persistence 73.2%76.2%78.1%73.2%71.8% Transfer 49.5%51.9%47.0%47.4%47.0% Basic Skills Improve 69.3%71.5% 71.9%66.7% Basic Skills Transition 35.9%34.4%36.5%37.1%33.5% CTE Completion 46.5%45.8%45.9%42.0%43.8% Equity – Completion 23.2%25.2%25.1%23.3%<=25.7% Equity - Transfer 24.6%24.1%24.4%24.5%<=25.5% 6

SMC Office of Institutional Research 7 Key Indicator /16 Target Goal In progress- needs attention In progress – on target Target achieved Persistence73.2%75% X Transfer Rate47.4% Remain Steady X Basic Skills Improve 71.9%73% X Basic Skills Transition 37.1%39% X CTE Completion42.0%47% X Equity - Completion 23.3%Decrease X Equity - Transfer24.5%Decrease X Institutional Priorities - Target Goals

SMC Office of Institutional Research Semesters to Associate Degree Completion 8

SMC Office of Institutional Research Key Findings SMC did not meet the institution-set standard for the CTE Completion indicator SMC exceeds the target on one of the indicators Equity Gap – Completion Rate SMC meets the target on two of the indicators Transfer Rate, Equity Gap – Transfer Rate SMC performs below the target on four of the indicators… CTE Completion, Persistence Rate, Basic Skills Improvement, Basic Skills Transition …However, SMC demonstrates improvement on two of them Basic Skills Improvement, Basic Skills Transition 9

SMC Office of Institutional Research Basic Skills Strategies ENGL 20, ENGL 85 MATH 85, MATH 49 (coming Fall 2014) Baltimore Model (ENGL 1 + non-credit support course) Accelerated Courses Summer Jams First Year Experience (FYE) Programs Prep 2 Test Assessment App Preparation for Assessment Test 10

SMC Office of Institutional Research Student Equity Interviews Interview with 17 African American and Latino students in Spring and Fall of 2013 (not generalizable to population) Themes from interviews with students: 1.Interviewees feel welcome at SMC. 2.Proactive and early advising is critical for students’ progress towards their goals. 3.Interviewees entered college without the essential college success skills and struggled to adjust to college life. However, the Counseling 20 course helped teach skills and build confidence in many students. 4.Supportive friends and family members play an important role in guiding and motivating the interviewees in college. 11

SMC Office of Institutional Research Student Equity Interviews (cont.) 5.Interviewees prefer to study alone in the library, typically in between or immediately after class 6.Interviewees value campus clubs and activities but are not actively involved in them. 7.Math is the biggest barrier for the interviewees. 8.Information becomes actionable for interviewees when reinforced by human contact. 9.The most successful interviewees have relationships with a faculty/staff member on campus. 10.According to the interviewees, good teachers are student-centered, promote student-to-student interactions, provide timely feedback, and connect students to resources 12

SMC Office of Institutional Research Impact of Student Behaviors Study Outcomes Student Behaviors Background Characteristics Gender Ethnicity Age Residence Low Income First gen BA+ attainment of zip code Ed goal Unit load Engl/math skill Orientat/assess Major Welcome Center # counseling visits Special prog Tutoring/SI GPA Probation Persisted to term Enrolled Engl/ESL/math # yrs since HS # course repeats Persistence Transfer Basic skills course improvement Basic skills transition to degree course CTE completion 13

SMC Office of Institutional Research Student Success Scorecard 14 Removed 3 metrics (voc and basic skills course success, basic skills progress) 800 page PDF Peer groups Self-assessment Remains 4 metrics (completion, persistence, 30 units, CDCP) – with methodology changes Requirement to report to Board of Trustees Added Web-based interface 2 metrics (CTE & remedial progress) Disaggregated data Measured against self Addition of student- counseling ratio Built upon existing ARCC framework

SMC Office of Institutional Research Student Success Scorecard Background Major limitation: excludes students without valid social security numbers (SSNs). Santa Monica College (SMC) enrolls a large proportion of students without valid SSNs In fall 2013, 14% of the student population did not report a valid SSN. Aligns with areas being measured by Institutional Effectiveness Dashboards 15

SMC Office of Institutional Research SMC Scorecard Cohort Year Metric Statewide Completion 62.9%56.3%53.3%52.5%47.7%48.1% Persistence 71.6%66.6%67.5%68.7%71.7%70.5% 30 Units 71.0%66.1%67.0%68.5%67.5%66.5% Remedial Progress - Math 33.2%29.9%28.2%29.2%28.8%30.6% Remedial Progress - English 48.2%42.5%42.6%43.7%42.8%43.7% Remedial Progress - ESL 64.6%60.7%61.7%62.1%58.7%27.1% CTE 58.0%52.6%53.2%51.7%49.4%53.9% CDCP NA 7.2%7.5%NA 16

SMC Office of Institutional Research Key Findings Five-year decreasing trend for the completion, remedial progress, 30 units, and CTE rates. The college demonstrated improvement in the persistence and CDCP rates over the last five and two years, respectively. SMC performed better than the system-wide averages on three of the eight metrics (persistence, 30 units, remedial progress ESL) 17

Questions/Discussion 18