1. P ROGRAM R EVIEW AS A C ATALYST FOR S TUDENT L EARNING O UTCOMES A SSESSMENT The Santa Monica College Counseling Department Approach Esau Tovar, Santa Monica College Accreditation Institute 2011 Academic Senate for California Community Colleges March 19, 2011

2. Program Review as a Guide to SLO Assessment Program Review at Santa Monica College: –Ongoing assessment of program effectiveness and improvement –Dialogue –Program goals aligned/support institutional goals –Student learning outcomes assessment is central to the PR process: Goals Program improvement Program effectiveness 2 Student Services Program Review Process Link

3. Program Review Key SLO Questions Discuss how the goals and Institutional Learning Outcomes of the College (see Vision, Mission, Goals, and ILOs) are integrated into the program. Describe how the department engages all members in the discussion, review, assessment and revision of program SLOs. How and when has your program assessed SLOs, and how have you responded to the results? How does the program ensure that SLOs are assessed consistently? What program changes have been made based on the result of the assessed outcomes? Discuss how program SLOs relate to the program goals or achievement outcomes. 3

4. Counseling Student Learning Outcomes—An Overview Student Learning Outcomes discussions and plans start in March 2006 anticipating our Fall 2007 Program Review. Have undergone several assessment cycles for a variety of SLOs; three cycles for educational planning SLO. Adopted SLOs have included attitudinal, behavioral, and cognitive domains for counseling service and instruction. Common in all: lots of discussion, debriefing, assessing, and willingness to sustain efforts. 4

5. March 2006: Preparing for Program Review March 2006: Goal Development May 2006: Adoption of SLOs 1 & 2 November- January: Data Collection March 2007: Data Analysis April 2007: Program Review Report Turned In Adoption of “Global” SLO for Instruction Starts in 2006 5 Timeline Toward SLO Adoption & Assessment— Some Milestones

6. 6 October 2007: Review Of Assessment Results & Future Directions October 2007: Changes To Probationary Student Orientation Implemented Refined SLOs 1 & ISLO October 2007: Special Programs Submit SLOs Establish Competencies For All Counseling Courses November 2007: SLO Mapping Toward Institutional Learning Outcomes Spring 2008: Submit Status Update In Preparation For Accreditation Self-Study

7. 7 Timeline Toward SLO Adoption & Assessment— Some Milestones May 2008: Decision To Assess SLO #1 And #2 In Student Success Seminar: 2 Presentations Conducted: Transfer And Educational Planning May - August 2008: Educational Planning Rubric Developed Educational Planning Quiz & Surveys Developed September-October 2008: Presentations Conducted Students Meet With Counselors To Review Educational Plans Late November 2008: Quiz & Survey Administered

8. 8 Timeline Toward SLO Adoption & Assessment— Some Milestones January 2009: Data Analyzed Report Prepared February 2009: SLO assessment report reviewed Future directions decided Spring 2009: Revisions made to educational planning and transfer presentations September 2009 – February 2010: Cycle 2 Starts-Ends Department discusses outcomes Decision to enter Cycle 3 (Fall 2010) Fall 2010: Start Cycle 3

9. Core SLOs for Realistic Appraisal of Educational Planning Original SLO Adopted: SLO #1: As a result of the counseling session(s), students will demonstrate a basic understanding of their individual interests and related academic and career goals which will help them to formulate a realistic educational plan. SLO #2: Students will identify their math and English course sequences, understand the possible consequences of not following appropriate course sequence and chart their individual math and English course sequences to achieve their educational goals. Revised Based on First Assessment Cycle: Student will formulate a realistic self-appraisal of their educational status and its relationship to their overall goals. + SLO#2 As a result of counselor visitations to the basic skills classes, students will be able to identify specific support services designed to supplement/enhance student success. SLO #2: Probationary students who receive counseling intervention via the “Back-to-Success” group counseling session will identify obstacles to academic success and apply strategies to manage these obstacles more effectively. –Assessment: Point of Service Survey -- Back-to-Success (BTS) sessions –Based on results: decision to move to a problem-based learning assessment model –New SLOs adopted and continue to assess over next two years SLO 2a. Students will define progress and academic probation SLO 2b. Students will identify the consequences of progress and academic probation ISLO #1: As a result of taking a course in the Counseling Department, students will identify their needs and apply the knowledge, skills and strategies that support their academic and life-long success 9

10. SMC’s Counseling Department Model for Assessing SLOs 10 SLO Mastery Application of Skills Learned Direct Evidence of Learning Perception of Needs

11. Logistics Required collaboration of counselors and Counseling 20—Student Success Seminar instructors. Willingness to dedicate 1-2 class sessions for: –Ed Planning presentation –Transfer presentation –Helps to have small number of engaging/knowledgeable presenters Willingness to administer post-assignment quiz and survey 11

12. Logistics (cont.) Require students to individually meet with counselors to review ed plan –Willingness to ask students to come back when it is clear little effort was placed in completing assignment Provide guidance on how to create ed plan –Review of 5 ed plan components. Identification of relevant: Degree/transfer General Education requirements (categorize) Major –specific requirements Prerequisites English/Math prerequisite sequencing 3 semester/session course planning –Counselor evaluation of ed plan—based on completed assignment and in-person discussion of student goals, etc. 12

13. Logistics (cont.) Data Collection/Analysis –Counselors must keep track of ed plans evaluated 13

14. Logistics (cont.) Data Collection/Analysis –Counselors must keep track of ed plans evaluated –Records kept by student name, ID number –Require ID number on Quiz and Survey –Integrate ALL data above into single student record for comprehensive analysis if desired –Method affords opportunity to identify both a control and an experimental group –Opportunity to incorporate course grade received and examine differences by select student characteristics or for group as a whole –Analysis requires someone with basic quantitative analysis experience 14

15. Logistics (cont.) Closing Assessment Cycles –Discussing findings and dedicating time to dialogue is crucial –Use findings to guide future directions Revise materials and methods used –Willingness to sustain efforts 15

16. SLO Assessment Cycle 1 – 3 Results Fall 2008 to Fall 2010: –Educational Planning SLO Competency: From 67% to 89% –Represents: 22-percentage point differential Net improvement rate of 33% SLO 1 met –English/Math Prerequisite Sequencing: From 65% to 85% –Represents: 20 percentage point differential Net improvement rate of 31% SLO 2 Met 16

17. Results—A Brief Overview Figure 1. Student educational plan MEAN differences by cohort year (all differences significant at p <.001) 17

18. Results—A Brief Overview Figure 2. Student educational plan PERCENTAGE differences by cohort year (all differences significant at p <.001) 18

19. Results—A Brief Overview 19 Figure 3. Percentage of Students Correctly Identifying Educational Planning Quiz Responses Figure 2. Percentage of Students Correctly Identifying Educational Planning Quiz Responses

20. Results—A Brief Overview Figure 4. Percentage of Students Needing Additional Assistance with Ed Plan Assignment 20

21. Conclusion/New Directions Fall 2010: Examined differences by race, ethnicity, gender, income Determined that we have achieved our goals for SLOs –Will not assess in the coming year –Will expand educational planning exercise to all students Develop a pilot self-paced, interactive exercise Begin to plan for a Fall 2012 Program Review –Spring 2011: Revise Departmental Goals Develop new SLOs 21