Structuring and Scaling Up Embedded Assessment of General Education Outcomes at St. Louis Community College Thomas M. Dieckmann, English Faculty Joyce Starr Johnson, Ph.D., Director of Curriculum and Assessment

St. Louis Community College expands minds and changes lives every day. We create accessible, dynamic learning environments focused on the needs of our diverse communities. Florissant Valley campus South County Education and University Center

About the College Forest Park campus Meramec campus Wildwood campus Florissant Valley’s Emerson Center for Engineering and Manufacturing

Enrollment Profile (Fall 2014) 21, 218 students enrolled in credit courses Median Age = 23 Mean Age = 27 59% Female Race and culture Caucasian = 55% African-American = 33% Asian = 3% Hispanic/Latino = 3% Represent nearly 100 countries Curricula College Transfer = 69% Occupational = 29% Self-Improvement = 1% Undecided = 1% Students in the Florissant Valley cafeteria

Where are you in your assessment journey? What are your challenges with assessment of student learning?

General Education Re-Design Developed Student Learning Outcomes and rubrics to assess the outcomes Disciplinary “champions” identified in 2011 to lead revision of course profiles for all courses designated as “General Education” Intensive revision process throughout academic year resulted in more than 120 courses being revised

General Education Redesign Skill Areas Communicating* Higher Order Thinking* Valuing* Managing Information* Knowledge Areas Social & Behavioral Sciences* Life & Physical Sciences* Mathematics* Humanities and Fine Arts* Global/Intercultural Studies Interdisciplinary Studies *Missouri State General Education Learning Outcomes

Beginning Assessment Planning STLCC joined HLC Academy for Assessment of Student Learning 4-year program, beginning Fall 2013

HLC Academy for Assessment of Student Learning Short-term Project Goal: – Assessment of Student Learning Outcomes (SLOs) in General Education – focus of today’s presentation Long-term Project Goals (in progress): – Articulation and assessment of SLOs for all transfer programs – Articulation and assessment of SLOs for all CTE and co- curricular programs

Project Outline: 1.Sharing and gathering information 2.Mapping curriculum for Gen Ed SLOs 3.Developing a system for collecting artifacts 4.Implementing system for analysis 5.Sharing data to affect change General Education SLO Assessment

1. Sharing and gathering information Information campaign during academic year stressed importance of faculty participation

2. Mapping Gen Ed SLOs

Piloted a process in Spring Developing a system for collecting artifacts

Spring 2014 Pilot Results – Rating Session Planning for Spring 2015 CategoryTarget # of artifacts Artifacts x2 for Inter-Rater Rel. Hours needed to review/rate (div. by 8) Number of Faculty reviewers (div. by 2) Estimated number of sessions (div. by 15 per session) ENG: COM:101 & COM: ENG: G/I Valuing IDS Higher Order Thinking Social and Behavioral Sciences Humanities and Fine Arts Life and Physical Sciences Mathematics IDS G/I

Spring 2014 Pilot Results – Artifact Collection Strategies Each of the four skill areas will strive to collect and assess artifacts from 85 sections: – 13 sections of ENG:101 (Communicating), approx. 320 – 6 sections of COM:101 (Communicating), approx. 140 – 6 sections of COM:107(Communicating), approx. 140 – 20 sections of ENG:102 (Managing Information), approx. 400 – 20 sections of G/I designated courses (Valuing), approx. 400 – 20 sections of IDS courses (Higher Order Thinking), approx. 400 Each of the five knowledge areas will strive to collect and assess a proportional number of artifacts based upon the number of credits they represent within Gen Ed. from the remaining 85 sections. – 24 sections, approx. 480 from Social & Behavioral Sciences (9 out of 32 credit hours, 28%) – 24 sections, approx. 480 from Humanities & Fine Arts (9 out of 32 credit hours, 28%) – 18 sections, approx. 360 from Life & Physical Sciences (7 out of 32 credit hours, 22%) – 10 sections, approx. 300 from Mathematics (4 out of 32 credit hours, 12%) – 8 sections, approx. 160 from Interdisciplinary Studies (3 out of 32, 10%) – 8 sections, approx. 160 from Global Intercultural (3 out of 32, 10%)

Spring 2014 Pilot Results – Creation of GEAF Position Respected faculty leaders were needed to vet faculty artifact submission for applicability to chosen SLOs Creation of General Education Assessment Facilitator (GEAF) position for each Gen Ed SLO category, to be filled by faculty members Work would include small stipend

GEAF Job Description The facilitator for each general education category coordinates the assessment processes of a category of general education and makes recommendations to the General Education Council on the basis of the assessment activities and findings. Duties beginning Fall 2015: – Review General Education assignments as they are submitted; remove those that don’t address the designated student learning outcomes (1-2 hours per semester) – Participate in orientation meeting for the reading/rating sessions (2 hours per year) – Serve as a liaison with faculty in the collection of artifacts – Hold 2-3 reading-rating sessions (4-6 hours per semester starting in Spring 2015) – Review findings and make recommendations to Gen Ed Council (2 hours per semester) – Attend occasional meetings, including those of the General Education Council as needed, and participate in discussions related to the general education categories

Current STLCC Assessment Team Personnel Director of Curriculum and Assessment (DCA) Director of Institutional Research and Planning General Education Council – Includes Vice Presidents for Academic Affairs from each of 4 campuses Faculty General Education Program Coordinator from 3 campuses HLC Academy for the Assessment of Student Learning Team In addition to VPAAs, also includes a faculty representative from 3 campuses 10 Faculty GEAFs (one for each Gen Ed SLO Area) Ad hoc members of Institutional Research and Planning and professional staff as necessary

Collection Activities

Fall 2014 Status Report All faculty received this document during Service Week presentation before the start of the Fall 2014 term

Instructions All faculty whose Gen Ed section was chosen for assessment received this document as Part 1 (of 2) of participation packet at beginning of Fall 2014 term

Cover Letter All faculty whose Gen Ed section was chosen for assessment received this document (with appropriate Gen Ed SLO listed) as Part 2 (of 2) of participation packet at beginning of Fall 2014 term

Fall 2014 Artifact Collection Process

Fall 2014 Artifact Collection Process: Challenges Some confusion amongst faculty that all student submissions for chosen assignment must be collected Some faculty failed to eliminate all identifying faculty or student information from artifacts Some chosen assignments unsuitable for assessment of selected SLOs Difficulties of collecting/submitting large and/or electronic artifacts, including videos oral presentations

Early Spring 2015 Work Faculty given reminder and update of assessment work during Service Week meetings Faculty provided with rating session schedule at beginning of semester and all Gen Ed faculty asked to participate in one session Courses randomly selected for second round of artifact collection during Spring semester and faculty of these courses notified

4. Implementing system for analysis Spring 2015 Rating Sessions Each session focused on one particular knowledge or skill goal area (e.g. “Communicating”, “Valuing”, “Higher-Order Thinking,” or “Mathematics”) and its defined SLOs Faculty allowed to self-select the session he/she would attend Sessions led by faculty GEAF Sessions held in computer labs on various campuses

Spring 2015 Rating Sessions (continued) Faculty participants provided with brief discussion of Gen Ed assessment history, plan, and rationale Participants introduced to appropriate rubric for area Participants given a sample assignment and student artifact to rate using the rubric Discussion (sometimes robust) followed of application of the rubric to the assignment to ensure inter-rater reliability Sample activity also provided participants practice with survey instrument

Spring 2015 Assessment Survey Instrument Opening Page screenshot

Spring 2015 Assessment Survey Instrument Gen Ed SLO category selection screenshot

Spring 2015 Assessment Survey Instrument “Communicating” Rubric screenshot

Spring 2015 Rating Sessions: Challenges Faculty participation for some sessions – For example, 3 “Communicating” sessions had a total of 11 faculty participants – Our planning based upon Spring 2014 pilot was based up 38 attendees Because of this, not all submitted artifacts were read/reviewed twice, or, in some cases, at all CategoryTarget # of artifacts Artifacts x2 for Inter-Rater Rel. Hours needed to review/rate (div. by 8) Number of Faculty reviewers (div. by 2) Estimated number of sessions (div. by 15 per session) ENG: COM:101 & COM:

Rating Session Participant Feedback: Areas of success Faculty reported 2-hour time block for rating sessions worked well and was appropriate Faculty reported appreciation for the assistance and leadership of the faculty GEAFs during the sessions Benchmark data was generated for future reference

Rating Session Participant Feedback: Areas for Improvement (continued) Faculty need to be provided with professional development regarding providing assignment instructions/documentation and selecting SLOs intended to be elicited by assignments Faculty should provide answer keys to assignments when appropriate Faculty with specific discipline expertise should rate artifacts coming out of their disciplines when disciplinary knowledge or vocabulary is being assessed

Rating Session Participant Feedback: Areas for Improvement (continued) Survey instrument needs some refinement to make assessment more efficient and less cumbersome Knowledge and Skill Area rubrics need some refinement in some categories to more clearly distinguish between levels of achievement Rating session scheduling should take into account particular departmental/disciplinary needs to ensure appropriate faculty can attend relevant sessions

Preliminary Findings Positive growth in four skill areas between introduction and reinforcement Baseline performance indicators for all 10 General Education categories

Current Status Collecting second round of artifacts for assessment in the Fall semester Next is step 5 (the most important) of our Assessment plan: 5) Sharing data to affect change – The long term goal is to ensure that students develop skills and knowledge that will improve their lives – Planning is underway for faculty development activities in the fall

Questions? Thomas Dieckmann Instructor II, English St. Louis Community College, Florissant Valley Joyce Starr Johnson, Ph.D. Director of Curriculum and Assessment St. Louis Community College