Using Outcomes Assessment Data to Change and Improve Introduction An example from Highline College Reflective practice using data Break Using data to direct and amplify change Connecting assessment data and your action plans Reflective practice – what is your data story?

Outcomes assessment: Why it matters Participation Geoscience course enrollment Broadening participation in geoscience courses Course Success Outcomes Success in geoscience courses Geoscience course success by student sub-group Pathways Progression Outcomes Enrollment in multiple geoscience courses Major in geoscience program Equity- and outcomes assessment, link to action plan Starting an OA project is always challenging – new and complex. You’ve been remarkably patient and responsive. As few hiccups along the way, but has gone remarkably smoothly. We consider this a pilot and we’re learning too. Connection to project goals – improve instruction, broaden participation, and enhance pathways What we do as scientists: Research, hypothesis testing, and your action plan Value to institutions and to geoscience community Value to YOU

Questions to consider Considering the outcomes measures we used, what did you (do you) expect to see? What surprised you in your data? Did you have an “aha” moment? Write Share Short report-outs

An Example from Highline College What I thought I knew… Geoscience has less ethnic diversity than the College as a whole African American students have lower success rates than other groups We have more women in our classes than the College as a whole International students struggle in our classes

Enrollment demographics Geoscience Vs Highline as a whole Students Asian African American Native American Latino/ Hispanic White International Other/Mixed Women Highline 10,765 (credit) 20% 17% <1% 25% 59% Geoscience 213 11% 2% 15% 40% 6% 7% Environmental Science 83 12% 13% 1% 8% 42% 63% Geology 109 22% 9% 3% 19% 38% 56% Oceanography 21 10% 0% 24% 43% 5% 57%

Enrollment demographics Geoscience Vs Highline as a whole Students Asian African American Native American Latino/ Hispanic White International Other/Mixed Women Highline 10,765 (credit) 20% 17% <1% 25% 59% Geoscience 213 11% 2% 15% 40% 6% 7% Environmental Science 83 12% 13% 1% 8% 42% 63% Geology 109 22% 9% 3% 19% 38% 56% Oceanography 21 10% 0% 24% 43% 5% 57%

Completion (C or Higher) By Ethnicity Course (Students) Asian African American Native American Latino/Hisp White International All Environmental Issues (27) 0% 75% 50% 92% 78% ENVS 301 (9) 100% 67% 89% Intro to Env Science (22) 80% 88% 86% Intro to Env Science (25) 91% SHAPING THE EARTH (24) 40% 60% 63% GEOLOGIC CATASTROPHES (29) 62% GEOLOGIC CATASTROPHES (23) 25% 73% 61% Puget Sound Landslides (9) Intro to Oceanography (21) 56% 57% ALL (213) 65% 58% 77% N (Total Students) 37 23 4 33 85 13 213

Completion (C or Higher) By Ethnicity Course (Students) Asian African American Native American Latino/Hisp White International All Environmental Issues (27) 0% 75% 50% 92% 78% ENVS 301 (9) 100% 67% 89% Intro to Env Science (22) 80% 88% 86% Intro to Env Science (25) 91% SHAPING THE EARTH (24) 40% 60% 63% GEOLOGIC CATASTROPHES (29) 62% GEOLOGIC CATASTROPHES (23) 25% 73% 61% Puget Sound Landslides (9) Intro to Oceanography (21) 56% 57% ALL (213) 65% 58% 77% N (Total Students) 37 23 4 33 85 13 213

Completion (C or Higher) by Ethnicity Courses with Changes Made Course (Students) Asian African American Native American Latino/Hisp White International All Environmental Issues (27) 0% 75% 50% 92% 78% ENVS 301 (9) 100% 67% 89% Intro to Env Science (22) 80% 88% 86% Intro to Env Science (25) 91% SHAPING THE EARTH (24) 40% 60% 63% GEOLOGIC CATASTROPHES (29) 62% GEOLOGIC CATASTROPHES (23) 25% 73% 61% Puget Sound Landslides (9) Intro to Oceanography (21) 56% 57% ALL (213) 65% 58% 77% N (Total Students) 37 23 4 33 85 13 213

Completion (C or Higher) rates by Ethnicity and Subject Asian African American Native American Latino/Hisp White International All Environmental Science 78% 82% 100% 86% 91% 70% Geology 83% 50% 67% 48% 76% 71% Oceanography 60% 56% 57%

Evening/Weekend vs. Daytime Classes Enrollment Asian African American Native American Latino/Hisp White International Other/2 Race All Evening+Weekend 5 (11%) 8 (18%) 3 (7%) 19 (42%) 2 (4%) 45 Day 32 (19%) 15 (9%) 4 (2%) 30 (18%) 66 (39%) 11 (7%) 10 (6%) 168 Completion (C or Higher) Asian African American Native American Latino/Hisp White International All Evening+Weekend 80% 75% 67% 95% 50% 84% Day 78% 60% 57% 76% 82% 73% Finish with “I know the information from my program much better and I can come up with ideas for changes that will make the most significant improvements.”

Reflective Practice Using Data Debra Bragg & Heather McCambly

SAGE 2YC Outcomes Assessment Template

Reflective Practice Gibbs’ (1998) Reflective Cycle Description: What happened? Feelings: What were you thinking and feeling? Evaluation: What was good and bad about the experience? Analysis: What sense can you make of the situation? Conclusion: What else could you have done? Action Plan: If it arose again what would you do? Reflective Practice

Equity-Minded Practice Color-blindness Diversity Equity Equity- Mindedness

What is Your Data Story? What do your data tell you about your geoscience program, courses and students? What data points are most compelling and most challenging? What type of story is emerging? Growth or decline Universal or singular Inclusive or exclusive Equity or inequity What story do you want to tell?

Discussion Questions What story do your data tell? What else would you like to know? What have you done to reflect on your data? What will you do to engage others in equity-minded reflection in the future?

Outcomes Assessment: Using Data to Direct and Amplify Change   Outcomes Assessment: Using Data to Direct and Amplify Change

Step 1: What are we changing?   Past What has changed? Future What will change? Self Others Step 2: What could these changes… change? Reiterating a change hypothesis: Based on the past and future strategies, described above, we could expect to see a change in X measure over time in Y classes/groups.

Gallery walk Walk the room and review other change agent’s posters Use post-its to note where you see strong alignments between hypotheses and actions, or where you have questions Note if you think a different type of data would be needed to align with the change work Gallery walk

Debrief Changes Outcomes Collected Data Are your actions or plans, your hypothesis, and the data you’re collecting in alignment? If not, what needs rethinking?: The data collection to capture meaningful outcomes? The action plan to amplify potential strategies for impact? Changes Outcomes Collected Data

Connecting assessment data and your action plans If-then statements If I change the introductory course to make it more relevant, then I will see more students, including more women, taking a second environmental science course If I change the experience in oceanography so that it is more supportive of students, then I will see more students, including more students of color, asking about oceanography careers and 4YCU oceanography programs If I include more exposure to career possibilities, workplaces, and professionals, then more of our students, including first-generation students, will transfer to geoscience programs at our neighboring 4YCU. A hypothesis that you will test, backwards design and alignment

Possible approaches Practices Events Programs Policies Complementary, collectively contribute to a comprehensive strategyPractices include what you do in your teaching and advising

Observations Students of color are underrepresented in geoscience courses, across the board While 20% of the students in my courses identify as Hispanic or Latino, only 50% of those students earn a C or higher Of the students who take more than one geoscience course, 80% are male

What might you choose to do in each case? Students of color are underrepresented in geoscience courses, across the board Think about what you could do to attract more students of color into geoscience courses. Survey current geoscience students about the departmental climate? Add science spotlights to your courses, to try to improve the experience of students of color who do take geoscience courses? Organize and promote participate in a career fair? Work with your institution’s advising staff for incoming students? While 20% of the students in my courses identify as Hispanic or Latino, only 50% of those students earn a C or higher Think about how you can support Hispanic/Latino students in geoscience courses. Add societally-relevant course content? Teach metacognitive strategies? Add academic support for all students? Ask your institution’s Multi-Cultural Affairs office for advice? Add a values-affirming writing assignment to all of your courses? Of the students who take more than one geoscience course, 80% are male Think about what you could do to interest more women in the geosciences. Actively recruit all female students who earn a grade of C or higher in any geoscience course to take additional geoscience courses? (What percentage of female students earn a grade of C or higher in their first geoscience courses?) Add science spotlights that focus on female geoscientists? Expand on measures of success. What change do you want to see.

Connecting assessment data and your action plans: How will you know if you are successful? Now that you’ve reflected on your data and your work in this session, what are some ideas for what you might do differently in the next year in your teaching? In your program? And how will you know the impact? What would you consider to be a measure of success? Your task: Generate some ideas for your action plan based on your outcomes assessment data, including the data related to equity. Make a direct link between each strategy and the outcomes. Write for 10 minutes Share Report out backwards design and alignment, outcomes (measures of success)