The Power of Being Explicit: Thoughts on Program Planning and Assessment Cathy Manduca SERC at Carleton College

Goals Based Design Goals: How do you want your students to be different at the end? (What are you and your colleagues trying to accomplish through your program?) Assessment: How will you tell if you did it? Design: How do you design your program so that students will meet your goals (and thus succeed with your assessment)? Assessing Geoscience Programs: February 2009

Goals and Success Core Geoscience Knowledge Develop Professionals/Leaders Foundation for Problem Solving/Addressing Prepare for Careers/Graduate Studies Critical thinking and communication skills Understand Complex systems Visualization skills Assessing Geoscience Programs: February 2009

Theory of Change and Logic Model: Tools from Program Assessment Theory of Change: provides a rational basis for program design Logic Model: describes inputs, program, outcomes, intended consequences For more info: google ‘theory of change’, ‘logic model’

Building Strong Geoscience Departments Theory of Change: If more departments know more about other departments, their programs, challenges, and successes, each will individually be better positioned to grow and change, and the community as a whole will be better able to advocate for geoscience education. Making use of ideas from other departments requires a thoughtful process that adapts them for the local departmental and institutional context. Assessing Geoscience Programs: February 2009

Interested Department Learn about other departments strategies and successes Consider how to use this info in own context Improvements in Department Program Logic Model: Develop action plan, consider barriers, build team approach, develop progress measures Build team Implement Action Plan Build team Implement Action Plan Measure progress Refine plan – use website Talk to colleagues Measure progress Refine plan – use website Talk to colleagues More characteristics of thriving departments in your department More thriving departments nationwide More characteristics of thriving departments in your department More thriving departments nationwide Better educational opportunities Better student learning Articulate current state and areas for action Articulate current state and areas for action

A hypothetical example Goal: Articulate role of geoscience in society Theory of Change: The ability to discuss a topic depends on knowledge, personal experience, and practice communicating Incoming student Equates geoscience with earthquakes Limited experience recognizing geoscience in public issues Incoming student Equates geoscience with earthquakes Limited experience recognizing geoscience in public issues Intro Course Activities on climate change Case study of wind energy in local region Intro Course Activities on climate change Case study of wind energy in local region Interest in geoscience and society motivates major Core Major Course Detailed understanding of mineral and energy resources Core Major Course Detailed understanding of mineral and energy resources REU Presentation for public audience REU Presentation for public audience Geology Club Leadership role on city council sustainability project Geology Club Leadership role on city council sustainability project Conversation in Lounge Spontaneous discussion of news Conversation in Lounge Spontaneous discussion of news Electives Service learning project on flood planes Policy-geoscience connections Global energy balance Electives Service learning project on flood planes Policy-geoscience connections Global energy balance Knowledge Practice making connections Practice speaking and making arguments Practice addressing real world problems Knowledge Practice making connections Practice speaking and making arguments Practice addressing real world problems Student can spontaneously discuss role of geoscience in a local or global issue with confidence and plausible basis

Using your logic model Assessing Geoscience Programs: February 2009 Building consensus: Do members of your department share your understanding of the goals, theory of change, logic model? Trouble shooting: Does it all make sense now that it is laid out? Problems with alignment? Program design: Could it be strengthened? Are their obvious connections/synergies among goals? Among elements? Basis for assessment Refine as a communication strategy

Your Turn 15 minutes Articulate a single departmental goal Articulate what success looks like Sketch the activities in your program that support the student in achieving the goal (make sure it includes the incoming state and the successful state) Add the pieces that say why/how these activities are supporting the goal Articulate the theory of change 20 minutes Discuss what you learned at your table: aha moments; what is useful in this activity; what was hard; how you solved problems

Geoscience Departments What is our/your Theory of Change: Role of courses Research experiences Repeated opportunities to practice Field work Role of mentoring and peer interaction Extra/co-curricular activities Assessing Geoscience Programs: February 2009