1. Educational Practices and Outcomes Assessment of SAGE 2YC
Connect to the webinar:
Please use phone or skype for audio

2. Educational Practices and Outcomes Assessment of SAGE 2YC November, 2016
Heather Macdonald, College of William & Mary
Debra Bragg and Heather McCambly, Bragg & Associates
Pamela Eddy, College of William & Mary
Ellen Iverson, SERC

4. Webinar Goals and Overview
Review SAGE 2YC project, goals, and theory of change
Introduce plan for collecting data that will allow us to measure project outcomes
Educational practices inventory
Outcomes assessment, common outcomes measures
Q & A about the plan

5. SAGE 2YC Project Team Evaluation & Research Project Leaders SERC Staff
Heather Macdonald
Eric Baer
Debra Bragg
Pamela Eddy
Carol Ormand
John McDaris
Heather McCambly
Ellen Iverson
Jan Hodder
Norlene Emerson

6. SAGE 2YC Teams and Their Work
SAGE 2YC Teams: 24 faculty Change Agents in 10 regions
+ Administrators
+ Institutional Research
Action plans developed at June 2016 workshop considering three project strands
Individual
Geoscience program/department
Regional workshop
Project strands
Supporting academic success of all students
Broadening participation
Facilitating students’ professional pathways

7. Faculty learn from research Faculty make changes in their own practice
SAGE 2YC Goals
Cycle of Innovation
Build a national network of 2YC geoscience faculty and administrators led by 2YC faculty leaders, called Change Agents, who catalyze change at multiple levels
Implement high-impact, educational practices that:
Support the academic success of all students
Broaden participation
Facilitate professional pathways into the geosciences
Investigate professional development models that support the cycle of innovation to improve geoscience education
Faculty learn from research
Faculty make changes in their own practice
Faculty share what they are learning with the geoscience education community

8. Change Agent Action Plans
Theory of Change Logic Model
Outcomes of Educational Practice Changes to 2YC Geoscience Education in:
Access
Participation
Course Success
Pathway Progression +
Educational
Practices
SAGE 2YC Leadership Team Support
Change Agents
Change Agent Action Plans
SAGE 2YC Professional Development
Cycle of Innovation

9. Educational Practices Inventory
Research-based practices related to the project goals
Academic success of all students
Broadening participation in the geosciences
Facilitating students’ professional pathways
Two inventories of faculty practices:
What individual faculty do
What faculty do collectively, in the context of their department or program
Individual inventory ~ 20 minutes

10. Outcomes Assessment Goals
Assess implementation of educational practices
Apply outcomes assessment to your geoscience program(s) and course(s)
Link outcomes assessment to the three strands of SAGE 2YC:
Academic success
Broadening participation
Pathways progression – transfer and career

11. Snapshot of Outcomes Assessment Measures
Participation
Geoscience Course Enrollment
Broadening Participation in Geoscience Courses
Course Success Outcomes
Success in Geoscience Courses
Geoscience Course Success by Student Sub-group
Pathways Progression Outcomes
Enrollment in Multiple Geoscience Courses
Major in Geoscience Program

12. Approach
Use Outcomes Assessment Worksheets, with instructions provided by the SAGE 2YC evaluators, Debra Bragg & Heather McCambly
Partner with institutional research (IR)
Engage college administration to help secure data, if needed
Work collaboratively with Debra Bragg and the leadership team (led by Heather Macdonald) on your assessment plan
Start the outcomes assessment in Spring 2017 and continue to the end of the grant
Partner with institutional research (IR) staff to collect, disaggregate, and format data
Does not require primary data collection by change agents, unless individuals elect to adopt an optional strategy/ies
A SAGE 2YC Outcomes Assessment Template has been developed and will be provided for clarity and as one formatting option, however you can work with your IR partners to develop formats the work best for you and them
Teams will collect data for all geoscience courses, but will work with IR to “flag” or group courses into “changed” or “unchanged” categories, if applicable. i.e., if there are courses you do not expect to change over the course of this grant, they will be measured but will not be considered as part of the group receiving a specific “treatment”
The following are common measures to be collected across all teams, however you may add additional (optional) strategies to measure changes specific to your project

13. Outcome Measure: Geoscience Course Enrollment
Method
List geoscience courses offered by your college
For each term, indicate whether course is offered and whether the course has received an “educational change”
For each geoscience course offered, count the total number of students enrolled in each course at census date.
Question: What is the enrollment in geoscience courses, including courses receiving an “educational change”, and how does course enrollment change from Spring 2017 to the end of the grant? Measure: Term-by-term enrollment at census date in geoscience courses, including geoscience courses identified as having received an “educational change”, from Spring 2017 to the end of the grant.

14. Outcome Measure: Broadening Participation in Geoscience Courses
Question: What is the enrollment of underrepresented student groups in geoscience courses, and does geoscience course enrollment by underrepresented student groups change from Winter/Spring 2017 to the end of the grant?
Measure: Term-by-term geoscience course enrollment by student sub-group (e.g., gender, race, ethnicity).
Method
For each course and by term offered, count students affiliated within sub-groups by gender, race, ethnicity, and other demographic variables (according to IPEDS definitions) at census date.

15. Outcomes Measure: Success in Geoscience Courses
Method
For each course and by term, count the number of students who completed and passed the course with a grade of “C” or higher.
Compute percentage of students deemed successful (#1) by the total enrollment in same geoscience course.
Question: How many students and what percentage of students in geoscience courses succeed, and does course success change from Winter/Spring 2017 to the end of the grant?
Measure: Term-by-term geoscience course completion with a grade of “C” or higher.

16. Outcome Measure: Geoscience Course Success by Student Sub-Group
Question: By sub-group, how many students succeed in geoscience courses, and does the percentage of course enrollee success change from Winter/Spring 2017 to the end of the grant?
Measure: Term-by-term geoscience course completion with a grade of “C” or higher by student sub-group.
Method
For course and by term, count the number of students within sub-groups who completed and passed the geoscience course with a grade of “C” or higher.
Divide #1 by the total count of students within sub-groups who enrolled in the same geoscience course.

17. Outcomes Measure: Pathways Progression
Question: How many students take more than one geoscience course, how many students major in geoscience, and do these numbers change from Winter/Spring 2017 to the end of the grant?
Method:
Student enrollment in more than one geoscience course.
Students with declared major or concentration in geoscience.
Method
By course and by term, count the number of students who enroll at census date who have enrolled in more than one geoscience course.
By course and by term, count the number of students who enroll at census data who are declared geoscience majors.
Added text in read

18. Outcomes Measure: Pathways Progression by Student Sub-Group
Question: How many students by sub-group who take more than one geoscience course and major in geoscience? How do these numbers change from Winter/Spring 2017 to the end of the grant?
Method:
Student sub-group enrollment in more than one geoscience course.
Student sub-group with declared major in geoscience.
Method
By course and by term, count the number of students by sub-group who enroll at census date who have enrolled in more than one geoscience course.
By course and by term, count the number of students by sub-group who enroll at census data who are declared geoscience majors.

19. Outcomes Measures Participation and Access Course Success and Access
Term-by-term enrollment at census date in geoscience courses
Term-by-term geoscience course enrollment by student sub-group
Course Success and Access
Term-by-term geoscience course completion with a grade of “C” or higher
Term-by-term geoscience course completion with a grade of “C” or higher by student sub-group
Pathways Progression and Access
Student enrollment in more than one geoscience course
Student sub-group enrollment in more than one geoscience course
Students with declared major or concentration in geoscience
Student sub-group with declared major in geoscience

20. Timeline for Outcomes Assessment
November 2016 – this webinar
Mid-December 2016 – Outcomes Assessment Guide and Worksheets shared with Change Agents
January 2017 – telephone conference calls to review outcomes assessment plan with Change Agents
Educational practices inventories
Individual: December January 2017
Department: Early winter, 2017
Winter/Spring 2017 – Change Agents launch outcomes assessment process
June 2017 – Change Agent Annual Workshop in Tacoma, WA

21. Outcomes Assessment Resources
Primary Sources:
Change Agent Action Plans
SAGE 2YC Proposal and Logic Model
Other Sources:
Gates Foundation/Institute for Higher Education & Policy’s scan of 20+ improvement initiatives
Alliance for Quality Career Pathways Framework
The Voluntary Framework for Accountability
Pathways to Results Equity & Outcomes Templates

22. Questions?
Contacts:
Outcomes Assessment: Debra Bragg,
Inventories: Pamela Eddy,
Overall project: Heather Macdonald,

23. Webinar evaluation form:
Thank you!
Webinar evaluation form: