Monitoring and measuring the development of earth education pathways: Strategies learned from the EarthConnections project Ellen Iverson, Karen Peterman, Barbara Nagle, Mitchell Awalt, Cathy Manduca

A Collective Impact Alliance Common Agenda Shared Measurement Systems Mutually Reinforcing Activities Continuous Communication Backbone Support Organization *Stanford Social Innovation Review: Kania and Kramer, 2011, Collective Impact: http://www.ssireview.org/articles/entry/collective_impact

Shared Measurements Align with shared vision Promote organizational learning Gauge engagement levels Demonstrate progress in measurable way

Shared Measurement System Three Strategies: Discourse-centered checkpoint process Web-based communications dashboard Student outcomes measures Icecream image provenance: http://www.uzanafest.com/free-printable-measurement-worksheets/ccss-2-md-1-worksheets-measuring-worksheets-5/

EarthConnections Pathways: 1) Are embedded in the local community 2) Use geoscience to address a local need or problem 3) Connect learning opportunities at multiple educational levels and into the geoscience workforce 4) Couple opportunities for classroom learning with opportunities to use knowledge in community service at each level 5) Use mentoring and signposting to support students in recognizing and navigating the pathway.

Checkpoint Process Regional alliance routine reporting Online and virtual discussion feedback given Visual and narrative pathway rubric

Checkpoint Process Regional alliance routine reporting Online and virtual discussion feedback given Visual and narrative pathway rubric

Checkpoint Process Regional alliance routine reporting Online and virtual discussion feedback given Visual and narrative pathway rubric

Pathways Rubric 1) Are embedded in the local community 2) Use geoscience to address a local need or problem 3) Connect learning opportunities at multiple educational levels and into the geoscience workforce 4) Couple opportunities for classroom learning with opportunities to use knowledge in community service at each level 5) Use mentoring and signposting to support students in recognizing and navigating the pathway.

Pathways Rubric 3) Connect learning opportunities at multiple educational levels and into the geoscience workforce https://serc.carleton.edu/earthconnections/planning_tools/monitor_pathways.html

Pathway Rubric

Pathway Rubric

Regional Alliances Checkpoint 7 (6/26/18) 3 4 5 Mini 7 Atlanta Oklahoma San Bernardino Embedded in a community A1 Geo Ed leaders identified A2 Geo Ed leaders invested A5 Articulat-ed over-laps A3 Outside leaders identified A4 Outside leaders invested A6 Emerged identity Use Geoscience to contribute to a local problem B1 Articulat-ed B2 Connect-ed to Geo B3 Decided focus B4 Designed B5 Evaluated B6 Ready to grow Multiple learning opportunities connected and sequenced C1 Under-stand levels C2 Decided focus C3 Planned C4 Piloted C5 Evaluated early indicators C6 Ready to scale Classroom learning coupled with opportunities in community service D1 Identified D2 Designed D3 Evaluated D4 Demon- strated Mentoring and signposting supports students E1 Identified Student skills E2 Identified Coach strategies E3 Piloted E4 Evaluated early indicators E5 Ready to scale

Regional Alliances Checkpoint 7 (6/26/18) 3 4 5 Mini 7 Atlanta Oklahoma San Bernardino Embedded in a community A1 Geo Ed leaders identified A2 Geo Ed leaders invested A5 Articulat-ed over-laps A3 Outside leaders identified A4 Outside leaders invested A6 Emerged identity Use Geoscience to contribute to a local problem B1 Articulat-ed B2 Connect-ed to Geo B3 Decided focus B4 Designed B5 Evaluated B6 Ready to grow Multiple learning opportunities connected and sequenced C1 Under-stand levels C2 Decided focus C3 Planned C4 Piloted C5 Evaluated early indicators C6 Ready to scale Classroom learning coupled with opportunities in community service D1 Identified D2 Designed D3 Evaluated D4 Demon- strated Mentoring and signposting supports students E1 Identified Student skills E2 Identified Coach strategies E3 Piloted E4 Evaluated early indicators E5 Ready to scale

Web-based Communications Dashboard Engagement Measures Number of email messages by list Number of email list members Visits to internal workspace pages Participation in meetings National alliance Alliance Leadership Regional Alliance

Web-based Communications Dashboard Leadership engagement Email list messages Meeting attendance Monthly workspace visits

Proposed Areas of Interest Ability of regional partners to identify shared goals related to supporting the movement of underrepresented students along the pathway toward the geoscience workforce, Improved geoscience capacity within the regional partner institutions, Student progress toward geoscience-related careers, and Impact on regional capacity to increase resilience in the face of geologic hazards and limited geologic resources.

Student Outcomes Measures Proven validity Demonstrated evidence that the relevant student measures are predictive of student success Aligned with IRB-approved research design Demonstrated usability with age groups and duration of activities

Student Outcomes Measures Values in Science Survey Engagement in Science Survey Career Interest Survey Values in Science Survey: Measures Technical Brief: Values in Science. (2016). Science Learning Activation Lab. Engagement in Science Survey: Measures Technical Brief: Engagement in Science. (2016). Science Learning Activation Lab Career Interest Survey: Tyler-Wood, T., Knezek, G., & Christensen, R. (2010). Instruments for assessing interest in STEM content and careers. Journal of Technology and Teacher Education, 18(2), 341-363.

Strengths and Limitations Promoted communications and intentional learning between alliances Provided enduring record of pathway development Identified challenges early Took time and effort for reporting Missed offline regional alliance engagement communications Failed to fully capture non-sequential development and resets

A Collective Impact Alliance Common Agenda Shared Measurement Systems Mutually Reinforcing Activities Continuous Communication Backbone Support Organization *Stanford Social Innovation Review: Kania and Kramer, 2011, Collective Impact: http://www.ssireview.org/articles/entry/collective_impact

Monitoring and measuring the development of earth education pathways: Strategies learned from the EarthConnections project https://serc.carleton.edu/earthconnections/planning_tools/index.html