How to Make Meaning from your Data: The InTeGrate Attitudinal Instrument Kim Kastens & Vali Mara InTeGrate External Evaluation Team For the Materials Development Workshop Carleton College June 21, 2016 Inputs Activities Outcomes Outputs

Progress on nationally- urgent problems: • Resources: - Minerals Increased quality, quantity and diversity of Geoscience professionals in workforce Increased motivation and ability of Geoscience professionals to address resource and environmental Grand Challenges Progress on nationally- urgent problems: • Resources: - Minerals - Energy - Water - Food • Environment: - Climate change - Natural hazards - Waste disposal - Environmental degradation Increased Geo- understanding of other professionals in workforce Increased Geo- understanding of the public (voters, consumers) Teachers with ability to tie Geo concepts to sustainability challenges

Evaluation examines the effectiveness of InTeGrate in the aggregate, and does not monitor individual students, teachers, or modules. We focus on the flock, not the birds. http://www.pbs.org/wgbh/nova/sciencenow/3410/03-ever-01.html

Students’ knowledge about the Earth, oceans, atmosphere and climate Geoscience Literacy Exam (GLE) Students’ interest in Earth-related careers and college major; also student demographics Students’ motivation to contribute to addressing grand challenges of environmental sustainability This talk InTeGrate Attitudinal Instrument (IAI) Essay questions on interdisciplinary problem solving & system thinking Students’ ability to contribute to addressing grand challenges of environmental sustainability You will be hearing about the embedded summative assessments and essay questions from your assessment team member. Embedded summative assessments Students’ mastery of the learning goals established for the module or course

InTeGrate Attitudinal Instrument (IAI) Administered online pre- and post-instruction Used in pilot tests of new materials, some Implementation Programs, and research project. Probes: Career interests and college major selection Concern about environmental issues and motivation towards sustainability behaviors Demographics New for 2016 Materials Developers’ Meeting: Toolkit for analyzing IAI data for a subsample you care about Built in Microsoft Excel Pre-populated with a project-wide data set from 1125 students who used InTeGrate modules from Fall 2012 to Spring 2014 One Excel spreadsheet per IAI item Paste in your data, and spreadsheet generates graphs and tables comparing your data with the project-wide sample.

Go to Excel now Demo SusGrph first Then demo SusEmp

SusGrph: People differ in how motivated they are to take action in their personal and professional lives to create a more environmentally sustainable society. Which of the following graphs best represents your degree of motivation before and after taking this course? Shown in Excel Demo (2014 data: question asked on post- only: non-respondents omitted)

CarGrph: Which of the following graphs most accurately depicts your level of interest in a career in Earth or Environmental Sciences before and after taking this course or studying this module?

CarGrph: Which of the following graphs most accurately depicts your level of interest in a career in Earth or Environmental Sciences before and after taking this course or studying this module?

SusEmp: As you consider employment after graduation, how important is it to you to work in an organization committed to environmentally sustainable practices (independent of the field)? Examples of environmentally sustainable practices would include minimizing energy and water use in the workplace. Please select one number: [Likert scale 1 = not important to 7 = very important] Shown in Excel Demo

UseKno: As you consider career directions after graduation, how important is it to you to do work in which you use your knowledge of the Earth and environment? Please select one number: [Likert scale 1 = not important to 7 = very important] 4.7 4.1 4.8 4.2

EnvCrn: Please indicate your level of concern about each of the following potential developments on the Earth. Focus on the impact on your region in your lifetime.

EnvCrn: Please indicate your level of concern about each of the following potential developments on the Earth. Focus on the impact on your region in your lifetime.

EnvCrn: Please indicate your level of concern about each of the following potential developments on the Earth. Focus on the impact on your region in your lifetime.

SusAct: Please indicate the extent to which you have engaged in the following activities during the past week:

SusAct: Please indicate the extent to which you have engaged in the following activities during the past week:

SusAct: Please indicate the extent to which you have engaged in the following activities during the past week: 6.0 6.5 Index is constructed by summing number of sustainable activities respondent engaged in at least once per week.

WhySus:. When you engage in behaviors such as those listed in the previous question, what factors or sources of information influence your decision to do so? (mark all that apply):

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( ) yes ( ) no If yes, how? If no, why not?

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( ) yes ( X ) no If yes, how? If no, why not?

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( ) yes ( X ) no

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( X ) yes ( ) no

Questions? Comments? Kim Kastens kastens@ldeo.columbia.edu Vali Mara vmara@ciesin.columbia.edu