1. Assessment and Evaluation Feedback to Teams
Summer 2016

2. Data Collected GLE Multiple Choice Pre/Post
Essays (Interdisciplinary, Systems)
Summative Assessments
Attitude and Career Interest Pre/Post
Demographic data

3. GLE Summary
Globally: Consistent 10% normalized gain regardless of starting point (= 1 point)
Individual modules/courses have small number students with data
Most gains in lower quartile
Module students generally show higher normalized gains than control students on individual items

4. GLE by Subgroup
Norm gain courses = 8%; Modules = 9%; Implementation = N/A; control = 8%. When analyzed by question, course & module gains were spread across GLE items; control gain attributable to Climate Level 2 question.

5. Most Compelling GLE Findings
Lowest quartile showed
the greatest gain

6. GLE by Unit Summary

7. Interdisciplinary Essay
Knowledge of Earth system interactions can influence how people make decisions about global challenges. Identify and describe a global challenge that society will likely face in the next 50 years. Explain how the science related to that challenge informs economic, social, and/or political decision making related to the global challenge you described.
This is the interdisciplinary question. Will score later, but for time being are investigating if InTeGrate-related concept shows up in answers.

8. Interdisciplinary Essay
Gen Ed Control:
n = 153
Coastal Processes:
n = 33
Sea Level Rise
Carbon, Climate & Energy
n = 63
Fossil Fuels
Course focusing on issues related to oceans and their impact on society
Module focusing on carbon cycle and fossil fuel formation
Climate Change
Fossil Fuels
Control – no InTeGrate materials; student-centered Earth Science class for non-majors; their answers reveal climate change and fuels most common.
Show Coastal and CC&E – one course, one module
Take home – Interdisciplinary essay topic reflect content of module/course
Global Change last topic
Fossil Fuels not covered

9. Complex System
A systems thinker can identify a system (a natural system, a human system, a linked human/environment system), understand how that system can be divided into interacting parts, and recognize that changes in one part of the system will affect other parts of the system.
Give an example of a real-world system and describe its parts.
Explain how parts of the system interact. Use systems concepts in your explanation (e.g., positive and negative feedbacks, equilibrium, rates, etc.)
Using your example system, discuss how an effect in one part of that system can be influenced by multiple causal factors.

10. Systems Essay
Scores based upon a 4-point rubric.
Systems thinking is still a difficult concept the requires greater effort to improve student understanding.

11. Summative Assessment InTeGrate Guiding Principles
Grand Challenges
Interdisciplinary
Habits of Mind
Uses Data
Systems Thinking
Module/Course Objectives
Varies in number

12. Evaluation examines the effectiveness of InTeGrate in the aggregate, and does not monitor individual students, teachers, or modules. We focus on the flock, not the birds.

13. What you can expect
Spreadsheet of GLE Pre- Post- answers by team (Kim will discuss by Skype on Tuesday)
Spreadsheet of Attitude Pre- Post- answers by team (Kim will discuss by Skype on Tuesday)
A link will be available on your reporting page.
Evaluation of assessments and student work

14. How to use this feedback
GLE/Attitude – analyze as you wish. Since numbers are small, analysis has limited utility
Essays – Consider overall quality of answers and how they do or do not relate to module
Evaluation of assessments – exploit your Assessment Consultant!!!
Address as warranted during revisions