1. Student Sense Making About Climate Change-Related Data Joyce Parker 1, Beth Covitt 2, May Lee 1, and Charles Anderson 1 1 Michigan State University, 2 University of Montana This research is supported by grants from the National Science Foundation: A Learning Progression-based System for Promoting Understanding of Carbon-transforming Processes (DRL 1020187), and Sustaining Responsive and Rigorous Teaching Based on Carbon TIME (NSF 1440988 ). Additional support comes from the Great Lakes Bioenergy Research Center (DOE Office of Science BER DE-‐FC02-‐07ER64494), funded by the United States Department of Energy, from Place-based Opportunities for Sustainable Outcomes and High-hopes, funded by the United States Department of Agriculture. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, the United States Department of Energy, or the United States Department of Agriculture. Context StudentsModels & ExplanationsData & Arguments from Evidence undergraduate, non-science majors = stand in for educated public  Explain Keeling curve  Interpret data about effects of increased CO 2  Relate data sets  Evaluate claims Interpret global & local CO 2 & climate data  Determine generalizability  Identify trends or patterns Preparation for Future Learning Identify holes in understanding Seek and use new info Research Goals Develop a learning progression (LP) framework that describes how students make sense of large-scale data, and in particular, climate data Extend LP framework to include preparation for future learning Explore how quantitative reasoning relates to capacity to employ scientific models in explanations Methodology iterative, design-based research use assessment responses to articulate a spectrum of less to more sophisticated sense-making; our four progress variables include: ⇢ Identifying patterns in large scale data – distinguishing signal from noise ⇢ Explaining patterns & relationships in data sets, evaluating claims – whether/how students employ scientific models in explanations ⇢ Systems thinking – whether/how students relate local data to global system ⇢ Preparation for future learning – ability to Identify weaknesses in own understanding, seek relevant info, and use new info to expand model Data Sources AOP - Undergraduate, non-science majors at freshmen orientation ISB – Undergraduate, non-science majors enrolled in required life science lab Jan 2015: ~500 pre- post sets from ISB students; analysis of ~100 matched pairs Summer/Fall 2015: 11 interviews of AOP and ISB students Initial Findings from Written Responses A plurality of students provided responses at the intermediate level for all progress variables. Students: identified broad trends in complex but non-noisy data like annual cycle and long-term increase in CO 2 levels (75%), or noisy data like downward trend in extent of Arctic ice (49%). overlooked specifics such as accelerating rate of increase in CO 2 or particular months CO 2 levels fall. explained most trends with simple narrative of human activity producing CO 2, which contributes to global warming. Only 6% accurately explained why CO 2 falls between May and October. 33% proposed seasonal changes in human activity (extending their simple model) to explain summer levels without noticing this could only explain a leveling, not a decrease in CO 2. looked for specific, but not pertinent info to improve explanations (49%). For example, wanted to know about effects of decreasing Arctic ice on local animals when asked what info would help better understand how/why ice is melting. Implications. Non-science majors and entering freshmen represent an educated, but non-expert public. Our finding that a plurality of students has an intermediate level of understanding, even after taking a data analysis lab course, is disturbing. Students see trends in broad strokes, whereas clues that might inform their models are often in the specifics. They have simple models to start from, and have difficulty extending them to accommodate new data. They do not recognize when their models do not adequately explain data. They are not adept at identifying pertinent info to extend understanding. These points imply that students will have trouble with future learning needed to make sense of the continually growing climate science knowledge domain. Carbon: Transformations in Matter and Energy Identification of Trends Explanations of Trends, Relationships among Data Sets, & Evaluations of Claims Relation of Local Data to Global System and Local Phenomena Preparation for Future Learning Sample Assessment Items What do you think is the cause of this pattern? [see previous Q] You pointed out that Figure 1 suggests that the carbon dioxide level is trending up and Figure 2 suggests that the amount of Arctic ice is trending down. Do you think that carbon dioxide levels and extent of Arctic sea ice are related? How? As stated earlier, the data in Figure 1 are based on measurements taken at the top of Mauna Loa, Hawaii. Do the data in Figure 1 tell us anything about how carbon dioxide concentrations in Michigan might be changing? Why or why not? You can see that the amount of CO 2 going into the atmosphere from fossil fuel use is small compared to the other inputs. Some people argue that means we don’t need to worry about it. Do you agree? Can you show me how you would use the internet to further explain your answer? What else would you like to know to better explain how carbon dioxide levels are changing and why? High Level Identifies reasonable trends in variable data. (Arctic Sea Ice, Gull Lake ice) Identifies multiple trends in complex data. (Keeling curve) Descriptions of trends include important specifics. (months that carbon dioxide is highest & lowest, increasing variability in amounts of ice) [Description of annual cycle of Keeling curve] Annually, CO 2 peaks in April and is lowest in October. This is consistent from 1960-2015. [The graph of CO 2 levels shows] a positive shaped line slowly increasing in slope. Uses an accurate model in explanations or evaluations. Model includes cause and mechanism All right, so the fossil fuels we burn, they obviously have carbon in them; they’re releasing the carbon dioxide from burning the fossil fuels, which we used all the time in modern culture. Yeah, so it’s [CO2 levels] increasing and decreasing semiannually, which I think is from my science class study; in the summer when the plants are growing they’re taking in more carbon dioxide would lead to this decrease here on a semiannual basis. And then when the plants are gone, like during the winter, there’s not as much carbon sequestration, which is leading to increased carbon dioxide. Recognizes which and how variables or local components are representative of global system. Correctly articulates structure and function of system components across temporal and spatial scales. [Does Hawaiian data tell us anything about MI CO 2 ?] Well I guess we’re both in the northern hemisphere, so I think it could give like a rough idea. I think for a more exact figure for Michigan, you’d have to probably do studies closer to Michigan. But it would kind of get you a general idea of what’s happening somewhere else in the northern hemisphere around the globe.  Identifies critical weaknesses in own understanding  Is specific about aspects does not understand  Seeks and uses new info to expand working model I know for sure that the carbon dioxide’s going up. I’m not sure if the Arctic [ice melting] is exactly the main cause or if it’s one of the secondary causes. But I mean I could confidently say that it’s a rising trend [in the Keeling curve] because the bottoms are also rising when you kind of show it, but I do find it very interesting. Oh that made it worse. Very interesting that it goes down as well. That is what kind of made me wonder, right, when I saw all the rise as well as dropping, what’s changing to make it drop and then rise? Intermediate Level Trend descriptions are incomplete, partially inaccurate, or non-specific. It [CO2 levels] like goes up and down, but gradually increases as it goes. [Description of trend in graph of extent of Arctic Sea Ice over time] Yeah, it looks like the square mileage of the, or the square kilometers of the ice is decreasing; not exactly on the same semiannual constant rate that the CO 2 was increasing. But over time, they have lost ice in the Arctic. Explanation includes cause without a mechanism. Explanation is inaccurate, but includes a cause and mechanism. [ Explaining long term trend in Keeling curve] Probably the melting icecaps in the Arctic. There’s a bunch of CO 2 that’s sequestered in the ice, and as it melts it’s released into the atmosphere. [Explaining annual cycle of Keeling curve] It gets warmer so less people use fossils and when its colder more people use them. Recognizes how some, but not all variables or local components are representative of global system. Correctly articulates structure and function of some system components across temporal and spatial scales. I think yeah, at some point, humanity will feel the effects of global warming. I don’t know, maybe not in my lifetime, but in my kids’ lifetime; my grandkids’ lifetime, there’ll be the increased temperature will lead to melting ice, which will lead to rising sea levels, which obviously will change the topography of the earth because there’ll be lands that are there now won’t be there in 50, 100 years. So I don’t know if necessarily it will affect me, but I think at some point we have to make a change, otherwise it will affect future generations.  Identifies less relevant weaknesses in own understanding  General rather than specific about aspects they do not understand  Seeks but doesn’t use new info to expand working model I would like to know more factors contributing to the changing levels [of CO2]. So what are you most confident about [in your explanation of the annual cycle of the Keeling curve]? RESPONDENT: The plant growth in the summer. INTERVIEWER: Okay, so plant growth in the summer. And what are you least confident about? RESPONDENT: The other factors and if that’s even a factor in the up and down pattern. Low Level Does not identify reasonable trends in data Describes variables, but not trend Identifies variability as a trend Makes errors reading graphs [Description of long term trend in Keeling curve] The first pattern is how CO 2 concentration changes during 1960 - 2015. [Description of annual cycle of Keeling curve] after every few years it switches from down to up then up to down Unclear cause/effect relationship due to: Lack of explanation beyond what is in Q Covering law = things just go together Language is vague, unclear Wrong cause, no mechanism Improper linkage btw explanation and data Explanations aren’t linked to data; [Annual cycle of Keeling curve] Because it’s like a cycle like regulates itself, well it tries to but because of the humans trying it’s harder for it to regulate itself. So I feel like it going down is an attempt to regulate itself and then it keeps going back up. [Long term trend of Keeling curve] Volcanic activity could be a factor. Nuclear power plant use could be a factor. I’m not sure if they have power plants there. Fails to recognize how local components are representative of global system Fails to articulate structure and function of system components across temporal and spatial scales Relates components through irrelevant factors [Can the Arctic Ice data tell us anything about weather changing in Michigan?] No, because we don’t have ice like in the Arctic, you know, we don’t permafrost so. [Do the Hawaiian CO2 data tell you anything about CO2 concentrations in Michigan?] No. Because we have a completely different climate and I mean I think that the changes would be a lot different because we were talking about how the data rises and drops every year seasonally and our seasons are completely different. I mean right now outside there was like a blizzard yesterday. It’s just completely different. So I don’t think it’s really applicable.  Fails to identify weaknesses in own understanding  Seeks only confirmatory or irrelevant info  "I don't know" ends reasoning rather than begins it [ What else would you like to know to better explain how arctic ice is changing and why?] I would like to know how it impacts my everyday life. I don’t really know much about CO2 levels because I am a business major but I am interested in learning more. Figure 1. This graph shows changes in concentration of carbon dioxide in the atmosphere over a 50-year span. The measurements are taken at an observatory at the top of Mauna Loa, Hawaii. Describe one pattern you see in this figure. What long-term trend, if any, do you see in these data in Figure 2?