This research is supported in part by two grants from the National Science Foundation: Developing a Research-based Learning Progression for the Role of Carbon in Environmental Systems (REC ) and Learning Progression on Carbon-Transforming Processes in Socio-Ecological Systems (NSF ). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. ENVIRONMENTAL LITERACY Student Reasoning Related to Matter and Energy Flow through Ecosystems: Lessons from Diagnostic Question Clusters Laurel Hartley 1, Charles W. Anderson 2, Brook Wilke 2, Jonathon Schramm 2, Charlene D’Avanzo 3, Barbara Abraham 4, Amy Arnett 5, Alan Dickman 6, Heather Griscom 7, April Cordero Maskiewicz 8, Chris Picone 9 1 University of Colorado Denver, 2 Michigan State University, 3 Hampshire College, 4 Hampton University, 5 Unity College, 6 University of Oregon, 7 James Madison University, 8 Point Loma Nazarene University, 9 Fitchburg State College Laurel Hartley 1, Charles W. Anderson 2, Brook Wilke 2, Jonathon Schramm 2, Charlene D’Avanzo 3, Barbara Abraham 4, Amy Arnett 5, Alan Dickman 6, Heather Griscom 7, April Cordero Maskiewicz 8, Chris Picone 9 1 University of Colorado Denver, 2 Michigan State University, 3 Hampshire College, 4 Hampton University, 5 Unity College, 6 University of Oregon, 7 James Madison University, 8 Point Loma Nazarene University, 9 Fitchburg State College IntroductionResults Methods Example Item Examples (Informal) Examples (Scientific) Data from Sample Item For Further Investigation When the leaves in a compost pile decay, they lose mass. What do you think happens to the mass of the leaves? Circle True (T) or False (F). T F The mass goes away when the leaves decompose. T F The mass is converted into heat energy. T F The mass is converted to soil minerals. T F The mass is converted into carbon dioxide and water. F, T, F, T The mass does not leave the pile of leaves, it simply decreases. F, F, T, T The mass does not just go away when the leaves decompose. Mass cannot be converted into energy. The leaves get broken down into soil minerals. When the leaves break down, CO 2 is released and H 2 O. On pre-tests, 10% (n = 309) of students correctly traced matter. On post-tests, 16% (n = 164) did so. Why do some students think plants get carbon from the soil? Do students commonly see overly simplified gas-gas and solid-solid cycles? Do students think atoms can become other atoms? Why do some students think plants respire and photosynthesize, but only emit O 2 ? Consider the three diagrams. They represent three situations in which 100 kg of green plants serve as the original source of food for each of the food chains. In situation II, for example, cattle eat 100 kg of green plants and then people eat the beef that is produced by the cattle as a result of having eaten the plants. In which of the three situations is the most energy available to people? A) I B) II C) III D) Situations I and II will roughly tie for the most energy. E) The same amount of energy will be available to people in all three situations A remote island in Lake Superior is uninhabited by humans. The primary mammal populations are white-tailed deer and wolves. The island is left undisturbed for many years. Select the best answer(s) below for what will happen to the average populations of the animals over time. _____a. On average, there will be more deer than wolves. _____b. On average, there will more wolves than deer _____c. On average, the populations of each would be about equal. _____d. The populations will fluctuate, with sometimes more deer, sometimes more wolves _____e. None of the above. B, Cattle for 100 kg provides more calories for less compared to fish and green plants. C, Deer and wolves will both run into natural change with the environment. For example global warming/food shortages but the environment will keep the populations in check. C, The sun provides the most energy we use. The plants use the sun so really all energy comes from the sun. Every chain away from the plants is less energy because each animal uses some of the energy provided from the link before them. A, Because of energy lost in eating, the wolves need a large supply of deer to support their few numbers. Even though both questions require students to understand movement of energy through trophic levels, students performed better on the first question than on the second question. While 64% (n = 207) of students knew that a shorter food chain between producers and humans resulted in more energy for the humans, only 5% (n = 179) of students knew that predator numbers could not sustainably be greater than or equal to prey numbers in a closed system. Do students who can trace energy when questions are posed using typical “school science” representations fail to use principled reasoning when questions are posed in other contexts? Do students think water and nutrients, in addition to sunlight, provide energy for plant growth? At what scale are students most likely to use energy as a “fudge factor” in their reasoning? Once carbon enters a plant, it can … B) become part of the plant cell walls, protein, fat, and DNA. Circle True or False. Explain Carbon is essential for life in plants but it isn't found in fat / CO 2 could possibly seep into the cell walls of a plant Then biological processes in the cell turn those simple compounds into all the other compounds that the tree needs to live...proteins, lipids, nucleic acids On pre-tests, 6% (n = 300) of students were able to trace carbon from its entrance into an organism to its incorporation into molecular structures. On the post-test, 31 % (n = 140) of students were able to trace carbon across scales. Do students who can trace matter into and out of an organism often not understand the mechanisms by which matter is used/partitioned once in an organism? Table 1. A selection of results for each principle from both overall written scores and individual assessments. The last column indicates the type of questions for further study that are emerging from this dataset. 1.Most college student answers were a hybrid of scientific reasoning and informal accounts (Fig. 3). 2.Students demonstrated similar types of reasoning, both correct and incorrect, across the range of institutions. 3.Despite the fact that the type & frequency of active instructional interventions used by participating faculty varied, we observed learning gains pre- and post-instruction (Fig. 3). 4.Our results both corroborate patterns in student thinking identified in previous studies and lead to further hypotheses about student reasoning (Table 1). Fig. 1. Contrasting account frameworks for reasoning about ecological processes.  We investigated college students’ ability to trace matter and energy through processes that generate, transform and oxidize organic carbon at multiple scales. Our work focuses on college student understanding of  Principles: Conservation of Matter and Energy  Processes: Generation, Transformation, and Oxidation of Organic carbon  Scale: Atomic-molecular, Cellular, Organismal, Ecosystem  Carbon-transforming processes are a prominent part of college-level biology curricula, but ideas are typically presented in disconnected ways. We believe that teaching students to explicitly and continuously apply the principles of conservation of matter and energy can lead to a deeper understanding of processes across multiple scales. Fig. 2. Description of DQC development process.  We believe the pathway to scientific literacy requires students to learn to replace their informal reasoning and discourse with principled reasoning and discourse (Fig. 1).  Objectives:  Look for patterns in student reasoning related to principles, processes, and scales and generate hypotheses for further research.  Examine the effects of instruction on student understanding of carbon-related processes. We developed diagnostic question clusters (DQCs) to investigate college students’ reasoning about the carbon cycle. DQCs included 7-10 multiple-choice, true/false and short-answer questions about single processes (e.g. respiration) and multiple processes (e.g. respiration and photosynthesis) posed at scales from molecular to ecosystem. Faculty from 12 institutions (research universities, liberal arts colleges, community colleges) administered 4 DQCs, two focused on carbon cycling and two on energy flow, to 440 students in biology and ecology courses. One carbon and one energy DQC were used as pre- tests. Some form of inquiry-based instruction on the topics followed, with details of implementation at the discretion of the faculty. All four DQCs were then administered as post-tests, allowing analysis of differences among students both pre- and post-instruction, as well as between institutions, course types, and pedagogical approaches. For further details about the DQCs and particular items, please see our web site at: At one university, interviews were conducted to further explore student responses to written questions. This research is supported in part by a grant from the National Science Foundation: Diagnostic Question Clusters to Improve Student Reasoning and Understanding in General Biology Courses (NSF ). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Despite the fact that the principles of matter and energy conservation across multiple scales are fundamental to understanding biology, and particularly ecology, this research indicates that students are not as well-grounded in those principles as faculty often assume. By helping to diagnose Conclusion A scientific explanation uses simple principles, the same at all scales, governing matter and energy transformations A informal explanation describes individuals, different at all scales, with different abilities/ purposes that create seemingly complex, unconnected results this “hidden curriculum” for faculty, DQCs can be an effective tool on which to base further instructional interventions. Our results also show that it is difficult for most students to replace their informal reasoning with principled, scientific reasoning. Fig. 3. Proportions of students responding with various reasoning strategies before and after instruction. We classified each of the 42 questions according to the principle, process, and scale it addressed and compared all pre-test responses to all post-test responses in each given category. ‘No Data’ primarily means that a student either skipped the question or answered “I don’t know.”