1. Long Term Ecological Research Math Science Partnership
The Effects of Teaching Materials and Teachers’ Approaches on Student Learning about Carbon-transforming Processes
2011 NARST Presenetation
Written by: Li Zhan, Dante Cisterna, Jennifer Doherty, and Charles W. Anderson (Michigan State University) and Yongsang Lee and Karen Draney (University of California, Berkeley)
Culturally relevant ecology, learning progressions and environmental literacy
Long Term Ecological Research Math Science Partnership
April 2011
Disclaimer: This research is supported by a grant from the National Science Foundation: Targeted Partnership: Culturally relevant ecology, learning progressions and environmental literacy (NSF ). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

2. The Effects of Teaching Materials and Teachers’ Approaches on Student Learning about Carbon-transforming Processes
You need to change the affiliation of Yong and Karen to University of California, Berkeley
Li Zhan, Dante Cisterna,
Jennifer Doherty, and Charles W. Anderson
Michigan State University
Yongsang Lee, Karen Draney
University of California, Berkeley
National Research in Science Teaching Conference
Orlando, FL. April, 5, 2011 2

3. Overview Research Background Research questions Methods
Results and findings
Implications and suggestions for future work
Our presentation starts by describing the research background that framed this study as well as the results of our student assessments. Then we continue with the research question posed for this study and its methods. Finally, we present the implications of our work for future research

4. Force-dynamic reasoning
Research Background
Curriculum Developers
&
Educational Researchers
Design
Consider
Students
Teaching Materials
Assessments
Affordances
This picture illustrates the framework of our research which focuses on students’ learning progressions. To achieve this goal we, as educational researchers and curriculum developers, intend changes in students’ understanding from a force-dynamic reasoning to scientific reasoning.
We designed a set of teaching materials which aims are the affordances of scientific reasoning. They are concretized in student units and teacher guides.
We intend that students, by using these teaching materials, may advance in their learning progression.
Also, in order to get evidence of students’ learning progressions we create and administer students’ pre and post assessments
Scientific reasoning
Force-dynamic reasoning
Learning progression

5. Students’ learning achievements (pre-post)
Discrepancy
Students of different teachers demonstrated different patterns of progress
Similarity
Students improved their achievement in each level
In this slide, we present the results of students learning achievement pre and posttest in the period This graph shows the changing results for elementary, middle and high school students; and college students. We can see that, in general, students improved their achievements in each level. The overall learning gain was 0.4 and 0.5 mean of EAP estimates (“describe better what this means”)
Also, the increases are similar for each level. As expected, the scores of ES students were lower than MS students and this latter group, lower that HS and college students.
The graphs below present pre-post change for ES, MS, and HS students. Even though all teachers had learning gains, they were different.
From these graphs, we can conclude that students of different teachers demonstrated different patterns of progress.

6. Force-dynamic reasoning
Research Background
Curriculum Developers
&
Educational Researchers
Teachers
Scaffold
Strategy
Understanding
Implement
Consider
Design
Teachers
Implement
Teaching Materials
Teaching Materials
Students
Assessments
Affordances
Based on student pre-post learning gains. We strongly appreciate teachers to properly implement our teaching materials. So, teachers also need to understand the meaning and purpose of those materials and implement it in a proper strategy in order to enhance students’ learning progression.
The teaching materials that we developed consisted of several units about carbon-transforming processes. In particular, we focused on the introductory unit named «Systems and Scales» which was used by all participants that includes the core scientific principles of carbon transforming process.
Scientific reasoning
Force-dynamic reasoning
Learning progression

7. Visible Thinking tools in Systems and Scales unit ------Process Tool
Car Running
Process:
Scale:
(Matter Input)
(Matter Output)
(Energy Output)
(Energy Input)
Chemical Energy
Heat
Motion
Octane (CH3(CH2)6CH3)
(liquid)
Water (H2O) (gas)
Oxygen (O2) (gas)
Carbon Dioxide (CO2) (gas)
Combustion
Atomic-molecular
Another feature of «Systems and scales» is the use of tools for reasoning, which also are visible thinking tools (or VTTs) . They are named «process tool» and «powers of 10».
This picture shows an example of the process tool for car running process. This process tool intends that students trace matter and energy separately and recognize inputs and outputs in such transformations.
Matter conservation / transformation
Energy conservation / transformation / degradation 7

8. Visible Thinking tools in Systems and Scales unit ------Powers of ten
The second VTT is Powers of 10, which intends to organize objects into dynamic systems at multiple scales. We set four benchmarks for scale (large, macroscopic, microscopic, and atomic molecular). This tool is useful to analyze and connect objects and processes in different scales.
Note that I changed “structures” to “emergent properties.” I think that this is important since the real upper anchor focuses on processes
Hierarchy of systems and scales:
The world is organized into dynamic systems that have emergent properties at multiple scales. The systems are dynamic in that matter and energy are constantly flowing through them and being changed by them. 8

9. Upper Anchor of teaching materials’ affordance
Integration of Powers of 10 and Process tool to analyze scientific events as transformations of matter and energy in the same process at multiple scales
Scale
Large
Macro
Micro
Atomic - molecular
Scientific events
This slide shows our upper anchor of the teaching materials to understand events based on scientific reasoning. We expect teachers integrate the two visible thinking tools; powers of 10, related to scale; and process tool, related to matter and energy transformation and conservation.
By using these tools, teachers analyze and distinguish changes in matter and energy. Also, that teachers differentiate those changes in objects like ……………… and processes like ……….. We included processes and principles like energy conservation and degradation and matter conservation.
At the same time, these scientific events are explained in different scales.
Note that I changed the text to reflect more clearly what it means to integrate the tools.

10. Research Questions
How do teachers interpret teaching materials? Do teachers’ interpretations match with the affordances of teaching materials?
What patterns do teachers show for interpreting teaching materials?
What are the possible connections between teachers’ interpretation of teaching materials and students’ success in learning?
These are our research questions… read them
I changed the wording a little.

11. Powers of Ten activities (scale)
Research methods
Participants: 12 Michigan teachers who implemented the unit “Systems and Scales”
Level
Number
Type of School
Elementary 4
2 rural public schools
1 suburban public school
Middle
3 rural public schools
High
1 regional math & science magnet school
Procedures: Interviews (unit implementation)
Stimulated recall
We worked with 12 Michigan teachers who participated in our project and implemented the teaching materials. Four teachers were from ES, MS and HS each.
We interviewed teachers about how they implemented the “system and scales” unit. This was organized in three set of activities: powers of ten, related to scale; What’s air?, related to matter conservation and transformation; as well energy.
The interview protocol asked teachers for the description of activities, perception of student learning, teachers’ appreciation of activities and suggestions for improvement
Powers of Ten activities (scale)
Air Activities
(matter)
Energy activities
(energy)
Description of activities - Teachers’ appreciation of activities
Perception of student learning - Suggestions for improvement

12. Data analysis Categorize teachers’ responses.
Develop coding rubric for scales performance and conservation & transformation performance based on teachers’ interpretation.
Coding teachers’ responses and reliability check.
Analyze coding results with students’ learning achievement. From pre and post-tests.
These are the procedures we used for our data analysis
Description and categorization of responses.
In case of responses related to VTTs (Powers of 10 and Process tool), we developed a rubric for interpretations: scale, and conservation and transformation
Coding of teachers’ responses (reliability check)
Connections with students’ pre/post results in order to explore possible relationships

13. Findings
So let’s see our findings.

14. General coding rubric for interpretation of visible thinking tools
General Description
Level 4
Process tools and Powers of ten are tools to analyze scientific events based on the integration of conservation/transformation and scales.
Level 3
Process tools and Powers of ten are separate dynamic models with connected components.
They are used to distinguish and connect components that represent scientific principles.
Level 2
Process tools and Powers of ten are static models with separated components: different parts of systems and processes.
They are used to differentiate components.
Level 1
VTTs are literally understood as a means to represent things: “pictures” of objects and processes.
No scientific relationships among objects.
So let’s see our findings.
I added a little to the descriptions. Check to see if it is appropriate

15. Examples of Powers of 10 Interpretation
Teacher A: (level 1)
“…I liked the video, and I think the students liked the video; and they understood, they had no problem putting the objects in the different categories…”
“… I just don't think they really see the associations between the numbers, I mean, they understand that gas is matter, and they know it's very small…”
Teacher B (level 3)
“…I think that is definitely one of the challenges is making those connections. I mean they get the big scale, the large scale objects, small scale but that all the large scale objects are made up of the small scale objects. I think we build that connection…”

16. Examples of Process Tool Interpretation
Teacher A: (level 2)
“…I would use pictures or a drawing - a very crude drawing of what we were talking about; and then the kids would talk to us about what went in and what would come out…. it was pretty basic stuff, but I think the kids understood that when something comes in, something would go out then”.
Teacher B (level 3)
“…because when you have something on the left side and you have to move it to the right side, you can see that nothing is happening to those balls. They are simply just being moving … before kids thought that things just disappeared. It went away. This was the first time in their life when they had to say, had to be able to tell me where it went. Never before in their education had they had anybody talked about transformation, movement, whatever we have. You know, conservation of matter, so it was great….”
You can see that studnets are interpreting quite different… you can say, since the patterns are different ES..
Within levels, we are traing to do suggestions…
It’s not clear to me why Teacher A is L1 and not L2. Isn’t she trying to connect what goes in with what comes out?
Li: I think we can change the the level:Teacher A (Level 2)

17. Teachers’ coding results Conservation and transformation
Elementary
Middle
High
Teacher
Scale
Conservation and transformation E1 1 M1 3 H1 E2 2 M2 H2 E3 M3 H3 E4 M4 H4

18. Suggestive Patterns Pattern 1 Pattern 2
None of the teachers’ interpretations reached the upper affordance of teaching materials
Pattern 2
Elementary teachers’ interpretation tends to be lower than middle and high school teachers
These are suggestive patterns of data. They are not based on statistical analysis.
Teachers differed in their interpretation
There are ”tendencies” according to level of teaching. The fact that ES tended to have lower results may be explained in terms that teachers explained their interpretation according to their students (or maybe their understanding)
I reworded the patterns a little

19. Suggestive Patterns Pattern 3
Teachers’ interpretation of visible thinking tools and students gains
H3: Yes. Especially at this point where it's so easy to combine matter and energy,,,,, the process tool to separate that butane is the chemical compound that contains chemical energy that starts things going in the right direction.
H2: And then, going into the process tools I thought is very beneficial for them, because it does separate the energy from the matter..
It isn’t clear to me what differences you want to point out between H2 and H3. I think that one thing you need to say is that the most effective elementary teachers focused on affordances that were appropriate for their students.

20. Implications To do list
Teachers’ interpretations of Tools for Reasoning influenced their classroom implementation
Teaching materials need to be scaffolded so that teachers can recognize and use the intended affordances of Tools for Reasoning
In order to help teachers to improve understanding and implementation of teaching materials, better support and professional development are needed
To do list
I changed the wording here.
Gather different types of evidences to connect teachers, teaching materials, and student learning.
Improve teaching materials to make their affordances more apparent to teachers and students.
Provide better support to teachers in teaching experiments.

21. Thank you!