1. Teachers’ Sensemaking about Accountability and Assessment
Elizabeth de los Santos1, Stefanie Marshall2, James Brian Hancock II2, Sarah Bodbyl2, Andrea Forsyth1, Qinyun Lin2, William Penuel3, and Charles W. Anderson2
1University of Nevada, Reno, 2Michigan State University, 3University of Colorado, Boulder
Problem Statement and
Research Questions
We designed Carbon TIME assessments to provide teachers with tools for formative and summative assessment purposes. Yet, teachers must use these assessments within the context of their local ecologies, where they must assign grades and may face competing purposes for assessment, including feeling pressure to conform to local norms and obligations such as formal teacher evaluations based on student data. We were interested in exploring how case study teachers approached Carbon TIME assessments, including local and personal factors that influenced their assessment practices.
How did teachers approach assessments in Carbon TIME classrooms?
How did teachers’ local context and personal emphases influence their purposes for assessment?
Opportunities for Assessment in the Carbon TIME Instructional Model
Add POSOH logo, disclaimer gives credits to projects that have not been funding this work.
Take off projects not funding this work.
Teachers have multiple opportunities for assessment within the Carbon TIME instructional model, which includes an inquiry-application sequence in which students first explore differing ideas (divergent thinking) and then strive for consensus after obtaining evidence (convergent thinking). Some tools and resources can serve both formative and summative purposes; additionally, teachers must assign grades for student participation and/or achievement.
Summative Assessment Opportunities
Explanations Tool
Post-Tests
Quizzes
Molecular modeling worksheets
Formative Assessment Opportunities
Pre-Tests
Expressing Ideas Tool
Predictions Tool
Evidence-based Arguments Tool
Teacher-developed exit tickets
Big Ideas probes
Discussion
Productive Sensemaking
Productive sensemaking about accountability and assessment resulted in successful navigation of multiple purposes for assessment and while meeting local obligations and/or personal emphases.
As the case of Ms. Callahan shows, high alignment and high salience can foster productive sensemaking if the local context is aligned with Carbon TIME goals and purposes for assessment.
As the case of Mr. Gilbert shows, low salience can also foster productive sensemaking, particularly if the local context is not aligned with Carbon TIME goals and purposes for assessment.
Unproductive Sensemaking
Unproductive sensemaking about accountability and assessment resulted in teachers’ attention to some purposes for assessment at the expense of others:
As the case of Ms. Estabrook shows, low salience of local norms and obligations provided her with opportunities to explore, but low alignment of her personal emphases and local obligations with Carbon TIME goals and purposes for assessment inhibited her sensemaking.
As the case of Mr. Ross shows, high salience yet low alignment resulted in unproductive sensemaking, with a focus on satisfying teacher evaluation requirements at the expense of insight into student thinking.
Conclusions
The interaction of teachers’ local obligations with their personal emphases can either enhance or inhibit productive sensemaking depending on teachers’ goals and purposes for engaging in sensemaking about accountability and assessment. The implication for science teacher educators is to support teachers in navigating multiple purposes for assessment, including assessment for grading, insight into student thinking, and students’ self-evaluation of learning over time, and address how local norms and obligations may conflict with Carbon TlME goals and purposes for assessment.
Research Design
Qualitative comparative case study design (Yin, 2014).
Fourteen secondary science teachers in three states with 1-30 years of experience; teaching contexts ranged from a math and science magnet school to an urban school with a diverse student population.
Case study teachers implemented at least three Carbon TIME units during the school year and attended face-to-face and online PD.
Data Sources: primarily 69 teacher interviews; also teacher-created artifacts, video-recordings of classroom instruction, student work.
Data Analysis: descriptive coding of transcripts to identify teachers’ talk about their grading and assessment practices, interpretive coding to identify occasions of sensemaking, and identification of themes within and across cases.
The Influence of Teachers’ Personal Emphases and Local Obligations on Their Sensemaking about Accountability and Purposes for Assessment
Mr. Gilbert
(2016—17)
Ms. Estabrook
Ms. Callahan
(2015—16 & 2016—17)
Mr. Ross
(2015—16)
School Context
9th and 10th grade biology classroom with primarily English language learners and recent refugees and immigrants at a large, suburban high school; 56% free-and-reduced lunch, 62% students of color.
10th grade biology classroom with special education students at a large, suburban high school; 56% free-and-reduced lunch, 62% students of color; worked closely with Mr. Gilbert to implement the curriculum.
9th grade biology classroom at a math and science magnet program located in an urban setting; students attended the program in the morning, then returned to their home schools; 21% students of color.
9th grade biology classroom at a midsize, suburban high school; 19% free-and-reduced lunch, 52% students of color.
Influence of Local Norms
and Obligations
Felt less pressure to conform to local obligations since he taught a special population of students
Felt less pressure to conform to local obligations since she taught a special population of students
Felt some pressure to conform to local obligations, particularly alignment with mathematics and technology teachers
Felt pressure to show student learning gains for teacher evaluation purposes and alignment with other local initiatives.
Personal Emphases
Primarily to his students in terms of supporting their learning and Carbon TIME
Primarily to her students in terms of engaging their interest and participation.
Primarily to local obligations, Carbon TIME, and her students in terms of supporting their learning over time
Primarily to local obligations and Carbon TIME
Purposes for Assessment
Assigning grades
Insight into student thinking
Students’ self-evaluation of growth over time
Used pre-and-post tests to assign grades and explained to students that he wanted to see growth over time.
Informed students that Carbon TIME is different than how he has taught in the past and that students’ grades on tests may be lower than they expect.
Recognized differences between Carbon TIME assessments and traditional assessments, such as multiple-choice tests.
Chose not to use the online tests because she thought her students would not do well on them.
Chose not to use the written Explanations Tool.
Preferred short quizzes (e.g., atoms and molecules quiz) and wanted more of them in the units.
Assigned grades based on participation and not evidence of learning.
Graded Process Tools using a check, check minus, and check plus system to provide feedback to students about her expectations.
Examined students’ ideas closely and focused on precision of language.
Had students use different color pens to keep track of different students’ ideas (individual vs. group vs. class).
Had students review all their Process Tools at the end of a unit to self-assess their growth over time.
Spent time developing an elaborate grading scheme using LP levels.
Focused on completion of the Process Tools rather than attention to students’ ideas or developing more sophisticated thinking over time.
Had common assessments and other district-level initiatives, such as IB.
Used pre-and-posts tests to show student learning gains for teacher evaluation purposes.
Illustrative Quote
“It [Carbon TIME] kind of moved me in the direction that I wanted to go for a while, which is more standards-based grading… I love [the assessments] because it asks them to think at a higher level… I firmly believe we we learned way more science than we learned the year before. My tests in the past were multiple-choice, and it was a lot of sit-and-get in my classroom, and they would be able to produce the multiple-choice answers for a day, but I’m not sure if you asked any of them this year what they knew about it that they could tell you anything” (SS interview, Oct 2016).
“I don’t have those formative assessments or checks along the way for understanding, so I think that’s something, like, coming up with, you know, did we do a question at the beginning of the hour? Did we do, like, I don’t have exit tickets planned, because I don’t know how long things take. It’s not until you get to the end of the test that you see how much did they really glean, you know? Do they just superficially glean things from it, do they have a solid understanding of what’s going on?” (PL interview, Nov 2016).
“After they filled it [Explanations Tool] and we talked about it, we had them go back and look at their Predictions Tool. And I think that their responses were just hilarious… how much that they have grown… it has allowed the students to see that even though they feel like they didn’t necessarily gain or learn from them right in the moment, they really did come a long way. Like, they really have a much more thorough, more detailed understanding of what’s happening in something as simple as soda water fizzing” (SS interview, Nov 2016).
“The [teacher] evaluation required me to show a student’s progress in scientific argumentation” (Post interview, June 2016).
“With the IB, inquiry is like, a huge focus, and so I’m just trying to figure out how to use the Carbon TIME stuff to cover what I have to teach and how I have to assess it” (Post interview, Oct 2016).
“I put a lot more focus on the essay, and I said, ‘for every single essay question, I want to see a claim, evidence, reason. That doesn’t have to be a whole paragraph, like 10 sentences” (PL interview, May 2016).
Outcomes of Sensemaking
Attention to assessment for showing student growth over time
Attention to assessment for grading purposes based on completion
Successful navigation of multiple purposes for assessment while meeting local obligations and personal emphases
Attention to assessment for grading purposes and teacher evaluation with less emphasis on insight into student thinking
Opportunities for Sensemaking
Critical noticing that involves action situated in context over time
Disclaimer: This research is supported by grants from the National Science Foundation: A Learning Progression-based System for Promoting Understanding of Carbon-transforming Processes (DRL ), and Sustaining Responsive and Rigorous Teaching Based on Carbon TIME (NSF ).