1. Bethany Rittle-Johnson Jon Star
It Pays to Compare: Effectively Using Comparison to Support Student Learning of Algebra
Bethany Rittle-Johnson
Jon Star

2. Our approach to improving students’ mathematics learning
Identify instructional practices used in exemplary and typical classrooms
Use cognitive science literature to focus on practices most likely to help student learning
Experimentally evaluate impact of the instructional practice on student learning and develop instructional guidelines
IES Conference 2008

3. Potential of comparison
Mathematics Education: Central tenet of reform efforts; used by teachers
Cognitive Science: A fundamental learning mechanism
IES Conference 2008

4. Central tenet of math reforms
Students benefit from sharing and comparing solution methods
“nearly axiomatic”, “with broad general endorsement” (Silver et al., 2005)
Noted feature of ‘expert’ math instruction
Present in classrooms in high performing countries such as Japan and Hong Kong
(Ball, 1993; Fraivillig, Murphy, & Fuson, 1999; Huffred-Ackles, Fuson, & Sherin Gamoran, 2004; Lampert, 1990; Silver et al., 2005; NCTM, 1989, 2000; Richland et al 2007; Stigler & Hiebert, 1999)
IES Conference 2008

5. Used in some Algebra textbooks
Sobel, M.A., Maletsky, E. M., Lerner, N., & Cohen, L.S. (1985) Algebra One, Harper and Row Inc.  
IES Conference 2008

6. But does comparison improve student learning?
No evidence that comparison improves student learning in mathematics
Cognitive science research suggests that it should…
IES Conference 2008

7. Comparison in cognitive science
“The simple, ubiquitous act of comparing two things is often highly informative to human learners…. Comparison is a general learning process that can promote deep relational learning and the development of theory-level explanations” (Gentner, 2005, pp. 247, 251)

8. Fundamental learning mechanism
Lots of evidence from cognitive science
Identifying similarities and differences in multiple examples is an important pathway to flexible, transferable knowledge
Mostly laboratory studies
Rarely done with school-age children or in mathematics
(Catrambone & Holyoak, 1989; Gentner, Loewenstein, & Thompson, 2003; Gick & Holyoak, 1983; Kurtz, Miao, & Gentner, 2001; Loewenstein & Gentner, 2001; Namy & Gentner, 2002; Oakes & Ribar, 2005; Schwartz & Bransford, 1998)
IES Conference 2008

9. Does comparison support math learning?
Goal of our IES grant
Investigate whether comparison can support conceptual and procedural knowledge of equation solving (and estimation)
Explore what types of comparison are most effective
Experimental studies in intact classrooms
IES Conference 2008

10. Why equation solving?
Often students’ first exposure to abstraction and symbolism of mathematics
Area of weakness for US students (Blume & Heckman, 1997; Schmidt et al., 1999)
According to NCTM and National Math Panel Report, linear equation solving should be a focal point of math instruction in middle school
Although real-world contexts and informal solution methods are powerful for simple problems, equations and equation solving are more effective for complex problems (Koedinger, Alibali & Nathan, 2008)
IES Conference 2008

11. Multiple methods for solving equations
Some are better than others
Students tend to memorize only one method
Example: Solving 3(x + 1) = 15
Method #1:
3(x + 1) = 15
3x + 3 = 15
3x = 12
x = 4
Method #2:
3(x + 1) = 15
x + 1 = 5
x = 4
IES Conference 2008

12. Study 1
Research question: Does comparing solution methods improve equation solving knowledge?
Research design: Randomly assigned to:
Comparison condition
Compare and contrast alternative solution methods
Sequential condition
Study same solution methods sequentially
Rittle-Johnson, B. & Star, J.R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology.
IES Conference 2008

13. Translation to the classroom
Students study and explain worked examples with a partner
Based on core findings in cognitive science -- the advantages of:
Worked examples (e.g. Sweller, 1988)
Generating explanations (e.g. Chi et al, 1989; Rittle-Johnson, 2006)
Peer collaboration (e.g. Fuchs & Fuchs, 2000)
IES Conference 2008

14. Comparison condition
IES Conference 2008

15. Sequential condition
next page
IES Conference 2008

16. Predicted outcomes
Students in comparison condition will make greater gains in:
Procedural knowledge, including success on novel problems
Procedural flexibility (e.g. use more efficient methods; evaluate when to use a procedure)
Conceptual knowledge (e.g. equivalence)
IES Conference 2008

17. Study 1 Method
Participants: 70 7th-grade students and their math teacher
Design:
Pretest - Intervention - Posttest
Replaced 2 lessons in textbook
Intervention occurred in partner work during 2 1/2 math classes
Intervention:
Randomly assigned to Compare or Sequential condition
Studied worked examples with partner
Solved practice problems on own
IES Conference 2008

18. Procedural knowledge assessment
Equation Solving
Intervention: /3 (x + 1) = 15
Posttest Familiar: -1/4 (x – 3) = 10
Posttest Novel: (t + 3) = 0.5
IES Conference 2008

19. Procedural flexibility
Use of more efficient solution methods on procedural knowledge assessment
Knowledge of multiple methods
Solve each equation in two different ways
Evaluate methods: Looking at the problem shown above, do you think that this way of starting to do this problem is a good idea? An ok step to make? Circle your answer below and explain your reasoning.
(a) Very good way
Ok to do, but not a very
good way
(c) Not OK to do
IES Conference 2008

20. Conceptual knowledge assessment
IES Conference 2008

21. Gains in procedural knowledge
Novel - procedural transfer
F(1, 31) =4.49, p < .05
IES Conference 2008

22. Flexible use of procedures
Comparison students more likely to use more efficient method and somewhat less likely to use the conventional method
Solution Method at Posttest (Proportion of problems)
Solution Method
Comparison
Sequential
Conventional
.61~
.66
Demonstrated efficient
.17*
.10
~ p = .06; * p < .05
IES Conference 2008

23. Gains in flexible knowledge of procedures
Most pronounced on Evaluating non-standard items
IES Conference 2008
F(1,31) = 7.73, p < .01

24. Gains in conceptual knowledge
No Difference
IES Conference 2008

25. Summary of Study 1
Comparing alternative solution methods is more effective than sequential sharing of multiple methods
Improves procedural transfer and flexibility
In mathematics, in classrooms
IES Conference 2008

26. Comparison can help: Now what?
Replicated findings for fifth graders learning computational estimation
Goal: Develop guidelines for using comparison to support mathematics learning
Starting Point: Standard classroom practices
IES Conference 2008

27. What are teachers doing?
US teachers use comparison in 8th grade math lessons (average 4 per lesson)
Types of comparisons (used with equal frequency):
Compare two similar problems with same basic solution
Compare two moderately similar problems or solutions
Compare a problem to a mathematical rule or principle
Compare a problem to a non-mathematical situation
(Richland , Holyoak & Stigler, 2004 analysis of TIMSS videos)
IES Conference 2008

28. What are teachers doing?
May not be using comparison well
Teachers, rather than students, initiate comparisons and make links between examples
When they present multiple solutions, rarely provide support for or discuss comparisons
Don’t know which types of comparison support learning
e.g. Comparisons to contexts from different domains rarely support learning in laboratory studies.
(Richland , Holyoak & Stigler, 2004; Richaland, Zur & Holyoak, 2007; Chazan & Ball, 1999)
IES Conference 2008

29. What about Algebra I textbooks?
Compare two similar problems with same basic solution method (Equivalent Equations)
Bellman,A.E., Bragg,S.C., Charles, R.I., Hall,B., Handlin, W.G., & Kennedy, D.
(2007) Algebra 1, Pearson Education Inc, Pearson Prentice Hall
IES Conference 2008

30. Algebra I textbooks
Compare problems with different structures (Different Problem Types)
Hollowell, K.A., Ellis, W., & Schultz, J.E. (1997). HRW Algebra. Holt, Rinehart, & Winston. 
IES Conference 2008

31. Algebra I textbooks
Compare different solution methods to same problem (Solution Methods)
Sobel, M.A., Maletsky, E. M., Lerner, N., & Cohen, L.S. (1985) Algebra One, Harper and Row Inc.  
IES Conference 2008 31

32. Comparison in Algebra 1 textbooks
Type of Comparison
Percent of worked examples
Equivalent equations (similar problems; same method)
33%
Different problem types (diff probs, solved same way) 1%
Solution methods (one problem solved in two ways)
19%
None - single worked examples
47%
This is number of individual worked examples - so in CI, these are members of pairs or triplets
Informal analysis of 10 Algebra I textbooks - chapter on multi-step linear equations
IES Conference 2008

33. What should be compared?
Variety of comparisons are being used in math classrooms
What are benefits and drawbacks to different types of comparisons?
Study 1 confirms that comparing solution methods aids learning, as suggested by expert teaching practices
Cognitive science literature suggests that comparing two problems solved with the same solution method should benefit learning
IES Conference 2008

34. Study 2
Research question: What are the relative merits of comparing solution methods vs. comparing problems?
Research design: Randomly assigned to:
Compare solution methods
Compare problems that:
Are very similar (Equivalent)
Have different problem features (Different problem types)
IES Conference 2008

35. Types of comparison Solution Methods Problem Types Equivalent
(one problem solved in 2 ways)
Problem Types
(2 different problems, solved in same way)
Equivalent
(two similar problems, solved in same way)
Take a vote -
Make this slide pretty!
IES Conference 2008

36. Study 2 Method
Participants: 161 7th & 8th grade students from 3 schools (more diverse sample)
Design:
Pretest - Intervention - Posttest - 2 week Retention
Replaced 3 lessons in textbook
Randomly assigned to
Compare Solution Methods
Compare Problem Types
Compare Equivalent
Intervention occurred in partner work
Assessment adapted from Study 1
Won’t present retention results today
IES Conference 2008

37. Conceptual knowledge results*
F (2, 153) = 5.76, p = .004, 2 = .07
IES Conference 2008

38. Procedural knowledge results
No differences, even on novel problem types
IES Conference 2008

39. Flexible use of procedures*
These are estimated marginal means
F (2, 153) = 4.96, p = .008, 2 = .06
IES Conference 2008

40. Flexible knowledge of procedures*
Students who compared solution methods scored higher than students who compared equivalent equations, p < .002, and marginally higher than students who compared problem types, p < Students who compared problem types had greater flexibility knowledge than those who compared equivalent equations, but this difference did not reach significance (p = .158).
F (2, 153) = 5.01, p = .008, 2 = .07
IES Conference 2008

41. Summary
Across studies, Comparing Solution Methods often supported the largest gains in conceptual knowledge, procedural knowledge and procedural flexibility
Supported attention to multiple methods and their relative efficiency, which both predicted learning
IES Conference 2008

42. Guidelines for using comparison
Provide a written record of examples
Leverage current use of worked examples in textbooks
Contrast important dimensions in the examples, such as problem features or solution methods
Contrasting correct and incorrect solution methods can help too (Kelley Durkin, IES pre-doc research)
Have students compare a familiar method to an unfamiliar method
Invite comparisons by using common labels and prompting for specific comparisons, including efficiency of the methods
Be sure students, not just teachers, are comparing and explaining
Incorporate some direct instruction
Using worked examples allows you to implement key features of effective comparison, rather than building them spontaneously during instruction - e.g. ensure contrast meaningful, not trivial differences
IES Conference 2008

43. What’s next?
Teacher Professional Development for using comparison in Algebra I courses
Type of comparison matched to prior knowledge and sequencing different types of comparison

44. Acknowledgements
For slides, papers or more information, contact:
Funded by a grant from the Institute for Education Sciences, US Department of Education
Thanks to research assistants at Vanderbilt:
Holly Harris, Shanelle Chambers, Jennifer Samson, Anna Krueger, Heena Ali, Kelley Durkin, Kelly Cashen, Calie Traver, Sallie Baxter, Amy Goodman, Adam Porter, John Murphy, Rose Vick, Alexander Kmicikewycz, Jacquelyn Beckley and Jacquelyn Jones
And at Michigan State:
Kosze Lee, Kuo-Liang Chang, Howard Glasser, Andrea Francis, Tharanga Wijetunge, Beste Gucler, and Mustafa Demir
And at Harvard:
Martina Olzog, Jennifer Rabb, Christine Yang, Nira Gautam, Natasha Perova, and Theodora Chang
IES Conference 2008