It Pays to Compare: Effectively Using Comparison to Support Student Learning of Algebra Bethany Rittle-Johnson Jon Star

IES Conference Our approach to improving students’ mathematics learning Identify instructional practices used in exemplary and typical classrooms Use cognitive science literature to focus on practices most likely to help student learning Experimentally evaluate impact of the instructional practice on student learning and develop instructional guidelines

IES Conference Potential of comparison Mathematics Education: Central tenet of reform efforts; used by teachers Cognitive Science: A fundamental learning mechanism

IES Conference Central tenet of math reforms Students benefit from sharing and comparing solution methods “nearly axiomatic”, “with broad general endorsement” (Silver et al., 2005) Noted feature of ‘expert’ math instruction Present in classrooms in high performing countries such as Japan and Hong Kong (Ball, 1993; Fraivillig, Murphy, & Fuson, 1999; Huffred-Ackles, Fuson, & Sherin Gamoran, 2004; Lampert, 1990; Silver et al., 2005; NCTM, 1989, 2000; Richland et al 2007; Stigler & Hiebert, 1999)

IES Conference Used in some Algebra textbooks Sobel, M.A., Maletsky, E. M., Lerner, N., & Cohen, L.S. (1985) Algebra One, Harper and Row Inc.

IES Conference But does comparison improve student learning? No evidence that comparison improves student learning in mathematics Cognitive science research suggests that it should…

Comparison in cognitive science “The simple, ubiquitous act of comparing two things is often highly informative to human learners…. Comparison is a general learning process that can promote deep relational learning and the development of theory-level explanations” (Gentner, 2005, pp. 247, 251)

IES Conference Fundamental learning mechanism Lots of evidence from cognitive science ◦ Identifying similarities and differences in multiple examples is an important pathway to flexible, transferable knowledge Mostly laboratory studies Rarely done with school-age children or in mathematics (Catrambone & Holyoak, 1989; Gentner, Loewenstein, & Thompson, 2003; Gick & Holyoak, 1983; Kurtz, Miao, & Gentner, 2001; Loewenstein & Gentner, 2001; Namy & Gentner, 2002; Oakes & Ribar, 2005; Schwartz & Bransford, 1998)

IES Conference Does comparison support math learning? Goal of our IES grant ◦ Investigate whether comparison can support conceptual and procedural knowledge of equation solving (and estimation) ◦ Explore what types of comparison are most effective ◦ Experimental studies in intact classrooms

IES Conference Why equation solving? Often students’ first exposure to abstraction and symbolism of mathematics Area of weakness for US students (Blume & Heckman, 1997; Schmidt et al., 1999) According to NCTM and National Math Panel Report, linear equation solving should be a focal point of math instruction in middle school Although real-world contexts and informal solution methods are powerful for simple problems, equations and equation solving are more effective for complex problems (Koedinger, Alibali & Nathan, 2008)

IES Conference Multiple methods for solving equations Method #1: 3(x + 1) = 15 3x + 3 = 15 3x = 12 x = 4 Method #2: 3(x + 1) = 15 x + 1 = 5 x = 4 ◦ Some are better than others ◦ Students tend to memorize only one method ◦ Example: Solving 3(x + 1) = 15

IES Conference Study 1 Research question: Does comparing solution methods improve equation solving knowledge? Research design: Randomly assigned to: ◦ Comparison condition  Compare and contrast alternative solution methods ◦ Sequential condition  Study same solution methods sequentially Rittle-Johnson, B. & Star, J.R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology.

IES Conference Translation to the classroom Students study and explain worked examples with a partner Based on core findings in cognitive science -- the advantages of: ◦ Worked examples (e.g. Sweller, 1988) ◦ Generating explanations (e.g. Chi et al, 1989; Rittle-Johnson, 2006) ◦ Peer collaboration (e.g. Fuchs & Fuchs, 2000)

IES Conference Comparison condition

IES Conference Sequential condition next page

IES Conference Predicted outcomes Students in comparison condition will make greater gains in: ◦ Procedural knowledge, including success on novel problems ◦ Procedural flexibility (e.g. use more efficient methods; evaluate when to use a procedure) ◦ Conceptual knowledge (e.g. equivalence)

IES Conference Study 1 Method Participants: 70 7th-grade students and their math teacher Design: ◦ Pretest - Intervention - Posttest ◦ Replaced 2 lessons in textbook ◦ Intervention occurred in partner work during 2 1/2 math classes Intervention: ◦ Randomly assigned to Compare or Sequential condition ◦ Studied worked examples with partner ◦ Solved practice problems on own

IES Conference Procedural knowledge assessment Equation Solving ◦ Intervention: 1/3 (x + 1) = 15 ◦ Posttest Familiar: -1/4 (x – 3) = 10 ◦ Posttest Novel: 0.25 (t + 3) = 0.5

IES Conference Procedural flexibility Use of more efficient solution methods on procedural knowledge assessment Knowledge of multiple methods ◦ Solve each equation in two different ways ◦ Evaluate methods: Looking at the problem shown above, do you think that this way of starting to do this problem is a good idea? An ok step to make? Circle your answer below and explain your reasoning. (a) Very good way (b) Ok to do, but not a very good way (c) Not OK to do

IES Conference Conceptual knowledge assessment

IES Conference Gains in procedural knowledge F(1, 31) =4.49, p <.05

IES Conference Flexible use of procedures Solution MethodComparisonSequential Conventional.61 ~.66 Demonstrated efficient.17 *.10 Solution Method at Posttest (Proportion of problems) ~ p =.06; * p <.05 Comparison students more likely to use more efficient method and somewhat less likely to use the conventional method

IES Conference Gains in flexible knowledge of procedures F(1,31) = 7.73, p <.01

IES Conference Gains in conceptual knowledge No Difference

IES Conference Summary of Study 1 Comparing alternative solution methods is more effective than sequential sharing of multiple methods ◦ Improves procedural transfer and flexibility ◦ In mathematics, in classrooms

IES Conference Comparison can help: Now what? Replicated findings for fifth graders learning computational estimation Goal: Develop guidelines for using comparison to support mathematics learning Starting Point: Standard classroom practices

IES Conference What are teachers doing? US teachers use comparison in 8th grade math lessons (average 4 per lesson) Types of comparisons (used with equal frequency): 1.Compare two similar problems with same basic solution 2.Compare two moderately similar problems or solutions 3.Compare a problem to a mathematical rule or principle 4.Compare a problem to a non-mathematical situation (Richland, Holyoak & Stigler, 2004 analysis of TIMSS videos)

IES Conference What are teachers doing? May not be using comparison well ◦ Teachers, rather than students, initiate comparisons and make links between examples ◦ When they present multiple solutions, rarely provide support for or discuss comparisons ◦ Don’t know which types of comparison support learning  e.g. Comparisons to contexts from different domains rarely support learning in laboratory studies. (Richland, Holyoak & Stigler, 2004; Richaland, Zur & Holyoak, 2007; Chazan & Ball, 1999)

IES Conference What about Algebra I textbooks? Compare two similar problems with same basic solution method (Equivalent Equations) Bellman,A.E., Bragg,S.C., Charles, R.I., Hall,B., Handlin, W.G., & Kennedy, D. (2007) Algebra 1, Pearson Education Inc, Pearson Prentice Hall

IES Conference Hollowell, K.A., Ellis, W., & Schultz, J.E. (1997). HRW Algebra. Holt, Rinehart, & Winston. Algebra I textbooks Compare problems with different structures (Different Problem Types )

IES Conference Algebra I textbooks Sobel, M.A., Maletsky, E. M., Lerner, N., & Cohen, L.S. (1985) Algebra One, Harper and Row Inc. Compare different solution methods to same problem (Solution Methods)

IES Conference Comparison in Algebra 1 textbooks Type of ComparisonPercent of worked examples Equivalent equations (similar problems; same method) 33% Different problem types (diff probs, solved same way) 1% Solution methods (one problem solved in two ways) 19% None - single worked examples47% Informal analysis of 10 Algebra I textbooks - chapter on multi-step linear equations

IES Conference What should be compared? Variety of comparisons are being used in math classrooms What are benefits and drawbacks to different types of comparisons? ◦ Study 1 confirms that comparing solution methods aids learning, as suggested by expert teaching practices ◦ Cognitive science literature suggests that comparing two problems solved with the same solution method should benefit learning

IES Conference Study 2 Research question: What are the relative merits of comparing solution methods vs. comparing problems? Research design: Randomly assigned to: ◦ Compare solution methods ◦ Compare problems that:  Are very similar (Equivalent)  Have different problem features (Different problem types)

IES Conference Types of comparison Solution Methods (one problem solved in 2 ways) Problem Types (2 different problems, solved in same way) Equivalent (two similar problems, solved in same way)

IES Conference Study 2 Method Participants: 161 7th & 8th grade students from 3 schools (more diverse sample) Design: ◦ Pretest - Intervention - Posttest - 2 week Retention ◦ Replaced 3 lessons in textbook ◦ Randomly assigned to  Compare Solution Methods  Compare Problem Types  Compare Equivalent ◦ Intervention occurred in partner work ◦ Assessment adapted from Study 1

IES Conference Conceptual knowledge results F (2, 153) = 5.76, p =.004,  2 =.07 *

IES Conference Procedural knowledge results No differences, even on novel problem types

IES Conference Flexible use of procedures F (2, 153) = 4.96, p =.008,  2 =.06 *

IES Conference Flexible knowledge of procedures F (2, 153) = 5.01, p =.008,  2 =.07 *

IES Conference Summary Across studies, Comparing Solution Methods often supported the largest gains in conceptual knowledge, procedural knowledge and procedural flexibility ◦ Supported attention to multiple methods and their relative efficiency, which both predicted learning

IES Conference Guidelines for using comparison Provide a written record of examples ◦ Leverage current use of worked examples in textbooks Contrast important dimensions in the examples, such as problem features or solution methods ◦ Contrasting correct and incorrect solution methods can help too (Kelley Durkin, IES pre-doc research) Have students compare a familiar method to an unfamiliar method Invite comparisons by using common labels and prompting for specific comparisons, including efficiency of the methods Be sure students, not just teachers, are comparing and explaining Incorporate some direct instruction

What’s next? Teacher Professional Development for using comparison in Algebra I courses Type of comparison matched to prior knowledge and sequencing different types of comparison

IES Conference Acknowledgements For slides, papers or more information, contact: Funded by a grant from the Institute for Education Sciences, US Department of Education Thanks to research assistants at Vanderbilt: ◦ Holly Harris, Shanelle Chambers, Jennifer Samson, Anna Krueger, Heena Ali, Kelley Durkin, Kelly Cashen, Calie Traver, Sallie Baxter, Amy Goodman, Adam Porter, John Murphy, Rose Vick, Alexander Kmicikewycz, Jacquelyn Beckley and Jacquelyn Jones And at Michigan State: ◦ Kosze Lee, Kuo-Liang Chang, Howard Glasser, Andrea Francis, Tharanga Wijetunge, Beste Gucler, and Mustafa Demir And at Harvard: ◦ Martina Olzog, Jennifer Rabb, Christine Yang, Nira Gautam, Natasha Perova, and Theodora Chang