1. The Power of Comparison in Learning & Instruction Learning Outcomes Supported by Different Types of Comparisons Dr. Jon R. Star, Harvard University Dr. Bethany Rittle-Johnson, Vanderbilt University

2. Plan for this talk  Overview of our studies  Review of five studies on comparison  Summary of findings across studies 5/27/2011 3 Association for Psychological Science

3. Overview of studies  How does comparison support learning of school mathematics within a classroom setting?  Redesigned 2 - 3 math lessons on a particular topic (e.g., equation solving, estimation)  Implemented during students’ mathematics classes  Five studies  Effectiveness of comparing correct methods (5 of 5)  Effectiveness of comparing problem types (2 of 5)  Focus on equation solving (4 of 5)  Focus on estimation (1 of 5) 5/27/2011 4 Association for Psychological Science (Rittle-Johnson & Star, in press)

4. Characteristics of all five studies  Packet of worked examples, prompts for explanation for each condition  Students work in pairs  Side-by-side comparison, labeled solution steps, some direct instruction  Explicit prompts to identify similarities/differences  Randomly assigned student pairs to condition within the same classroom  Measures of procedural knowledge, conceptual knowledge, and procedural flexibility 5/27/2011 5 Association for Psychological Science

5. Measures  Procedural knowledge (e.g., equation solving)  Ability to adapt known procedures to novel problems (i.e., transfer)  Conceptual knowledge (e.g., equivalence, like terms, composite variables)  “Is 98 = 21x equivalent to 98 + 2x = 21x + 2x?”  Procedural flexibility  Flexible Use - Use of more efficient solution methods on procedural knowledge assessment (i.e., fewer solution steps)  Flexible Knowledge  Knowledge of multiple methods (e.g., solve each equation in two different ways when prompted)  Ability to evaluate methods (e.g., “Looking at the problem shown above, do you think that this first step is a good way to start this problem?”) 6 5/27/2011 Association for Psychological Science

6. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Five Contrasting Cases studies 5/27/2011 7 Association for Psychological Science  Study 1: Does comparing solution methods facilitate conceptual and procedural knowledge? (Rittle-Johnson & Star, 2007)

7. Study 1  Research question: Does comparing solution methods improve equation solving knowledge?  Research design  Random assignment to  Compare condition: Students compare and contrast alternative solution methods for equation solving  Sequential condition: Students study same solution methods sequentially  Pretest - Intervention – Posttest  Participants  70 7th-grade students and their math teacher 8 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2007)

8. Equation solving strategies 9 5/27/2011 Association for Psychological Science

9. Study 1: Compare condition 10 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2007)

10. Study 1: Sequential condition 11 5/27/2011 Association for Psychological Science next page (Rittle-Johnson & Star, 2007)

11. Study 1: Results 12 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2007) F(1, 31) =4.49, p <.05

12. Study 1: Results 13 5/27/2011 Association for Psychological Science F(1,31) = 7.73, p <.01 Solution MethodCompariso n Sequential Conventional.61~.66 Demonstrated efficient.17*.10 Flexible Use of Procedures ~ p =.06; * p <.05 (Rittle-Johnson & Star, 2007)

13. Study 1: Results 14 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2007) No Difference

14. Study 1: Results summary  Students in the compare condition  Showed greater gains in procedural knowledge and flexibility  Were more likely to use more efficient method and somewhat less likely to use the conventional method  Maintained conceptual knowledge 15 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2007)

15. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Five Contrasting Cases studies 5/27/2011 16 Association for Psychological Science  Study 2: It pays to compare: An experimental study on computational estimation (Star & Rittle-Johnson, 2009)

16. Study 2  Research question: Does comparing solution methods improve knowledge of computational estimation?  Research design  Random assignment to  Compare condition: Students compare two solution methods for estimating multi-digit multiplication problems  Sequential condition: Students reflect on same solution methods one at a time  Pretest - Intervention – Posttest – Retention test  Participants  157 fifth and sixth graders from urban, private school or small, rural school  Moderate to low prior knowledge of strategies for estimation 17 5/27/2011 Association for Psychological Science (Star & Rittle-Johnson, 2009)

17. Study 2: Strategies for estimation 18 5/27/2011 Association for Psychological Science Exact Calculation 26 X 42 = 1092 Round one 30 x 42 = 1260, or 26 x 40 =1040 Round both30 x 40 = 1200 Truncate 2 x 4 or 8, and add two zeros to make 800 (Star & Rittle-Johnson, 2009)

18. Study 2: Compare condition 19 5/27/2011 Association for Psychological Science (Star & Rittle-Johnson, 2009)

19. Study 2: Sequential condition 20 5/27/2011 Association for Psychological Science next page (Star & Rittle-Johnson, 2009)

20. Study 2: Results 21 5/27/2011 Association for Psychological Science No Difference (Star & Rittle-Johnson, 2009)

21. Study 2: Results 22 5/27/2011 Association for Psychological Science F(1, 150) = 14.058, p <.001, η 2 =.086 (Star & Rittle-Johnson, 2009)

22. Study 2 Results  Effect of condition depended on prior knowledge 23 5/27/2011 Association for Psychological Science F(1, 150) = 14.069, p<.001, η 2 =.089 (Star & Rittle-Johnson, 2009)

23. Study 2: Results summary  Students in the compare condition became more flexible estimators  Comparing methods aided retention of conceptual knowledge if students had above average knowledge of estimation at pretest 24 5/27/2011 Association for Psychological Science (Star & Rittle-Johnson, 2009)

24. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Five Contrasting Cases studies 5/27/2011 25 Association for Psychological Science  Study 3: The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving (Rittle-Johnson, Star, & Durkin, 2009)

25. Study 3  Research question: Do children with different levels of prior knowledge benefit equally from comparing solution methods?  Research design  Random assignment to  Compare solution methods  Compare problem types  Study sequentially (no comparison)  Identified whether students used algebra at pretest  40% did not attempt algebra  60% attempted algebra  20% of students accurately used algebra  Participants: 236 7th & 8th-grade students in classes with limited algebra instruction 26 5/27/2011 Association for Psychological Science (Rittle-Johnson, Star, & Durkin, 2009)

26. Study 3: Compare conditions 27 5/27/2011 Association for Psychological Science (Rittle-Johnson, Star, & Durkin, 2009)

27. Study 3: Sequential condition 28 5/27/2011 Association for Psychological Science next page (Rittle-Johnson, Star, & Durkin, 2009)

28. Study 3: Results 29 5/27/2011 Association for Psychological Science F(2, 213)= 8.466, p<.001

29. Study 3: Results 30 5/27/2011 Association for Psychological Science F(2, 224)= 2.548, p=.080F(2, 210)= 7.292, p <.001

30. Study 3: Results 31 5/27/2011 Association for Psychological Science F(2, 226) =2.497, p <.085

31. Study 3: Results summary  Prior knowledge matters!  Students with little prior knowledge may not benefit from comparing solution methods  For Students without prior knowledge of algebra  Sequential study of examples or comparing problem types was best for procedural and conceptual knowledge, and flexibility  Sequential study produced fewer signs of confusion  For students who had attempted algebra, comparing solution methods tended to be most effective 32 5/27/2011 Association for Psychological Science (Rittle-Johnson, Star, & Durkin, 2009)

32. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Five Contrasting Cases studies 5/27/2011 33 Association for Psychological Science  Study 4: Developing procedural flexibility: When should multiple solution methods be introduced? (Rittle-Johnson, Star, & Durkin, in press)

33. Study 4  Research questions  What is the impact of early introduction to multiple procedures in equation solving?  How can comparison support learning of these procedures?  Research design  Random assignment to  Immediate compare [Immediate CP]  Delay – compare [Delay CP]  No compare [No CP]  Slower lesson pace, longer intervention time  Pretest - Intervention – Posttest – One-month-Retention Test  Participants: 198 8th grade students in TN classes with limited algebra instruction (i.e., novices) 34 5/27/2011 Association for Psychological Science (Rittle-Johnson, Star, & Durkin, in press)

34. Conditions: Days 1 & 2 35 5/27/2011 Association for Psychological Science Compare two ways to solve same problem Compare different problems solved same way Compare two ways to solve same problem Study examples sequentially Study examples sequentially & review Day 1 Day 2 Day 1 Day 2

35. Study 4: Results 5/27/2011Association for Psychological Science 36 (Rittle-Johnson, Star, & Durkin, in press)

36. Study 4: Results 5/27/2011Association for Psychological Science 37 (Rittle-Johnson, Star, & Durkin, in press)

37. Study 4: Results summary  Immediate comparison of multiple procedures supports attention to and adaption of efficient procedures, which benefits flexibility  Regardless of prior knowledge, students in the no-delay – compare condition…  Had greater procedural flexibility than students in the other conditions  Even novices gain learn from comparing methods if given adequate instructional support. 38 5/27/2011 Association for Psychological Science (Rittle-Johnson, Star, & Durkin, in press)

38. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Five Contrasting Cases studies 5/27/2011 39 Association for Psychological Science  Study 5: Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. (Rittle-Johnson & Star, 2009)

39. Study 5  Research question: Is it best to compare solution methods or are other types of comparison also effective?  Research design  Pretest - Intervention – Posttest – Retention Test  Random assignment to  Compare equivalent problems  Compare problem types  Compare solution methods  Intervention  In addition to partner work, solved practiced problems on own  Participants: 162 7th & 8th grade students from 3 schools 40 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2009)

40. Study 5: Compare conditions 41 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2009)

41. Study 5: Comparison conditions 42 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2009)

42. Study 5: Results 43 5/27/2011 Association for Psychological Science F (2, 153) = 5.76, p =.004, η 2 =.07No Differences

43. Study 5: Results 44 5/27/2011 Association for Psychological Science F (2, 153) = 5.01, p =.008, η 2 =.07F (2, 153) = 4.96, p =.008, η 2 =.06

44. Study 5: Results summary  Comparing Solution Methods supported the largest gains in conceptual knowledge and procedural flexibility  Supported attention to multiple methods and their relative efficiency, which both predicted learning  Comparing problem types supported students’ conceptual knowledge and procedural flexibility to a lesser extent 45 5/27/2011 Association for Psychological Science (Rittle-Johnson & Star, 2009)

45. Summary of findings across studies 46 5/27/2011 Association for Psychological Science  Learning through comparison works!  Supported by all five studies  Rittle-Johnson, B., & Star, J. R. (2007)  Rittle-Johnson, B., & Star, J. R. (2009)  Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009)  Rittle-Johnson, B., Star, J., & Durkin, K. (in press).  Star, J. R., & Rittle-Johnson, B. (2009)  The power of comparison varies by  What is compared  Who compares  When comparison occurs

46. Summary of findings across studies 47 5/27/2011 Association for Psychological Science What to compare Solution methods Rittle- Johnson & Star, 2007 Star & Rittle- Johnson, 2009 Rittle- Johnson, Star, & Durkin, 2009 Rittle- Johnson & Star, 2009 Problem Types Rittle- Johnson, Star, & Durkin, 2009 Rittle- Johnson & Star, 2009

47. Summary of findings across studies 48 5/27/2011 Association for Psychological Science Who compares Students with prior knowledge Rittle-Johnson, Star, & Durkin, 2009 Rittle-Johnson, Star, & Durkin, in press Novices, with modifications Rittle-Johnson, Star, & Durkin, in press

48. Summary of findings across studies 49 5/27/2011 Association for Psychological Science When comparison occurs Immediate comparison Rittle-Johnson, Star, & Durkin, in press

49. Thanks! Questions? Dr. Jon R. Star, Harvard University jon_star@harvard.edu Dr. Bethany Rittle-Johnson, Vanderbilt University b.rittle-johnson@vanderbilt.edu 50 5/27/2011 Association for Psychological Science Acknowledgements: Thanks to Kelley Durkin, Courtney Pollack, Katie Lynch, Natasha Perova, and many other research assistants at Vanderbilt, Michigan State, and Harvard This research is supported by a grant from the U.S. Department of Education, Institute for Education Sciences

50. References  Rittle-Johnson, B., & Star, J. R. (in press). The power of comparison in learning and instruction: Learning outcomes supported by different types of comparisons. In J. P. Mestre & B.H. Ross (Eds.), Psychology of Learning and Motivation: Cognition in Education (Vol. 55).  Rittle-Johnson, B., & Star, J. R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561-574.  Rittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101(3), 529-544.  Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving. Journal of Educational Psychology, 101(4), 836-852.  Rittle-Johnson, B., Star, J., & Durkin, K. (in press). Developing procedural flexibility: When do novices learn from comparing procedures? British Journal of Educational Psychology.  Star, J. R., & Rittle-Johnson, B. (2009). It pays to compare: An experimental study on computational estimation. Journal of Experimental Child Psychology, 102, 408 – 426. 51 5/27/2011 Association for Psychological Science