1. Using Comparison to Support Mathematics Knowledge: From the Lab to the Classroom Bethany Rittle-Johnson Jon Star Kelley Durkin

2. Goal of This Research Program  Focus on a basic learning process to:  Better understand how and when it aids learning  Contribute to improvements in mathematics education  Chosen Process: Comparison

3. Why Comparison? Comparison is Ubiquitous In Our Lives  “It’s not fair! She got more than me.”

4. Comparison is a Key Learning Process  “The simple, ubiquitous act of comparing two things is often highly informative to human learners.”  (Gentner, 2005, pp. 247, 251).

5. Comparison Aids Learning Across Variety of Tasks and Ages  Comparing multiple examples supports abstraction of common structure  Analogy stories in adults (Gick & Holyoak, 1983; Catrambone & Holyoak, 1989)  Perceptual Learning in adults (Gibson & Gibson, 1955)  Negotiation Principles in adults (Gentner, Loewenstein & Thompson, 2003)  Cognitive Principles in adults (Schwartz & Bransford, 1998)  Category Learning and Language in preschoolers (Namy & Gentner, 2002)  Spatial Mapping in preschoolers (Loewenstein & Gentner, 2001)  Spatial Categories in infants (Oakes & Ribar, 2005)

6. Comparison is Used in Mathematics Education  Expert teachers do it (e.g. Lampert, 1990)  Teachers in higher performing countries help students do it (Richland, Zur & Holyoak, 2007)  Reform curriculum advocate for it (e.g. NCTM, 2000; Fraivillig, Murphy & Fuson, 1999)

7. Our Research on Comparison  How does comparison support learning of school mathematics within a classroom setting?  Initial short-term classroom studies:  Redesigned 2 - 3 math lessons on a particular topic (i.e., equation solving, estimation)  Implemented during students’ mathematics classes by researchers  Year-long classroom study:  Designed supplemental curricular materials for Algebra I courses  Implemented by classroom teachers for school year (Rittle-Johnson & Star, 2012)

8. Short-Term Classroom Research  Five studies  Effectiveness of comparing correct methods (5 of 5)  Effectiveness of comparing problem types (2 of 5)  Focus on equation solving (4 of 5)  Focus on estimation (1 of 5)

9. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Five Short-Term Studies  Study 1

10. Study 1  Research design  Random assignment to  Compare condition: Students compare and contrast alternative solution methods for equation solving  Sequential condition: Students study same solution methods sequentially  Pretest - Intervention – Posttest  Participants  70 7th-grade students (Rittle-Johnson & Star, 2007)

11. Translation to the classroom  Students study and explain worked examples with a partner  Randomly assigned student pairs to condition within the same classroom

12. Study 1: Compare condition (Rittle-Johnson & Star, 2007)

13. Study 1: Sequential condition next page (Rittle-Johnson & Star, 2007)

14. Measures  Procedural knowledge (e.g., equation solving)  Problem-solving accuracy: ability to implement step-by-step methods, including adapting known procedures to novel problems (i.e., transfer)  Conceptual knowledge (e.g., equivalence, like terms, composite variables)  “Is 98 = 21x equivalent to 98 + 2x = 21x + 2x?”  Procedural flexibility  Flexible Use - Use of more efficient solution methods on procedural knowledge assessment (i.e., fewer solution steps)  Flexible Knowledge  Knowledge of multiple methods (e.g., solve each equation in two different ways when prompted)  Ability to evaluate methods (e.g., “Looking at the problem shown above, do you think that this first step is a good way to start this problem?”)

15. Study 1: Results (Rittle-Johnson & Star, 2007) F(1, 31) =4.49, p <.05

16. Study 1: Results F(1,31) = 7.73, p <.01 Solution MethodCompariso n Sequential Most Efficient.17*.10 Flexible Use of Procedures * p <.05 (Rittle-Johnson & Star, 2007)

17. Study 1: Results (Rittle-Johnson & Star, 2007) No Difference

18. Study 1: Results summary  Students in the compare condition  Showed greater gains in procedural knowledge and flexibility  Were more likely to use more efficient method and somewhat less likely to use the conventional method  Maintained conceptual knowledge (Rittle-Johnson & Star, 2007)

19. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Study 2: New Topic

20. Study 2: Compare condition (Star & Rittle-Johnson, 2009)

21. Study 2: Results summary  Similar findings for new topic.  Students in the compare condition:  Tended to have greater procedural knowledge  Had greater procedural flexibility  Retained conceptual knowledge better  Only if students had above average knowledge of estimation at pretest (Star & Rittle-Johnson, 2009)

22. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Study 3: Importance of Prior Knowledge

23. Study 3  Research question: Do children with different levels of prior knowledge benefit equally from comparing solution methods?  Participants: 236 7th & 8th-grade students in classes with limited algebra instruction  Identified whether students used algebra at pretest  40% did not attempt algebra  60% attempted algebra  Note: Only 20% of students accurately used algebra vs. 96% of students in Study 1 (Rittle-Johnson, Star, & Durkin, 2009)

24. Study 3: Results

27. Study 3: Results summary  Prior knowledge matters!  Students with little prior knowledge may not benefit from comparing solution methods  For Students without prior knowledge of algebra  Sequential study of examples was best for procedural and conceptual knowledge and flexibility  Sequential study produced fewer signs of confusion  For students who had attempted algebra, comparing solution methods tended to be most effective (Rittle-Johnson, Star, & Durkin, 2009)

28. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Study 4: Adjusting for Limited Prior Knowledge

29. Study 4  Research questions  Can comparing methods support learning in novices?  Modified materials to accommodate novices:  Slower lesson pace, cover less material, longer intervention time  Pretest - Intervention – Posttest – One-month-Retention Test  Participants: 198 8th grade students with limited algebra instruction (i.e., novices) (Rittle-Johnson, Star, & Durkin, 2011)

30. Study 4: Results Compare group was more flexible Conceptual and Procedural Knowledge were similar across conditions

31. Study 4: Results summary  Regardless of prior knowledge, students in the compare condition had greater procedural flexibility  Even novices can learn from comparing methods if given adequate instructional support, although the benefits are more modest.

32. Study 1 (Rittle-Johnson & Star, 2007) Study 2 (Star & Rittle- Johnson, 2009) Study 3 (Rittle-Johnson, Star, & Durkin, 2009) Study 4 (Rittle-Johnson, Star, & Durkin, in press) Study 5 (Rittle-Johnson & Star, 2009) Study 5: Compared to What?

33. Types of Comparison  Solution Methods  1 problem  solved in 2 ways  Problem Types  2 different problems,  solved in same way  Equivalent  two similar problems,  solved in same way

34. Study 5  Research question: Is it best to compare solution methods or are other types of comparison also effective?  Research design  Pretest - Intervention – Posttest – Retention Test  Random assignment to  Compare equivalent problems  Compare problem types  Compare solution methods  Participants: 162 7th & 8th grade students with some previous Algebra instruction (not novices) (Rittle-Johnson & Star, 2009)

35. Study 5: Results F (2, 153) = 5.76, p =.004, η 2 =.07No Differences

36. Study 5: Results F (2, 153) = 5.01, p =.008, η 2 =.07F (2, 153) = 4.96, p =.008, η 2 =.06

37. Study 5: Results summary  Comparing Solution Methods supported the largest gains in conceptual knowledge and procedural flexibility  Supported attention to multiple methods and their relative efficiency, which both predicted learning  Comparing problem types supported students’ conceptual knowledge and procedural flexibility to a lesser extent  Similar findings in Rittle-Johnson, Star & Durkin (2009), in which students had less prior knowledge (Rittle-Johnson & Star, 2009)

38. Summary of findings across short-term studies  Learning through comparison works!  The power of comparison varies by  What is compared  Who compares

39. Year-Long Classroom Study  Support teachers’ use of comparison throughout Algebra I  Supplemental materials – 150 pairs of worked examples with comparison prompts for variety of Algebra I topics.  Expanded the types of comparisons we promoted.

40. 1. Present examples side-by-side 2. Use common language to draw attention to similarities. 3. Prompt for specific comparisons, tailored to your learning goals. 4. Be sure students, not just teachers, are comparing and explaining. Guidelines for supporting comparison

41. 5. Include a lesson summary, highlighting key points of the comparison.

42. Reasons to Compare Methods Comparison goal: Which is better?Which is correct? Why does it work? Features of examples Same problem solved in two different, correct ways Same problem solved correctly and solved incorrectly. Same problem solved in two different, correct ways Features of comparison prompts Compare efficiencyCompare accuracyCompare similarities to identify key concepts Learning goalProcedural flexibility Procedural accuracy Conceptual understanding

43. Additional Type of Comparison: Comparing Problems When can you use it?What does it mean? Features of examples Different problems, solved in the same or different ways Different problems, solved in a similar way Features of comparison prompts Compare features of problems to learn when a particular procedure can be used Compare impact of a problem feature on the answer Learning goalProcedural transfer – use procedure on a broad range of problems Conceptual understanding

44. Study Design  Participants: Algebra I teachers  45 Treatment Teachers  32 Control Teachers (waitlist design - received training and materials following year)  Treatment  One week of professional development in summer  Asked to use our materials about twice a week. Teacher chooses which to use.

45. Actual Use in Treatment Classrooms AverageMinimumMaximum Number of days used 20*456 Number of minutes used 28023864 Percent of instructional time 3.50.310.7 *Based on teacher logs; on average, use less than once a week (36 weeks in school year)

46. Quality of Use: Video Coding AverageMinimumMaximum Critical Features Present 2.613 Critical & Desired Features Present 4.516 Sample Critical Feature: Touched on the primary instructional aim of all three phases (understand, compare, make connections) Sample Desired Feature: Class discussions were present not only in the ‘Make Connections’ phase, but in the other phases as well.

47. 2 Outcome Measures  Standardized Algebra Achievement (Acuity)  Researcher-designed measure tapping conceptual, procedural and flexibility knowledge for Algebra  Administered at beginning and end of school year

48. Preliminary Outcomes  No main effect of condition on either measure  Perhaps not surprising given how little materials were being used in some treatment classrooms (typically used less than once a week, for about 3.5% of estimated instructional time)  But, frequency and quality of use of our materials predicted results on researcher-designed measure

49. Frequency of Use Predicts Posttest Scores Partial r (31) =.41, p =.02

50. Quality of Use Predicts Posttest Scores Partial r (31) =.31, p =.08

51. Classroom Study Summary  Use of our supplemental curricular materials that supported a variety of comparison types:  Large variability in frequency of use of treatment materials  Quality of use was fairly high, with some variability it use of desired features of use  Impact of our materials (preliminary):  Students in treatment classrooms did not outperform students in control classrooms  Greater frequency and quality of use of treatment materials predicted better outcomes on researcher-designed measure - promising! Lots of analyses still to do

52. Discussion  Comparison can support mathematics learning  Consistent benefits when used intensively for short period of time, especially comparing correct solution methods  Potential benefits when used throughout school year by teachers – if used frequently!  There are different types of comparisons that should support different learning outcomes.  Future research needed to flesh out how different types of comparisons support different learning outcomes

53. Bridging Between Cognitive Science and Education  Usually requires collaboration between cognitive scientists and education researchers.  Look for convergence across the cognitive science and education literatures to highlight particularly promising learning approach to pursue.  Conduct experimental research in classrooms to naturally constrain the research to at least some typical classroom conditions.  Cognitive science does not only inform educational practice; educational practice reveals new constraints and new ideas that need to be tested and incorporated into theories of learning.