1. The Role of Comparison in the Development of Flexible Knowledge of Computational Estimation Jon R. Star (Harvard University) Bethany Rittle-Johnson (Vanderbilt University) AERA, New York City. Tuesday, March 25, 2008. Session 33.074 Developing Mathematical Understanding paper session, Crowne Plaza Times Square, Room 509/510, 4:05 – 5:35 pm

2. Thanks to... Research supported by Institute of Education Sciences (IES) Grant # R305H050179 All participating teachers and schools in Nashville, Tennessee and Hale, Michigan Graduate and undergraduate research assistants at Vanderbilt, Michigan State, and Harvard March 25, 20082AERA CCS4

3. Comparison Is a fundamental learning mechanism Lots of evidence from cognitive science –Identifying similarities and differences in multiple examples appears to be a critical pathway to flexible, transferable knowledge Mostly laboratory studies Not done with school-age children or in mathematics March 25, 20083AERA CCS4 (Gentner, Loewenstein, & Thompson, 2003; Kurtz, Miao, & Gentner, 2001; Loewenstein & Gentner, 2001; Namy & Gentner, 2002; Oakes & Ribar, 2005; Schwartz & Bransford, 1998)

4. Central tenet of math reforms Students benefit from sharing and comparing of solution methods “nearly axiomatic,” “with broad general endorsement” (Silver et al., 2005) Noted feature of ‘expert’ math instruction Present in high performing countries such as Japan and Hong Kong March 25, 2008AERA CCS44 (Ball, 1993; Fraivillig, Murphy, & Fuson, 1999; Huffred-Ackles, Fuson, & Sherin Gamoran, 2004; Lampert, 1990; Silver et al., 2005; NCTM, 1989, 2000; Stigler & Hiebert, 1999)

5. “Contrasting Cases” Project Experimental studies on comparison in academic domains and settings largely absent Goal of present work –Investigate whether comparison can support learning and transfer, flexibility, and conceptual knowledge –Experimental studies in real-life classrooms –Algebra equation solving –Computational estimation March 25, 2008AERA CCS45

6. Rittle-Johnson & Star, 2007 Experimental study in algebra classrooms with 70 7th grade students on equation solving Intervention (Comparison condition) –Comparing and contrasting alternative solution methods Control (Sequential condition) –Reflecting on same solution methods one at a time March 25, 2008AERA CCS46 Rittle-Johnson, B, & Star, J.R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561-574.

7. Results (from 2007 study) At posttest, students in comparison condition made significantly greater gains in procedural knowledge and flexibility and comparable gains in conceptual knowledge The intervention worked! But need to replicate and extend these findings... March 25, 2008AERA CCS47

8. Computational estimation Widely studied in 80’s and early 90’s –Less so in recent years Process of mentally generating an approximate answer for a given arithmetic problem (Rubenstein, 1985) –(Distinct from “mental computation,” which means finding the exact answer) Estimates of 2-digit multiplication problems 13 x 27 March 25, 2008AERA CCS48

9. Strategies for estimating 13 x 27 Round both (to the nearest 10) 10 x 30 or 300 Round one (to the nearest 10) 10 x 27 or 270 (Alternatively, 13 x 30, or 390) Trunc (truncate) (Sowder & Wheeler, 1989) 1 x 2, or 2 Then append 2 zeros, or 200 March 25, 2008AERA CCS49

10. Why estimation for replication? Estimation is different from algebra equation solving in several ways that play a potentially important role whether comparison will help March 25, 2008AERA CCS410

11. Equation solving vs. estimation Equation solving –Problems have a single correct answer Estimation –Correctness or “goodness” of estimate depends on two sometimes competing goals –Simplicity: how easy it is to compute –Proximity: how close the estimate is to the exact answer March 25, 2008AERA CCS411

12. Complexity of “goodness” Sometimes easy-to-compute estimate is not very proximal to the exact value, and vice versa Some strategies present false illusion of consistent proximity Is round one always more proximal than round two? –Intuitively, yes? The less you round, the closer you get –Actually no! It depends on the problem –Try it for 39 x 41 versus 39 x 37 March 25, 2008AERA CCS412

13. Solution efficiency Algebra equation solving –Easy and visually apparent to judge relative efficiency of two compared solutions –For example, just count the number of steps! Estimation –Not at all clear how one would judge efficiency of two compared solutions –Efficiency is more of an individual or subjective judgment March 25, 2008AERA CCS413

14. Summary of rationale Evidence for effectiveness of comparison in algebra, but replication needed Estimation is a good domain in which to replicate –Certain features of estimation raise legitimate questions about whether comparison of strategies can have the same positive impact March 25, 2008AERA CCS414

15. Method Estimation: 158 5th-6th grade students 69 5th graders in urban private school –4 classes, taught by the same teacher 44 5th and 45 6th graders in rural public school –Two 5th classes, taught by same teacher –Two 6th classes, taught by same teacher March 25, 2008AERA CCS415

16. Design Pretest - Intervention – Posttest – Retention test –3-day intervention replaced lessons in textbook Intervention occurred in partner work during math classes –Random assignment of pairs to condition –Both conditions present in all classrooms Students studied worked examples with partner and also solved practice problems on own March 25, 2008AERA CCS416

17. Comparison materials March 25, 2008AERA CCS417

18. Sequential materials March 25, 2008AERA CCS418 page next page next

19. Results Procedural knowledge –Ability to compute accurate estimates Flexibility –Knowledge of multiple strategies and ability to select most appropriate strategies for a given problem and problem-solving goal –Direct measure (from procedural knowledge items) –Independent measure Conceptual knowledge March 25, 2008AERA CCS419

20. Procedural knowledge Mental –Estimate 32 x 17 mentally and quickly Familiar –Estimate 12 x 24 and 113 x 27 Transfer –Estimate 1.19 x 2.39 and 102 ÷ 27 March 25, 2008AERA CCS420

21. Procedural knowledge results No difference between conditions on: Accuracy of estimates –Assessed accuracy a number of ways, none of which showed a difference for intervention students Both conditions improved the accuracy of students’ estimates March 25, 2008AERA CCS421

22. Accuracy of estimates March 25, 2008AERA CCS422

23. Conceptual knowledge sample What does “estimate” mean? March 25, 2008AERA CCS423

24. Conceptual knowledge results Comparison improved conceptual knowledge for students who began with some initial procedural knowledge, but not for those who began with very little procedural knowledge In other words, comparison was particularly beneficial for students who began the study with some initial ability to compute estimates March 25, 2008AERA CCS424

25. Conceptual knowledge March 25, 2008AERA CCS425

26. Flexibility sample items Independent measure Estimate 12 x 36 in three different ways March 25, 2008AERA CCS426

27. Flexibility results Comparison led to greater flexibility Independent measure: Comparison students... –Were more likely to be able to produce estimates for the same problem in multiple ways –Were better at making judgments about which strategy would led to an easier or a closer estimate for a given problem March 25, 2008AERA CCS427

28. Flexibility independent measure March 25, 2008AERA CCS428

29. Flexible strategy use Direct measure –Strategies on procedural knowledge assessment Comparison students were: –More likely to use trunc, optimizing their strategy use for ease more than sequential student March 25, 2008AERA CCS429

30. Flexibility Recall that students in both conditions saw the same strategies demonstrated on the same problems Yet comparison students were –Better at generating multiple ways to find estimates –Were more likely to use the easiest strategy for a given problem –Were better at predicting which strategy would led to a close or easy estimate for a given problem March 25, 2008AERA CCS430

31. In sum... It pays to compare! Comparison led to: –Greater flexibility (2007 and present study) –Improved procedural knowledge (2007 study) –Improved conceptual knowledge (for students with modest procedural knowledge at pre-test; present study) –In two very different mathematical domains, algebra and computational estimation March 25, 2008AERA CCS431

32. National Math Panel (p. 27) “Teachers should broaden instruction in computational estimation beyond rounding. They should insure that students understand that the purpose of estimation is to approximate the exact value and that rounding is only one estimation strategy.” March 25, 2008AERA CCS432

33. National Math Panel (p. 27) “Textbooks need to explicitly explain that the purpose of estimation is to produce an appropriate approximations. Illustrating multiple useful estimation procedures for a single problem, and explaining how each procedure achieves the goal of accurate estimation, is a useful means for achieving this goal. Contrasting these procedures with others that produce less appropriate estimates is also likely to be helpful.” March 25, 2008AERA CCS433

34. Thanks! This presentation and other related papers and presentations can be found at: http://gseacademic.harvard.edu/~starjo/ or by contacting Jon Star (jon_star@harvard.edu)