1. Strategy Flexibility Matters for Student Mathematics Achievement: A Meta-Analysis Kelley Durkin Bethany Rittle-Johnson Vanderbilt University, United States Jon R. Star Harvard University, United States

2. Defining Strategy Flexibility Simplest definition: Knowing more than one strategy for solving a particular type of problem (e.g., Heirdsfield & Cooper, 2002) Most complex definition: Being able to use a variety of strategies and information from the problem context, the learner’s environment, and the sociocultural context to select the most appropriate problem solving procedure (e.g., Verschaffel, Luwel, Torbeyns, & Van Dooren, 2007) 2

3. Recent Focus on Strategy Flexibility Previously, flexibility rarely measured as an instructional outcome (Star, 2005). Standardized tests in the U.S. include sections on: – Concepts – Procedures – Problem solving – But not flexibility Recently, flexibility examined as a separate outcome (Star, 2007; Verschaffel et al., 2007). 3

4. Importance of Strategy Flexibility Helps adapt existing procedures to unfamiliar problems (e.g., Blöte, Van der Burg, & Klein, 2001) Greater understanding of domain concepts (e.g., Hiebert & Wearne, 1996) Crucial component of expertise in problem solving (Dowker, 1992; Dowker, Flood, Griffiths, Harris, & Hook, 1996; Star & Newton, 2009) 4

5. Current Study Is strategy flexibility related to other mathematical constructs? – Conceptual Knowledge Success recognizing and explaining key domain concepts (Carpenter et al., 1998; Hiebert & Wearne, 1996) – Procedural Knowledge Success executing action sequences to solve problems (Hiebert & Wearne, 1996; Rittle-Johnson, Siegler, & Alibali, 2001) – General Mathematics Achievement Meta-analysis of our past work 5

6. Our Definition of Strategy Flexibility Knowing multiple strategies and their relative efficiencies (Flexibility Knowledge) AND Adapting strategy choice to specific problem features (Flexible Use) (e.g., Blöte et al., 2001; National Research Council, 2001; Rittle-Johnson & Star, 2007) 6

7. Method Overview Selected Studies Measures Analysis Strategies 7

8. Included Studies Study AuthorsYearTopicNGrade Rittle-Johnson & Star2007Equation Solving707 Star & Rittle-Johnson2008Equation Solving1556 Rittle-Johnson & Star2009Equation Solving1627 & 8 Rittle-Johnson, Star, & Durkin 2009Equation Solving2367 & 8 Star et al.2009Estimation655 Star & Rittle-Johnson2009Estimation1575 & 6 Rittle-Johnson, Star, & Durkin 2011Equation Solving1988 Schneider, Rittle-Johnson & Star 2011Equation Solving2937 & 8 8

9. Measures Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Tests 9

10. Measures Flexibility Knowledge – Knowing multiple procedures and the relative efficiency of the procedures 5(x + 3) + 6 = 5(x + 3) + 2x 6 = 2x a. What step did the student use to get from the first line to the second line? b. Do you think that this is a good way to start this problem? Circle One: (a) a very good way (b) OK to do, but not a very good way (c) Not OK to do c. Explain your reasoning. 10

11. Measures Flexible Use – Students using the most appropriate strategy depending on problem features 3(h + 2) + 4(h + 2) = 35 7(h + 2) = 35 Sometimes know a more appropriate strategy for solving a problem before actually use it (Blöte et al., 2001; Siegler & Crowley, 1994) 11

12. Measures Conceptual Knowledge – Ability to recognize and explain key domain concepts Which of the following is a like term to (could be combined with) 7(j + 4)? (a) 7(j + 10) (b) 7(p + 4) (c) j (d) 2(j + 4) (e) a and d Procedural Knowledge – Ability to execute action sequences to solve problems 3(h + 2) + 4(h + 2) = 35 12

13. Measures Standardized Tests National Tests Comprehensive Testing Program (CTP) Measures of Academic Progress (MAP) State Tests Massachusetts Comprehensive Assessment System (MCAS) Tennessee Comprehensive Assessment Program (TCAP) Collected scores from school records 13

14. Coding and Analysis Strategies Calculated correlation between each pair of outcomes for each study Fischer’s z to transform correlations to get effect sizes, ES r, for each study (Lipsey & Wilson, 2001). The mean correlation effect size was calculated using a random effects model. 14

15. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 15

16. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 16

17. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 17 Flexibility knowledge and flexible use strongly related

18. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 18

19. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 19 Conceptual knowledge had moderately strong relationships to flexibility

20. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 20 Conceptual knowledge had moderately strong relationships to flexibility Procedural knowledge had moderately strong relationships to flexibility

21. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 21 Similar to correlation between conceptual and procedural knowledge

22. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 22

23. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 23 Standardized test measures significantly correlated with flexibility

24. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 24 Standardized test measures significantly correlated with flexibility Standardized test measures significantly correlated with other outcomes

25. Results Mean correlations between outcomes Flexibility Knowledge Flexible Use Conceptual Knowledge Procedural Knowledge Standardized Test Flexibility Knowledge 1.635.563.610.535 Flexible Use 1.541.627.404 Conceptual Knowledge 1.544.520 Procedural Knowledge 1.475 Note: All correlations were significant (p <.001) 25 Standardized test measures significantly correlated with flexibility Standardized test measures significantly correlated with other outcomes Correlations between flexibility and standardized tests similar to other correlations

26. 3 Main Findings Flexibility knowledge and flexible use are separate constructs Flexibility is related to other constructs Standardized tests relate to flexibility as well as they relate to other constructs 26

27. The Construct of Strategy Flexibility May be important to measure flexible use and flexibility knowledge separately. – Appears measures of knowledge and use are tapping different aspects of flexibility. Conceptual and procedural knowledge are related to flexibility (Schneider et al., 2011). 27

28. Relation to Standardized Tests Standardized test scores relate to flexibility just as well as they relate to conceptual and procedural knowledge. Teachers can feel pressured to teach to the test, and the lack of flexibility items on assessments could lead to less time on flexibility in the classroom. Push for standardized tests to include items that assess flexibility. Flexibility a valued outcome when evaluating interventions. 28

29. Conclusion Strategy flexibility is important for developing expertise and efficient problem solving Need to measure and encourage students’ strategy flexibility in the future 29

30. 30 Acknowledgements E-mail: kelley.durkin@vanderbilt.edu Visit our Contrasting Cases Website at http://gseacademic.harvard.edu/contrastingcases/index.html for more information http://gseacademic.harvard.edu/contrastingcases/index.html Thanks to the Children’s Learning Lab at Vanderbilt University Funded by a grant from the Institute for Education Sciences, U.S. Department of Education – The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education. 30

31. References Blöte, A. W., Van der Burg, E., & Klein, A. S. (2001). Students' flexibility in solving two-digit addition and subtraction problems: Instruction effects. Journal of Educational Psychology, 93(3), 627-638. Carpenter, T. P., Franke, M. L., Jacobs, V. R., Fennema, E., & Empson, S. B. (1998). A longitudinal study of invention and understanding in children's multidigit addition and subtraction. Journal for Research in Mathematics Education, 29(1), 3-20. Dowker, A. (1992). Computational estimation strategies of professional mathematicians. Journal for Research in Mathematics Education, 23(1), 45-55. Dowker, A., Flood, A., Griffiths, H., Harris, L., & Hook, L. (1996). Estimation strategies of four groups. Mathematical Cognition, 2(2), 113-135. Heirdsfield, A. M., & Cooper, T. J. (2002). Flexibility and inflexibility in accurate mental addition and subtraction: Two case studies. The Journal of Mathematical Behavior, 21, 57-74. 31

32. References Hiebert, J., & Wearne, D. (1996). Instruction, understanding and skill in multidigit addition and subtraction. Cognition and Instruction, 14, 251- 283. Lipsey, M. W., & Wilson, D. B. (2001). Practical Meta-Analysis (Vol. 49). Thousand Oaks, CA: Sage Publications. National Research Council. (2001). Adding it up: Helping children learn mathematics. Washington, DC: National Academy Press. Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93, 346-362. Rittle-Johnson, B., & Star, J. R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561-574. 32

33. References Rittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101, 529-544. Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving. Journal of Educational Psychology, 101(4), 836-852. Rittle-Johnson, B., Star, J. R., & Durkin, K. (2011, June 28). Developing procedural flexibility: Are novices prepared to learn from comparing procedures? British Journal of Educational Psychology. Advance online publication. Schneider, M., Rittle-Johnson, B., & Star, J. R. (2011, August 8). Relations among conceptual knowledge, procedural knowledge, and procedural flexibility in two samples differing in prior knowledge. Developmental Psychology. Advance online publication. 33

34. References Siegler, R. S., & Crowley, K. (1994). Constraints on learning in nonprivileged domains. Cognitive Psychology, 27(2), 194-226. Star, J. R. (2005). Reconceptualizing procedural knowledge. Journal for Research in Mathematics Education, 36, 404-411. Star, J. R. (2007). Foregrounding Procedural Knowledge. [Peer Reviewed]. Journal for Research in Mathematics Education, 38(2), 132-135. Star, J. R., & Newton, K. J. (2009). The nature and development of experts’ strategy flexibility for solving equations. ZDM-International Journal on Mathematics Education, 41, 557-567. Star, J. R., & Rittle-Johnson, B. (2008). Flexibility in problem solving: The case of equation solving. Learning and Instruction, 18, 565 - 579. Star, J. R., & Rittle-Johnson, B. (2009). It pays to compare: An experimental study on computational estimation. Journal of Experimental Child Psychology, 102, 408 - 426. 34

35. References Star, J. R., Rittle-Johnson, B., Lynch, K., & Perova, N. (2009). The role of prior knowledge in the development of strategy flexibility: The case of computational estimation. ZDM, 41(5), 569-579. Verschaffel, L., Luwel, K., Torbeyns, J., & Van Dooren, W. (2007). Developing adaptive expertise: A feasible and valuable goal for (elementary) mathematics education? Ciencias Psicologicas, 2007(1), 27-35. 35