1. The Effects of Feedback During Exploratory Math Practice
SREE Fall 2011
The Effects of Feedback During Exploratory Math Practice
Emily R. Fyfe & Bethany Rittle-Johnson
Vanderbilt University
Marci S. DeCaro
University of Louisville

2. How do children learn best?
Two schools of thought have emerged
Advocates of Direct Instruction
(Kirschner, Sweller, & Clark, 2006)
Advocates of Discovery Learning
(Bruner, 1961; Kuhn, 1989; Piaget, 1973)

3. An Integrated Perspective
No need for this strict dichotomy
“Proponents of constructivism and direct instruction…each have something to learn from the other” (Rieber, 1992)
“A mixture of guidance and exploration is needed” (Mayer, 2004)
“There’s a place for both direct instruction and student-directed inquiry” (Kuhn, 2007)
“Characterizing discovery and direct instruction as diametrically opposed…has done a disservice to both approaches” (Wilson,
et al, 2010)

4. Combining Instruction and Discovery
Exploration prior to instruction facilitates learning
(DeCaro & Rittle-Johnson, 2011; Schwartz & Bransford, 1998)
Hints or coaching during problem solving is better than pure problem solving alone
(Mayer, 2004)
Recent meta-analysis indicates that guided discovery is better than unassisted discovery or direct instruction
(Alfieri et al, 2010)

5. Questions remain…
Which elements of instruction are most suitable to incorporate within exploration? For whom is guided exploration most advantageous and why?

6. Feedback During Exploration
Feedback represents one element of instruction that may be particularly effective in combination with exploration
What is Feedback?
Any information that the learner can use to confirm, reject, or modify prior knowledge
Accuracy information, hints, etc.

7. (Ahmad, 1988; Kluger & DeNisi, 1996; Luwel et al., 2011)
Types of Feedback
Outcome Feedback
Strategy Feedback
Provides information about learner’s answer
Used extensively
Related to strong, positive effects in past work compared to no feedback
Provides information about how answer was obtained
Only examined in a few previous studies
Better than outcome feedback in terms of strategy selection
(Ahmad, 1988; Kluger & DeNisi, 1996; Luwel et al., 2011)

8. Why feedback?
May reduce disadvantages of discovery by guiding the learner’s search for information
Helps identify errors and encourages search for plausible alternatives (e.g., new strategies)
(Hattie & Timperley, 2007; Mory, 2004)
But…
Past research indicates variable efficacy
(Kluger & DeNisi, 1996)
May only benefit a subset of learners

9. What about prior knowledge?
Expertise reversal effect
Instructional technique is effective for novices, but loses its benefits for high-knowledge learners
(Kalyuga, Ayres, Chandler, & Sweller, 2003)
Example: worked examples vs. problem solving
Low knowledge learners benefit from more external guidance; high knowledge learners benefit from less
Perhaps feedback during exploration only helps children who have low prior knowledge

10. Why the reversal? Due to existing cognitive resources
(Paas, Renkl, & Sweller, 2003)
Novices lack schemas; need external guidance to reduce cognitive load
High-knowledge learners have schemas; additional guidance creates more cognitive load

11. Goals of this study
Examine the effects of feedback during exploratory math practice for children with varying levels of prior knowledge
Specifically
Compare feedback vs. no feedback
Compare outcome vs. strategy feedback
Look at effects of prior knowledge

12. Hypotheses Ho 1: Feedback > No Feedback Ho 2:
Strategy Feedback > Outcome Feedback
One sentence about why strat feedback is better
Ho 3:
Feedback better for children with low prior knowledge

13. Domain: Mathematical Equivalence
Concept that two sides of an equation represent the same amount and are interchangeable
Commonly represented by equal sign (=)
Important for algebra: hammer this point
= 3 + _
6 + 4 = _ + 8

14. Why Study Math Equivalence?
Fundamental concept in arithmetic and algebra
Very difficult for children in U.S.
Interpret equal sign as an operator symbol that means “the total” as opposed to relational symbol
(McNeil, 2008; Rittle-Johnson & Alibali, 1999)
In one study, only 24% of U.S. children in 3rd and 4th grade solved math equivalence problems correctly
(McNeil & Alibali, 2000)

15. Participants Worked with 91 children (2nd & 3rd grade)
M age = 8 yrs, 7 mo
53 females, 38 males
45% white, 40% black, 15% other
47% receive free or reduced lunch

16. Design and Procedure Session 1: Pretest (~25 minutes)
Excluded if score >80% on pretest measures
Session 2: Intervention & Posttest (~50 minutes)
Session 3: Two-week Retention Test (~25 minutes)

17. Tutoring Intervention
Exploratory Practice
Attempt to solve 12 math equivalence problems
Randomly assigned to 1 of 3 conditions
No Feedback (n = 31)
Outcome Feedback (n = 32)
Strategy Feedback (n = 28)
Midtest
Brief conceptual instruction
(DeCaro & Rittle-Johnson, 2011)
Be 7:30 in

18. Exploratory Practice Find the number that goes in the blank.
= 3 + ☐
How did you solve that problem?
No Feedback: “OK, let’s move on to the next problem.”
Outcome Feedback: “Good try, but that’s not the correct answer. The correct answer is 12.”
Strategy Feedback: “Good try, but that’s not a correct way to solve that problem.”

19. Assessment of Math Equivalence
Procedural Knowledge
Use correct strategy to solve problems
Conceptual Knowledge
Understand concept of equivalence
Used at Pretest, Midtest, Posttest, & Retention Test
= 7 + _
= _ + 3
Emphasize how we measure procedural knowledge
What does the equal sign mean?
4 + 8 = 8 + 4
True or False?
(Rittle-Johnson, Matthews, Taylor, & McEldoon, 2011)

20. Analysis & Results Contrast-based ANCOVA (West, Aiken, & Krull, 1996)
Two contrast-coded condition variables
Feedback (no feedback vs. two feedback conditions combined)
Feedback Type (outcome feedback vs. strategy feedback)
Two condition x prior knowledge interactions
Three covariates
Interaction follow-up
Categorize as low vs. high knowledge (median split)
Simple main effects of condition
Also note the results I’ll be presenting. First I’ll discuss children’s procedural knowledge on the posttest assessments. Then I’ll show some results from the intervention activities.
Contrast-based ANCOVA
Two contrast-coded condition variables
Feedback (No feedback vs. Feedback conditions combined)
Feedback type (Outcome feedback vs. Strategy feedback)
Two condition x prior knowledge interactions
Three covariates

21. Procedural Knowledge
Repeated Measures ANCOVA: Midtest, Posttest, and Retention Test.
Less time.

22. Procedural Knowledge
Overall feedback x prior knowledge interaction, F(1, 83) = 7.05, p = .01
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 3.84, p = .05
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.61, p = .04 *
Less time. *

23. Procedural Knowledge Results
Expertise reversal effect Feedback during exploratory math practice is more beneficial than no feedback, but only for children with low knowledge For children with high prior knowledge, the reverse is true; they benefit more from no feedback

24. Intervention Activities
Subjective Cognitive Load (NASA-TLX, Hart & Staveland, 1988)
I had to work hard to solve those problems.
I was stressed and irritated when I had to solve those problems.
Mean rating on agreement scale from 1 to 5.
Problem-Solving Strategy Use
Variability of correct & incorrect strategies

25. Cognitive Load *
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 1.15, p = .29
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 6.05, p = .02

26. Cognitive Load
Supports expertise reversal effect explanation Feedback may have hurt high-knowledge learners’ performance because of increased cognitive load An effect not found for low-knowledge learners

27. Strategy Coding Scheme
Sample explanation ( = __ + 8)
Correct Strategies
Equalize
I added 4, 5, and 8 and got 17. And 9 plus 8 is 17.
Add-Subtract
I added 4, 5, and 8 and got 17. And 17 minus 8 is 9.
Grouping
I took out the 8’s and I added 4 plus 5.
Incorrect Strategies
Add-All
I added the 4, 5, 8 and 8.
Add-to-Equal
I just added the first three, the 4, 5, and 8.
Carry
I saw a 4 here, so I wrote a 4 in the blank.
Ambiguous
I used 8 plus 8, and then 5.
Spend less time on this slide

28. Perseveration
Perseveration = Using the same incorrect strategy on all the problems. *
* P = .01

29. Incorrect Strategy Variability* *
Ranged from 0 to 5.
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 5.66, p = .02
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.56, p = .04

30. Correct Strategy Variability*
Ranged from 0 to 3.
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.76, p = .03
High Knowledge: Feedback vs. No Feedback, F(1, 83) = 0.16, p = .90

31. Strategy Variability
For children with low prior knowledge, feedback prevented perseveration and led to the generation of diverse strategies
May explain why these children learned more when they received feedback than when they did not
For children with higher prior knowledge, feedback led to the generation of more incorrect, but not correct strategies
May help explain why feedback had negative impact
Feedback facilitated the generation of incorrect, but not correct strategies for children with high prior knowledge
May help explain why feedback had negative impact

32. Summary
Feedback led to higher procedural knowledge of math equivalence than no feedback, but only for children with low prior knowledge For children with high prior knowledge, no feedback was better No differences between outcome feedback and strategy feedback

33. Implications Theoretical Practical
Extends expertise reversal effect to feedback
Clarifies research on discovery learning
Practical
Pay more attention to when you give feedback during tutoring and teaching

34. Thank You
Children’s Learning Lab Bethany Rittle-Johnson Marci DeCaro Laura McLean Maryphyllis Crean Lucy Rice
Funding Sources ExpERT Training Grant, through IES to Fyfe NSF CAREER grant to Rittle-Johnson

35. (Kirschner, Sweller, & Clark, 2006; Mayer, 2004; Sweller, 1988)
Instruct vs. Discover
Direct Instruction
Discovery Learning
Told or shown how to solve the problems
Provides structure and reduces task ambiguity
But…
Limits self-discovery
Might limit learner engagement
Explore the problems on your own with no guidance
Find target information and new strategies independently
But…
Overwhelms working memory
Might never locate target info or invent correct strategy
(Kirschner, Sweller, & Clark, 2006; Mayer, 2004; Sweller, 1988)

36. Exploratory Practice
Children’s performance on 12 problems during intervention (Percent Correct)

37. Let’s take a look at this problem.
Instruction
Let’s take a look at this problem.
3 + 4 = 3 + 4
There are two sides to this problem, one on the left side of the equal sign and one on the right side of the equal sign…
The equal sign means that the left side of the equal side is the SAME AMOUNT AS the right side of the equal sign. That is, things on both sides of the equal sign are equal or the same.

38. Conceptual Knowledge
Feedback type x prior knowledge interaction, F(1, 83) = 4.82, p = .03.
Low knowledge: No effect of feedback type, F(1, 83) = 0.51, p = .48
High knowledge: Main effect of feedback type, F(1, 83) = 8.21, p = .005

39. Intervention – Strategy Use
Incorrect
Correct * * *
Note. Differences are between no feedback condition and two feedback conditions combined: * p < .05

40. Strategy Variability * *
Note. Difference is between no feedback condition and two feedback conditions combined: * p < .001