CLT Conference Heerlen Ron Salden, Ken Koedinger, Vincent Aleven, & Bruce McLaren (Carnegie Mellon University, Pittsburgh, USA) Does Cognitive Load Theory account for the beneficial effects of worked examples in tutored problem solving?

Worked examples and tutored problem solving Worked examples mostly investigated in untutored problem solving environments Cognitive Tutor is Intelligent Tutoring System  provides step-by-step guidance during complex problem solving practice

Worked examples and tutored problem solving Cognitive Tutor provides a tougher control condition of tutored problem solving – It is this tutored part that in our view reduces extraneous load – And, sometimes, increases germane load

Longstanding tradition in improving students’ learning Grounded in cognitive theory (ACT-R, Anderson & LeBière, 1998) Methods for reducing WM load – Scaffolding or prompting of sub-goals – Step by step feedback & hints (i.e., guided learning) Use cognitive model of student thinking Many full-year classroom evaluations show improved math competence (Koedinger & Aleven, 2007) Cognitive Tutors

Elements that reduce extraneous cognitive load.

7/12/2015Pittsburgh Science of Learning Center6 1a. Corrective feedback Standard Cognitive Tutor: control condition

7/12/2015Pittsburgh Science of Learning Center7 Standard Cognitive Tutor: control condition 1b. Implicit positive feedback

2. Stepwise hints: last hint level is bottom-out hint  problem fading into example Standard Cognitive Tutor: control condition

3. Problem sub-goals are given Standard Cognitive Tutor: control condition

7/12/2015Pittsburgh Science of Learning Center10 Standard Cognitive Tutor: control condition 4. Student’s self explanation This feature is not so much about reducing extraneous load but about increasing germane load

Studies Shih et al (Geometry) McLaren et al (Chemistry) Salden et al (Geometry) Not addressed in this talk Anthony et al (Algebra) obtained similar results as the other three studies  Does Cognitive Load Theory explain beneficial effects of examples in tutored problem solving?

Shih et al Study: re-analyzing prior study (Aleven & Koedinger, 2002) – Logged response data on bottom-out hint usage One type of “Gaming the system” behavior – Can be hint abuse due to students skipping abstract hints to obtain the concrete answer – Can also be helpful when bottom-out hints act as worked examples

Shih et al Developed a model to distinguish between good student use of bottom-out hints from bad student use of bottom-out hints Two key elements of model are time spent on: – Reflecting about prior step (after bottom-out hint) – Thinking about next step (prior to next action) Subtraction method to isolate reflection (self- explanation) time – Use other data, when bottom-hints are not requested, to estimate next step time

Shih et al results High correlation of time spent reflecting on bottom-out hint with learning (pre-to-post gain) Spending time on hints is beneficial to learning for all students Difference between students’ hint usage: – Good usage = spending more time on bottom-out hint – Bad usage = spending less time on bottom-out hint Thus students who study bottom-out hint as worked example obtain higher learning gains

McLaren et al Conducted three studies comparing – Tutored Alone vs. Worked Examples + Tutored Examples are alternated with isomorphic problems

Stoichiometry Tutor: control condition

Worked Example condition Students watch video of a worked example plus do prompted self-explanations following the example:

McLaren et al results No differences on posttest performance BUT, students in Examples condition did learn more efficiently, using 21% less time to finish same problem set

Salden et al Conducted lab and classroom study comparing: – Tutored problem solving – Fixed example fading – Adaptive example fading Adaptive fading based on students’ self- explanations of the example steps – Students who self explain well receive fewer examples than students who self explain poorly

7/12/2015Pittsburgh Science of Learning Center20 Standard Cognitive Tutor: control condition

Worked out value step with calculation shown by Tutor Example-enhanced Tutor: experimental condition Student still has to self explain the worked out value step!

Salden et al results Lab study: – Adaptive fading condition needed fewer examples than fixed fading condition – Adaptive fading > both fixed conditions on posttest and delayed posttest Classroom study: – Adaptive fading condition needed fewer examples on several theorems than fixed fading condition – Adaptive fading > problem solving on delayed posttest

Summary of results Shih et al: Students can effectively use bottom-out hints as worked examples and achieve higher learning gains McLaren et al: Students working with examples can complete learning phase needing 21% less time while obtaining the same learning outcomes Salden et al: Students learning from adaptively faded examples obtained higher immediate and delayed posttest performance Fourth study by Anthony et al (using Algebra Tutor): Students who learned with examples attained better long term retention  Also measured mental effort: examples = tutored problem solving

Does CLT explain these beneficial effects of worked examples in tutored problem solving? Cognitive Tutor is a harder control condition than untutored environments – Students can effectively use bottom-out hints as worked examples The tutoring seems to reduce possible extraneous cognitive load – Anthony study even showed no difference in mental effort between control and experimental condition Stepwise feedback & hints, self-explanation prompts geared to increase germane cognitive load

Does CLT explain these beneficial effects of worked examples in tutored problem solving? Possible explanations Without the information (guidance) provided by examples, students waste time tackling new skills during problem solving – McLaren study: examples lead to same learning gains but needed 21% less time – Two Freiburg lab studies: examples lead to same learning gains needing roughly 17.5% and 25% less time Motivation – Goal of understanding v. performing (Shih et al) – Frustration after unsuccessful solution attempt Where is the cognitive load?

Questions?