Does Learning from Examples Improve Tutored Problem Solving? Ron Salden & Vincent Aleven (CMU) Alexander Renkl, Rolf Schwonke & Carmen Krieg (Freiburg University) Jörg Wittwer (Kiel University) 9/19/2018 Pittsburgh Science of Learning Center

Pittsburgh Science of Learning Center Theory Combination of two successful research traditions Cognitive Tutors Worked Out Examples 9/19/2018 Pittsburgh Science of Learning Center

Scientific and applied questions Scientific question Under what conditions can Tutored learning from examples be as effective or more effective than standard Tutored learning? Applied question Can we speed up learning while retaining the learning outcomes? 9/19/2018 Pittsburgh Science of Learning Center

Placing Study within Robust Learning Framework Primary research cluster Interactive communication, coordinative learning, refinement & fluency Independent variable(s) IC: Reflective dialog, scripting collaboration, peer tutoring, peer observation of tutoring, varying contribution of student and system… CL: self-explanation, integrate conceptual & procedural, fading of worked out steps, multi-modal… R&F: Feature focusing, example comparison, cognitive mastery, optimal spacing, … Dependent variables used Normal post-test, long term retention, transfer measuring procedural and conceptual knowledge, future learning 9/19/2018 Pittsburgh Science of Learning Center

Control: standard problem solving with self explanation Contiguous interface: student works in diagram, entries appear in table after submitting answers and rules to the diagram Clicking on the question mark opens a work field 9/19/2018 Pittsburgh Science of Learning Center

Example condition: Worked out answer steps Formula and answer are worked out but student still has to find out what rule is being applied on this step Givens in problem statement are completely worked out to avoid extraneous load 9/19/2018 Pittsburgh Science of Learning Center

The sequencing of problems and fading of worked-out steps Examples Problem solving Principles Problems P1 P2 P3 W S P4 P5 P6 P7 Note. W stands for worked out examples and S for problem solving 9/19/2018 Pittsburgh Science of Learning Center

Pittsburgh Science of Learning Center Dependent variables Transfer test measuring: procedural knowledge - problem solving and reasoning conceptual knowledge - drawing problems - Multiple choice questions concerning geometric principles Note: hard to do long term retention test in lab study, we will include this in upcoming in vivo studies 9/19/2018 Pittsburgh Science of Learning Center

Pittsburgh Science of Learning Center Main results lab study 1 Example condition spent less time (t(48)= -3.11, p < .001, r = .41) to obtain the same learning outcomes as the control condition Using time on Tutor: Higher learning efficiency for the example condition on transfer test measuring conceptual knowledge (t(48)= 1.73, p < .05, r = .24) and procedural knowledge (t(48)= 1.82, p < .05, r = .25) 9/19/2018 Pittsburgh Science of Learning Center

Pittsburgh Science of Learning Center Main results lab study 2  Possible confounder Study 1: German participants inexperienced with Cognitive Tutor, thus more detailed instructions were given in Study 2 Example condition showed higher gain on transfer test for conceptual understanding (sense-making) than control condition (t(28)= 1.85, p < .05, r = .33) 9/19/2018 Pittsburgh Science of Learning Center

Main results lab study 2 continued Example condition spent less time (t(28)= -3.14, p < .001, r = .51) to complete the curriculum than control condition Using time on Tutor: Higher learning efficiency for example condition on transfer test measuring conceptual knowledge (t(28)= 3.48, p < .001, r = .55) 9/19/2018 Pittsburgh Science of Learning Center

Interpretation of Results within Robust Learning Framework Worked out examples lead to same level of foundational skills in less time Furthermore, study 2 shows that fading worked out examples can improve sense-making and consequently lead to better robust learning 9/19/2018 Pittsburgh Science of Learning Center

Ongoing + upcoming studies Results LearnLab studies with Tutor different than results in literature with no Tutor Examples No examples Tutor No Tutor Reported lab studies, other LearnLab studies by McLaren et al, Ringenberg et al. Sweller et al., Renkl & Atkinson Anderson et al. Possibly the standard Cognitive Tutor is “too supportive” which cancels out potential effects of Example enriched Tutor Conducting lab experiment with Cognitive Tutor resembling problem solving format used in literature (both Tutor conditions completed November 30) 9/19/2018 Pittsburgh Science of Learning Center

Upcoming studies continued LearnLab experiment with adaptive fading of worked out steps -fading occurs based on student’s explanations -delayed post test incl. not enough info items to test long term retention Start date: January 11, 2007 9/19/2018 Pittsburgh Science of Learning Center