Improving a Mathematical Intelligent Tutoring System Experiments & Equation Solver Improvements July 27th, 2012 Jennifer Ferris-Glick & Hee Seung Lee

Mathematical Intelligent Tutoring Systems How do we improve out dated textbook methods of learning and expand on intelligent tutoring systems?

Objective of Study To investigate the role of verbal explanation - When does it help learning? To Explore the degree to which we can teach without verbal instruction - Is verbal direction necessary? - How can we help students discover/learn underlying problem structure and induce correct rules for problem solving without explicit verbal directions?

* + * 4 5 Linearize task : solving algebra problems using data-flow diagrams : allows to study algebra learning anew in college population

Findings from Linearize study Showing intermediate cognitive steps Preventing superficial analogy from an example to problem Building appropriate problem representations Avoiding excessive floundering Giving a chance to reflect on solved problems

Showing Intermediate Cognitive Steps Students still have to fill in gaps between steps when students study a worked example Students learned better when intermediate cognitive steps were clearly revealed than when students had to determine hidden steps.  Force students to study step-by-step solutions

Preventing Superficial Analogy Given a highly similar example, students tend to draw a superficial analogy from the example to problem. –When superficial solution is available, students fail to abstract deep problem structure. –When examples look too dissimilar from a target problem, students sometimes simply ignore or do not spend enough time to study provided examples.  Examples should be dissimilar, and structurally similar to a target problem.

Building Appropriate Problem Representations Revealing hidden structure helped transfer - Making connections to Algebra activated prior knowledge and helped understand the problem structure Highlighting relevant features (e.g., color coding)  Highlight features to direct student’s attention and reveal hidden problem structure

Restricting Floundering & Supporting Reflection Activity When students show too much floundering, they may not remember solution paths. In order to reduce floundering: –Immediate feedback on wrong attempt –Providing hints when students show floundering –Auto-solving with a time limit Giving students an opportunity to make sense of already solved problems  Allow only a certain number of off-track activities. When a certain step was solved automatically, show corresponding verbal explanations for a while.

General Improvements to Carnegie Learning Cognitive Tutor (all conditions) Showing Intermediate Cognitive Steps –Force step-by-step guided example before solving Maybe show previously solved equations –Hint improvements Some changes but not full review Preventing superficial analogy –Present side-by-side example –Similarity of example Randomly selected from a pool of problems in each section (not too similar, not too dissimilar) Appropriate Problem Representation –Aligning equations on the equals sign –Coloring variable sides of equation Restrict Floundering –Reduce off-path steps either allow 0 or 1 off-path step Reflection Time –Provide side-by-side already worked out problems

Target Material One-step linear equations Solving with addition and subtraction (no type in) –o w + 6 = -1 Solving with addition and subtraction (type in) –o -4 = 1 + z Solving with multiplication and division (no type in) –o 2x = -1 Solving with multiplication and division (type in) –o 8w = 5

Target Material Two-step linear equations Solving One-step equations (Type in) –o Mix of types from unit 5 Solving with multiplication (no type in) –o -2 = 5x + 10 Solving with multiplication (type in) –o -2 = 5x + 10 Solving with division (no type in) –o x/6 + 7 = 8 Solving with division (type in) –o x/6 + 7 = 8 Solving with a variable in the denominator (no type in) –o 10 = -2/x Solving with a variable in the denominator (type in) –o 2/z = -4

Target Material Linear equations with similar terms Solving two-step equations (type in) –o Mix of types from unit 6 Combining like variable terms and a constant with integers (no type in) –o 2 – 9x + 6x = 2 Combining like variable terms and a constant with integers (type in) –o -6x x = -6 Combining like variable terms with decimals (no type in) –o 9 = 9.7x – 2x Combining like variable terms with decimals (type in) –o -2.1z – 8z = 2 Combining like variable terms and a constant with decimals (no type in) –o x + 3.7x = 4.6 Combining like variable terms and a constant with decimals (type in) –o -6.6z – 2.8 – 6.7z = 2.8

Target Material Linear equations and the distributive property Solving Linear Equations (Type In) –o 7 + 2x = -3 Using Multiplication and Integers (No Type In) –o 4(z - 3) = 3 Using Multiplication and Integers (Type In) –o -6(z – 6) = 10 Using Multiplication and Decimals (No Type In) –o -4.5 = -2.6(z + 2.9) Using Multiplication and Decimals (Type In) –o 8.3(z – 5.9) = 6.8 Using Multiplication and Large Decimals (No Type In) –o = (z ) Using Multiplication and Large Decimals (Type In) –o (w ) = Using Division (No Type In)

Experimental Conditions Kanawha County School District 3 Conditions divided by class (4 classes for each condition): Discovery, Instruction, Standard Presented in line with curriculum.

Experimental Conditions DiscoveryDirect InstructionCurrent tutor Hints “How” hints only (on general interface) “How” and “why” hints available “How” and “why”, no changes to 2011 std release Just in Time (JITs) Messages NoneNormal Step by Step Guided Problem “How” hints only Force step-by- step in the beginning of new section “How” and “why” Force step-by- step in the beginning of new section Optional Side by Side Worked Examples Side-by-side worked example with history tags None

Example Types Step-by-Step Guided Problem One per section – selected based on sections “new” material Students simply follow directions in a step-by- step manner to solve the problem Side-by-Side Worked Examples Already solved problems appear next to a target problem

w + 6= Instruction condition Provide “why” and “how” hints Directly tell students what to do To isolate w, subtract 6 from both sides. Select “subtract both sides” and enter 6. Step-by-Step Guided Problem Instruction

w + 6= Discovery condition Provide “how” hints Provide instruction on general interface Select an item from Transform menu and enter a number. Step-by-Step Guided Problem Discovery

w w + 6 – 6 ==== -6 subtract 6 from both sides -1 – 6 Both conditions Provide guided problem Align equation on the equal sign Coloring variable side of the equation Highlighting changes between steps Color-Coding

w w ==== +6 add 6 from both sides Wrong Attempts This is not the best way to solve the problem. Click “undo” and ask for hint if you need help. Both conditions Reduce off-path steps using “undo” button.

Both conditions Provide already solved problems (screen-shot) Selected from the pool of problems in the same section Side by Side Worked Example Instruction & Discovery w w w =.===.== +6 add 6 from both sides combine like terms 5 ExampleProblem 8=x - 2

Post-Test 2 types of test questions: (paper-based, within subject) -Plain equations -Equations with solver interface scaffolding