Cognitive Tutors ITS, Sept 30, 2004

Overview Production system models –For LISP, geometry, and algebra 8 principles from ACT theory Gains: 1/3 time to mastery Evolved to view of ITS as non-human tool or “teacher assistant” New development model

ACT theory of skill acquisition Procedural-declarative distinction –Cognitive skill: convert declarative knowledge into production rules Knowledge compilation –Employing declarative knowledge in problem- solving context Strengthening –Knowledge acquires strength with practice

Resulting instruction Initial brief declarative knowledge presentation Guided practice Note: complexity of learning a cognitive skill is inherit in domain, not the process of cognitive skill acquisition

Declarative v. Procedural Side-angle-side theorem Skills: –Placing triangles into correspondence –Determining the included angle –Setting subgoals –Making inferences

Model-tracing approach Model production rules of skill Recognize on-path actions Focus instruction on getting students back on path when off Error feedback and help

8 principles 1.Rep. student competence as production set 2.Communicate goal structure 3.Provide instruction in problem-solving context 4.Promote abstract understanding 5.Minimize working memory load 6.Provide immediate error feedback 7.Adjust grain size with learning 8.Facilitate successive approx. to target skill

Initial results Geometry tutor –+14 points on 80 point test –Project teacher only one with + results LISP tutor –30% faster learning, one std. dev. + Algebra tutor –Hierarchical knowledge shown –Skills transfer not effective

Predicting learning 1.Production practice 2.Within-problem practice effects (strengthening) 3.Acquisition factor 4.Retention factor

Knowledge tracing Bayesian probability a rule was learned Successfully predicted post-test results Applies for measuring mastery

Locus of feedback control Immediate Error-flagging Demand None First three better than last

Feedback content Error flagging (Short) explanatory feedback –Fewer errors per production –More likely to correct error on first attempt No long-term effect except time & perception of tutor Carefully crafted => long-term effect

Impractical? Didn’t address curriculum What happens after tutors? Inflexible in application How to support deployment?

New development model 1.Interface construction 2.Curriculum specification 3.Cognitive modeling 4.Design of instruction 5.Classroom deployment