Simulation-based tutoring systems CPI 494/598, April 16 Kurt VanLehn

Classification of step-based tutors User interface Forms e.g., CTAT Canvas e.g., Andes Text e.g., Criterion Dialogue e.g., AutoTutor Simulation e.g., ??? Tutoring on Process During e.g., Cog Tutors After e.g., ??? Both e.g., Andes Product e.g., Criterion Both e.g., AutotTutor

Prototypes (ill-defined classes) of simulation-based tutoring systems Equipment operation & maintenance – E.g., nuclear reactor operation Vehicle driving – E.g., flight simulator Tactical decision making – E.g., defend ship from possible aerial attack Negotiation, leadership, other soft skills – E.g., convince an NGO’s clinic to move Lab/field experimentation – E.g., discover cause of an epidemic

What’s different between simulation- based interface vs. others? Time matters in the sim Heavy reliance on delayed feedback Game-based expectations of sim quality Game-based technology e.g., game engines Just making the sim work – huge! Game-like expense e.g., $1M each! Most research done outside universities – Mostly military

Steamer – Very early example (Hollan, Hutchins, Weitzman 1984) Steam plant occupies 1/3 of space on Navy ships; miles of pipe; Engineer of the watch supervises 16 to 25 people who run it Years of experience required

Operator’s screen: conceptual, not physical, fidelity of controls

One actual operator’s console physical & conceptual fidelity

First, build the simulation Underlying mathematical model – huge! – Built before Steamer Objects for each gauge & control authored – Visual presentation – Hooked up to underlying math model Augment math model: models of faults/failures Math may not be perfect – Especially in abnormal operating conditions or failures Validate the sim – check that standard operating procedures work – Compare real to sim behaviors; many conditions

Sim is not an ITS but… Can be used for training anyway – Drill to memorize standard op procedures – Exercise in handling standard breakdowns Instructor(s) supervises trainees – Decides on exercise – Sets sim into initial state – Watches trainees – Gives immediate feedback occasionally – Gives delayed feedback: After Action Review (AAR)

Problem: After training, still can’t handle non-routine situations Need a “mental model” of the whole steam plan – Helps to diagnose what is causing trouble – Helps to hypothesize & test (mentally) repairs This sim is a black box; need a “glass box” – Shows multiple levels of detail – Should it to be active; can manipulate & watch – Animation; real time?

Top level view of the whole steam plant

One of about 100 subsystem diagrams

Controlling sim via the schematics Can click on schematics to e.g., open a valve Can do “abstract” actions that are hard or impossible to do in real world – Replace boiler – Raise chill-water temp to 100 degrees

Problem: Can’t understand time courses Neg feedback loops in steam plant control e.g., fuel flow to boiler Need to see first and second derivatives as the sim is running – Add new “gauges” to the sim – Some were later added to the real steam plant Need to see graphs

New gauges at bottom; graph at top

Was Steamer ever an ITS? Only for safety issues – Wired to give student immediate negative feedback if certain safety conditions violated – E.g., starting a pump without first opening the input and output values Doesn’t know what student is attempting to do – Can’t offer immediate or delayed feedback – Can’t offer hints – Even if it knew goals, can’t connect goals to actions – Has no representation of knowledge-to-be-learned Except mental model of the plant itself, which is huge

RIDES (Munro Johnson Pizzini Surmon Towne Wogulis, 1997) 15 years later Like Steamer, has authoring system for widgets Has underlying language for implementing Sim Integrates widgets & sim

Has panels with widgets some are controls, some are gauges

Has better tools for authoring instruction

Instructional design procedure Specify objectives; specify scenes Develop sim – Graphical objects – Behavior rules for each – Debug sim Develop lessons – Author procedures using the sim – Add texts – Debug lessons

Object palette: Note how mouse controls rotation

Authoring a procedure Put sim in an initial state; name the procedure Do it End it

Put sim in an initial state

Half done with the procedure

Three modes for instruction Provided automatically Demo mode – Tutor executes procedure – Student controls pace Practice model – Student tries to execute – Gets immediate feedback before sim reacts – If authored hints exists, gets them Test mode – No feedback & no hints – Sim does whatever

Practice mode

Curriculum editor

Is Rides an ITS Authoring tool? Procedural training = CTAT = ITS Drill on identifying parts = tutor, but not so intelligent?