1. Simulation-based tutoring systems CPI 494/598, April 16 Kurt VanLehn

2. Classification of step-based tutors User interface Forms e.g., CTAT Canvas e.g., Andes Text e.g., Criterion Dialogue e.g., AutoTutor Simulation e.g., ??? Tutoring on Process During e.g., Cog Tutors After e.g., ??? Both e.g., Andes Product e.g., Criterion Both e.g., AutotTutor

3. Prototypes (ill-defined classes) of simulation-based tutoring systems Equipment operation & maintenance – E.g., nuclear reactor operation Vehicle driving – E.g., flight simulator Tactical decision making – E.g., defend ship from possible aerial attack Negotiation, leadership, other soft skills – E.g., convince an NGO’s clinic to move Lab/field experimentation – E.g., discover cause of an epidemic

4. What’s different between simulation- based interface vs. others? Time matters in the sim Heavy reliance on delayed feedback Game-based expectations of sim quality Game-based technology e.g., game engines Just making the sim work – huge! Game-like expense e.g., $1M each! Most research done outside universities – Mostly military

5. Steamer – Very early example (Hollan, Hutchins, Weitzman 1984) Steam plant occupies 1/3 of space on Navy ships; miles of pipe; Engineer of the watch supervises 16 to 25 people who run it Years of experience required

6. Operator’s screen: conceptual, not physical, fidelity of controls

7. One actual operator’s console physical & conceptual fidelity

8. First, build the simulation Underlying mathematical model – huge! – Built before Steamer Objects for each gauge & control authored – Visual presentation – Hooked up to underlying math model Augment math model: models of faults/failures Math may not be perfect – Especially in abnormal operating conditions or failures Validate the sim – check that standard operating procedures work – Compare real to sim behaviors; many conditions

9. Sim is not an ITS but… Can be used for training anyway – Drill to memorize standard op procedures – Exercise in handling standard breakdowns Instructor(s) supervises trainees – Decides on exercise – Sets sim into initial state – Watches trainees – Gives immediate feedback occasionally – Gives delayed feedback: After Action Review (AAR)

10. Problem: After training, still can’t handle non-routine situations Need a “mental model” of the whole steam plan – Helps to diagnose what is causing trouble – Helps to hypothesize & test (mentally) repairs This sim is a black box; need a “glass box” – Shows multiple levels of detail – Should it to be active; can manipulate & watch – Animation; real time?

11. Top level view of the whole steam plant

12. One of about 100 subsystem diagrams

13. Controlling sim via the schematics Can click on schematics to e.g., open a valve Can do “abstract” actions that are hard or impossible to do in real world – Replace boiler – Raise chill-water temp to 100 degrees

14. Problem: Can’t understand time courses Neg feedback loops in steam plant control e.g., fuel flow to boiler Need to see first and second derivatives as the sim is running – Add new “gauges” to the sim – Some were later added to the real steam plant Need to see graphs

15. New gauges at bottom; graph at top

16. Was Steamer ever an ITS? Only for safety issues – Wired to give student immediate negative feedback if certain safety conditions violated – E.g., starting a pump without first opening the input and output values Doesn’t know what student is attempting to do – Can’t offer immediate or delayed feedback – Can’t offer hints – Even if it knew goals, can’t connect goals to actions – Has no representation of knowledge-to-be-learned Except mental model of the plant itself, which is huge

17. RIDES (Munro Johnson Pizzini Surmon Towne Wogulis, 1997) 15 years later Like Steamer, has authoring system for widgets Has underlying language for implementing Sim Integrates widgets & sim

18. Has panels with widgets some are controls, some are gauges

19. Has better tools for authoring instruction

20. Instructional design procedure Specify objectives; specify scenes Develop sim – Graphical objects – Behavior rules for each – Debug sim Develop lessons – Author procedures using the sim – Add texts – Debug lessons

21. Object palette: Note how mouse controls rotation

22. Authoring a procedure Put sim in an initial state; name the procedure Do it End it

23. Put sim in an initial state

24. Half done with the procedure

25. Three modes for instruction Provided automatically Demo mode – Tutor executes procedure – Student controls pace Practice model – Student tries to execute – Gets immediate feedback before sim reacts – If authored hints exists, gets them Test mode – No feedback & no hints – Sim does whatever

26. Practice mode

27. Curriculum editor

28. Is Rides an ITS Authoring tool? Procedural training = CTAT = ITS Drill on identifying parts = tutor, but not so intelligent?