1. Sterling Somers : sterling@sterlingsomers.comsterling@sterlingsomers.com Dr. Robert West : robert_west@carleton.carobert_west@carleton.ca Both @ Institute of Cognitive Science Carleton University Ottawa, ON, Canada Macro Cognition : Using SGOMS to Pilot a Flight Simulator

2. SGOMS is an extension of GOMS to deal with interruptions, teamwork, and re-planning Adds a control structure above unit tasks – planning unit SGOMS in ACT-R A particular way to program ACT-R to do complex tasks Adds more buffers to ACT-R Planning unit buffer Unit task buffer Method buffer These can be thought of collectively as the goal module To test SGOMS we need to show that it works across tasks SGOMS

3. Different SGOMS models in the lab

4. Why Aviation? 1.Real World 2.Dynamic 3.Both Planned and Reactive Behaviour 4.Replanning 5.Expertise 6.Continuous / Feedback

5. Interface Simulator FlightGear Free Open Source (hack simulation loop?) Custom Python API Property tree via UDP/HTTP/Telnet http/telnet simultaneously (read/write) State / Control of aircraft Problem Real time VS ACT-R time (for now…) Simple Vision Python bindings to OpenCV Very efficient Redundancy check possible

6. Vision Top Down: SOS Vision Vision Module: When not busy Request from environment Motion Detection Python bindings OpenCV Screen shot VS centre of screen comparison / frame Motion Detection Screen Shot at frame 1 Screen Shot at frame 2 (arrow indicates direction of movement)

7. Vision : Dynamic Salience / Context Dependence

8. Operator Pool Start EnginePress Both BrakesRecall HeadingMaintain Heading To Full ThrottleSet TrimLook HeadingRudder Left/Right Motor Methods To Full Throttle: Continue increasing throttle until lever stops moving Planning Unit: Prepare for take off DM Unit Task: Get Started Press Both Brakes + Start Engine + Cue: Motor module: not busy Auditory Module: no engine noise Unit Task: Do settings Set Trim + Planning Unit: To take off speed DM Unit Task: Get to speed Release Brakes + To Full Throttle + Unit Task: Watch Speed Read Air Speed + +

9. Planning Unit: To take off speed DM Unit Task: Get to speed Release Brakes + To Full Throttle + Unit Task: Watch Speed Read Air Speed + Vision Module Look at [target] Look out window Top-down request Opportunistic, when no top-down request See movement + Reasonable sized move Interruption Maintain Heading Rudder, etc. + Clear Buffer + (resulting in resume)

10. Interruption Interruptions and re-planning take place on different levels Case 1 – Bottom up scanning – Small – Expected – Context remains the same (still taking off) – Handle it within the unit task Case 2 – Bottom up scanning – Large – Unexpected – Context is changed (abort take off) – New unit task needed SGOMS Hypothesis – there is a set processing structure used across tasks

11. Thank you. Sterling Somers : sterling@sterlingsomers.comsterling@sterlingsomers.com Dr. Robert West : robert_west@carleton.carobert_west@carleton.ca Both @ Institute of Cognitive Science Carleton University Ottawa, ON, Canada