1. Time-constrained reasoning Learn to be a split-second expert Niels Taatgen and John Anderson Carnegie Mellon University Funded by grants from ONR (ACT-R), NWO (Set) and NASA (CMU-ASP)

2. Overview Time-constrained reasoning Time-constrained reasoning ACT-R essentials ACT-R essentials Set (a game) Set (a game) CMU-ASP (complex dynamic task) CMU-ASP (complex dynamic task)

3. Goals of the ACT-R project Explain as much as possible of cognition using a single theory Explain as much as possible of cognition using a single theory The theory has to be explicated in a cognitive architecture, a simulation environment that mimics actual human behavior The theory has to be explicated in a cognitive architecture, a simulation environment that mimics actual human behavior Single mechanisms should explain many phenomena Single mechanisms should explain many phenomena

4. Skill Acquisition through Production Compilation People become faster at performing a certain task through learning People become faster at performing a certain task through learning The general theory is that they transform declarative into procedural knowledge The general theory is that they transform declarative into procedural knowledge Speed-up is explained by the fact that declarative knowledge no longer has to be retrieved Speed-up is explained by the fact that declarative knowledge no longer has to be retrieved A second aspect of becoming faster is that some things that had to be done serially can now be done in parallel leading to qualitative changes in behavior A second aspect of becoming faster is that some things that had to be done serially can now be done in parallel leading to qualitative changes in behavior

5. Time-constrained reasoning Initial knowledge Initial knowledge Speed-up by production compilation Speed-up by production compilation Qualitative improvements in performance Qualitative improvements in performance Embedded learning Embedded learning Being fast means: explaining why people can learn to be very fast at certain complex tasks (so nothing fast computers solve) Being fast means: explaining why people can learn to be very fast at certain complex tasks (so nothing fast computers solve)

6. Goals of the ACT-R project Model Human behavior in order to: Better understand human reasoning Better understand human reasoning Use models of human reasoning in: Use models of human reasoning in: HCI HCI Intelligent agents Intelligent agents Tutoring Tutoring The simulations predictions are as fine-grained as possible: reaction times reaction times errors errors choices choices learning curves learning curves eye movements eye movements fMRI data fMRI data

7. ACT-Rs foundations Rational Analysis: Knowledge is treated as having a potential benefit for the agent (so truth is not fundamental) Rational Analysis: Knowledge is treated as having a potential benefit for the agent (so truth is not fundamental) Procedural/Declarative distinction Procedural/Declarative distinction Hybrid: both symbolic and subsymbolic Hybrid: both symbolic and subsymbolic Strong focus on learning Strong focus on learning Strong focus on interaction with the environment Strong focus on interaction with the environment

8. Summary of ACT-Rs memory Declarative memory Declarative memory contains facts and past goals (called chunks) contains facts and past goals (called chunks) activation determines which chunk is selected activation determines which chunk is selected chunks that are retrieved or recreated often receive high activations chunks that are retrieved or recreated often receive high activations Procedural memory Procedural memory contains rules contains rules utility determines which rule is selected utility determines which rule is selected High utility = high probability of success and low costs High utility = high probability of success and low costs

9. Overview ACT-R in a Diagram Matching (Striatum) Selection (Pallidum) Execution (Thalamus) Productions (Basal Ganglia) Retrieval Buffer (VLPFC) Goal Buffer (DLPFC) Manual Buffer (Motor) Visual Buffer (Parietal) Declarative Module (Temporal/Hippocampus) Intentional module (not identified) Visual Module (Occipital/Parietal) Manual Module (Motor/Cerebellum) External World ACT-R Cycle: MatchingSelectionExecution

10. Overview ACT-R in a Diagram Matching (Striatum) Selection (Pallidum) Execution (Thalamus) Productions (Basal Ganglia) Retrieval Buffer (VLPFC) Goal Buffer (DLPFC) Manual Buffer (Motor) Visual Buffer (Parietal) Declarative Module (Temporal/Hippocampus) Intentional module (not identified) Visual Module (Occipital/Parietal) Manual Module (Motor/Cerebellum) External World ACT-R Cycle: Matching: Production rules that match the current contents of the buffers are determined

11. Overview ACT-R in a Diagram Matching (Striatum) Selection (Pallidum) Execution (Thalamus) Productions (Basal Ganglia) Retrieval Buffer (VLPFC) Goal Buffer (DLPFC) Manual Buffer (Motor) Visual Buffer (Parietal) Declarative Module (Temporal/Hippocampus) Intentional module (not identified) Visual Module (Occipital/Parietal) Manual Module (Motor/Cerebellum) External World ACT-R Cycle: Selection: Select the production rule with the highest Utility

12. Overview ACT-R in a Diagram Matching (Striatum) Selection (Pallidum) Execution (Thalamus) Productions (Basal Ganglia) Retrieval Buffer (VLPFC) Goal Buffer (DLPFC) Manual Buffer (Motor) Visual Buffer (Parietal) Declarative Module (Temporal/Hippocampus) Intentional module (not identified) Visual Module (Occipital/Parietal) Manual Module (Motor/Cerebellum) External World ACT-R Cycle: Execution: The selected production rule modifies contents of buffers Modules operate asynchronously from central cognition

13. Production Compilation: Learning new production rules Declarative Procedural Rule 1 Rule 2 Fact from Declarative memory Combine two existing rules that are used in sequence into a new rule, while substituting a fact that is retrieved from memory Solidify recurring reasoning patterns Rule 1 Rule 2 Fact from Declarative memory Rule1 & Rule2

14. Set Game Game Predictions Predictions Experiment Experiment Model Model Application Application Evaluation Evaluation With Marcia van Oploo, Jos Braaksma and Jelle Niemantsverdriet

15. Computer Games Computer chess: design a program that plays chess as good as possible Computer chess: design a program that plays chess as good as possible But chess players complain that computers play boring chess But chess players complain that computers play boring chess Different goal: opponents that play as humanly as possible. Different goal: opponents that play as humanly as possible.

16. The Game of Set! Game consists of 81 cards Game consists of 81 cards Each card has four attributes: color, shape, filling and number Each card has four attributes: color, shape, filling and number

17. Goal of the Game Twelve cards are put on the table Twelve cards are put on the table Find a Set: three cards in which for each attribute, the attribute values for each of the cards are all different, or all the same Find a Set: three cards in which for each attribute, the attribute values for each of the cards are all different, or all the same

18. Goal of the Game Not a Set!

19. Some Sets are more difficult then others One attribute different Two attributes different Three attributes different Four attributes different

20. 123 4 5 67 8 910 1112

21. Set! as a game to play against the computer For a computer, the game is trivial (as opposed to chess, etc.) For a computer, the game is trivial (as opposed to chess, etc.) The challenge is to program an opponent that acts as a human player The challenge is to program an opponent that acts as a human player So the computer opponent has to be fast at sets that people are fast at, and slow at sets that people are slow at So the computer opponent has to be fast at sets that people are fast at, and slow at sets that people are slow at

22. The Predictions 1. The easy sets will be found faster than the hard sets. 2. Experts on the game will mainly excel in finding the hard sets, and will be approximately equally good as beginners on the easy sets

23. The Experiment 8 subjects, 4 beginners, 4 experts 8 subjects, 4 beginners, 4 experts 20 set-problems (12 cards, find the set as fast as possible) 20 set-problems (12 cards, find the set as fast as possible) 5 problems of each of the four levels of difficulty 5 problems of each of the four levels of difficulty

24. Experimental results Experiment confirms both hypotheses: 1. Difficult problems take longer 2. Beginners and Experts are equally good at easy problems, but Experts excel on hard problems

25. The model Why do hard sets take longer? Why do hard sets take longer? Checking for unequal attributes takes longer Why are experts better at hard problems? Why are experts better at hard problems? They are better at multi- tasking, which pays off in hard problems

26. How the model works First, pick a random card and stick it in the goal buffer First, pick a random card and stick it in the goal buffer Goal 3 Visual 3

27. How the model works Second, search for a card of the same color. If this fails, search for an arbitrary different card Second, search for a card of the same color. If this fails, search for an arbitrary different card We dont put this card in the goal, but leave it in the visual buffer We dont put this card in the goal, but leave it in the visual buffer Goal 3 Visual 2

28. How the model works Now the model is going to do two things in parallel: Now the model is going to do two things in parallel: Check in declarative memory whether or not we tried this combination of two cards before Check in declarative memory whether or not we tried this combination of two cards before Make a prediction what the third card has to look like Make a prediction what the third card has to look like Goal 3 Visual 2 Retrieval buffer: Have we tried these two cards before? Declarative Memory busy retrieving the fact

29. How the model works Predicting the third card Predicting the third card For each attribute, we determine what it has to be like in the third card, and put this back into the goal For each attribute, we determine what it has to be like in the third card, and put this back into the goal When the attribute for goal and visual are equal, this attribute is also the desired attribute for the new card When the attribute for goal and visual are equal, this attribute is also the desired attribute for the new card Goal 3 Visual 2 Retrieval buffer: Have we tried these two cards before? Declarative Memory busy retrieving the fact

30. How the model works Predicting the third card Predicting the third card When the attributes are different, we have to determine the third value When the attributes are different, we have to determine the third value Goal 3 Visual 2 Retrieval buffer: Have we tried these two cards before? 1 1 Declarative Memory busy retrieving the fact

31. How the model works After predicting the third card After predicting the third card Now that the third card has been predicted, the model tries to find it on the screen. Now that the third card has been predicted, the model tries to find it on the screen. If this fails, it starts all over again If this fails, it starts all over again If this succeeds, it announces it has found a Set! If this succeeds, it announces it has found a Set! Goal 31

32. Beginners vs. Experts Beginner: Beginner: IF value in the goal is val1 and the value in the visual buffer is val2 THEN send a retrieval request for a value that is different from val1 and val2 IF the retrieval buffer contains val3, different from val1 and val2 THEN put val3 in the goal Expert: Expert: IF value in the goal is red and the value in the visual buffer is blue THEN put green in the goal

33. How the model works Beginners: Beginners: At the moment that (in this example) the filling has to be determined, a declarative retrieval is needed At the moment that (in this example) the filling has to be determined, a declarative retrieval is needed This is however impossible, because declarative memory is still engaged in another retrieval! This is however impossible, because declarative memory is still engaged in another retrieval! So the beginner has to wait until the first declarative retrieval is done So the beginner has to wait until the first declarative retrieval is done Experts Experts Have proceduralized the retrieval for the third attribute value Have proceduralized the retrieval for the third attribute value Goal 3 Visual 2 Retrieval buffer: Have we tried these two cards before? Retrieval buffer: What is the third value? Declarative Memory busy retrieving the fact

34. Novice Expert Nothing going on here except wait for retrieval Third card is predicted Third card is sought

35. Results of the Model

36. Conclusion (intermediate) Experts are not just faster than beginners Experts are not just faster than beginners They can do certain reasoning steps effortlessly which are effortful to novices They can do certain reasoning steps effortlessly which are effortful to novices In this task the beginners can still do it, but one can image tasks where you have to be a (partial) expert to be able to do it at all In this task the beginners can still do it, but one can image tasks where you have to be a (partial) expert to be able to do it at all

37. The Application

38. CMU-ASP task Subjects have to classify planes (tracks) on a radar-screen Subjects have to classify planes (tracks) on a radar-screen They have to do three things to classify a track: They have to do three things to classify a track: Select one by clicking on it Select one by clicking on it Use one of two classification methods, each of which sometimes successful and sometimes not Use one of two classification methods, each of which sometimes successful and sometimes not Enter the classification into the system Enter the classification into the system

39. Clicking on a track selects it One classification method is to look at altitude and speed Another is to ask for a radar signature by pressing some keys and waiting for the information to come up Finally, a series of keypresses is used to enter the information

40. General modeling approach Represent instructions as declarative knowledge Represent instructions as declarative knowledge Have task-general production rules that interpret these instructions Have task-general production rules that interpret these instructions Production compilation produces task- specific rules Production compilation produces task- specific rules

41. Representation of instructions (Anderson) Hand to F-keys Identify Tracks Look for a track Hook track Id itRepeat Do RadarClassify Select EWS Select Query Encode Look at alt & speed

42. This model misses many interesting phenomena Moving hands from the mouse to the keyboard and vice versa ahead of time Moving hands from the mouse to the keyboard and vice versa ahead of time Deciding which classification strategy to use first Deciding which classification strategy to use first Comparing tracks before selecting one without using to much time Comparing tracks before selecting one without using to much time To summarize: optimize parallelization of behavior To summarize: optimize parallelization of behavior

43. From a strong to weak Hierarchy Select Track Look for Track Hook Track Use Radar Id Find approp. key Read Air ID Recall classifi- cation Enter Classifi- cation Find approp. key Check alt/speed Look for alt Check range Look for speed Hand to F-keys Identify Tracks Look for a track Hook track Id itRepeat Do RadarClassify Select EWS Select Query Encode Look at alt & speed

44. Instructions involve multiple steps Look at a track 1. Find a visual- location of type track 2. Move attention to it 3. Store the location in the goal Hook a track 1. Move the hand to the mouse 2. Move the mouse to the location of the track 3. Click the mouse Carrying out these steps in order is inefficient!

45. Example productions (p retrieve-next-clause =goal> =goal> isa task isa task rule =id rule =id step done step done-retrieval>==> +retrieval> +retrieval> isa clause isa clause rule =id rule =id) IF the goal is a task THEN retrieve the next instruction Note: spreading activation will insure the right instruction is retrieved

46. Example productions (p look-for-visual-track =goal> =goal> isa task isa task step done step done =retrieval> =retrieval> isa clause isa clause relation look-for relation look-for arg2 =type arg2 =type =visual-location> =visual-location> isa visual-location isa visual-location kind =type kind =type -visual> -visual>==> +visual> +visual> isa visual-object isa visual-object screen-pos =visual-location screen-pos =visual-location -retrieval>) -retrieval>) IF the goal is a task AND an instruction is retrieved to look for something of a certain type AND we have found a location on the screen with something of that type THEN move the eyes to the location and attend it

47. Example productions (p hook-new-mouse-not-at-destination =goal> =goal> isa task isa task var1 =object var1 =object step done step done =retrieval> =retrieval> isa clause isa clause relation hook-new relation hook-new arg1 var1 arg1 var1 =manual-state> =manual-state> isa module-state isa module-state modality free modality free !eval! (not (cursor-at =object)) ==> -retrieval> -retrieval> +manual> +manual> isa move-cursor isa move-cursor object =object) object =object) IF the goal is a task AND an instruction has been retrieved that specifies that an object has to be clicked AND the motor module is available AND the cursor is not on the object THEN move the mouse to the object

48. Additional rule If the visual system finds another track while one is already in the goal, then it compares the new track to the old track and retains the best If the visual system finds another track while one is already in the goal, then it compares the new track to the old track and retains the best

49. Novice Move hand to mouse Move mouse to trackClick mouse Attend a track

50. Learned rules (p Production1013 (p Production1013 =goal> =goal> isa TASK isa TASK rule Find-Track rule Find-Track step Done step Done =manual-state> =manual-state> isa MODULE-STATE isa MODULE-STATE modality Free modality Free !eval! (hand-not-on-mouse) !eval! (hand-not-on-mouse) ==> ==> +manual> +manual> isa HAND-TO-MOUSE) isa HAND-TO-MOUSE) IF the goal is to find a track AND the motor module is free AND the hand is not on the mouse THEN move the hand to the mouse

51. Learned rules (p Production544 (p Production544 =goal> =goal> isa TASK isa TASK rule Find-Track rule Find-Track step Done step Done =visual-location> =visual-location> isa VISUAL-LOCATION isa VISUAL-LOCATION kind Square-Track kind Square-Track =visual-state> =visual-state> isa MODULE-STATE isa MODULE-STATE modality Free modality Free -visual> -visual> ==> ==> +visual> +visual> isa VISUAL-OBJECT isa VISUAL-OBJECT screen-pos =visual-location) screen-pos =visual-location) IF the goal is to find a track AND a visual-location of type track has been found AND it is not attended THEN attend the track

52. Expert

53. Other changes in the models behavior The model sometimes opts for doing a Radar sweep first and then looking at the altitude and speed (subjects do too) The model sometimes opts for doing a Radar sweep first and then looking at the altitude and speed (subjects do too) The model sometimes prefers using the left hand for keying and the right hand for mousing The model sometimes prefers using the left hand for keying and the right hand for mousing

54. From strong to weak task hierarchy Production compilation not only speeds up performance, but also leads to qualitative changes in behavior Production compilation not only speeds up performance, but also leads to qualitative changes in behavior Bottom-up behavior next to Top-down behavior Bottom-up behavior next to Top-down behavior Is this real reasoning? Maybe reasoning with the small r, but nevertheless crucial in understanding the flexibility of human learning Is this real reasoning? Maybe reasoning with the small r, but nevertheless crucial in understanding the flexibility of human learning

55. Related work Mike Freed: APEX (but no learning) Mike Freed: APEX (but no learning) Ron Chong: EPIC-Soar Ron Chong: EPIC-Soar Kieras and Meyer are working on it (EPIC), but they will have to incorporate learning into EPIC Kieras and Meyer are working on it (EPIC), but they will have to incorporate learning into EPIC Rick Lewis uses an APEX-like approach in ACT-R Rick Lewis uses an APEX-like approach in ACT-R

56. Three models 1. Wait 35 sec in expert setting and 45 sec in beginner setting 2. Based on the model just discussed 3. The model plus two additional strategies

57. Usability test