1. Functional Constraints on Architectural Mechanisms Christian Lebiere (cl@cmu.edu) Carnegie Mellon University Bradley Best (bjbest@adcogsys.com) Adaptive Cognitive Systems

2. Introduction Goal: Strong Cogsci – single integrated model of human abilities that is robust, adaptive and general Not just an architecture that supports it (Newell test evaluation) but a system that actually does it Not strong AI (means matter), weak Cogsci (general) Plausible strategies: Build a single model from scratch – traditional AI strategy Incremental assembly – successful CS system strategy But little/no reuse of models limits complexity! 7/29/0922009 ACT-R Workshop

3. Model Fitting Constraint Fitting computational models to human data is the “coin of the realm” of cognitive modeling Is it a sufficient constraint to achieve convergence toward the goal of model integration and robustness Good news: cognitive architectures are increasingly converging toward a common modular organization Bad news: still very little model reuse – almost every task results in a new model developed tabula rasa Question: have we gotten right the tradeoff between precision (fitting data) and generality (reuse/integration)? 7/29/0932009 ACT-R Workshop

4. You Can’t Play 20 Models… 35 years ago Newell raised a similar issue with convergence in experimental psychology He diagnosed much of the issue with the lack of emphasis on the control structure to solve a problem He offered 3 prognoses for “putting it together”: – Complete processing models (and PS suggestion) – check! – Analyze a complex task (chess suggestion) – progress but… – One program for many tasks (integration, e.g WAIS) – fail? What have been the obstacles to putting it together? 7/29/0942009 ACT-R Workshop

5. Obstacles to Integration Models tend to be highly task-specific – they usually cannot be used directly even for closely related tasks They tend to represent the final point of the process from initial task discovery to final asymptotic behavior Modeler’s meta-cognitive knowledge of the task gets hardwired into the model Experience with High-Level Language (HLSR) compilation Task discovery processes, including metacognitive processes, should be part of the model/architecture Tackles broader category of tasks through adaptation 7/29/0952009 ACT-R Workshop

6. Forcing Functions for Integration Model comparison challenges (e.g. DSF) that feature: – Breadth of applicability (e.g. multiple conditions) – Unknown conditions and/or data (tuning vs testing sets) – Integration of multiple functionalities (control, prediction) Unpredictable domains, e.g. adversarial behavior: – Breadth and variability of behavior – Constant push for adaptivity and unpredictability – Strong incentive to maximize functionality Architectural implications to model integration? – Focus on both control and representation structure 7/29/0962009 ACT-R Workshop

7. A Tour of Four Modules Procedural Module Declarative Module Retrieval Buffer Intentional Module Goal Buffer Vision Module Visual Buffer Motor Module Manual Buffer Working Memory Module Imaginal Buffer Environment 7/29/0972009 ACT-R Workshop All modules have shortcomings in robustness and generality Ability to craft models for lab tasks does not guarantee plausible behavior in open-ended situations

8. Module 1: Declarative Base-level learning can lead to looping if unchecked – Most active chunk is retrieved, then its activation boosted… Very hard to control if compiling higher-level model – Many logical conditions require repeated retrieval loops Old solution: tag chunk on retrieval (e.g. list learning) New solution: declarative finsts to perform tagging 7/29/0982009 ACT-R Workshop +retrieval> isa item index =index :recently-retrieved nil (sgp :declarative-num-finsts 5 :declarative-finst-span 10) +retrieval> isa item index =index - retrieved =goal =retrieval> retrieved =goal

9. Base-Level Inhibition (BLI) Also in other domains: arithmetic, web navigation, physical environments Provides inhibition of return resulting in soft, adaptive round-robin in free-recall procedures w/o requiring any additional constraints 7/29/0992009 ACT-R Workshop

10. Emergent Robustness Running the retrieval mechanism unsupervised leads to the gradual emergence of an internal power law distribution It differs from both the pathological behavior of the default BLL, and from the hard and fixed round-robin of the tag/finst version Frequencies of Free Recall as a Function of Item Rank 7/29/09102009 ACT-R Workshop

11. Module 2: Procedural Procedural module – Production rule set need careful crafting to cover all cases – Degenerate behavior in real environments (stuck, loop, etc) – Esp. difficult in continuous domains (ad hoc thresholds, etc) Generalization of production applicability – Often need to use declarative module to leverage semantic generalization through partial matching mechanism Unification between symbolic (matching) and subsymbolic (selection) processes is desirable for robustness, adaptivity and generalization 7/29/09112009 ACT-R Workshop

12. Production Partial Matching (PPM) Same principle as partial matching in declarative memory – Unification is good and logical given representation (neural models) Matching Utility Dynamic generalization: production condition defines ideal “prototype” situation, not range of application conditions Adaptivity: generalization expands with success as utility rises, contracts with failure as production over-generalizes Safe version: explicit ~ test modifier similar to -,, etc Learning new productions can collapse across range and learn differential sensitivity to individual buffer slot values 7/29/09122009 ACT-R Workshop

13. Building Sticks Task Standard Production Model (Lovett, 1996) 4 productions – Force-over – Force-under – Decide-over – Decide-under Hardwired range Utility Learning Instance-based Model (Lebiere, 1997) Chunks: under, over, target & choice slots Partial matching on closeness of over and under to target Base-level learning w/ degree of match New Partial-Matching Production Model 2 productions –Over: match over stick against target –Under: match under stick against target Utility learning mixed with degree of match 7/29/09132009 ACT-R Workshop

14. Procedural or Instance-based? One of Newell’s decried “oppositions” reappeared in the computational modeling context Neuroscience (e.g., fMRI) might provide arbitrating data between modules but likely not within module Correct solution is likely a combination of initial declarative retrieval to procedural selection Need a smooth transition from declarative to procedural mechanism without modeler-induced discontinuity in terms of arbitrary control structure 7/29/092009 ACT-R Workshop14

15. Module 3: Working Memory Current WM: Named, fixed buffers, types, slots – Pros Precise reference makes complex information processing not only possible but relatively easy Familiar analogy to traditional programming – Cons Substantial modeling effort required – Modeling often time-consuming and error-prone Hard limit on flexibility of representation – Fine in laboratory tasks, more problematic in open-ended, dynamic, unpredictable environments

16. Representation Implications Explicit slot (and also type, buffer) management – Add more slots to represent all information needed Pro: slots have clear semantics Con: profligate, dilution of spreading activation – Reuse slots for different purposes over time Pro: keep structures relatively compact Con: uncertain semantics (what is in this slot right now?)‏ – Use different (goal) types over time Pro: cleaner semantics, hierarchical control Con: increase management of context transfer – More buffers or reuse buffers as storage Less of that for now but same general drawbacks as slot, type Integration issues (episodic memory)‏

17. Working Memory Module Replace chunk structures in buffers with sets of values associated with fast decaying short-term activation – Faster decay rate than LTM and no reinforcement Generalize pattern matching to ordered set of values – Double match of semantic and position content Assumptions about context permanence – Short-term maintenance w/ quick decay (sequence learning) – Explicit rehearsal possible but impact on strength and ordering

18. N-Back Task Nback working memory task: is current stimulus same as the one n back? Default ACT-R model holds and shifts items in buffer: perfect recall! Working memory model adds item to WM, then decays and partial match Performance decreases with noise and n up to plateau – good fit to data (p back4 =goal> isa nback stimulus =stimulus match nil +intentional> =back1 =back2 =back3 =back4 ==> !output! (Stimulus =stimulus retrieving 4-back =back4) =goal> match =back4) (p back4 =goal> isa nback stimulus =stimulus match nil =imaginal> isa four-back back1 =back1 back2 =back2 back3 =back3 back4 =back4 ==> !output! (Stimulus =stimulus matching 4-back =back4) =goal> match =back4)

19. Module 4: Episodic Memory Need integration of information in LTM across modalities Main role of episodic memory is support goal management Store snapshots of working memory – Concept of chunk slot is replaced with activation – Similar to connectionist temporal synchrony binding Straightforward matching of WM context to episodic chunks – Double, symmetrical match of semantic and activation content Issues: – Creation signal: similar to current chunk switch in buffer – Reinforcement upon rehearsal? – Relation to traditional LTM? Similar to role of HC in training PC?

20. List Memory Pervasive task requires multi-level indexing representation – “micro-chunks” vs traditional representation Captures positional confusion and failures Is it strategy choice or architectural feature? How best to provide this function pervasively 7/29/09202009 ACT-R Workshop +retrieval> isa item parent =group position fourth :recently-retrieved nil

21. Related Work Instruction following (Anderson and Taatgen) General model for simple step-following tasks Minimal control principle (Taatgen) Limit modeler-imposed control structure Threading and multitasking (Salvucci and Taatgen) Combine independent models and reduce interference Metacognition (Anderson) Enable model to discover original solution to new problem Call for new thinking on “an increasingly watered down set of principles for the representation of knowledge” (Anderson) 7/29/09212009 ACT-R Workshop

22. Conclusion Available data is often not enough to discriminate between competing models of single tasks – Newell might have been too optimistic about the ability to uniquely infer the control method given data and system More data can help but often leads to more specialized and complex models and away from integration Focus on functionality, esp. Newell’s 2 nd (complex tasks) and 3 rd (multiple tasks) criteria for further discrimination Focusing on tasks that require open-ended behavior can enhance the robustness and generality of cognitive architectures without compromising their fidelity 7/29/09222009 ACT-R Workshop