A Soar’s Eye View of ACT-R John Laird 24 th Soar Workshop June 11, 2004

2 Soar / ACT-R Comparison What changes relative to ACT-R would significantly alter Soar? Not just extensions (activation, RL, EpMem) but fundamental changes. What changes relative to Soar would significantly alter ACT-R? Soar ACT-R

3 Obvious Similarities Soar 9ACT-R 5 Input/OutputBuffers & Async.Buffers & Async. Short-term memoriesGraph StructureChunks in buffers ActivationBase Activation Long-term memoriesProduction RulesProduction Rules EpisodesDeclarative Memory Rule Utilities Chunk Associations Sequential controlOperatorProduction Goal StructuresState stack Goal & Declarative Memory LearningChunkingProduction Composition Reinforcement Utility Learning Episodes Chunks -> Decl. Memory Goal & Chunk Association Base Activation

4 Short-term Memories Soar Unbounded graph structure Multi-valued attributes: sets Decision on ^operator of state I-support and o-support Explicitly represent state Short-term identifiers Generated each time retrieved Values can be long-term symbols ACT-R Chunks (flat structures) in buffers One chunk/buffer Chunk types with fixed slots Goal, Declarative Memory, Perception All persistent until replaced/modified Long-term identifiers for each chunk Provides hierarchical structure state declarative memory red #3 ‘x’ #9 #45 goal perception #3 #45 visualization

5 Implications for Soar Unbounded working memory No easy way to move subset of short-term memory to long-term memory piece by piece Can’t maintain connections between objects without long- term memory symbols Makes it possible to determine results automatically Supports automatic removal of irrelevant data state

6 Implications for ACT-R Bounded representation Long-term memory symbols allow dynamic encapsulation Can learn to test only chunk id instead of substructure Flat representation Hard to represent sets Requires “unpacking” of object symbols to access features But can learn rules that access symbols directly How can it recognize structured objects from perception? (Blending?) Unitary object representation primacy (vs. independent features) All features are equally important (activation is object based) Chunk types are architecturally meaningful goal declarative memory perception #3 #45

7 Implications for ACT-R II Persistence Easy to have inconsistent beliefs Consistency always competes with other reasoning Working Memory = retrieved LTM Declarative Memory (Changes in working memory change declarative memory) No memory of old values in chunks Difficult to maintain independent copies of same object Hypothetical reasoning goal declarative memory perception #3 #45

8 Fundamental Issue: Long-Term Object Identity Architectural (ACT-R) vs. Knowledge-based (Soar) Connecting to perception Connecting to other long-term memories Copying structures

9 Decision Making SoarACT-R Generate featuresParallel rulesSequential rules Generate alternativesParallel rulesMatch rule conditions Compare & rate alternativesParallel rulesRule utility SelectArchitectureArchitecture ApplyParallel rulesRule actions Dimensions for comparison: Simple metrics # of reasoning steps required # of sequential rule firings # knowledge units (rules) required –ACT-R often trades off chunks + interpretation + learning for rules. Capabilities Expressibility Use context Open to meta-reasoning Modification through learning

10 Execution Steps SoarACT-R Generate features (F)Parallel rulesSequential rules Generate options (O)Parallel rulesMatch rule conditions Compare & rate options (C)Parallel rulesRule utility SelectArchitectureArchitecture Apply (A)Parallel rulesRule actions # of rule firingsF + O + C + AF + 1 # of sequential steps1F + 1 This is complicated by declarative memory retrievals in ACT-R – but they are not really procedural knowledge directly involved in decision making, although they are sometimes involved indirectly.

11 Propose and Apply Knowledge Units For a single O that can be selected in S Situations and has A was of Applying: Soar: O + A rules ACT-R: O * A rules O: Independent Proposals A: Independent Applications Op

12 Selection Knowledge Units In Soar, independent numeric indifferent rules combine values for decision Allows linear combinations of desirability In ACT-R, only a single utility value is associated with each rule No run time combination Conflates legality (proposal) and desirability Must have separate rule for each unique context application pair QiQi QjQj QkQk Q ArchitectureArchitecture

13 Expressibility Soar allows “open decisions” Which knowledge contributes is determine at run time Does not require pre-compilation of important features. Separates knowledge about “can” do an action from “should” Makes easy to express and add knowledge to modify method Symbolic preferences Possibility of one-shot learning for decision making Can be told not to do an action (and overcome statistical) Can learn to not do an action

14 Use Run-time Context SoarACT-R Generate alternativesYes – rulesYes – rule conditions Compare/rate alternativesYes – rulesNo – rule utility SelectArchitectureArchitecture ApplyYes – rulesNo – rule action

15 Meta-Reasoning Soar has tie impasses & subgoals Can detect when knowledge is uncertain/incomplete Can use arbitrary reasoning to analyze and make decision Including look-ahead planning with hypothetical states Can return results that modify the decision Learning can directly modify decision ACT-R Difficult to detect uncertainty a & reason about decision Could create impasse when utilities are close or uncertain Difficult to modify decision without experience How could other reasoning change a production rule selection?

16 Predictions! ACT-R Something to deal with meta-cognition Detecting uncertainty and deliberate reasoning to deal with it (and the learning). Planning Integration of emotion/pain/pleasure for learning Episodic memory Soar Long-term declarative memory & architectural declarative learning Some one will build ASCOT-ARR! ACT-R memory structure with Soar operators

17 Gold and Coal Goal: Having alternative architectures Provides inspiration for architectural modification Provides comparison Forces us to examine arbitrary decisions Coal: Most comparisons to date are: Informal (such as this) Not theory directed (AMBER) Confound programming & architecture Not exactly same task