Soft Computing Lab. Yongjun Kim A Gentle Introduction to Soar, an Architecture for Human Cognition: 2006 Update Jill Fain Lehman, John Laird, Paul Rosenbloom 26th Mar., 2009 Soft Computing Lab. Yongjun Kim

Outline Introduction Soar (State operator and result) Architecture The Idea of Architecture What Cognitive Behaviors have in common Behavior as Movement through Problem Spaces Tying the Content to the Architecture Memory, Perception, Action, and Cognition Detecting a Lack of Knowledge Learning Putting it all together: a Soar Model of Joe Rookie Stepping back: the Soar Architecture in review From Architecture to Unified Theories of Cognition Discussion

Introduction Many intellectual disciplines contribute to cognitive science: Psychology, linguistics, anthropology, artificial intelligence, etc. Each discipline provides expertise and contributes microtheories. Descriptions of regularities in behavior Theories that try to explain those regularities How to know the contributions of each discipline fit in the big picture? Go ahead and try to put the whole picture together. Try to build unified theories of cognition (UTCs). A set of general assumptions for cognitive models that account for all of cognition. Soar was developed and used as a candidate UTC in 1980’s. Try to find a set of computationally-realizable mechanisms and structures that can answer all the questions about cognitive behavior.

The Idea of Architecture Architecture is the fixed set of mechanisms and structures. An architecture stands as a theory of what is common among behavior. Any complex system can be decomposed into architecture and content. Architecture requires content to produce behavior. Cognitive Architecture A theory of the fixed mechanisms and structures that underlie human cognition. Soar is used as a cognitive architecture.

What Cognitive Behaviors have in common Soar theory assumes that cognitive behavior has at least the following characteristics: It is goal-oriented. It takes place in a rich, complex, detailed environment. It requires a large amount of knowledge. It requires the use of symbols and abstractions. It is flexible, and a function of the environment. It requires learning from the environment and experience. Need to explore the architecture in terms of some particular content in order to see how the architecture contributes to behavior. Have to be goal-oriented about something. Need a scenario.

What Cognitive Behaviors have in common A Simple Scenario from Baseball.

What Cognitive Behaviors have in common A Simple Scenario from Baseball. Behave in a goal-oriented manner: Joe’s goal is to win the game. A number of subgoals : get the batter out, strike the batter out with a curve ball, etc. Operate in a rich, complex, detailed environment: Many relevant aspects of Joe’s environment. Positions of people, the number of balls and strikes, etc. Use a large amount of knowledge: Need to draw on statistics about his own team, his own pitching record, etc. Behave flexibly as a function of the environment: Need to respond to his own perceptions of the environment. Is it windy?, is the batter left-or right-handed? etc. Use symbols and abstractions: Need to draw on his previous experience by abstracting away from this day and place. Learn from the environment and experience: Joe need to throw Sam a fast ball next time.

What Cognitive Behaviors have in common A Simple Scenario from Baseball. For Joe to act like a rookie pitcher, many different kinds of knowledge should be given to it. Need to find some way to represent and process Joe’s knowledge in Soar. Assume that there is an underlying structure to behavior and knowledge. This structure provides a means for organizing knowledge as a sequence of decisions through a problem space.

Behavior as Movement through Problem Spaces The Space of Possible Actions for Joe. Must make his decisions with respect to the situation at the moment. Support two points of view : a static / a dynamic view of Joe’s life.

Behavior as Movement through Problem Spaces The Abstract Form of a Problem Space. Consist of states, features, values and operators. The state is a representation of all the aspects of the situation (internal and external). Only one state exists at any time and prior states are not directly accessible. f2 : batter status v6 : out

Behavior as Movement through Problem Spaces The Abstract Form of a Problem Space. Movement could be entirely random. To keep behavior goal-directed, the succession of operators and the resulting state transformations must be guided by the principle of rationality: If an agent has knowledge that an operator application will lead to one of its goals then the agent will select that operator. Tying the Content of Joe’s World to the Soar Architecture. Map knowledge into goals, states and operators. Determine what knowledge becomes part of the state and the operators. Find how to know what an operator application will do. Find how to know when the goal has been achieved.

Tying the Content to the Architecture Knowledge Representation in the Architecture Must be domain independent. What is common across all domains and problems? In Soar, it is the decomposition of knowledge into goals, problem spaces, states, and operators.

Tying the Content to the Architecture Guidelines for Tying the Domain Content to the Architecture The domain knowledge of the objects and people in the game (K1) => the features and values in the state Knowledge of actions (K5 and K7) => operators Knowledge about objectives (K4) => goals Knowledge of the rules of the game (K3) + K1 + K4 + K5 + K7 => problem space The Operator Selection Operators that share common tests for goals and situations can be considered to be part of the same problem space. Given the initial state and the goal in the game, the operators will be the various kinds of pitches.

Tying the Content to the Architecture The Effect of Operators Can be defined in two ways: Defined by the execution of the operator in the external world using knowledge of physical actions (K7). Defined by knowledge of abstract events or particular episodes (K2). Goal Evaluation Determining that the current state is a desired state relies on knowledge of the rules of the game (K3). The environment can give signals of success and failure. Umpire’s judge (“You’re out!”) The Modification of Goal and Problem Space Done by augmenting the state with goals and problem spaces. Ex. Joe’s team is ahead in the fifth inning, but rain is on the horizon => out quickly. Ex. A member of the opposing team on base => short windup.

Memory, Perception, Action, and Cognition Soar’s Memory Consist of long-term memory (LTM) and working memory (WM). LTM has three different types: procedural, semantic, and episodic.

Memory, Perception, Action, and Cognition Memory type Long-term memory (LTM) Procedural : knowledge about how and when to do things. How to ride a bike, how to solve an algebra problem, etc. Semantic : knowledge consists of facts about the world. Bicycles have two wheels, a baseball game has nine innings, etc. Episodic : knowledge consists of things you remember. The time you fell off your bicycle and scraped your elbow. LTM is not directly available, but must be searched to find what is relevant to the current situation. Procedural knowledge is primarily responsible for controlling behavior and maps directly onto operator knowledge. Working memory (WM) Knowledge that is most relevant to the current situation. In Soar, WM is represented as a set of the features and values that make up the current state (and substates). Can be used to retrieve other knowledge from LTM. Working memory elements in Soar arise in one of two ways: Through perception. Through retrievals from long-term memory.

Memory, Perception, Action, and Cognition Examples of LTM procedural knowledge There are dependencies between the rules. However, Soar doesn’t recognize them. Rules are processed by the architecture in a general way.

Memory, Perception, Action, and Cognition The Decision Cycle Generate behavior out of the content in LTM and WM. Do its work in five phases: Input WM elements are created. Elaboration WM elements are matched against the “if” parts of the rules in LTM. Decision Decide suggestions according to preferences (symbolic/numeric). Application Output Support limited parallelism. Multiple actions can be packaged together as a single operator.

Detecting a Lack of Knowledge Impasse in the Decision Cycle Happen when the decision cycle can’t decide a single operator due to lack of knowledge for preferences (e.g., without r5). Soar automatically creates a substate. The goal is to select between two operators for the original state. Semantic and episodic memories are usually used in substates. The reminding is goal-driven.

Detecting a Lack of Knowledge Memory Search in Impasse Assume that Joe has the following fact in episodic memory. A Cue must be created that can be used to search the memory. In some cases, no likely match will be returned. Then, the model can modify the cue. The next three rules define the evaluate operator that creates preferences to resolve the tie.

Detecting a Lack of Knowledge Resolving an Operator-Tie Impasse

Detecting a Lack of Knowledge Working Memory Hierarchy Working memory consists of a state/substate hierarchy. The hierarchy grows as impasses arise and shrinks as impasses are resolved. If multiple changes are suggested in different states, the change to the state that is highest in the hierarchy is selected. If a change occurs to a context high up in the hierarchy, then all the substates below the changed state disappear. Impasse Type in Soar Other types of impasses can be occurred. E.g., an operator tie, an operator no-change impasse, etc. The full set of impasses defined in Soar is fixed and domain-independent.

Learning Four Learning Mechanisms Chunking Reinforcement learning Episodic learning Semantic learning Three Questions for Learning Systems: What do they learn? Soar systems learn structures for its LTM: rules, declarative facts, and episodes. What is the source of knowledge for learning? Different between the learning mechanisms. When do they learn?

Learning Chunking Reinforcement Learning Is the most developed learning mechanism. Is deductive and compositional. Resolving impasses can lead to learn new rules (called chunks). Reinforcement Learning Knowledge source is feedback from the environment (reward). Learn rules that generate preferences based on future expected rewards. Two parts in Soar Must learn rules that test the appropriate features of the states and operators. Initially create rules based on the rules that propose operators, and specialize them to consistently predict the same value. Must learn the appropriate expected rewards for each rule. Done by comparing the prediction of a rule with what happens during the next decision.

Learning Episodic Memory Semantic Memory Knowledge source is the stream of experience. Episodes are recorded automatically as a problem is solved. An episode consists of a subset of the WM elements that exist at the time of recording. Comparison with chunking and reinforcement learning Passive learning mechanism. The contents of an episode are determined only indirectly by reasoning. No distinction between conditions for retrieval and what should be retrieved. Semantic Memory Knowledge source is the co-occurrence of structures in WM. Knowledge about the rules of baseball, what is a home run, etc. When to store a structure in semantic memory is a research issue. Deals with static structure instead of derivation-based rules. Comparison with episodic memory More general than episodes. Place and time information is disassociated.

Putting it all together: a Soar Model of Joe Rookie To Build a Full Model. Specify the domain knowledge and which memories it is stored in. Tie the domain knowledge to state structures and operators. Specify the relationships between different levels by the impasses and the kinds of knowledge that will be missing, and learned.

Stepping back: the Soar Architecture in review A Cognitive Architecture Is a fixed set of mechanisms and structures that process content to produce behavior. Is a theory, or point of view, about what cognitive behaviors have in common. Soar Architecture States and Operators Working Memory Long-term memory The Perception/Motor Interface The Decision Cycle Impasses Four Learning Mechanism

From Architecture to Unified Theories of Cognition The model of Joe Rookie is a content theory. Can explain why human needs change with time. Can explain the specific factors that motivate people. Soar has contributed a number of content theories to the field. NTD-Soar, Instructo-Soar, IMPROV, TacAir-Soar, Soar MOUTBOT, etc. Content theories model aspects of human language (or concept learning, or multi-tasking) within a framework. The theory of the resulting model will be compatible with what is assumed to be architectural in the other content theories. Content theories constitute a burgeoning unified theory of cognition (UTC). Soar as a UTC.

Discussion What is the similarity and difference between Soar and ACT-R? Both try to model human. Soar has focused on memory system, but ACT-R on brain image. Do you think Soar can be a UTC? A UTC must explain the following things. How intelligent organisms flexibly react to stimuli from the environment. How they exhibit goal-directed behavior and acquire goals rationally. How they represent knowledge (or which symbols they use). Learning. It is known that there is efforts to model emotions into Soar. Do you think emotions can play a big role in cognition?