Pat Langley Institute for the Study of Learning and Expertise Palo Alto, CA Cumulative Learning of Relational and Hierarchical Skills.

Slides:



Advertisements
Similar presentations
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Advertisements

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley Arizona State University and Institute for the Study of Learning and Expertise Expertise, Transfer, and Innovation in.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California Elena Messina.
Center for the Study of Language and Information Stanford University, Stanford, California March 20-21, 2004 Symposium on Reasoning and Learning in Cognitive.
Pat Langley Institute for the Study of Learning and Expertise Palo Alto, California and Center for the Study of Language and Information Stanford University,
Pat Langley Seth Rogers Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA.
Some Information 1.Parking is free in Stanford lots over the weekend. 2.There is no registration fee thanks to ONR and NSF. 3.Participation in the meeting.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Varieties of Problem Solving in a Unified Cognitive Architecture.
Pat Langley Dan Shapiro Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Extending the I CARUS Cognitive Architecture Thanks to D. Choi,
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Pat Langley Dongkyu Choi Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Mental Simulation and Learning in the I CARUS Architecture.
General learning in multiple domains transfer of learning across domains Generality and Transfer in Learning training items test items training items test.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Modeling Social Cognition in a Unified Cognitive Architecture.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Information Processing Technology Office Learning Workshop April 12, 2004 Seedling Overview Learning Hierarchical Reactive Skills from Reasoning and Experience.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA
IL Kickoff Meeting June 20-21, 2006 DARPA Integrated Learning POIROT Project 1 Learning Hierarchical Task Networks by Analyzing Expert Traces Pat Langley.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona Institute for the Study of Learning and Expertise Palo Alto, California.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Institute for the Study of Learning and Expertise Palo Alto,
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona A Cognitive Architecture for Integrated.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA A Unified Cognitive Architecture for Embodied Agents Thanks.
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Cognitive Architectures and Virtual Intelligent Agents Thanks.
Pat Langley Computer Science and Engineering Arizona State University Tempe, Arizona USA Institute for the Study of Learning and Expertise Palo Alto, California.
Pat Langley Institute for the Study of Learning and Expertise Palo Alto, California A Cognitive Architecture for Complex Learning.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA
1 ISLE Transfer Learning Team Main Technology Components The I CARUS Architecture Markov Logic Networks Executes skills in the environment Long-TermConceptualMemoryShort-TermConceptualMemory.
ARCHITECTURES FOR ARTIFICIAL INTELLIGENCE SYSTEMS
Level 1 Recall Recall of a fact, information, or procedure. Level 2 Skill/Concept Use information or conceptual knowledge, two or more steps, etc. Level.
Business Architecture
Part2 AI as Representation and Search
TWS Aid for Scorers Information on the Background of TWS.
Certified Business Process Professional (CBPP®) Study Session Part 4 Sept. 15, 2010.
Show Me an Evidential Approach to Assessment Design Michael Rosenfeld F. Jay Breyer David M. Williamson Barbara Showers.
Design Patterns for Metamodel Design Domain-Specific Modeling Workshop Portland, Oregon October 23, 2011 Hyun Cho and Jeff Gray University of Alabama Department.
Knowledge Acquisitioning. Definition The transfer and transformation of potential problem solving expertise from some knowledge source to a program.
Marakas: Decision Support Systems, 2nd Edition © 2003, Prentice-Hall Chapter Chapter 7: Expert Systems and Artificial Intelligence Decision Support.
Polyscheme John Laird February 21, Major Observations Polyscheme is a FRAMEWORK not an architecture – Explicitly does not commit to specific primitives.
Certified Business Process Professional (CBPP®) Exam Overview
Evaluation: A Challenging Component of Teaching Darshana Shah, PhD. PIES
ARTIFICIAL INTELLIGENCE [INTELLIGENT AGENTS PARADIGM] Professor Janis Grundspenkis Riga Technical University Faculty of Computer Science and Information.
Human Learning Brian Newberry.
Author: James Allen, Nathanael Chambers, etc. By: Rex, Linger, Xiaoyi Nov. 23, 2009.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA
EDU 385 EDUCATION ASSESSMENT IN THE CLASSROOM
SOFTWARE DESIGN.
Putting Research to Work in K-8 Science Classrooms Ready, Set, SCIENCE.
Synthetic Cognitive Agent Situational Awareness Components Sanford T. Freedman and Julie A. Adams Department of Electrical Engineering and Computer Science.
Chapter 7 Learning Objectives 1.Discuss how to link training programs to organizational needs. 2.Explain how to assess the need for training. 3.Explain.
What is the Skills framework?
Chapter 10 Designing Adaptive Organizations. Organizing The deployment of organizational resources to achieve strategic goals  Division of labor  Lines.
Formal Verification. Background Information Formal verification methods based on theorem proving techniques and model­checking –To prove the absence of.
Cognitive Architectures and General Intelligent Systems Pay Langley 2006 Presentation : Suwang Jang.
INTRODUCTION: Project management involves the planning, monitoring, and control of the people, process, and events that occur as – software evolves from.
Joscha Bach Nick Cassimatis Ken Forbus Ben Goertzel Stacey Marsella John Laird Pat Langley Christian Lebiere Paul Rosenbloom Matthias Scheutz Satinder.
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, CA
Understanding Depth of Knowledge. Depth of Knowledge (DOK) Adapted from the model used by Norm Webb, University of Wisconsin, to align standards with.
Teaching Games for Understanding
Learning Teleoreactive Logic Programs by Observation
Object oriented system development life cycle
Measuring Project Performance: Tips and Tools to Showcase Your Results
2018 OSEP Project Directors’ Conference
Design and Implementation
KNOWLEDGE REPRESENTATION
Organizational Structure
LEARNER-CENTERED PSYCHOLOGICAL PRINCIPLES. The American Psychological Association put together the Leaner-Centered Psychological Principles. These psychological.
Presentation transcript:

Pat Langley Institute for the Study of Learning and Expertise Palo Alto, CA Cumulative Learning of Relational and Hierarchical Skills from Problem Solving This research was funded by Grant HR from the DARPA Information Processing Technology Office, which may not agree with the points made in this talk.

Research Objectives We are designing and implementing new learning methods that: operate over relational, hierarchical knowledge structures operate over relational, hierarchical knowledge structures support reasoning, reactive control, and problem solving support reasoning, reactive control, and problem solving are embedded within a broader architectural framework are embedded within a broader architectural framework utilize existing knowledge to increase learning rates utilize existing knowledge to increase learning rates acquire this knowledge in an incremental, cumulative manner acquire this knowledge in an incremental, cumulative manner are applicable to a variety of challenging domains are applicable to a variety of challenging domains We hope to develop learning mechanisms that support horizontal and vertical transfer both within and across domains.

The I CARUS Architecture * Long-TermConceptualMemory Long-Term Skill Memory Short-TermConceptualMemory Goal/SkillStack Categorization and Inference SkillExecution Perception Environment PerceptualBuffer * without learning Means-EndsAnalysis MotorBuffer SkillRetrieval

Organization of Long-Term Memory concepts skills Each concept is defined in terms of other concepts and/or percepts. Each skill is defined in terms of other skills, concepts, and percepts. I CARUS organizes both concepts and skills in a hierarchical manner.

Concepts from In-City Driving Domain (in-segment (?self ?sg) :percepts((self ?self segment ?sg) (segment ?sg))) (aligned-with-lane (?self ?lane) :percepts((self ?self) (lane-line ?lane angle ?angle)) :positives((in-lane ?self ?lane)) :tests((> ?angle 0.05) ( ?angle 0.05) (< ?angle 0.05)) ) (on-street (?self ?packet) :percepts((self ?self) (packet ?packet street ?street) (segment ?sg street ?street)) :positives((not-delivered ?packet) (current-segment ?self ?sg))) (increasing-direction (?self) :percepts((self ?self)) :positives((increasing ?b1 ?b2)) :negatives((decreasing ?b3 ?b4)) )

Organization of Long-Term Memory conceptsskills For example, the skill highlighted here refers directly to the highlighted concepts. I CARUS interleaves its long-term memories for concepts and skills.

Skills from In-City Driving Domain (turn-around-on-street (?self ?packet) :percepts((self ?self segment ?segment direction ?dir) (building ?landmark)) :start ((on-street-wrong-direction ?packet)) :effects((on-street-right-direction ?packet)) (building ?landmark)) :start ((on-street-wrong-direction ?packet)) :effects((on-street-right-direction ?packet)) :ordered((get-in-U-turn-lane ?self) (prepare-for-U-turn ?self) :ordered((get-in-U-turn-lane ?self) (prepare-for-U-turn ?self) (steer-for-U-turn ?self ?landmark)) ) (steer-for-U-turn ?self ?landmark)) ) (get-aligned-in-segment (?self ?sg) :percepts((lane-line ?lane angle ?angle)) :requires((in-lane ?self ?lane)) :effects((aligned-with-lane ?self ?lane)) :actions(( steer ( times ?angle 2))) ) (steer-for-right-turn (?self ?int ?endsg) :percepts((self ?self speed ?speed) (intersection ?int cross ?cross) (segment ?endsg street ?cross angle ?angle)) :start((ready-for-right-turn ?self ?int)) :effects((in-segment ?self ?endsg)) :actions(( times steer 2)) ) :actions(( times steer 2)) )

Basic I CARUS Processes concepts skills Concepts are matched bottom up, starting from percepts. Skill paths are matched top down, starting from intentions. I CARUS matches patterns to recognize concepts and select skills.

A Trace of Means-Ends Problem Solving The resulting traces provide the material for learning new relational skills and concepts in terms of simpler components. An impasse causes I CARUS to invoke a means-ends problem solver.

Learning Skills from Means-Ends Traces A concept chaining I CARUS learns skills for ordering subgoals from concept chaining.

Learning Skills from Means-Ends Traces A B skill chaining I CARUS learns skills for ordering subskills from skill chaining.

Learning Skills from Means-Ends Traces A B C concept chaining Each level of skill learning builds upon results from prior levels.

Learning Skills from Means-Ends Traces A B D C skill chaining This leads I CARUS to extend its skill hierarchy in a cumulative way.

Learning Skills from Means-Ends Traces A B D E C concept chaining This in turn supports transfer both within and across problems.

Transfer Results in FreeCell FreeCell is a complex solitaire game in which all cards are visible. We let I CARUS practice on versions with a small set of cards, then examined its transfer to problems with more cards.

Transfer Results in FreeCell Experiments revealed substantial transfer to the harder problems. This held both for the percentage of problems solved and for the effort required on successful attempts.

Directions for Future Research vertical transfer to domains that utilize others as components; vertical transfer to domains that utilize others as components; horizontal transfer to domains to share knowledge elements; horizontal transfer to domains to share knowledge elements; horizontal transfer to tasks that require representation mapping. horizontal transfer to tasks that require representation mapping. Our initial results suggest I CARUS can transfer knowledge learned on simple problems to complex ones from the same domain. In future work, we intend to examine the additional issues of: The final problem is a key challenge in developing robust methods for reusing learned knowledge. We hope to evaluate our ideas on both action-oriented domains like strategy games and inferential tasks like physics problems.

The General Game-Playing Testbed supports a wide variety of wide variety of N-person games; supports a wide variety of wide variety of N-person games; describes each game setting in a standard logical formalism; describes each game setting in a standard logical formalism; specifies the rules of each game in a related formalism; specifies the rules of each game in a related formalism; manages matches between players and records activities; manages matches between players and records activities; provides sample games for debugging candidate systems. provides sample games for debugging candidate systems. Genesereth and Love (2005) have developed a framework that: They have designed this framework to encourage research on general approaches to intelligent behavior. However, it also provides an excellent testbed for evaluating the ability of learning systems to transfer within and across domains. See for more details and examples.