Metareasoning: An Overview of Select Systems Joshua Jones 3/4/2011.

Slides:



Advertisements
Similar presentations
Pat Langley School of Computing and Informatics Arizona State University Tempe, Arizona USA Modeling Social Cognition in a Unified Cognitive Architecture.
Advertisements

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California USA
ARCHITECTURES FOR ARTIFICIAL INTELLIGENCE SYSTEMS
Verification and Validation
Chapter 2: Marr’s theory of vision. Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010 Overview Introduce Marr’s distinction between.
Advanced Educational Psychology --mgmsantos. Misconceptions: 1. That nothing needs to be done for students with a high intelligence because they will.
A model of Consciousness With neural networks By: Hadiseh Nowparast.
Ontologies for Reasoning about Failures in AI Systems Matthew D. Schmill, Tim Oates, Dean Wright University of Maryland Baltimore County Don Perlis, Shomir.
Background information Formal verification methods based on theorem proving techniques and model­checking –to prove the absence of errors (in the formal.
Learning Networks connecting people, organizations, autonomous agents and learning resources to establish the emergence of effective lifelong learning.
SSP Re-hosting System Development: CLBM Overview and Module Recognition SSP Team Department of ECE Stevens Institute of Technology Presented by Hongbing.
1 © 1998 HRL Laboratories, LLC. All Rights Reserved Construction of Bayesian Networks for Diagnostics K. Wojtek Przytula: HRL Laboratories & Don Thompson:
1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.
Software Metrics II Speaker: Jerry Gao Ph.D. San Jose State University URL: Sept., 2001.
1 Planning. R. Dearden 2007/8 Exam Format  4 questions You must do all questions There is choice within some of the questions  Learning Outcomes: 1.Explain.
Introduction to CSE 591: Autonomous agents - theory and practice. Chitta Baral Professor Department of Computer Sc. & Engg. Arizona State University.
Writing Good Software Engineering Research Papers A Paper by Mary Shaw In Proceedings of the 25th International Conference on Software Engineering (ICSE),
Architectural Design Principles. Outline  Architectural level of design The design of the system in terms of components and connectors and their arrangements.
MICHAEL T. COX UMIACS, UNIVERSITY OF MARYLAND, COLLEGE PARK Toward an Integrated Metacognitive Architecture Cox – 8 July 2011.
Models of Human Performance Dr. Chris Baber. 2 Objectives Introduce theory-based models for predicting human performance Introduce competence-based models.
Physical Symbol System Hypothesis
Protégé An Environment for Knowledge- Based Systems Development Haishan Liu.
Meaningful Learning in an Information Age
1 FM Overview of Adaptation. 2 FM RAPIDware: Component-Based Design of Adaptive and Dependable Middleware Project Investigators: Philip McKinley, Kurt.
Course Instructor: Aisha Azeem
Verification and Validation
UML Class Diagrams: Basic Concepts. Objects –The purpose of class modeling is to describe objects. –An object is a concept, abstraction or thing that.
Sepandar Sepehr McMaster University November 2008
Tutoring and Learning: Keeping in Step David Wood Learning Sciences Research Institute: University of Nottingham.
Taxonomies of Learning Foundational Knowledge: Understanding and remembering information and ideas. Application: Skills Critical, creative, and practical.
Verification and Validation Yonsei University 2 nd Semester, 2014 Sanghyun Park.
Exploring Design Innovation: The AI Method and Some Results Ashok Goel Georgia Tech May 18, 2006.
Knowledge representation
SLB /04/07 Thinking and Communicating “The Spiritual Life is Thinking!” (R.B. Thieme, Jr.)
CSA3212: User Adaptive Systems Dr. Christopher Staff Department of Computer Science & AI University of Malta Lecture 9: Intelligent Tutoring Systems.
HCI in Software Process Material from Authors of Human Computer Interaction Alan Dix, et al.
K. J. O’Hara AMRS: Behavior Recognition and Opponent Modeling Oct Behavior Recognition and Opponent Modeling in Autonomous Multi-Robot Systems.
The roots of innovation Future and Emerging Technologies (FET) Future and Emerging Technologies (FET) The roots of innovation Proactive initiative on:
Chapter 13 Artificial Intelligence and Expert Systems.
Curiosity-Driven Exploration with Planning Trajectories Tyler Streeter PhD Student, Human Computer Interaction Iowa State University
Generic Tasks by Ihab M. Amer Graduate Student Computer Science Dept. AUC, Cairo, Egypt.
Review of Parnas’ Criteria for Decomposing Systems into Modules Zheng Wang, Yuan Zhang Michigan State University 04/19/2002.
Finite State Machines (FSM) OR Finite State Automation (FSA) - are models of the behaviors of a system or a complex object, with a limited number of defined.
What’s Ahead for Embedded Software? (Wed) Gilsoo Kim
PnP Networks Self-Aware Networks Self-Aware Networks Self-Healing and Self-Defense via Aware and Vigilant Networks PnP Networks, Inc. August, 2002.
RULES Patty Nordstrom Hien Nguyen. "Cognitive Skills are Realized by Production Rules"
Reinforcement Learning AI – Week 22 Sub-symbolic AI Two: An Introduction to Reinforcement Learning Lee McCluskey, room 3/10
Formal Verification. Background Information Formal verification methods based on theorem proving techniques and model­checking –To prove the absence of.
Artificial Intelligence: Research and Collaborative Possibilities a presentation by: Dr. Ernest L. McDuffie, Assistant Professor Department of Computer.
“Intelligent User Interfaces” by Hefley and Murray.
Cognitive Architectures and General Intelligent Systems Pay Langley 2006 Presentation : Suwang Jang.
Projection and the Reality of Routines – reflections of a computational modeller Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University.
Intelligent Agents Chapter 2. How do you design an intelligent agent? Definition: An intelligent agent perceives its environment via sensors and acts.
Pattern Recognition. What is Pattern Recognition? Pattern recognition is a sub-topic of machine learning. PR is the science that concerns the description.
Learning Analytics isn’t new Ways in which we might build on the long history of adaptive learning systems within contemporary online learning design Professor.
Social Cognitive Theory
Network Management Lecture 13. MACHINE LEARNING TECHNIQUES 2 Dr. Atiq Ahmed Université de Balouchistan.
Artificial Intelligence
Big data classification using neural network
What is cognitive psychology?
Neural Network Architecture Session 2
Artificial Intelligence Lecture No. 5
WELCOME TO COSRI IBADAN
Bloom's Hierarchy “Although it received little attention when first published, Bloom's Taxonomy has since been translated into 22 languages and is one.
Program comprehension during Software maintenance and evolution Armeliese von Mayrhauser , A. Marie Vans Colorado State University Summary By- Fardina.
Intelligent Agents Chapter 2.
Developing metacognition – what does the evidence suggests?
Software Architecture
Mike Anderson, Darsana P. Josyula, Don Perlis
Presentation transcript:

Metareasoning: An Overview of Select Systems Joshua Jones 3/4/2011

Outline Overview of metareasoning Gaia – A game-playing metareasoning system. Abstraction Networks (ANs) – Domain knowledge adaptation. MetaCognitive Loop (MCL) – A general- purpose metareasoning shell. Key points

Metareasoning Metareasoning (metacog) is knowledge (i.e. awareness) of one's cognitive processes and the efficient use of this self-awareness to self- regulate these cognitive processes. (Wikipedia) Typically the intent in practical systems is to control and regulate some object-level reasoning process.

Metareasoning Model of a Metareasoning Agent Adapted From Cox and Raja, 2007.

Metareasoning Alternative models exist...

What kinds of things might the Meta- Layer do? Select among various methods to achieve a goal. Decide when to terminate or alter reasoning. Fix errors in object-level knowledge and/or process. Make judgments about skill at a task or quality of knowledge about a subject. (And accordingly direct computation/learning.) Adapt a reasoning process to a new situation.

Psychological Studies When solving problems, there is some evidence that it helps to explicitly force yourself to consider your thought process. It seems that being explicitly prompted to attend to monitoring one's own reasoning leads to some improvements. M. T. Cox. Metacognition in computation: A selected research review. Artificial Intelligence, 169(2):104–141, 2005.

Psychological Studies Some contradictory evidence – people often make serious errors in making metacognitively based judgments. The cost of metareasoning itself must also be considered, though its value is often in controlling cost of object-level processing.

Outline Overview of metareasoning Gaia – A game-playing metareasoning system. Abstraction Networks (ANs) – Domain knowledge adaptation. MetaCognitive Loop (MCL) – A general- purpose metareasoning shell. Key points

Gaia Gaia is an adaptive game-playing system. Gaia uses an explicit teleological model of the object-level game player. Gaia makes proactive adaptations in response to rule changes. An earlier system, REM, worked in a similar way and could also make retrospective adaptations.

Gaia/REM References REM: Murdock, J. W., and Goel, A. K. ICAPS-2003, JETAI Gaia: Joshua Jones, Chris Parnin, Avik Sinharoy, Spencer Rugaber and Ashok Goel. Teleological Metareasoning for Automating Software Adaptation. Third IEEE International Conference on Self-Adaptive and Self-Organizing Systems, 2009.

Problem A piece of software is placed in a similar, but new environment.

Problem It must be adapted to better meet the new demands...

Problem How can this be done automatically?

FreeCiv ● Turn-based multiplayer strategy game ● Massive number of game states and many(compound) actions ● Interrelated subgoals (technology, population growth, military...)

Metacognition Model of a Metareasoning Agent Adapted From Cox and Raja, 2007.

Adaptation Scenario Adding a rule: luxury resources on map (e.g. gems, silks) will now contribute to happiness at a city if a city is connected (via roads, railroads) to the resource. This rule is present in Civ3, but not in FreeCiv.

Automation What knowledge is needed to make such an adaptation? Goals – What is the purpose of system components? Procedures – How are the actions that achieve these goals structured? Plus, connections between goals and procedures.

Task-Method-Knowledge (TMK) TMK models provide the agent with knowledge of its own design. TMK encodes:  Tasks (including Subtasks and Primitive Tasks) Requirements and results (goals)  Methods Composition and control (procedures)  Knowledge Domain concepts and relations

Task-Method-Knowledge (TMK) Explicitly connects a functional view of the system with a procedural view. Thus, we have a teleological model that allows a reasoner to move between goals and processes. This process where an agent reasons over and changes its own design is metareasoning – reasoning about reasoning.

Partial TMKL Agent Model ● The “largepox” strategy is the one affected by the change we are considering – “luxuries increase happiness”:

Proposed Proactive Adaptation Process

Outline Overview of metareasoning Gaia – A game-playing metareasoning system. Abstraction Networks (ANs) – Domain knowledge adaptation. MetaCognitive Loop (MCL) – A general- purpose metareasoning shell. Key points

Abstraction Networks Hierarchical classification structures. Annotated with predictive grounding knowledge called Empirical Verification Procedures. Metaknowledge is used for diagnosis; statistical techniques are used for repair.

AN Reference Joshua Jones. Empirically-Based Self-Diagnosis and Repair of Domain Knowledge. Ph.D. Thesis, College of Computing, Georgia Institute of Technology. May Ashok Goel and Joshua Jones. Meta-Reasoning for Self- Adaptation in Intelligent Agents. To appear in Meta- Reasoning, M. Cox and A. Raja (editors). MIT Press, 2011.

Reflection on object-level knowledge What if a metareasoning process such as Gaia identifies a primitive task as the source of error? Primitive tasks are those that are directly implemented by object-level knowledge. Want reflective process to then descend into the task and reason about the knowledge. Take as an example the primitive 'select city build action' task (under ‘build city’).

City Location Quality Subproblem Predict shield (resource) production of a potential city over time VS.

Object-level domain knowledge representation A commonly used pattern of hierarchical classification is “structured matching” (Bylander, et al., 1991) In structured matching, state features are progressively aggregated and abstracted. Structured matching is efficient – linear in space and inference time with respect to input dimension.

FreeCiv City Location Quality Knowledge Structure

Domain Characterization Task: Compositional Classification There is a set of “empirically determinable” features that share mutual information amongst themselves, and with the class label Assume a known independence structure Some features are only determinable after classification, and may have some cost

Empirical Verification Procedures Gaia and REM provide theories of the kind of knowledge needed for metalevel reasoning over process – but what metaknowledge is needed for metareasoning over object-level knowledge? Use metaknowledge that grounds the semantics of object-level knowledge in perception. Can be exploited by the reflective layer to perform diagnosis over the object-level knowledge hierarchy

Inference – observe feature values at leaves

Inference – observe feature values at leaves – use local knowledge at each node with observed children to predict the value of the associated feature

Inference – observe feature values at leaves – use local knowledge at each node with observed children to predict the value of the associated feature – iterate the step above for all nodes with predicted or observed children

Learning Exploit the empirical determinability of the represented features Learning technique used within nodes is flexible, as is the internal knowledge representation Different learning progressions based upon other design choices and problem characteristics; in all cases the EVPs are key in assigning fault to specific nodes.

Learning – Nodes are recursively traversed from root, descending until verification succeeds

Learning – Nodes are recursively traversed from root, descending until verification succeeds

Learning – Nodes are recursively traversed from root, descending until verification succeeds

Experimentation Rote “table update” learners, neural networks and kNN learners have been placed within nodes in our knowledge structures. We have experimented in synthetic domains, in a variety of degraded settings, as well as in the FreeCiv example described, financial prediction and sports prediction. Some positive results with several domain/learner configurations, please see papers for detail. Some theoretical results also.

Metareasoning Architecture Ground-level actions may be in service of reflective layer diagnosis; thus the metareasoning level may in some cases drive action selection.

Metareasoning Architecture Meta-knowledge here takes the form of an explicit representation of object-level knowledge semantics via connection to ground level percepts.

Metareasoning Architecture Meta-knowledge is distributed over object- level knowledge structures, not confined to separate metareasoning level representations.

Outline Overview of metareasoning Gaia – A game-playing metareasoning system. Abstraction Networks (ANs) – Domain knowledge adaptation. MetaCognitive Loop (MCL) – A general- purpose metareasoning shell. Key points

MetaCognitive Loop MCL is a general-purpose shell that can be “attached” to various object-level reasoners. Some interfacing knowledge must be authored. MCL uses only weak models (vs. previous methods) and lightweight inference.

MCL Reference Matthew Schmill, Michael Anderson, Scott Fults, Darsana Josyula, Tim Oates, Donald Perlis, Hamid Haidarian Shahri, Shomir Wilson, Dean Wright. The Metacognitive Loop and Reasoning about Anomalies. In Cox, M., Raja, A. (Ed.), Metareasoning: Thinking about Thinking. MIT Press, MA, USA, 2011.

MCL Overview MCL performs retrospective adaptation or intervention. MCL uses expectations embedded in object- level code to detect failure. Expectation failures are abstracted to indications, which are mapped to failures (causes) and finally to responses. Responses must then be made concrete and implemented by the object-level host.

The MCL Ontologies and Fringes. (Figure due to Schmill)

Experimentation MCL has been tested with example cases in domains such as: – Video games (Bolo) – Robotics domains (Simulated/Actual) – Language domains (Chatbots/Alfred) The goal is to establish generality.

Outline Overview of metareasoning Gaia – A game-playing metareasoning system. Abstraction Networks (ANs) – Domain knowledge adaptation. MetaCognitive Loop (MCL) – A general- purpose metareasoning shell. Key points

Key Points Metareasoning can be used to monitor, repair and guide the processing of a system. It is a particular approach that separates this processing distinctly from the object-layer. It may take many forms: – Proactive vs. Retrospective – “Strong model” vs. “Weak model” – Process vs. Knowledge

Key Points In each case we have seen, predictive knowledge is used. These predictions are directly tied to the function of the associated knowledge or procedural structure. This predictive annotation can be seen as grounding the attached structure – and this grounding makes it possible to reason about the annotated entity.