Learning Agents Prof. Gheorghe Tecuci Learning Agents Laboratory Computer Science Department George Mason University

Instructable agents: the Disciple approach Overview Learning strategies Instructable agents: the Disciple approach Basic bibliography

Learning strategies Introduction Inductive learning from examples Deductive (explanation-based) learning Analogical learning Abductive learning Multistrategy learning

What is Machine Learning Machine Learning is the domain of Artificial Intelligence which is concerned with building adaptive computer systems that are able to improve their competence and/or efficiency through learning from input data or from their own problem solving experience. What does it mean to improve competence? What does it mean to improve efficiency?

Two complementary dimensions for learning Competence A system is improving its competence if it learns to solve a broader class of problems, and to make fewer mistakes in problem solving. Efficiency A system is improving its efficiency, if it learns to solve the problems from its area of competence faster or by using fewer resources.

The architecture of a learning agent Implements a general problem solving method that uses the knowledge from the knowledge base to interpret the input and provide an appropriate output. Implements learning methods for extending and refining the knowledge base to improve agent’s competence and/or efficiency in problem solving. Learning Agent Input/ Problem Solving Engine Sensors Learning Engine User/ Environment Output/ Ontology Rules/Cases/Methods Knowledge Base Effectors Data structures that represent the objects from the application domain, general laws governing them, actions that can be performed with them, etc.

Learning strategies A Learning Strategy is a basic form of learning characterized by the employment of a certain type of inference (like deduction, induction or analogy) and a certain type of computational or representational mechanism (like rules, trees, neural networks, etc.). Rote learning Learning from instruction Learning from examples Explanation-based learning Conceptual clustering Quantitative discovery Abductive learning Learning by analogy Instance-based learning Case-based learning Neural networks Genetic algorithms and evolutionary computation Reinforcement learning Bayesian learning Multistrategy learning

Successful applications of Machine Learning Learning to recognize spoken words (all of the most successful systems use machine learning); Learning to drive an autonomous vehicle on public highway; Learning to classify new astronomical structures (by learning regularities in a very large data base of image data); Learning to play games; Automation of knowledge acquisition from domain experts; Instructable agents.

government_of_Britain_1943 government_of_Britain_1943 Basic ontological elements: instances and concepts An instance is a representation of a particular entity from the application domain. A concept is a representation of a set of instances. state_government state_government government_of_US_1943 instance_of instance_of government_of_Britain_1943 government_of_US_1943 government_of_Britain_1943 “instance_of” is the relationship between an instance and the concept to which it belongs. “state_government” represents the set of all entities that are governments of states. This set includes “government_of_US_1943” and “government_of_Britain_1943” which are called positive examples. An entity which is not an instance of a concept is called a negative example of that concept.

democratic_government democratic_government Concept generality A concept P is more general than another concept Q if and only if the set of instances represented by P includes the set of instances represented by Q. state_government Example: democratic_government representative_ democracy totalitarian_ government parliamentary_ democracy state_government “subconcept_of” is the relationship between a concept and a more general concept. subconcept_of democratic_government

A generalization hierarchy governing_body ad_hoc_ governing_body established_ governing_body other_type_of_ governing_body state_government group_governing_body feudal_god_ king_government other_state_ government dictator other_ group_ governing_ body democratic_ government monarchy deity_figure government_ of_Italy_1943 representative_ democracy parliamentary_ democracy democratic_ council_ or_board autocratic_ leader totalitarian_ government government_ of_US_1943 government_ of_Britain_1943 chief_and_ tribal_council theocratic_ government police_ state military_ dictatorship fascist_ state religious_ dictatorship theocratic_ democracy communist_ dictatorship religious_ dictatorship government_ of_Germany_1943 government_ of_USSRy_1943

Learning strategies Introduction Inductive learning from Examples Deductive (explanation-based) learning Analogical learning Abductive learning Multistrategy learning

Empirical inductive concept learning from examples Illustration Given Positive examples of cups: P1 P2 ... Negative examples of cups: N1 … Learn A description of the cup concept: has-handle(x), ... Approach: Compare the positive and the negative examples of a concept, in terms of their similarities and differences, and learn the concept as a generalized description of the similarities of the positive examples. This allows the agent to recognize other entities as being instances of the learned concept.

The learning problem Given • a language of instances; • a language of generalizations; • a set of positive examples (E1, ..., En) of a concept • a set of negative examples (C1, ... , Cm) of the same concept • a learning bias • other background knowledge Determine • a concept description which is a generalization of the positive examples that does not cover any of the negative examples Purpose of concept learning: Predict if an instance is an example of the learned concept.

Generalization and specialization rules Indicate various generalizations of the following sentence: Students who have lived in Fairfax for 3 years. Learning a concept from examples is based on generalization and specialization rules: A generalization rule is a rule that transforms an expression into a more general expression; - A specialization rule is a rule that transforms an expression into a less general expression.

Turning constants into variables Climbing the generalization hierarchy Generalization (and specialization) rules Turning constants into variables Climbing the generalization hierarchy Dropping condition Generalizing numbers Adding alternatives

democratic_government representative_democracy parliamentary_democracy Climbing/descending the generalization hierarchy Generalizes an expression by replacing a concept with a more general one. democratic_government representative_democracy parliamentary_democracy The set of single state forces governed by representative democracies ?O1 is single_state_force has_as_governing_body ?O2 ?O2 is representative_democracy generalization specialization representative_democracy  democratic_government democratic_government  representative_democracy The set of single state forces governed by democracies ?O1 is single_state_force has_as_governing_body ?O2 ?O2 is democratic_government

… Basic idea of version space concept learning UB + LB + UB LB _ + UB Consider the examples E1, … , E2 in sequence. UB + Initialize the lower bound to the first positive example (LB=E1) and the upper bound (UB) to the most general generalization of E1. LB + UB LB If the next example is a positive one, then generalize LB as little as possible to cover it. _ + UB LB If the next example is a negative one, then specialize UB as little as possible to uncover it and to remain more general than LB. _ + UB=LB … Repeat the above two steps with the rest of examples until UB=LB. This is the learned concept.

The candidate elimination algorithm Initialize S to the first positive example and G to its most general generalization 2. Accept a new training instance I • If I is a positive example then - remove from G all the concepts that do not cover I; - generalize the elements in S as little as possible to cover I but remain less general than some concept in G; - keep in S the minimally general concepts. • If I is a negative example then - remove from S all the concepts that cover I; - specialize the elements in G as little as possible to uncover I and be more general than at least one element from S; - keep in G the maximally general concepts. 3. Repeat 2 until G=S and they contain a single concept C (this is the learned concept)

Illustration of the candidate elimination algorithm Language of generalizations: (shape, size) shape: {ball, brick, cube, any-shape} size: {large, small, any-size} Language of instances: (shape, size) shape: {ball, brick, cube} size: {large, small} Learning process: G = {(any-shape, any-size)} S = {(ball, large)} +(ball, large) 1 -(brick, small) G = {(ball, any-size) (any-shape, large)} 2 Input examples: ball small + shape size class large brick – cube -(cube, large) G = {(ball, any-size)} 3 +(ball, small) S = {(ball, any-size)} || 4

General features of the empirical inductive methods Require many examples. Do not need much domain knowledge. Improve the competence of the agent. The version space method relies on an exhaustive bi-directional search and is computationally intensive. This limits its practical applicability. Practical empirical inductive learning methods (such as ID3) rely on heuristic search to hypothesize the concept. To learn a good concept description through this learning strategy requires a very large set of positive and negative examples. On the other hand, this is the only information that the agent needs. That is, the agent does not require prior knowledge to perform this type of learning. The result of this learning strategy is the increase of the problem solving competence of the agent. Indeed, the agent will learn to do things it was not able to do before. For instance, it will learn to recognize cups, something that it was not able to do before. Most empirical inductive learning algorithms use heuristic methods to search an incredibly large space of possibilities for the learned concept. The version space method, which does not rely on heuristic search (it performs an exhaustive search), assumes that the representation language is complete, and a single concept exists, has a very important theoretical value, but a more limited practical relevance.

Learning strategies Introduction Inductive learning from Examples Deductive (explanation-based) learning Analogical learning Abductive learning Multistrategy learning

Explanation-based learning problem Given A training example cup(o1) Ü color(o1, white), made-of(o1, plastic), light-mat(plastic), has-handle(o1), has-flat-bottom(o1), up-concave(o1),... Learning goal A specification of the desirable features of the concept to be learned (e.g. the learned concept should have only features from the example) Background knowledge Complete knowledge that allows proving that the training example represents the concept: cup(x) Ü liftable(x), stable(x), open-vessel(x). liftable(x) Ü light(x), graspable(x). stable(x) Ü has-flat-bottom(x). … Determine A concept definition representing a deductive generalization of the training example that satisfies the learning goal. cup(x) Ü made-of(x, y),light-mat(y),has-handle(x),has-flat-bottom(x),up-concave(x). Purpose of learning Improve the problem solving efficiency of the agent.

Explanation-based learning method 1. Construct an explanation that proves that the training example is an example of the concept to be learned. 2. Generalize the explanation as much as possible so that the proof still holds, and extract from it a concept definition that satisfies the learning goal. A example of a cup cup(o1): color(o1, white), made-of(o1, plastic), light-mat(plastic), has-handle(o1), has-flat-bottom(o1), up-concave(o1),... cup(o1) stable(o1) liftable(o1) graspable(o1) light(o1) light-mat(plastic) made-of(o1,plastic) ... has-handle(o1) cup(x) stable(x) liftable(x) graspable(x) light(x) light-mat(y) made-of(x,y) has-handle(x) The proof identifies the characteristic features: Proof generalization generalizes them: • has-handle(o1) is needed to prove cup(o1) • color(o1,white) is not needed to prove cup(o1) • made-of(o1, plastic) is needed to prove cup(o1) • made-of(o1, plastic) is generalized to made-of(x, y); • the material needs not be plastic. The goal of this learning strategy is to improve the efficiency in problem solving. The agent is able to perform some task but in an inefficient way. We would like to teach the agent to perform the task faster. Consider, for instance, an agent that is able to recognize cups. The agent receives a description of a cup that includes many features. The agent will recognize that this object is a cup by performing a complex reasoning process, based on its prior knowledge. This process is illustrated by the proof tree from the left hand side of this slide. The object o1 is made of plastic which is a light material. Therefore o1 is light. o1 has a handle and therefore it is graspable. Being light and graspable, it is liftable. An so on … being liftable, stable and an open vessel, it is a cup. However, the agent can learn from this process to recognize a cup faster. Notice that the agent used the fact that o1 has a handle in order to prove that o1 is a cup. This means that having a handle is an important feature. On the other hand the agent did not use the the color of o1 to prove that o1 is a cup. This means that color is not important. Notice how the agent reaches the same conclusions as in learning from examples, but through a different line of reasoning, and based on a different type of information. The next step in the learning process is to generalize the tree from the left hand side into the tree from the right hand side. While the tree from the left hand side proves that the specific object o1 is a cup, the tree from the right hand side shows that any object x that is made of some light material y, has a handle and some other features is a cup. Therefore, to recognize that an object o2 is a cup, the agent only needs to look for the presence of these features discovered as important. It no longer needs to build a complex proof tree. Therefore cup recognition is done much faster. Finally, notice that the agent needs only one example to learn from. However, it needs a lot of prior knowledge to prove that this example is a cup. Providing such prior knowledge to the agent is a very complex task.

Discussion cup(o1) stable(o1) liftable(o1) graspable(o1) light(o1) light-mat(plastic) made-of(o1,plastic) ... has-handle(o1) cup(x) stable(x) liftable(x) graspable(x) light(x) light-mat(y) made-of(x,y) ... has-handle(x) The proof identifies the characteristic features Proof generalization generalizes them How does this learning method improve the efficiency of the problem solving process? A cup is recognized by using a single rule rather than building a proof tree. Do we need a training example to learn an operational definition of the concept? Why? The learner does not need a training example. It can simply build proof trees from top-down, starting with an abstract definition of the concept and growing the tree until the leaves are operational features. However, without a training example the learner will learn many operational definitions. The training example focuses the learner on the most typical example.

General features of explanation-based learning • Needs only one example • Requires complete knowledge about the concept (which makes this learning strategy impractical). • Improves agent's efficiency in problem solving • Shows the importance of explanations in learning

Learning strategies Introduction Inductive learning from Examples Deductive (explanation-based) learning Analogical learning Abductive learning Multistrategy learning

Learning by analogy Learning by analogy means acquiring new knowledge about an input entity by transferring it from a known similar entity. The hydrogen atom is like our solar system. The Sun has a greater mass than the Earth and attracts it, causing the Earth to revolve around the Sun. The nucleus also has a greater mass then the electron and attracts it. Therefore it is plausible that the electron also revolves around the nucleus.

Discussion Examples of analogies: Pressure Drop is like Voltage Drop A variable in a programming language is like a box. Provide other examples of analogies. Which is the central intuition supporting learning by analogy: If two entities are similar in some respects then they could be similar in other respects as well.

Learning by analogy: the learning problem Given: • A partially known target entity T and a goal concerning it. • Background knowledge containing known entities. Find: • New knowledge about T obtained from a source entity S belonging to the background knowledge. Partially understood structure of the hydrogen atom under study. Knowledge from different domains, including astronomy, geography, etc. In a hydrogen atom the electron revolves around the nucleus, in a similar way in which a planet revolves around the sun.

Learning by analogy: the learning method • ACCESS: find a known entity that is analogous with the input entity. • MATCHING: match the two entities and hypothesize knowledge. • EVALUATION: test the hypotheses. • LEARNING: store or generalize the new knowledge. Based on what is known about the hydrogen atom and the solar system identify the solar system as a source for the hydrogen atom. One may map the nucleus to the sun and the electron to the planet, allowing one to infer that the electron revolves around the nucleus because the nucleus attracts the electron and the mass of the nucleus is greater than the mass of the electron. A specially designed experiment shows that indeed the electron revolves around the nucleus. Store that, in a hydrogen atom, the electron revolves around the nucleus. By generalization from the solar system and the hydrogen atom, learn the abstract concept that a central force can cause revolution.

Discussion How does analogy help? Why not just study the structure of the hydrogen atom to discover that new knowledge? We anyway need to perform an experiment to test that the electron revolves around the hydrogen atom. How could we define problem solving by analogy? How does analogy help to solve new problems?

General features of analogical learning • Requires a huge amount of background knowledge from a wide variety of domains. • Improves agent's competence through knowledge base refinement. • Reduces the solution finding to solution verification. • Is a very powerful reasoning method with incomplete knowledge

Learning strategies Introduction Inductive learning from Examples Deductive (explanation-based) learning Analogical learning Abductive learning Multistrategy learning

Abductive learning The learning problem Illustration: Finds the hypothesis that explains best an observation (or collection of data) and assumes it as being true. Illustration: There is smoke in the East building. Fire causes smoke. Hypothesize that there is a fire in the East building. • Let D be a collection of data • Find all the hypotheses that (causally) explain D • Find the hypothesis H that explains D better than other hypotheses • Assert that H is true The learning method

Abduction (cont.) Consider the observation: “University Dr. is wet” Use abductive learning. Raining causes the streets to be wet. I hypothesize that it has rained on the University Dr. Which are other potential explanations? Provide other examples of abductive reasoning. What real world applications of abductive reasoning can you imagine?

Learning strategies Introduction Inductive learning from Examples Deductive (explanation-based) learning Analogical learning Abductive learning Multistrategy learning

Multistrategy learning Multistrategy learning is concerned with developing learning agents that synergistically integrate two or more learning strategies in order to solve learning tasks that are beyond the capabilities of the individual learning strategies that are integrated.

Learning from examples Complementariness of learning strategies Learning from examples Explanation- based learning Multistrategy learning Examples many one several needed Knowledge complete incomplete knowledge very little needed Type of induction deduction induction and/ or deduction inference Effect on improves competence efficiency competence or/ and efficiency agent's behavior

Instructable agents: the Disciple approach Overview Learning strategies Instructable agents: the Disciple approach Basic bibliography

Instructable agents: the Disciple approach Introduction Modeling expert’s reasoning Rule learning Integrated modeling, learning, solving Application and evaluation Multi-agent problem solving and learning

How are agents built and why it is hard The knowledge engineer attempts to understand how the subject matter expert reasons and solves problems and then encodes the acquired expertise into the system's knowledge base. This modeling and representation of expert knowledge is long, painful and inefficient. Why?

The Disciple approach: Problem statement Elaborate a theory and methodology for the mixed-initiative, end-to-end development of knowledge bases and agents by subject matter experts, with limited assistance from knowledge engineers. The expert teaches the agent to perform various tasks in a way that resembles how the expert would teach a person. The agent learns from the expert, building, verifying and improving its knowledge base Approach Mixed-initiative reasoning between an expert that has the knowledge to be formalized and a learning agent shell that knows how to formalize it. Interface Problem Solving Learning Ontology + Rules How is this approach addresses the knowledge acquisition bottleneck?

Vision on the evolution of computer systems Mainframe Computers Personal Computers Software systems developed by computer experts and used by persons that are not computer experts Learning Agents Software systems developed and used by persons that are not computer experts Software systems developed and used by computer experts

General architecture of Disciple-RKF Intelligent User Interface Teaching, Learning and Problem Solving Modeling Task Learning Scenario Elicitation Mixed Initiative Manager Ontology Editors and Browsers Mixed_Initiative Problem Solving Rule Learning Natural Language Generation Autonomous Problem Solving Rule Refinement Disciple-RKF is the most current shell from the Disciple family. Its architecture is organized into three main components: 1) the intelligent user interface, 2) the teaching, learning and problem solving component, and 3) the knowledge base management component. The intelligent user interface component allows the experts to communicate with Disciple in a manner as close as possible to the way they communicate in their environment. The teaching, learning and problem solving component is responsible for knowledge formation and problem solving, and contains specific modules for domain modeling, rule learning and refinement, as well as for mixed-initiative and autonomous problem solving. The knowledge base management component contains modules for managing the knowledge base of Disciple and for importing knowledge from external knowledge repositories. Knowledge Base Management Ontology Problem Solving Ontology Import Solving Instances Rules Instances Rules

Knowledge base: Object ontology + reasoning rules A hierarchical representation of the types of objects from the COG domain. A hierarchical representation of the types of features of the objects from the COG domain.

Task reduction rule IF: Test whether the will of the people can make a state accept the strategic goal of an opposing force The will of the people is ?O1 The state is ?O2 The opposing force is ?O3 The goal is ?O4 Plausible Upper Bound Condition ?O1 is will_of_agent ?O2 is force has_as_people ?O6 has_as_governing_body ?O5 ?O3 is (strategic_COG_relevant_factor agent) ?O4 is force_goal ?O5 is representative_democracy has_as_will ?O7 ?O6 is people has_as_will ?O1 ?O7 is will_of_agent reflects ?O1 Plausible Lower Bound Condition ?O1 is will_of_the_people_of_US_1943 ?O2 is US_1943 has_as_people ?O6 has_as_governing_body ?O5 ?O3 is European_Axis_1943 ?O4 is Dominance_of_Europe_by_European_Axis ?O5 is government_of_US_1943 has_as_will ?O7 ?O6 is people_of_US_1945 has_as_will ?O1 ?O7 is will_of_the_government_of_US_1943 reflects ?O1 THEN: Test whether the will of the people, that controls the government, can make a state accept the strategic goal of an opposing force The will of the people is ?O1 The government is ?O5 The state is ?O2 The opposing force is ?O3 The goal is ?O4

Teaching of the agent by the subject matter expert. The developed Disciple approach Modeling the problem solving process of the subject matter expert and development of the object ontology of the agent. Teaching of the agent by the subject matter expert.

Instructable agents: the Disciple approach Introduction Modeling expert’s reasoning Rule learning Integrated modeling, learning, solving Application and evaluation Multi-agent problem solving and learning

Application domain Identification of strategic Center of Gravity candidates in war scenarios The center of gravity of an entity (state, alliance, coalition, or group) is the foundation of capability, the hub of all power and movement, upon which everything depends, the point against which all the energies should be directed. Carl Von Clausewitz, “On War,” 1832. If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster.

… Modeling the identification of center of gravity candidates I need to Identify and test a strategic COG candidate for the Sicily_1943 scenario What kind of scenario is Sicily_1943? Sicily_1943 is a war scenario Therefore I need to Identify and test a strategic COG candidate for Sicily_1943 which is a war scenario Which is an opposing force in the Sicily_1943 scenario? Allied_Forces_1943 Therefore I need to Identify and test a strategic COG candidate for Allied_Forces_1943 Is Allied_Forces_1943 a single-member force or a multi-member force? Allied_Forces_1943 is a multi-member force Therefore I need to … Identify and test a strategic COG candidate for Allied_Forces_1943 which is a multi-member force European_Axis_1943 Therefore I need to Identify and test a strategic COG candidate for European_Axis_1943 …

… I need to Test the will of the people of US_1943 which is a strategic COG candidate with respect to the people of US_1943 What is the strategic goal of European_Axis_1943? ‘Dominance of Europe by European Axis’ Therefore I need to Test whether the will of the people of US_1943 can make US_1943 accept the strategic goal of European_Axis_1943 which is ‘Dominance of Europe by European Axis’ Let us assume that the people of US_1943 would accept ‘Dominance of Europe by European Axis’. Could the people of US_1943 make the government of US_1943 accept ‘Dominance of Europe by European Axis’? Yes, because US_1943 is a representative democracy and the government of US_1943 reflects the will of the people of US_1943 Therefore I need to Test whether the will of the people of US_1943 that controls the government of US_1943 can make US_1943 accept the strategic goal of European_Axis_1943 which is ‘Dominance of Europe by European Axis’ Let us assume that the people of US_1943 would accept ‘Dominance of Europe by European Axis’. Could the people of US_1943 make the military of US_1943 accept ‘Dominance of Europe by European Axis’? Yes, because US_1943 is a representative democracy and the will of the military of US_1943 reflects the will of the people of US_1943 Therefore conclude that The will of the people of US_1943 is a strategic COG candidate that cannot be eliminated

Modeling advisor: helping the expert to express his reasoning

Instructable agents: the Disciple approach Introduction Modeling expert’s reasoning Rule learning Integrated modeling, learning, solving Application and evaluation Multi-agent problem solving and learning

Rule learning Natural Language Logic I need to IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Which is a member of Allied_Forces_1943? explanation ?O1 has_as_member ?O2 US_1943 Plausible Upper Bound Condition ?O1 is multi_member_force has_as_member ?O2 ?O2 is force Therefore I need to Identify and test a strategic COG candidate for US_1943 Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance has_as_member ?O2 ?O2 is single_state_force IF Identify and test a strategic COG candidate corresponding to a member of the ?O1 THEN Identify and test a strategic COG candidate for a force The force is ?O2 Question Which is a member of ?O1 ? Answer ?O2 FORMAL STRUCTURE OF THE RULE THEN Identify and test a strategic COG candidate for ?O2 Rule learning INFORMAL STRUCTURE OF THE RULE

1. Formalize the tasks I need to I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943 Therefore I need to Therefore I need to Identify and test a strategic COG candidate for US_1943 Identify and test a strategic COG candidate for a force The force is US_1943

Task formalization interface

2. Find an explanation of why the example is correct I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Which is a member of Allied_Forces_1943? US_1943 Therefore I need to Identify and test a strategic COG candidate for US_1943 The explanation is the best possible approximation of the question and the answer, in the object ontology. has_as_member Allied_Forces_1943 US_1943

Explanation generation and selection interface

3. Generate the plausible version space rule has_as_member Allied_Forces_1943 US_1943 IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 Rewrite as explanation ?O1 has_as_member ?O2 Most general generalization Plausible Upper Bound Condition ?O1 is multi_member_force has_as_member ?O2 ?O2 is force Condition ?O1 is Allied_Forces_1943 has_as_member ?O2 ?O2 is US_1943 Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance has_as_member ?O2 ?O2 is single_state_force Most specific generalization THEN Identify and test a strategic COG candidate for a force The force is ?O2 Which is the justification for these generalizations?

Analogical reasoning ?O2 has_as_ member ?O1 force multi_member_force similar explanation less general than Analogy criterion ?O2 has_as_ member ?O1 force multi_member_force instance_of Germany_1943 European_Axis_1943 similar initial example explanation explains Identify and test a strategic COG candidate for a force The force is US_1943 I need to Therefore I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943 US_1943 has_as_ member Allied_Forces_1943 similar example explains? similar Identify and test a strategic COG candidate for a force The force is Germany_1943 I need to Therefore I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is European_Axis_1943

Generalization by analogy explains ?O2 has_as_ member ?O1 force multi_member_force instance_of Identify and test a strategic COG candidate for a force The force is ?O2 I need to Therefore I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 initial example explains Identify and test a strategic COG candidate for a force The force is US_1943 I need to Therefore I need to Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943 US_1943 has_as_ member Allied_Forces_1943 Any value of ?O1 should be an instance of: DOMAIN(has_as_member)  RANGE(The force is) = multi_member_force  force = multi_member_force Any value of ?O2 should be an instance of: RANGE(has_as_member)  RANGE(The force is) = force  force = force Knowledge-base constraints on the generalization:

Instructable agents: the Disciple approach Introduction Modeling expert’s reasoning Rule learning Integrated modeling, learning, solving Application and evaluation Multi-agent problem solving and learning

Mixed-Initiative Problem Solving Control of modeling, learning and solving Input Task Mixed-Initiative Problem Solving Ontology + Rules Generated Reduction New Reduction Accept Reduction Reject Reduction This slide shows the interaction between the expert and the agent when the agent has already learned some rules. This interaction is governed by the mixed-initiative problem solver. The expert formulates the initial task. Then the agent attempts to reduce this task by using the previously learned rules. Let us assume that the agent succeeded to propose a reduction to the current task. The expert has to accept it if it is correct, or he has to reject it, if it is incorrect. If the reduction proposed by the agent is accepted by the expert, the rule that generated it and its component tasks are generalized. Then the process resumes, the agent attempting to reduce the new task. If the reduction proposed by the agent is rejected, then the agent will have to specialize the rule, and possibly its component tasks. In this case the expert will have to indicate the correct reduction, going through the normal steps of modeling, formalization, and learning. Similarly, when the agent cannot propose a reduction of the current task, the expert will have to indicate it, again going through the steps of modeling, formalization and learning. The control of this interaction is done by the mixed-initiative problem solver tool. Solution Rule Refinement Task Refinement Modeling Rule Refinement Formalization Learning

… 2 1 3 5 4 Learns Provides an example Applies Refines ? US_1943 Identify and test a strategic COG candidate for US_1943 Which is a member of Allied_Forces_1943? I need to Therefore I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Provides an example 1 Rule_15 Learns 2 I need to Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943 … Rule_15 ? Applies Germany_1943 Identify and test a strategic COG candidate for Germany_1943 Which is a member of European_Axis_1943? Therefore I need to 3 Rule_15 Refines 5 Accepts the example 4

Rule refinement with a positive example Positive example that satisfies the upper bound IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 I need to Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943 explanation ?O1 has_as_member ?O2 Therefore I need to Identify and test a strategic COG candidate for Germany_1943 Plausible Upper Bound Condition ?O1 is multi_member_force has_as_member ?O2 ?O2 is force Condition satisfied by positive example ?O1 is European_Axis_1943 has_as_member ?O3 ?O2 is Germany_1943 less general than Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance has_as_member ?O2 ?O2 is single_state_force explanation European_Axis_1943 has_as_member Germany_1943 THEN Identify and test a strategic COG candidate for a force The force is ?O2

Minimal generalization of the plausible lower bound Plausible Upper Bound Condition ?O1 is multi_member_force has_as_member ?O2 ?O2 is force less general than (or at most as general as) New Plausible Lower Bound Condition ?O1 is multi_state_alliance has_as_member ?O2 ?O2 is single_state_force minimal generalization Plausible Lower Bound Condition (from rule) ?O1 is equal_partners_multi_state_alliance has_as_member ?O2 ?O2 is single_state_force Condition satisfied by the positive example ?O1 is European_Axis_1943 has_as_member ?O2 ?O2 is Germany_1943

Forces hierarchy … multi_state_alliance composition_of_forces force single_member_force multi_member_force single_state_force single_group_force multi_state_force multi_group_force … US_1943 Germany_1943 multi_state_alliance multi_state_coalition dominant_partner_ multi_state_alliance dominant_partner_ multi_state_coalition multi_state_alliance is the minimal generalization of equals_partners_multi_state_alliance that covers European_Axis_1943 equal_partners_ multi_state_alliance equal_partners_ multi_state_coalition European_Axis_1943 Allied_Forces_1943

Refined rule less general than IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 explanation ?O1 has_as_member ?O2 explanation ?O1 has_as_member ?O2 Plausible Upper Bound Condition ?O1 is multi_member_force has_as_member ?O2 ?O2 is force Plausible Upper Bound Condition ?O1 is multi_member_force has_as_member ?O2 ?O2 is force less general than Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance has_as_member ?O2 ?O2 is single_state_force Plausible Lower Bound Condition ?O1 is multi_state_alliance has_as_member ?O2 ?O2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ?O2 THEN Identify and test a strategic COG candidate for a force The force is ?O2

Which are some learning strategies used by Disciple? Rule learning Rule refinement

Instructable agents: the Disciple approach Introduction Modeling expert’s reasoning Rule learning Integrated modeling, learning, solving Application and evaluation Multi-agent problem solving and learning

Use of Disciple in Army War College courses 319jw Case Studies in Center of Gravity Analysis (COG), Term II and Term III Students develop input scenarios Students use Disciple as an intelligent assistant that supports them to develop a Center of Gravity analysis report for a war scenario. 589 Military Applications of Artificial Intelligence (MAAI) ,Term III Students develop agents Students act as subject matter experts that teach personal Disciple agents their own reasoning in Center of Gravity analysis.

COG Winter 2002: Expert assessments The use of Disciple is an assignment that is well suited to the course's learning objectives Disciple helped me to learn to perform a strategic center of gravity analysis of a scenario The use of Disciple was a useful learning experience Disciple should be used in future versions of this course

589 MAAI Spring 2002: SME assessment I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer

Instructable agents: the Disciple approach Introduction Modeling expert’s reasoning Rule learning Integrated modeling, learning, solving Application and evaluation Multi-agent problem solving and learning

Modeling, problem solving, and learning as collaborative agents Rules similar with the current example Rule Analogy Engine Explanation Generator New correct examples for learning Rule Analyzer Learning Partially learned rules with informal and formal descriptions Example Composer Mixed Modeling New positive and negative examples for rule refinement Example Analyzer Initiative Problem Solving Analysis of the instantiations for a rule Informal description of the reasoning Mixed-Initiative PS New situations that need to be modeled Reasoning Tree Analyzer Solution Analyzer Reasoning tree similar with the current model

Question-answering based task reduction A complex problem solving task is performed by: successively reducing it to simpler tasks; finding the solutions of the simplest tasks; successively composing these solutions until the solution to the initial task is obtained. S1 Q1 … A11 S11 A1n S1n T1n T11a S11a T11b S11b … Q11b S11b … A11b1 S11b1 … A11bm S11bm T11b1 T11bm Let T1 be the problem solving task to be performed. Finding a solution is an iterative process where, at each step, we consider some relevant information that leads us to reduce the current task to a simpler task or to several simpler tasks. The question Q associated with the current task identifies the type of information to be considered. The answer A identifies that piece of information and leads us to the reduction of the current task.

Intelligence Analysis

PERSONALIZED LEARNING ASSISTANT Information- Interaction A possible multi-agent architecture for Disciple PERSONALIZED LEARNING ASSISTANT EXPERT Information- Interaction Agent Support Agent Support Agent Support Agent Local Shared KB Global KB Expert’s Model Ontology Rules KB Awareness Agent External- Expertise Agent KB KB KB KB EXPERT PERSONALIZED LEARNING ASSISTANT

Bibliography Mitchell T.M., Machine Learning, McGraw Hill, 1997. Shavlik J.W. and Dietterich T. (Eds.), Readings in Machine Learning, Morgan Kaufmann, 1990. Buchanan B., Wilkins D. (Eds.), Readings in Knowledge Acquisition and Learning: Automating the Construction and the Improvement of Programs, Morgan Kaufmann, 1992. Langley P., Elements of Machine Learning, Morgan Kaufmann, 1996. Michalski R.S., Carbonell J.G., Mitchell T.M. (Eds), Machine Learning: An Artificial Intelligence Approach, Morgan Kaufmann, 1983 (Vol. 1), 1986 (Vol. 2). Kodratoff Y. and Michalski R.S. (Eds.) Machine Learning: An Artificial Intelligence Approach (Vol. 3), Morgan Kaufmann Publishers, Inc., 1990. Michalski R.S. and Tecuci G. (Eds.), Machine Learning: A Multistrategy Approach (Vol. 4), Morgan Kaufmann Publishers, San Mateo, CA, 1994. Tecuci G. and Kodratoff Y. (Eds.), Machine Learning and Knowledge Acquisition: Integrated Approaches, Academic Press, 1995. Tecuci G., Building Intelligent Agents: An Apprenticeship Multistrategy Learning Theory, Methodology, Tool and Case Studies, Academic Press, 1998.