 2004, G.Tecuci, Learning Agents Center CS 785 Fall 2004 Learning Agents Center and Computer Science Department George Mason University Gheorghe Tecuci.

 2004, G.Tecuci, Learning Agents Center CS 785 Fall 2004 Learning Agents Center and Computer Science Department George Mason University Gheorghe Tecuci tecuci@gmu.edu http://lac.gmu.edu/

 2004, G.Tecuci, Learning Agents Center Overview A Disciple agent for center of gravity analysis Limits of the classical knowledge engineering approaches Advanced approaches to agent development Recommended readings Research problems and research visions Demo: Training a Disciple agent Design principles for instructable agents Demo: Use of a Disciple agent as a decision-making assistant Learning agent shells

 2004, G.Tecuci, Learning Agents Center How are agents built A knowledge engineer attempts to understand how a subject matter expert reasons and solves problems and then encodes the acquired expertise into the agent's knowledge base. The expert analyzes the solutions generated by the agent (and often the knowledge base itself) to identify errors, and the knowledge engineer corrects the knowledge base.

 2004, G.Tecuci, Learning Agents Center Limited ability to reuse previously developed knowledgeThe knowledge acquisition bottleneckThe knowledge maintenance bottleneckThe scalability of the agent building processFinding the right balance between using general tools and developing domain specific modules Portability of the tools and of the developed agents Limiting factors in developing intelligent agents

 2004, G.Tecuci, Learning Agents Center Advanced approaches to KB and agent development Limited ability to reuse previously developed knowledge Problem: Ontology reuse (import, merge, export, OKBC protocol, CYC) Solution: Example: Ontologies of military units and equipment developed for a particular military planning agent could be reused by a course of action critiquing agent or other military agent.

 2004, G.Tecuci, Learning Agents Center The knowledge acquisition bottleneck Problem: Automation of knowledge acquisition through machine learning Solution: Example: A subject matter expert teaching an agent through examples and explanations, similarly to how the expert would teach an apprentice. Advanced approaches to KB and agent development

 2004, G.Tecuci, Learning Agents Center The knowledge maintenance bottleneck Problem: Use of machine learning methods by the agent, to continuously update its knowledge base in response to changes in the application domain or in the requirements of the system. Solution: Example: A subject matter expert providing feedback to the agent and guiding it to update its knowledge base. Remark: Software maintenance is estimated to be about four times more expensive that software development. With learning agents that are directly taught by humans, there is no longer a distinction between building the agent and maintaining it. Advanced approaches to KB and agent development

 2004, G.Tecuci, Learning Agents Center Finding the right balance between using general tools and developing domain specific modules Problem: Customizable learning agent shell. It is applicable to a wide variety of application domains. Requires limited customization. Solution: Example: Disciple learning agent shell Advanced approaches to KB and agent development

 2004, G.Tecuci, Learning Agents Center Expert system shell Problem Solving Engine Expert System Shell Empty Knowledge Base An expert system is a system that can help solve complex, real-world problems, in specific scientific, engineering, medical specialties, etc., by using large bodies of domain knowledge (facts and procedures) obtained from human experts, that have proven useful for solving typical problems in their domain. An expert system shell is a system that consists of an inference engine for a certain class of tasks (like planning, design, diagnosis, monitoring, prediction, interpretation, etc.) and supports representation formalisms in which a knowledge base can be encoded. If the inference engine is adequate for a certain expert task (e.g. planning), then the process of building the expert system is reduced to the building of the knowledge base.

 2004, G.Tecuci, Learning Agents Center Learning agent shell: definition A learning agent shell is a tool for building agents. It contains a general problem solving engine, a learning engine and an empty knowledge base structured into an object ontology and a set of rules. Building an agent for a specific application consists in customizing the shell for that application and in developing the knowledge base. The learning engine facilitates the building of the knowledge base by subject matter experts and knowledge engineers. Interface Problem Solving Learning Ontology + Rules

 2004, G.Tecuci, Learning Agents Center The Disciple learning agent shell: - can use imported ontological knowledge; - solves problems through task reduction; - can be taught directly by subject matter experts to become a knowledge-based assistant. Mixed-initiative reasoning between the expert that has the knowledge to be formalized and the agent that knows how to formalize it. Disciple learning agent shell The expert teaches the agent to perform various tasks in a way that resembles how the expert would teach a person. The agent learns from the expert, building, verifying and improving its knowledge base Interface Problem Solving Learning Ontology + Rules

 2004, G.Tecuci, Learning Agents Center The complex knowledge engineering activities, traditionally performed by a knowledge engineer with assistance from a subject matter expert, are replaced with equivalent ones performed by the subject matter expert and a learning agent, through mixed-initiative reasoning, and with limited assistance from the knowledge engineer. Define domain model Create ontology Define rules Verify and update rules KE SME Traditionally KE Agent SMEAgent SME Specify instances Learn ontological elements Import and create initial ontology Agent Learn rules SME Agent Define and explain examples SME AgentSMEAgent Critique examples Refine rules Explain critiques SME Agent Extend domain model SME KE Define initial model With Disciple Main idea of the Disciple mixed-initiative approach

 2004, G.Tecuci, Learning Agents Center Disciple-WA (1997-1998): Estimates the best plan of working around damage to a transportation infrastructure, such as a damaged bridge or road. A Disciple agent for action planning Disciple-WA demonstrated that a knowledge engineer can use Disciple to rapidly build and update a knowledge base capturing knowledge from military engineering manuals and a set of sample solutions provided by a subject matter expert. Evolution of KB coverage and performance from the pre-repair phase to the post-repair phase. Development of Disciple’s KB during evaluation. 72% increase of KB size in 17 days High knowledge acquisition rate; High problem solving performance (including unanticipated solutions). Demonstrated at EFX’98 as part of an integrated application led by Alphatech. Disciple-WA features:

 2004, G.Tecuci, Learning Agents Center Disciple-COA (1998-1999): Identifies strengths and weaknesses in a Course of Action, based on the principles of war and the tenets of army operations. A Disciple agent for course of action critiquing Development of Disciple’s KB during evaluation. 46% increase of KB size in 8 days Evolution of KB coverage and performance from the pre-repair phase to the post-repair phase for the final 3 evaluation items. Disciple-COA demonstrated the generality of its learning methods that used an object ontology created by another group (TFS/Cycorp). It also demonstrated that a knowledge engineer and a subject matter expert can jointly teach Disciple. High knowledge acquisition rate; Better performance than the other evaluated systems; Better performance than the evaluating experts (many unanticipated solutions). Disciple-COA features: 100%

 2004, G.Tecuci, Learning Agents Center KB development during experimentation. KB extended with 26 rules and 28 tasks in 3 hours Showed that a subject matter expert, who does not have prior knowledge engineering experience, can be rapidly trained to teach Disciple to critique COAs, based on a given model of the COA critiquing process. IKB Disciple COA Questionnaire All comments consider the fact that Disciple is a research prototype. Degree of agreement with a statement: 1 (not at all) to 5 (very). Do you think that Disciple is a useful tool for Knowledge Acquisition? Do you think that Disciple is a useful tool for Problem Solving? Do you think that Disciple has potential to be used in other tasks you do? Were the procedures/ processes used in Disciple compatible with Army doctrine and/or decision making processes? Could Disciple be used to support operational requirements in your organization? QuestionsAnswers Rating 5. Absolutely! The potential use of this tool by domain experts is only limited by their imagination - not their AI programming skills. 5 4 Yes, it allowed me to be consistent with logical thought. Rating 5. Yes, Absolutely! I’ll take 10 of them! 5 Not at this point of development. Rating 5. As a minimum yes, as a maximum—better! This again was done very well. 4 Rating 5. Again the use of the tool is only limited to one’s imagination but potential applications include knowledge bases built to support distance/individual learning, a multitude of decision support tools (not just COA Analysis), and autonomous and semi-autonomous decision makers - all these designed by the domain expert vs an AI programmer. Absolutely. It can be used to critique any of the BOS's for any mission. 5 Yes 4 Rating 5. Yes. 5 (absolutely) 4 Yes. As it develops and becomes tailored to the user, it will simplify the tedious tasks. LTC John N. Duquette LTC Jay E. Farwell MAJ Michael P. Bowman MAJ Dwayne E. Ptaschek Knowledge acquisition experiment at BCBL, Ft. Leavenworth

 2004, G.Tecuci, Learning Agents Center KBs Integration Integrated KB KB1 Disciple-RKF Assistant Disciple-RKF Assistant Problem solver for a non-expert Tutor to a student Assistant of an expert KBn Disciple-RKF Assistant... Expert Parallel Agent Training and KB Development Agent Use Three agent training and knowledge bases development experiments (2001, 2002, 2003). Knowledge bases integration experiment at the US Army War College (2003). Disciple agents regularly used in two courses at US Army War College (2001-2004). Each SME teaches a personal Disciple- RKF learning agent how to solve problems, in a way that resembles how the expert would teach a human apprentice. The mediator team integrates the knowledge bases developed by each subject matter expert and personal Disciple-RKF agent. Disciple-RKF with the integrated KB is used in practical applications. Successful experiments and transition to the US Army War College Goal: Develop the technology that enables teams of subject matter experts to build integrated knowledge bases and agents incorporating their problem solving expertise. Disciple-RKF: An agent for center of gravity analysis

 2004, G.Tecuci, Learning Agents Center Knowledge bases and agent development by subject matter experts, using learning agent technology. Experiments in the USAWC courses. Formalization of the Center of Gravity (COG) analysis process 319jw Case Studies in Center of Gravity Analysis Use of Disciple in a sequence of two joint warfighting courses 589jw Military Applications of Artificial Intelligence Students developed scenarios Students developed agents Synergistic collaboration and transition at the USAWC George Mason University - US Army War College Artificial Intelligence Research Military Strategy Research Military Education & Practice Disciple

 2004, G.Tecuci, Learning Agents Center The center of gravity of an entity (state, alliance, coalition, or group) is the foundation of capability, the hub of all power and movement, upon which everything depends, the point against which all the energies should be directed. Carl Von Clausewitz, On War, 1832. If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster. Giles and Galvin, USAWC 1996. Sample Domain: Center of Gravity Analysis The center of gravity of an entity is its primary source of moral or physical strength, power or resistance. Joe Strange, Centers of Gravity & Critical Vulnerabilities, 1996.

 2004, G.Tecuci, Learning Agents Center Government Military People Economy Alliances Etc. Which are the critical capabilities? Are the critical requirements of these capabilities satisfied? If not, eliminate the candidate. If yes, do these capabilities have any vulnerability? Approach to center of gravity analysis based on the concepts of critical capabilities, critical requirements and critical vulnerabilities, which have been recently adopted into the joint military doctrine. Identify potential primary sources of moral or physical strength, power and resistance from: Test each identified COG candidate to determine whether it has all the necessary critical capabilities: Identify COG candidatesTest COG candidates First computational approach to COG analysis

 2004, G.Tecuci, Learning Agents Center Is guided by Disciple to describe the relevant aspects of a strategic environment. Studies the logic behind COG identification and testing. Critiques Disciple’s analysis and finalizes the analysis report. Develops a formal representation of the scenario.Identifies and tests strategic COG candidates. Generates a COG analysis report. Disciple Student Student – Disciple collaboration

 2004, G.Tecuci, Learning Agents Center The student is guided by Disciple to describe the relevant aspects of a strategic environment.

 2004, G.Tecuci, Learning Agents Center Disciple identifies and tests COG candidates The students study the logic behind COG identification and testing

 2004, G.Tecuci, Learning Agents Center Disciple generates a COG analysis report

 2004, G.Tecuci, Learning Agents Center War on Terror 2003 LTC Thomas T. Smith LTC Joseph P. Schweitzer LTC Michael S. Yarmie CDR John J. Welsh Iraq 2003 Israel-PLO 2003 COL Christian E. de Graff LTC Robert D. Grymes North Korea 2003 COL Douglas J. Lee COL Robert F. Barry North Korea: military of North Korea US Led Coalition: will of the people of United States Al Qaeda 2003: Terrorist Cells of Al Qaeda Muslim non-state actors neutral to Al Qaeda US Coalition 2003: will of the people of US Muslim non-state actors neutral to Al Qaeda Iraq: Saddam Hussein US led coalition: will of the people of United States will of the people of Great Britain Israel: financial capacity of Israel Palestine: external support from Arab Countries to Palestine Liberation Organization Spring 2003 scenarios and COGs selected

 2004, G.Tecuci, Learning Agents Center Demonstration Disciple Strategic leader’s assistant

Structure the architecture into a reusable domain-independent learning agent shell and domain specific modules Generality-Power Tradeoff Disciple Agent Domain Independent Modules Domain Dependent Plug-in Modules Learning Agent Shell Graphical User Interface Customized User Interface Customized Problem Solver Problem Solver Knowledge Acquisition and Learning Knowledge Base Manager Knowledge Repository

Cognitive Functions Implement each cognitive module as a collaborative agent, in a mixed-initiative framework Make separate modules for each cognitive function, such as communication, problem solving, learning, and knowledge management Disciple: each module is implemented as a set of collaborative agents

Problem Solving Paradigm Use a general problem solving paradigm, that can be applied to a wide range of application domains and develop a methodology to help the subject matter experts express their reasoning and teach the agent using it Disciple: the task reduction paradigm A complex problem solving task is performed by: successively reducing it to simpler tasks; finding the solutions of the simplest tasks; successively composing these solutions until the solution to the initial task is obtained. S 1 S 11 S 1n S 111 S 11m T 11m T 111 T 1n T 11 T1T1 … …

 2004, G.Tecuci, Learning Agents Center Question-answering based task reduction S 1 S 11a S 1n S 11b1 S 11bm T 11bm T 11b1 T 1n T 11a … … T1T1 Q1Q1 S 11b T 11b A 1n S 11 A 11 … … A 11b1 A 11bm S 11b Q 11b Let T1 be the problem solving task to be performed. Finding a solution is an iterative process where, at each step, we consider some relevant information that leads us to reduce the current task to a simpler task or to several simpler tasks. The question Q associated with the current task identifies the type of information to be considered. The answer A identifies that piece of information and leads us to the reduction of the current task.

Rule_3 Rule_2 Task Reduction Example: COG Analysis Rule_1 Rule_4

Rules Knowledge Base Structuring Structure the knowledge base into its more general and reusable components, and its more specific components Disciple: separation between the ontology that defines the concepts and features from an application domain (which is a more general component and may be reused from existing knowledge repositories) the set of problem solving rules (which is a more specific component and is learned from the subject matter expert) Knowledge Base Ontology

Disciple: Ontology Fragment A hierarchical representation of the objects and types of objects. A hierarchical representation of the types of features.

Disciple: Example of a Task Reduction Rule Question Which is a member of ?O1 ? Answer ?O2 INFORMAL STRUCTURE IF Identify and test a strategic COG candidate corresponding to a member of the ?O1 THEN Identify and test a strategic COG candidate for ?O2 US_1943 Which is a member of Allied_Forces_1943? We need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Therefore we need to EXAMPLE OF REASONING STEP FORMAL STRUCTURE IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 THEN Identify and test a strategic COG candidate for a force The force is ?O2 Plausible Upper Bound Condition ?O1ismulti_member_force has_as_member ?O2 ?O2 isforce Plausible Lower Bound Condition ?O1isequal_partners_multi_state_alliance has_as_member ?O2 ?O2issingle_state_force Identify and test a strategic COG candidate for US_1943 LEARNED RULE

Partially Learned Knowledge Universe of Instances Concept Plausible Upper Bound Plausible Lower Bound Plausible version space (PVS) Allow the representation, use, and refinement of partially learned knowledge Disciple: use of plausible version spaces (PVS) to represent and use partially learned knowledge: Rules with PVS conditions Tasks with PVS conditions Features with the domain and range represented as PVS conditions

Integrated Problem Solving and Learning Develop a methodology where the subject matter expert and the agent solve problems in cooperation and the agent learns from the problem solving contributions of the expert, and from its own problem solving attempts

Disciple: Problem-Solving and Learning US_1943 Identify and test a strategic COG candidate for US_1943 Which is a member of Allied_Forces_1943? We need to Therefore we need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Provides an example 1 Rule_4 Learns 2 Rule_4 ? Applies Germany_1943 Identify and test a strategic COG candidate for Germany_1943 Which is a member of European_Axis_1943? Therefore we need to 3 Accepts the example 4 Rule_4 Refines 5 We need to Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943 … Modeling Learning Problem Solving Refining

Integrated Teaching and Learning Develop a methodology where the subject matter expert helps the agent to learn (e.g. by providing examples, hints and explanations), and the agent helps the subject matter expert to teach it (e.g. by asking relevant questions)

 2004, G.Tecuci, Learning Agents Center Find an explanation of why the example is correct US_1943 has_as_member Allied_Forces_1943 The explanation is an approximation of the question and the answer, in the object ontology. US_1943 Identify and test a strategic COG candidate for US_1943 Which is a member of Allied_Forces_1943? We need to Therefore we need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943

 2004, G.Tecuci, Learning Agents Center Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943 Identify and test a strategic COG candidate for a force The force is US_1943 We need to Therefore we need to Generate the PVS rule Condition ?O1 is Allied_Forces_1943 has_as_member ?O2 ?O2 is US_1943 Most general generalization IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 THEN Identify and test a strategic COG candidate for a force The force is ?O2 Plausible Upper Bound Condition ?O1ismulti_member_force has_as_member ?O2 ?O2 isforce Plausible Lower Bound Condition ?O1isequal_partners_multi_state_alliance has_as_member ?O2 ?O2issingle_state_force explanation ?O1 has_as_member ?O2 Most specific generalization US_1943 has_as_member Allied_Forces_1943 Rewrite as has_as_member domain: multi_member_force range: force

Multistrategy Learning Integrate several learning strategies, taking advantage of their complementary strengths to compensate for each other’s weaknesses

Disciple End to End Agent Development Methodology

 2004, G.Tecuci, Learning Agents Center Demonstration Disciple Demo Teaching Disciple how to determine whether a strategic leader has the critical capability to be protected.

 2004, G.Tecuci, Learning Agents Center Elaborate a theory, methodology and system for the development of knowledge bases and agents by subject matter experts, with limited assistance from knowledge engineers. Intelligent Agent Knowledge Base Present research problem

 2004, G.Tecuci, Learning Agents Center 1. Automating the domain modeling process that consists of making explicit, at an informal level, the way the expert solves problems. 4. Learning complex problem solving rules directly from a subject matter expert. 5. Learning object concepts that extend the generic ontology directly from a subject matter expert. 2. Building the initial generic object ontology through import from external repositories and direct elicitation from a subject matter expert. 3. Populating the generic object ontology with instances and relationships that describe a specific situation or scenario. What are the main technical challenges

 2004, G.Tecuci, Learning Agents Center 1.Develop a general approach to domain modeling that allows a subject matter expert to express the way he or she performs a task based on the task reduction paradigm. 2.Structure the knowledge base into an object ontology that can be imported/reused and a set of problem solving rules that can be learned from a subject matter expert. 3.Develop methods to import/reuse ontological knowledge from previously developed knowledge bases or repositories. 4.Develop a learnable knowledge representation that can express partially learned knowledge and can be used in reasoning. 5.Develop multistrategy learning methods that synergistically integrate several learning strategies. 6.Develop methods for integrated teaching and learning where the SME helps the agent to learn, and the agent helps the SME to teach it. 7.Use of plausible reasoning to hypothesize solutions based on incomplete and partially incorrect knowledge. How are these challenges addressed

 2004, G.Tecuci, Learning Agents Center Learning Agent Modeling Non-disruptive Learning Ontology Elicitation Rule & Ontology Learning Rule & Ontology Refining User Model Learning Exception Handling KB Maintenance Implicit reasoning of human expert Explicit reasoning in natural language Ontology extensions Learned rules, ontology Refined rules, ontology User modelCases, rules Rules w/o exceptions 1. Multistrategy teaching and learning 2. Mixed-initiative problem solving and learning 3. Autonomous (and interactive) multistrategy learning Analogy based methods Explanation based methods Natural Language based methods Abstraction based methods Plausible version spaces Learning from instruction Learning from examples Learning from explanations Learning by analogy Mixed-initiative learning Routine, innovative, inventive, and creative reasoning Automatic inductive learning Case-based learning Abductive learning Ontology discovery KB optimization KB maintenance Research goal: Life-long continuous agent learning 4. KB maintenance and optimization

 2004, G.Tecuci, Learning Agents Center This research aims at changing the way future knowledge-based agents will be built, from being programmed by computer scientists and knowledge engineers, to being taught by subject matter experts and typical computer users. Develop a capability that will allow subject matter experts and typical computer users to build and maintain knowledge bases and agents, as easily as they use personal computers for text processing. Long term research vision

 2004, G.Tecuci, Learning Agents Center Vision on the future of software development Mainframe Computers Software systems developed and used by computer experts Personal Computers Software systems developed by computer experts and used by persons that are not computer experts Learning Agents Software systems developed and used by persons that are not computer experts

 2004, G.Tecuci, Learning Agents Center Vision on the use of Disciple in Education teaches Disciple Agent KB The expert/teacher teaches Disciple through examples and explanations, in a way that is similar to how the expert would teach a student. teaches Disciple Agent KB teaches Disciple Agent KB … Disciple tutors the student in a way that is similar to how the expert/teacher has taught it. teaches Disciple Agent KB

 2004, G.Tecuci, Learning Agents Center Recommended reading G. Tecuci, Building Intelligent Agents, Academic Press, 1998, pp. 13-33. Tecuci G., Boicu M., Boicu C., Marcu D., Stanescu B., Barbulescu M., The Disciple-RKF Learning and Reasoning Agent, submitted to publication, September 2004. Boicu M., Tecuci G., Stanescu B., Marcu D., Barbulescu M., Boicu C., "Design Principles for Learning Agents," in Proceedings of AAAI-2004 Workshop on Intelligent Agent Architectures: Combining the Strengths of Software Engineering and Cognitive Systems, July 26, San Jose, AAAI Press, Menlo Park, CA, 2004. http://lac.gmu.edu/publications/data/2004/2004_Disciple-architecture.pdf

 2004, G.Tecuci, Learning Agents Center CS 785 Fall 2004 Learning Agents Center and Computer Science Department George Mason University Gheorghe Tecuci.

Similar presentations

Presentation on theme: " 2004, G.Tecuci, Learning Agents Center CS 785 Fall 2004 Learning Agents Center and Computer Science Department George Mason University Gheorghe Tecuci."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

 2004, G.Tecuci, Learning Agents Center CS 785 Fall 2004 Learning Agents Center and Computer Science Department George Mason University Gheorghe Tecuci.

Similar presentations

Presentation on theme: " 2004, G.Tecuci, Learning Agents Center CS 785 Fall 2004 Learning Agents Center and Computer Science Department George Mason University Gheorghe Tecuci."— Presentation transcript:

Similar presentations

About project

Feedback