Learning Agents Center George Mason University Computer Science Department Partners Day Symposium May 4, 2004 Gheorghe Tecuci, Mihai Boicu, Dorin Marcu, Bogdan Stanescu, Cristina Boicu, Marcel Barbulescu

Knowledge Representation, Reasoning, and Learning Overview Experiments of Agent Development and Use Long Term Research Vision Acknowledgements Research Problem, Approach, and Application

The knowledge engineer attempts to understand how the subject matter expert reasons and solves problems and then encodes the acquired expertise into the agent's knowledge base. This modeling and representation of expert’s knowledge is long, painful and inefficient (known as the “knowledge acquisition bottleneck”). How are agents built and why it is hard Knowledge Engineer Domain Expert Knowledge Base Inference Engine Intelligent Agent Programming Dialog Results

The expert teaches the agent how to perform various tasks in a way that resembles how the expert would teach a person. 1. Mixed-initiative problem solving 2. Teaching and learning 3. Multistrategy learning Interface Problem Solving Learning Ontology + Rules Research Problem and Approach Research Problem: Elaborate a theory, methodology and family of systems for the development of knowledge-base agents by subject matter experts, with limited assistance from knowledge engineers. Approach: Develop a learning agent that can be taught directly by a subject matter expert while solving problems in cooperation. The agent learns from the expert, building, verifying and improving its knowledge base

Knowledge bases and agent development by subject matter experts, using learning agent technology. Experiments in the USAWC courses. Formalization of the Center of Gravity (COG) analysis process 319jw Case Studies in Center of Gravity Analysis Use of Disciple in a sequence of two joint warfighting courses 589jw Military Applications of Artificial Intelligence Students developed scenarios Students developed agents Synergistic collaboration and transition to the USAWC George Mason University - US Army War College Artificial Intelligence Research Military Strategy Research Military Education & Practice Disciple

Government Military People Economy Alliances Etc. Which are the critical capabilities? Are the critical requirements of these capabilities satisfied? If not, eliminate the candidate. If yes, do these capabilities have any vulnerability? Application to current war scenarios (e.g. War on terror, Iraq) with state and non-state actors (e.g. Al Qaeda). Identify potential primary sources of moral or physical strength, power and resistance from: Test each identified COG candidate to determine whether it has all the necessary critical capabilities: Identify COG candidatesTest COG candidates Sample Domain: Center of Gravity Analysis Centers of Gravity: Primary sources of moral or physical strength, power or resistance of the opposing forces in a conflict.

Problem Solving Approach: Task Reduction A complex problem solving task is performed by: successively reducing it to simpler tasks; finding the solutions of the simplest tasks; successively composing these solutions until the solution to the initial task is obtained. Object Ontology Reduction Rules Composition Rules Knowledge Base

Question Which is a member of ?O1 ? Answer ?O2 IF Identify and test a strategic COG candidate corresponding to a member of the ?O1 THEN Identify and test a strategic COG candidate for ?O2 US_1943 Which is a member of Allied_Forces_1943? We need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Therefore we need to EXAMPLE OF REASONING STEP IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1 THEN Identify and test a strategic COG candidate for a force The force is ?O2 Plausible Upper Bound Condition ?O1ismulti_member_force has_as_member ?O2 ?O2 isforce Plausible Lower Bound Condition ?O1isequal_partners_multi_state_alliance has_as_member ?O2 ?O2issingle_state_force Identify and test a strategic COG candidate for US_1943 Problem Solving and Learning LEARNED RULE FORMAL STRUCTURE INFORMAL STRUCTURE ONTOLOGY FRAGMENT

Disciple Agent KB Problem solving Disciple was taught based on the expertise of Prof. Comello in center of gravity analysis. Disciple helps the students to perform a center of gravity analysis of an assigned war scenario. Teaching Learning The use of Disciple is an assignment that is well suited to the course's learning objectives Disciple should be used in future versions of this course Use of Disciple at the US Army War College 319jw Case Studies in Center of Gravity Analysis Disciple helped me to learn to perform a strategic COG analysis of a scenario Global evaluations of Disciple by officers from the Spring 03 course

Use of Disciple at the US Army War College 589jw Military Applications of Artificial Intelligence course Students teach Disciple their COG analysis expertise, using sample scenarios (Iraq 2003, War on terror 2003, Arab-Israeli 1973) Students test the trained Disciple agent based on a new scenario (North Korea 2003) I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer Spring 2001 COG identification Spring 2002 COG identification and testing Spring 2003 COG testing based on critical capabilities Global evaluations of Disciple by officers during three experiments

Extended KB stay informed be irreplaceable communicate be influential Integrated KB Initial KB have support be protected be driving force 432 concepts and features, 29 tasks, 18 rules For COG identification for leaders 37 acquired concepts and features for COG testing COG identification and testing (leaders) Domain analysis and ontology development (KE+SME) Parallel KB development (SME assisted by KE) KB merging (KE) Knowledge Engineer (KE) All subject matter experts (SME) DISCIPLE-COG Training scenarios: Iraq 2003 Arab-Israeli 1973 War on Terror 2003 Team 1 Team 2Team 3Team 4Team 5 5 features 10 tasks 10 rules Learned features, tasks, rules 14 tasks 14 rules 2 features 19 tasks 19 rules 35 tasks 33 rules 3 features 24 tasks 23 rules Unified 2 features Deleted 4 rules Refined 12 rules Final KB: +9 features  478 concepts and features +105 tasks  134 tasks +95 rules  113 rules DISCIPLE-COG Testing scenario: North Korea 2003 Correctness = 98.15% 5h 28min average training time / team 3.53 average rule learning rate / team Parallel development and merging of knowledge bases

Disciple-WA ( ): Estimates the best plan of working around damage to a transportation infrastructure, such as a damaged bridge or road. Demonstrated that a knowledge engineer can use Disciple to rapidly build and update a knowledge base capturing knowledge from military engineering manuals and a set of sample solutions provided by a subject matter expert. Disciple-COA ( ): Identifies strengths and weaknesses in a Course of Action, based on the principles of war and the tenets of army operations. Demonstrated the generality of its learning methods that used an object ontology created by another group (TFS/Cycorp). Demonstrated that a knowledge engineer and a subject matter expert can jointly teach Disciple. Other Disciple agents

Disciple’s vision on the future of software development Mainframe Computers Software systems developed and used by computer experts Personal Computers Software systems developed by computer experts and used by persons that are not computer experts Learning Agents Software systems developed and used by persons that are not computer experts

Vision on the use of Disciple in Education teaches Disciple Agent KB The expert/teacher teaches Disciple through examples and explanations, in a way that is similar to how the expert would teach a student. teaches Disciple Agent KB teaches Disciple Agent KB … Disciple tutors the student in a way that is similar to how the expert/teacher has taught it. teaches Disciple Agent KB

This research was sponsored by the Defense Advanced Research Projects Agency, Air Force Research Laboratory, Air Force Material Command, USAF under agreement number F , by the Air Force Office of Scientific Research under grant number F and by the US Army War College. Acknowledgements