Case Injected Genetic Algorithms for Affordable Human Modeling Start Date: 11/15/02 Sushil J. Louis University of Nevada, Reno John McDonnell SPAWAR San Diego

2 Case Injected Genetic AlgoRithms (CIGARs) combine genetic algorithms and case-based reasoning to address three problems Affordability: saves moneyAffordability: The system will be used for both decision support and training. Dual use saves money and familiarizes trainees with battlefield systems Human Modeling:Human Modeling: CIGAR uses cases captured from humans during decision support, war-gaming, and training to bias genetic algorithm search toward human solutions Quality of opponent: reduces knowledge acquisition costQuality of opponent: CIGAR automatically acquires knowledge (generates cases) by playing against itself. This bootstrapping leads to better quality opponents and reduces knowledge acquisition cost

3 Human Case Injected Genetic AlgoRithms (CIGAR)s combine Genetic Algorithm (GA) search with Case-Based Memory Problem 1Problem 2 CIGAR Technical Objective: Technical Objective: Prototype and validate CIGAR techniques for more robust, more affordable human behavior modeling Problem 3 We think of CIGAR as an optimization engine that acquires cases from problem solving or from humans and learns to increase performance with experience GAs adapt solutions (cases) acquired from previously attempted problems to solve subsequent problems

4 Cigar achieves more affordable decision support and training for naval applications The Real-time Executive Decision Support (REDS) effort at SPAWAR will use CIGAR as an optimization engine in strike force weapon-target pairing. Transition objective: Transition objective: Generate multiple weapon-target pairing options in less than 4 minutes for 20 weapon-target pairs. Include SEAD support and METOC information Platforms Targets and Threats Decision Support Decision Support: Assist decision maker in configuration Training: Training: Provide trainee with quality opponent (strike planner) Decision Support Decision Support: Assist decision maker in time- critical strike Training: Training: Provide trainee with quality opponent ( target/threat configurator)

5 CIGAR is affordable The system uses the same graphical user interface for decision support and for training. Decision makers use the interface to specify solutions to decision problems – these solutions are cases and are acquired as a by- product of operational use. Trainees use the same interface for training/wargaming. This dual use translates into significant cost savings and acquires domain knowledge Fighting Training Motto: Fight as you train, train as you fight CIGAR

6 CIGAR produces high quality solutions System use or operation by humans acquires cases representing domain knowledge. CIGAR also acquires cases as it solves problems generated by a problem generator. This offline knowledge acquisition will lead to better performance for training and for decision support. Problem 1Problem 2 CIGAR Problem 3 Problem Generator Replacing the problem generator with CIGAR, we can evolve quality opponents CIGAR

7 CIGAR acquires knowledge during problem solving Periodically save members of the GA’s population to the case- base –A member of the population is a candidate solution to the problem Periodically inject appropriate cases into the GA’s population replacing low-fitness members Genetic Algorithm Case- base CBR module Preprocessor Candidate solutions Cases Save best individual Inject closest to the best

8 How does CIGAR operate ? Which cases do we inject? –Inject cases that are closest to the current best member of the population. Genetic algorithms usually use binary encodings. For these encodings, our distance metric is therefore Hamming Distance – the number of differing bits. GA theory points to other injection strategies –Probabilistic version: The probability of injection of a case in the case base is inversely proportional to distance from the current best member of the population relative to the distances of other cases. How often should we inject cases? –Takeover time – number of generations needed for an individual to take over the population. P(Case i ) = (l – d i )/∑(l – d j ) Sum over all cases Chromosome length Hamming distance from best member

9 Expected behavior versus actual behavior Number of problems attempted Learning system/CIGAR No learning Quality Number of problems attempted Learning system/CIGAR No learning Time Expected behavior of a learning systemCIGAR behavior on 50 design problems Avg. best fitness found within a max of 30 gener ations Avg. time taken to find best fitness RIGA = Randomly Initialized Genetic Algorithm This performance is with (1) a simple problem generator and (2) a case base that grows large few However, we need to deal with few cases captured from humans and obtain (1) human-like and (2) high-quality solutions

10 CIGAR solutions are similar to injected cases The graph below displays hamming distance as a function of chromosome location. –At a number of locations, CIGAR solutions are more similar to each other Other analysis shows that CIGAR solutions are descendants of injected cases  When injected cases come from humans, CIGAR will tend to produce solutions similar to humans Chromosome position (locus) Avg. hamming distance few and Can injecting a few cases captured from humans result in (1) high-quality and (2) timely solutions?

11 Performance on weapon-target pairing Objective function to maximize Given allocation X, U(X) depends on pilot proficiency with weapon & weapon’s effectiveness on target. V(X) describes marginal effect of using multiple weapons Y(X) depends on routing, SEAD, METOC… effectivenessvalue & risk Note that in this case CIGAR takes decreasing time but there is little difference in quality ?

12 We have built a foundation for delivering on research objectives. Year 1 deliverables: Deliverables related to technical objective –Affordability Deploy a prototype GUI for weapon-target pairing support on the web Demonstrate dual use for decision support and training/war- gaming –Human Modeling A set of tools for case-base analysis fewAnalysis of empirical results from injecting cases acquired from a human expert. Techniques for dealing with few human cases Techniques for combining CIGAR and human cases Deliverables related to transitioning objective –Provide an optimization engine that integrates into the REDS-KSA architecture

13 Years 2 and 3 Related to the technical objective –Prototype and deploy a CIGAR system as the red-force against weapon-target pairing –Demonstrate competent red-force scenario generation against weapon-target pairing –Demonstrate techniques for co-evolving blue-red forces –Test and validate approaches to combining human generated cases with automatically acquired cases Related to the transitioning objective –Demonstrate < 4 minutes for 20 weapon-target pairs with SEAD support/routing and METOC data –Other military applications

14 Simulation System Architecture Physics Gfx Battle Authoring Offense Planning Decision Support Defense Planning Decision Support Comm. Hub Defense CIGAR Offense CIGAR GUIs

15 Questions? Tools being developed

16

17 Solution distribution