1 Interactive Evolutionary Computation: A Framework and Applications November 10, 2009 Sung-Bae Cho Dept. of Computer Science Yonsei University

2 Agenda Overview Methodology –IGA –Knowledge-based encoding –Partial evaluation based on clustering –Direct manipulation of evolution with NK-landscape model Applications –Media retrieval –Media design Concluding remarks

3 User Modeling AI FLGA NN (Soft Computing) Interaction/Evolution Interactive Computational Intelligence Overview Human Interface Computer (Emotion) (Efficiency)

4 Difficulty of optimization problems in interactive system –Outputs must be subjectively evaluated (graphics or music) Definition –Evolutionary computation that optimizes systems based on subjective human evaluation –Fitness function is replaced by a human user Interactive Evolutionary Computation (IEC) Overview Target system f(p1,p2,…,pn) Interactive EC System output Subjective evaluation From Takagi (2001) My goal is …

5 Inherent Problems in IEC Human fatigue problem –Common to all human-machine interaction systems IEC –Acceleration of EC convergence with small population size and a few generation numbers –Usually within 10 or 20 search generations Limitation of the individuals simultaneously displayed on a screen Limitation of the human capacity to memorize the time- sequentially displayed individuals Requirement to minimize human fatigue Overview I’m tired

6 Proposed Framework Overview Conventional IEC Media Retrieval & Design Proposed IEC Media Retrieval & Design Domain Knowledge Partial Evaluation Direct Manipulation

7 Interactive Genetic Algorithm Methodology

8 Knowledge-based Encoding Search Space New Search Space Domain Knowledge Methodology Removing impractical solutions Using partially known information about the final solution Focusing on a specific search space Encoding in a modular manner Effective encoding?

9 Partial Evaluation based on Clustering Methodology Issues  Clustering algorithm selection, Indirect fitness allocation strategy

10 Direct Manipulation of Evolution Methodology IGA Direct Manipulation It is very similar to the final solution but a bit different. Proof of usefulness of the direct manipulation  NK-landscape

11 Overview Applications Media Retrieval Media Design Image Retrieval Video Retrieval Music Retrieval Fashion Design Flower Design Intrusion Pattern Design Humanized Media Retrieval & Design

12 Content-based Image Retrieval Keyword-based method –Much time and labor to construct indexes in a large databases –Performance decrease when index constructor and user have different point of view –Inherent difficulty to describe image as a keyword Content-based method –Image contents as a set of features extracted from image –Specifying queries using the features Applications: Image Retrieval

13 Motivation Systems for content-based image retrieval –QBIC (IBM, 1993) –QVE (Hirata and Kato, 1993) –Chabot (Berkeley, 1994) Needs for image retrieval based on human intuition –Difficult to get perfect expression by queries –Lack of expression capability Applications: Image Retrieval

14 System Structure Applications: Image Retrieval

15 Chromosome Structure Applications: Image Retrieval Original image Transformed imageChromosome RGBRGB

16 Convergence Test The case of gloomy image retrieval Applications: Image Retrieval Initial populationThe 8th population

17 Subjective Test by Sheffe’s Method Applications: Image Retrieval Confidence interval95%99%

18 Motivation Evolution of computing technology (Huge computing storage, networking) –Necessary for management of video data transfer, processing and retrieval Change of view point –Query by keyword  Query by content  Difficult to represent human’s semantic information –Query by semantics Emotion-based retrieval Applications: Video Retrieval

19 Hierarchical Structure of Video Applications: Video Retrieval

20 System Architecture Applications: Video Retrieval

21 Chromosome Structure Applications: Video Retrieval

22 System Interface Applications: Video Retrieval

23 User’s Satisfaction Applications: Video Retrieval

24 Emotional Music Retrieval Music information retrieval  Immature field Query by singing Query by content –Uploading pre-recording humming via web browser –Typing the string to represent the melody contour Preprocessing –Query by humming  transforms user’s humming into symbolized representation Conventional music retrieval system –Fast and effective extraction of musical patterns and transformation of user’s humming into systematic notes Problem : User cannot remember some parts of melody Applications: Music Retrieval

25 Power Spectrum Analysis Applications: Music Retrieval a : classic b : jazz c : rock d : news channel

26 System Architecture Applications: Music Retrieval

27 Chromosome Structure Applications: Music Retrieval

28 Convergence Test (1) Applications: Music Retrieval

29 Convergence Test (2) Applications: Music Retrieval

30 Change of Consumer Economy Applications : Fashion Design Before the Industrial Revolution : Customers have few choices on buying their clothes After the Industrial Revolution : Customers can make their choices with very large variety Near Future : Customers can order and get clothes of their favorite design Manufacturer Oriented Consumer Oriented

31 Need for Interactive System Almost all consumers are non-professional at design To make designers contact all consumers is not effective Need for the design system that can be used by non-professionals Applications : Fashion Design

32 Fashion Design Definition –To make a choice within various styles that clothes can take Three shape parts of fashion design –Silhouette –Detail –Trimming Applications : Fashion Design

33 System Architecture Applications : Fashion Design

34 Knowledge-based Encoding Search space size =34*8*11*8*9*8 =1,880,064 A : Neck and body style(34) E : Skirt and waistline style(9) C : Arm and sleeve style(11) B : Color(8) D : Color(8) … … … A BCDEF Total 23 bits F : Color(8) Applications : Fashion Design

35 Direct Manipulation Interface Applications : Fashion Design

36 Fitness Changes for Encoding Schemes Applications : Fashion Design

37 Hypercube Analysis Applications : Fashion Design One-Mutant Neighbors using Genetic Operator Shortest Path Using DM Method

38 Motivation Proposed Method –Representation of knowledge-based genotype using the structure of real flower –Automatic generation of creatures Process –Directed graph: Define the genotype of flower structure –IGA: Evaluation of created individuals and automatic creation –Math Engine: Representation of phenotype Objective –Creation of character morphology similar to real flower –Find optimal solution in small solution space using knowledge- based genotype representation such as directed graph –Automatic creation of character using IGA Applications : Flower Design

39 Related Works Tree, flower and feather modeling –L-System Box-based artificial characters (Karl Sims) evolution –Graph model representation Golem –Robotic form and controller evolution –Graph-based data structure Applications : Flower Design

40 Overall Procedure Applications : Flower Design

41 Chromosome Structure Applications : Flower Design

42 Convergence Test Applications : Flower Design

43 Subjective Test by Sheffe’s Method Applications : Flower Design

44 Motivation (1) Various intrusion detection systems –For evaluating, IDS uses known intrusions Possibility of having different patterns within identical intrusion type Difficult to detect transformed intrusions Problem –For better evaluation, intrusion patterns are needed more Difficult to generate all possible proper intrusion patterns manually –Automatic generation is needed Applications : Intrusion Pattern Design

45 Motivation (2) Evolutionary pattern generation –Needs a variety of intrusion patterns It is possible to generate new patterns by combining primitives –Generation of transformed intrusion patterns using simple genetic algorithm IGA-based intrusion pattern generation –Difficult to estimate the fitness of patterns automatically –Using interactive genetic algorithm for the evaluation of patterns Applications : Intrusion Pattern Design

46 Method Intrusive behavior can be conducted in 5 steps –Proposed by DARPA –Possible path of U2R attacks Applications : Intrusion Pattern Design

47 System Architecture Initial Population Known Intrusion Patterns IGA GA Fitness Evaluation Exploit Code Intrusion Patterns Generate Audit Data Applications : Intrusion Pattern Design

48 Concluding Remarks Humanized CI framework using IEC –Knowledge-based encoding –Partial evaluation based on clustering –Direct manipulation of evolution with NK-landscape Applications in media retrieval & design –Image, video and music retrievals –Fashion, flower and intrusion pattern designs Contribution of humanized CI to engineering fields  Developing Emotional HC Interface by Evolving models through Interaction with Humans