1. Doshisha Univ., Kyoto Japan
NCGA : Neighborhood Cultivation Genetic Algorithm for Multi-Objective Optimization Problems
○ Shinya Watanabe
Tomoyuki Hiroyasu
Mitsunori Miki
Thank you Chairperson. I’m Shinya Watanabe and a graduate student of Doshisha University in Kyoto Japan. Now I’m talking about our study, the title is “NCGA : Neighborhood Cultivation Genetic Algorithm for Multi-Objective Optimization Problems”.
Intelligent Systems Design Laboratory，
Doshisha University，Kyoto Japan
Doshisha Univ., Kyoto Japan

2. Pareto optimal solutions Doshisha Univ., Kyoto Japan
Multi-objective Optimization Problems
●Multi-objective Optimization Problems (MOPs)
In the optimization problems, when there are several objective functions, the problems are called multi-objective or multi-criterion problems.
Design variables
X={x1, x2, …. , xn}
Feasible region
(x) f 2
Objective function
Pareto optimal solutions
F={f1(x), f2(x), … , fm(x)}
In the optimization problems, when there are several objective functions, the problems are called multi-objective or multi-criterion problems(MOPs).
There are trade off relation between the objective functions. Therefore the optimum solution is not only one. In MOPs, the concept of the pareto optimal solution is used. This figure shows the Pareto-optimal solution of this problem.
In this figure, horizontal axis represents f1(x). and vertical axis represents f2(x). Therefore, the lower left solutions are better. In this figure, Pareto-optimal solutions are illustrated as Red line.
The first goal of solving the MOPs is to obtain Pareto-optimal solutions.
// This slid shows a basic definition (ディフィニジュオン) of multi-objective optimization. But I think everyone in this room knows this matter.
I’ll skip this slid.
Constraints f
Gi(x)<0 ( i = 1, 2, … , k)
(x) 1
Doshisha Univ., Kyoto Japan

3. Doshisha Univ., Kyoto Japan
EMO
EMO
Evolutionary Multi-criterion Optimization
MOPs solved by Evolutionary algorithms
Typical method on EMO
VEGA　 :Schaffer (1985)
MOGA :Fonseca (1993)
DRMOGA :Hiroyasu, Miki, Watanabe (2000)
SPEA2 :Zitzler (2001)
NPGA :Erickson, Mayer, Horn (2001)
NSGA-II :Deb, Goel (2001)
MOPs solved by Evolutionary algorithms are often called EMO.
These are a part of leading researches in this category.
In these methods, SPEA2 and NSGA-II are called especial(エスペシャル) good methods.
Doshisha Univ., Kyoto Japan

4. Doshisha Univ., Kyoto Japan
Neighborhood Cultivation GA (NCGA)
NCGA : Neighborhood Cultivation GA
The features of NCGA
The neighborhood crossover.
Archive of excellent solutions.
A Method which cuts down reserved excellent solutions.
Use of the reserved excellent solutions for searching solutions.
Unification mechanism of the values of each objective.
In this presentation, we propose new GA for MOPs.
This algorithms is called “Neighborhood Cultivation GA : NCGA”.
NCGA has these mechanisms of previous methods, like SPEA2 and NSGA-II.
In addition to these mechanism, NCGA uses a new mechanism , “neighborhood crossover”.
Now , I will talk about neighborhood crossover.
Doshisha Univ., Kyoto Japan

5. Doshisha Univ., Kyoto Japan
Neighborhood Cultivation GA (NCGA)
A neighborhood crossover
In MOPs GA, the searching area is wide and the searching area of each individual is different.
f2(x)
f1(x)
If the distance between two selected parents is so large, the crossover may have no effect for local search.
In MOPs GA, the searching area is wide. And the searching area of each individuals are different.
If the distance between two selected parents is so large, crossover will have no effect for local search.
Like this figure.
So , parents of crossover should be in their adjacent (アジェイセン(ト) ).
Doshisha Univ., Kyoto Japan

6. Doshisha Univ., Kyoto Japan
Neighborhood Cultivation GA (NCGA)
A neighborhood crossover
Two parents in the crossover are chosen from the top of the sorted individuals.
f2(x)
f1(x)
In order not to make the same couple,
One of the objectives is changed at each generation.
The sorting of a population includes a little probabilistic change.
So, we propose a new mechanism, neighborhood crossover.
In this methods, Population is sorted according to one of the objectives.
And then , Two parents in the crossover are chosen from the top of the sorted individuals. This figure shows what couple of parents are chosen in this method.
In this method , two operations are used in order not to make the same couple. These operations are shown like this.
Doshisha Univ., Kyoto Japan

7. Doshisha Univ., Kyoto Japan
Neighborhood Cultivation GA (NCGA)
The differences from the recent major algorithms like SPEA2 and NSGA-II.
NCGA has the neighborhood crossover mechanism.
NCGA has only one selection in one generation.
Many methods have two types of selection (the environment selection and the mating selection). But, NCGA has the environment selection only.
These are the differences from the recent major algorithm like SPEA2, NSGA-II.
NCGA has the the neighborhood crossover mechanism.
NCGA has only one selection in one generation.
Many method has two types selection (environment selection and mating selection). But NCGA has only the environment selection. Because some same individuals in the population interrupt the effect of neighborhood crossover. //母集団の中に同じ個体が存在していると，近傍交叉が行えないから．
In neighborhood crossover, parents are chosen from the top of the sorted individuals.
If there are some same individuals , crossover is performed with same parents, therefore the effect of crossover is lost.
同じ個体ペアによる交叉となり，交叉の意味が失われる．
Doshisha Univ., Kyoto Japan

8. Doshisha Univ., Kyoto Japan
Comparison method
Sampling of the Pareto frontier Lines of Intersection (ILI) (Knowles and Corne 2000)
= 5/12=0.42
= 7/12=0.58
This slid shows the comparison method presented by Knowles and Corne 2000.
This figure shows the concept of this method.
In this figure, two solution sets of X and Y derived by the different methods are illustrated.
At first, the nondominated surfaces defined by the derived solution sets are formed.
In this figure, red and blue lines mean the nondominated surfaces.
Secondly, sampling lines against the nondominated surfaces are drawn at even intervals.
On each line, the intersections of the line and the surfaces are obtained.
These intersections are compared.
For example, this line intersects Y’s surface at this point and intersects with X’s surface at this point.
Y’s intersection is closer than X’s intersection from the origin .
Therefore on this sampling line, Y’s intersection is better than X’s intersection.
This evaluation is repeated for all sampling line.
Finally, the value of this method (I_LI) can be derived, by adding up the evaluation values.
The result of this figure would be around [40,60]. This indicates that the X outperforms the Y on about 40 % of the nondominated surface, while the Y outperforms X on around 60% on the surface.
So, we can say the solutions of Y are better than that of X.
Doshisha Univ., Kyoto Japan

9. Doshisha Univ., Kyoto Japan
Applied models and Parameters
Applied models
GA Operator
Crossover
One point crossover
Mutation
Bit flip
SPEA2
NSGA-II
NCGA
non-NCGA
(NCGA except neighborhood crossover )
Parameters
population size
100
250
From this slid, I will talk about numerical examples.
This slid shows applied models and parameters.
We applied 4 models to the problem.
These are SPEA2, NSGA-II , NCGA ,and non-NCGA.
non-NCGA is the same algorithms of NCGA except neighborhood crossover. non-NCGA uses normal-crossover.
This table shows used parameters. And this shows used GA operators.
crossover rate
1.0
mutation rate
0.01
terminal condition
250
2000
number of trials 30
Doshisha Univ., Kyoto Japan

10. Doshisha Univ., Kyoto Japan
Test Problems
Discontinuous Function
Fdiscon (Deb’00)
We applied NCGA to 6 test problems.
But now time is very limited. So, In this presentation, I’ll show you the results in 3 test problems.
This function is discontinuous function proposed by Deb .
This function has 100 design variables.
This figure shows the pareto optimum solution of this problem.
Doshisha Univ., Kyoto Japan

11. Doshisha Univ., Kyoto Japan
Pareto solutions of Fdiscon
At first, I’d like to show you the derived pareto solutions by each model.
In this figure, horizontal axis represetns f1(x). and vertical axis represents f2(x).
Therefore, the lower left solutions are better.
In these figure, all the Pareto optimum solutions that are derived in 30 trials are figured out.
From this figure, you may think that all results are almost same. But as a matter of fact , the results of NCGA has the best accuracy of all methods.
Doshisha Univ., Kyoto Japan

12. Doshisha Univ., Kyoto Japan
Comparison result of Fdiscon (ILI)
And from the results of I_LI, it is clear that the solutions of NCGA is superior to the that of the others.
These results represent a comparison between NCGA and the other method.
Right side results represents NCGA results.
and Left side results represents the other method results.
From these results, we can say NCGA gets better solutions than the other method.
Doshisha Univ., Kyoto Japan

13. Doshisha Univ., Kyoto Japan
Test Problems
Continuous Function
KUR
And This KUR is continuous function.
This function has a multi-modal function in f2(x) and pair-wise interactions among the variables in f1(x).
Doshisha Univ., Kyoto Japan

14. Doshisha Univ., Kyoto Japan
Pareto solutions of KUR
These pareto solutions are in KUR.
In this problem, results shows that NCGA is obviously superior(スピィリアラァ) to the other methods. Red solutions of NCGA are wider and deeper than that of the other methods.
Doshisha Univ., Kyoto Japan

15. Doshisha Univ., Kyoto Japan
Comparison result of KUR (ILI)
From this I_LI results, we can verify that NCGA can get the best solutions of all method.
In comparison with non-NCGA, NCGA can get better solutions obviously.
This is the reason that KUR has pair-wise interactions among the variables in f1.
Therefore, in this problem, neighborhood crossover can perform to get good solutions.
Doshisha Univ., Kyoto Japan

16. Doshisha Univ., Kyoto Japan
Test Problems
Combination problem
KP 750-2
Objectives
Constraints
And this problem is combination problem.
KP750-2 is the Multi-objective 0/1 knapsack problem.
There are 750 items and two objects.
This equations shows formulation of this problem.
pi,j = profit of item j according to knapsack i
wi,j = weight of item j according to knapsack i
ci,= capacity of knapsack i
Doshisha Univ., Kyoto Japan

17. Doshisha Univ., Kyoto Japan
Pareto solutions of KP750-2
These pareto solutions are in knapsack problem.
This problem is one of maximum problem.
So, the upper right solutions are better.
You can see that NCGA could get the wide spread solutions compared to the other methods.
Doshisha Univ., Kyoto Japan

18. Doshisha Univ., Kyoto Japan
Comparison result of KP750-2 (ILI)
From this results of ILI, we can see that the solutions of NCGA are better than those of the other methods.
In comparison with non-NCGA, NCGA can get better solutions obviously. This means neighborhood crossover can perform efficient search in this problem.
Also, in this result, I don’t show you the comparison result of all method except NCGA .
The solutions of non-NCGA is worse than that of SPEA2 and NSGA-II.
This result emphasizes how efficient neighborhood crossover is.
Doshisha Univ., Kyoto Japan

19. Doshisha Univ., Kyoto Japan
Conclusion
We proposed a new model for Multi-objective GAs.
NCGA: Neighborhood Cultivation GA
Effective method for multi objective GA
The neighborhood crossover
Archive of excellent solutions.
A Method which cuts down reserved excellent solutions.
Use of the reserved excellent solutions for searching solutions.
Unification mechanism of the values of each objective.
Now I’d like to show you the conclusions.
In this study, We proposed new model for Multi-objective GA.
That is Neighborhood Cultivation GA, NCGA.
NCGA has not only important mechanism of the other method but also the mechanism of neighborhood crossover.
Doshisha Univ., Kyoto Japan

20. Doshisha Univ., Kyoto Japan
Conclusion
NCGA was applied to some test functions and the results were compared to the other methods; such as SPEA2, NSGA-II and non-NCGA.
In almost test functions, NCGA derives the good results.
Comparing NCGA to NCGA without neighborhood crossover, NCGA is obviously superior to in all problems.
To discuss the effectiveness of the proposed method, NCGA was applied to test functions and results were compared to the other methods. Through the numerical examples, the following topics are made clear.
In some the test functions, NCGA derived the good results.
Compared to the other method, the results of NCGA are superior(スピィリアラァ) to the others.
Comparing to NCGA and NCGA without neighborhood crossover,NCGA is obviously superior to NCGA without NCGA in all problems.
This results emphasize that the neighborhood crossover is useful to derive the good solutions.
That’s all. Thank you.
NCGA is an effective algorithm
for multi-objective problems.
Doshisha Univ., Kyoto Japan

21. Doshisha Univ., Kyoto Japan
Test Problems
Continuous Function
ZDT4
From this slid, I will talk about numerical examples.
This ZDT4 is continuous function.
A feature of this function is multi-model.
This figure shows the real pareto solution line of this problem.
Doshisha Univ., Kyoto Japan

22. Doshisha Univ., Kyoto Japan
Pareto solutions of ZDT4
At first, I’d like to show you the derived pareto solutions by each model in ZDT4.
In this figure, horizontal axis represetns f1(x). and vertical axis represents f2(x).
Therefore, the lower left solutions are better.
In these figure, all the Pareto optimum solutions that are derived in 30 trials are figured out.
So, there are many line of pareto solutions in these figure .
From this figure, the accuracy of NCGA’s pareto solutions is the best of all.
Doshisha Univ., Kyoto Japan

23. Doshisha Univ., Kyoto Japan
Comparison result of ZDT4 (ILI)
// 再度よくチェックする必要あり．
This is the I_LI value results.
I_LI is the comparison method of two derived solutions.
// And in this method, the higher value means
These results are made a comparison between NCGA and the other method.
Right values of these results represents NCGA results.
and Left results of these results represents the other method results.
From these results, we can say NCGA gets better solutions than the other method.
Doshisha Univ., Kyoto Japan

24. Doshisha Univ., Kyoto Japan
ILI of KP750-2
Doshisha Univ., Kyoto Japan

25. Doshisha Univ., Kyoto Japan
URL of reference
About EMO
About 0/1 Knapsack problem
NCGA source program
My address
Doshisha Univ., Kyoto Japan

26. Doshisha Univ., Kyoto Japan
Performance Measure
The Ratio of Non-dominated Individuals (RNI) is derived from two types of Pareto solutions.
Method A
(x) f 2 f 2
(x)
Method A
Method B
0.333
0.666 f
(x) 1
I’d like to talk about Performance Measure that we have used. The Ratio of Non-dominated Individuals (RNI) is very simple measure method.
RNI is derived from two types of Pareto solutions. At first, these two types of pareto solutions are mixed . Secondly, new pareto solutions are selected from the mixed solutions. Then the number of solutions is counted.
RNI of method A is the ratio of this number against the total number of the solutions. In this example, there are 9 pareto solutions. Among these solutions, 3 are the solutions derived by Method A.
Method A, it means that the solutions of Method B is superior(スピィリアラァ) to Method A. Therefore RNI of Method A is 3/9 ( three - ninth ). This figure is the ratio of Method A and B.
This shows that RNI of method B is closer to 1 than that of
Method B
(x) f 2 f
(x) 1 f
(x) 1
Doshisha Univ., Kyoto Japan

27. Doshisha Univ., Kyoto Japan
EMO
The following topics are the mechanisms that the recent GA approaches have.
Archive of the excellent solutions
Cut down (sharing) method of the reserved excellent solutions
An appropriate assign of fitness
Reflection to search solutions mechanism of the reserved excellent solutions
Unification mechanism of values of each objective
And we found that the following topics are the mechanisms that the recent GA approaches have.
These mechanisms derive the good Pareto-optimum solutions. Therefore, the developed algorithm should have these mechanisms.
But, these mechanisms do not have local search effect.
So, Multi-objective GA should have any mechanism of local search.
Doshisha Univ., Kyoto Japan

28. Doshisha Univ., Kyoto Japan
Performance Assessment
Measures
The Ratio of Non-dominated Individuals :RNI
The Performance measure perform to compare two type of Pareto solutions.
Two types of pareto solutions derived by difference methods are compared.
Cover Rate Index
Diversity of the Pareto optimum.
Error
The distance between the real pareto front and derived solutions.
Various rate
Diversity of the pareto optimum individuals.
I’d like to talk about Performance Measures that we have proposed.
In MOPs, it is very important to measure Pareto solutions.
Because if we don't have any performance measure of pareto solutions, we cannot quantitatively evaluate performance of the proposed method.
These measures are evaluation criteria of Pareto optimal solutions .
Now , I am allowed to have little time for this presentation.
So, In this presentation, I’d like to explain only RNI.
Doshisha Univ., Kyoto Japan

29. Doshisha Univ., Kyoto Japan
Cluster System
Spec. of Cluster (16 nodes)
Processor Pentium Ⅲ(Coppermine)
Clock MHz
# Processors 1 × 16
Main memory 256Mbytes × 16
Network Fast Ethernet (100Mbps)
Communication TCP/IP, MPICH 1.2.1
OS Linux 2.4
Compiler gcc
The numerical examples were performed on the PC Cluster System. This table shows the specification of our cluster system.
Doshisha Univ., Kyoto Japan