1. CEC 2003 Tatsuya Okabe, Yaochu Jin and Bernhard Sendhoff
A Critical Survey of Performance Indices for Multi-Objective Optimization
CEC 2003
Tatsuya Okabe, Yaochu Jin and Bernhard Sendhoff

2. Various Performance Indices
Cardinality-based
Accuracy
Distance-based
Volume-based
Distribution & spread
Distribution
Spread

3. Cardinality-based
No information about the accuracy and distribution of the solutions.
Insensitive to small improvements.
Often fail to indicate the relative quality of two solution sets.

4. Accuracy
Reference point or true optimal point must be given: impossible for many real-world apps.
Influenced by the spread and distribution of solutions.

5. Empirical Comparison
Artificial solution sets

6. Comparison Results
Cardinality-based
Accuracy: distance
Accuracy: volume
Distribution
Spread
Distribution & Spread
Cardinality based PI cannot distinguish solutions far from true Pareto front.
Accuracy PI influenced by distribution & spread: ranks for S1, S2, S5, S6 are different.

7. Conclusion
No single existing PI is able to account for all aspects of the quality of solution sets for multi-objective optimization problems!