CEC 2003 Tatsuya Okabe, Yaochu Jin and Bernhard Sendhoff A Critical Survey of Performance Indices for Multi-Objective Optimization CEC 2003 Tatsuya Okabe, Yaochu Jin and Bernhard Sendhoff

Various Performance Indices Cardinality-based Accuracy Distance-based Volume-based Distribution & spread Distribution Spread

Cardinality-based No information about the accuracy and distribution of the solutions. Insensitive to small improvements. Often fail to indicate the relative quality of two solution sets.

Accuracy Reference point or true optimal point must be given: impossible for many real-world apps. Influenced by the spread and distribution of solutions.

Empirical Comparison Artificial solution sets

Comparison Results Cardinality-based Accuracy: distance Accuracy: volume Distribution Spread Distribution & Spread Cardinality based PI cannot distinguish solutions far from true Pareto front. Accuracy PI influenced by distribution & spread: ranks for S1, S2, S5, S6 are different.

Conclusion No single existing PI is able to account for all aspects of the quality of solution sets for multi-objective optimization problems!