1. Analysis of scores, datasets, and models in visual saliency modeling Ali Borji, Hamed R. Tavakoli, Dicky N. Sihite, and Laurent Itti,

2. Toronto dataset

7. Why important? Current status Methods: numerous / 8 categories (Borji and Itti, PAMI, 2012) Databases: Measures: scan-path analysis correlation based measures ROC analysis Visual Saliency How good my method works?

8. Benchmarks Judd et al. http://people.csail.mit.edu/tjudd/SaliencyBenchmark/ http://people.csail.mit.edu/tjudd/SaliencyBenchmark/ Borji and Itti https://sites.google.com/site/saliencyevaluation/ https://sites.google.com/site/saliencyevaluation/ Yet another benchmark!!!?

9. Dataset Challenge Dataset bias : Center-Bias (CB), Border effect Metrics are affected by these phenomena. MIT Le Meur Toronto

10. Tricking the metric Solution ? sAUC Best smoothing factor More than one metric

11. The Benchmark Fixation Prediction

12. The Feature Crises intensity orientation color size depth Low level people symmetry car text signs High level Features Does it capture any semantic scene property or affective stimuli? Challenge of performance on stimulus categories & affective stimuli Challenge of performance on stimulus categories & affective stimuli

13. The Benchmark Image categories and affective data

14. vs 0.64 (non-emotional )

15. The Benchmark predicting scanpath bB cC dD aAbBcCaA aAdDbBcCaA aAcCaA aAcCbBcCaAaA …. aA bBbBcC matching score

16. The Benchmark predicting scanpath (scores)

17. Category Decoding

18. Lessons learned We recommend using shuffled AUC score for model evaluation. Stimuli affects the performance. Combination of saliency and eye movement statistics can be used in category recognition. There seems the gap between models and IO is small (though statistically significant). It somehow alerts the need for new dataset. The challenge of task decoding using eye statistics is open yet. Saliency evaluation scores can still be introduced

19. Questions ??