Analysis of scores, datasets, and models in visual saliency modeling Ali Borji, Hamed R. Tavakoli, Dicky N. Sihite, and Laurent Itti,

Toronto dataset

Why important? Current status Methods: numerous / 8 categories (Borji and Itti, PAMI, 2012) Databases: Measures: scan-path analysis correlation based measures ROC analysis Visual Saliency How good my method works?

Benchmarks Judd et al. Borji and Itti Yet another benchmark!!!?

Dataset Challenge Dataset bias : Center-Bias (CB), Border effect Metrics are affected by these phenomena. MIT Le Meur Toronto

Tricking the metric Solution ? sAUC Best smoothing factor More than one metric

The Benchmark Fixation Prediction

The Feature Crises intensity orientation color size depth Low level people symmetry car text signs High level Features Does it capture any semantic scene property or affective stimuli? Challenge of performance on stimulus categories & affective stimuli Challenge of performance on stimulus categories & affective stimuli

The Benchmark Image categories and affective data

vs 0.64 (non-emotional )

The Benchmark predicting scanpath bB cC dD aAbBcCaA aAdDbBcCaA aAcCaA aAcCbBcCaAaA …. aA bBbBcC matching score

The Benchmark predicting scanpath (scores)

Category Decoding

Lessons learned We recommend using shuffled AUC score for model evaluation. Stimuli affects the performance. Combination of saliency and eye movement statistics can be used in category recognition. There seems the gap between models and IO is small (though statistically significant). It somehow alerts the need for new dataset. The challenge of task decoding using eye statistics is open yet. Saliency evaluation scores can still be introduced

Questions ??