Importance of region-of-interest on image difference metrics Marius Pedersen The Norwegian Color Research Laboratory Faculty of Computer Science and Media Technology Gjøvik University College, Gjøvik, Norway Supervisors: Jon Yngve Hardeberg and Peter Nussbaum Thesis presentation, 7. June 2007, Gjøvik

2 Outline Background Research questions Experimental setup Psychophysical experiment Image difference metrics Region-of-interest Images Workflow Results Questionnaire results How do we look at images? Image difference metrics Conclusion

3 Background When we print an image we want the output to be as close to the original as possible. How perceivable are changes made to an image by the observers? Image difference metrics have been developed to answer this question, their goal is to predict the perceived image difference. The image difference metrics used today do not predict the perceived image difference very well. When observers view an image some regions are more important than others.

4 Research questions Question 1: - Can region-of-interest improve overall image difference metrics in complex images? Question 2: - How do observers look at images given different tasks?

5 The experiment A psychophysical experiment using 4 different scenes was carried out with 25 observers. Using an eye tracker to record the gaze position of observers. 4 different image difference metrics - ΔE* ab - S-CIELAB - SSIM - iCAM Different region-of-interest - Freeview - Psychophysical experiment - Gaze marking - Observer marked

6 Images Changes to images made only in lightness. 4 global changes and 4 local changes. 3 and 5 ΔE* ab globally and 3 ΔE* ab locally.

7 Experiment workflow Freeview task - Observers were told to look freely at the images. Psychophysical experiment - Choose the image most similar to the original in a pair comparison task. Gaze marking - Look at the regions important for your decision in the experiment. Observer marking - Observers marked important regions on paper with a pen. Questionnaire

8 Questionnaire results 25 observers ranging from 20 to 38 years, with a mean age of 24. Recruited from the school 56% experts and 44% non- experts. 24% had participated in psychophysical experiments.

9 Psychophysical experiment results Small global changes are rated better than higher global changes. Overall results show that regions are rated generally better than global changes Highly visible changes in small regions are given a low score.

10 How do we look at images Difference between experts and non-experts when it comes to marking important areas. - Expert mark smaller and more precise areas. Same observations made with observers with psychophysical experience. Experts use longer time to evaluate difference.

11 How do we look at images Region-of-interest change when observer are given different tasks. 2-D correlation coefficient used as a measure of similarity between groups and maps. Freeview Psychophysical experiment Gaze marking Observer marking

12 Image difference metrics results In the normal computation S- CIELAB, ΔE*ab and the hue angle algorithm outperform SSIM and iCAM. Pearson product-moment correlation coefficient used a measure of performance. Scene 3 has a small but highly visible region, all metrics have problems here.

13 Area based image difference In metrics performing well only a minor improvement is found. While in metrics with a lower performance a bigger improvement is found. Also the mean squared difference from the regression line supports the finding of improvement in the low performing metrics.

14 Conclusion Q1: Can region-of-interest improve overall image difference metrics in complex images? - Region-of-interest can improve overall image difference metrics, especially in metrics with a low performance. Q2: How do observers look at images given different tasks? - Observer have different region-of-interest in different tasks. * In a freeview task semantic regions as faces draw attention * In a pair comparison task attention is drawn toward other areas where the observer locates difference but faces still draw attention. * Gaze marking cannot replace region-of-interest marking by hand. * Manual marking only reflects some areas of the gaze.

15 Questions? Thanks for your attention.