A Novel Method for Generation of Motion Saliency Yang Xia, Ruimin Hu, Zhenkun Huang, and Yin Su ICIP 2010

Outline Introduction Itti’s model Proposed Visual Saliency – Generation of motion feature map – Enhancement of motion sub-saliency map Experiment Results Conclusion

Introduction Visual saliency – Bottom up saliency – Top down saliency – Applications Image segmentation, motion detection, image/video compression…… Motion saliency – Motion object is more salient to human vision system(HVS) than spatial contrast in video.

Itti’s Model For Image – Spatial features Color, Intensity, and Orientation – Feature maps – Combining the normalized activation maps Visual Saliency Model For Video – Temporal features Flicker and Motion

Itti’s Model dx, dy : horizontal and vertical shift distance f : the frame rate

Itti’s Model Multi reference frame Enhance motion saliency map by spatial saliency information

Proposed Visual Saliency [1] J. Harel, C. Koch, and P. Perona, “graph-based visual saliency,” in Advances in Neural Information Processing Systems 19, Cambridge, MA: MIT Press, 2007 (Reference frame) np

Proposed Visual Saliency

Enhancement of Motion Sub-saliency Map Top 25% of locations which have larger saliency values in SM S Top 5% of locations which have larger saliency values in SM M

Experimental Results Dataset in CAVIAR—ThreePastShop1cor – ROC(Relative Operating Characteristic) score between estimated saliency maps(ESMs) and ground-truth saliency maps(GSMs) Anchor1: Itti’s model Anchor2: Itti’s model using activation operator based on graph theory SMRF: saliency model with the multi-reference frames SMRF+STE: plus spatio-temporal enhancement

Experimental Results motion channel five channel anchor1 SMRF+STE anchor2 anchor1 anchor2 SMRF SMRF+STE

Conclusion First analyze the drawback of Itti’s motion saliency model. Propose a novel motion saliency model in which motion saliency map is obtained through the multi reference frames, and enhanced by spatial saliency information.