Saliency & attention (P) Lavanya Sharan April 4th, 2011

Looking for people Ehinger et al. (2009) Question: Where do people look and can we model it?

Looking for people Ehinger et al. (2009) 14 observers with tracking accuracy 0.75 deg. Observers asked to look for pedestrians as fast as possible and answer. 912 images. Target (pedestrian) present in half the images. Images 23.5 x 17.7 deg, target when present 0.9 x 1.8 deg.

Inter-observer agreement Ehinger et al. (2009) Question: Do observers agree with each other? Use fixations from n-1 observers for one image, apply gaussian blur and use this region to predict fixation of remaining observer on that image. Repeat for n observers (Torralba et al. 2006). Do this for all images.

Inter-observer agreement Ehinger et al. (2009)

Cross-image control Ehinger et al. (2009) Question: Do fixated locations depend on image content? Or do observers always look at the same places? Use fixations from n-1 observers for one image, apply gaussian blur and use this region to predict fixation of remaining observer on a different, randomly selected image. Repeat for n observers (Torralba et al. 2006). Do this for all images.

Performance at predicting fixation locations Ehinger et al. (2009) Inter-observer agreement (Target absent AUC=0.93, target present AUC = 0.95) Cross-image control (Target absent AUC=0.68, target present AUC = 0.62)

Modeling eye movements during pedestrian detection Ehinger et al. (2009)

Low-level saliency. Use Torralba et al model. Target features. Use Dalal et al person detector. Scene context. Use Torralba et al. model but train on context (i.e. sidewalks not skies are good locations) + ‘Context Oracle’ (7 observers indicate good candidates for locations) Use validation set to decide relative weighting.

Modeling eye movements during pedestrian detection Ehinger et al. (2009)

Performance of saliency component

Modeling eye movements during pedestrian detection Ehinger et al. (2009) Performance of target features component

Modeling eye movements during pedestrian detection Ehinger et al. (2009) Performance of scene context component

Modeling eye movements during pedestrian detection Ehinger et al. (2009) Performance of full model

What does the model get wrong? Ehinger et al. (2009)