Studying Visual Attention with the Visual Search Paradigm Marc Pomplun Department of Computer Science University of Massachusetts at Boston

Slides:



Advertisements
Similar presentations
Selective Visual Attention: Feature Integration Theory PS2009/10 Lecture 9.
Advertisements

Reminder: extra credit experiments
Palmer (after Broadbent) interval 750 ms test 100 ms cue 250 ms Relevant size 28 (Palmer, after Shaw)
Attention and neglect.
Visual Saliency: the signal from V1 to capture attention Li Zhaoping Head, Laboratory of natural intelligence Department of Psychology University College.
Human (ERP and imaging) and monkey (cell recording) data together 1. Modality specific extrastriate cortex is modulated by attention (V4, IT, MT). 2. V1.
Chapter 6: Visual Attention. Overview of Questions Why do we pay attention to some parts of a scene but not to others? Do we have to pay attention to.
Visual Attention: Outline Levels of analysis 1.Subjective: perception of unattended things 2.Functional: tasks to study components of attention 3.Neurological:
Attention - Overview Definition Theories of Attention Neural Correlates of Attention Human neurophysiology and neuroimaging Change Blindness Deficits of.
Chapter 6: Visual Attention. Scanning a Scene Visual scanning – looking from place to place –Fixation –Saccadic eye movement Overt attention involves.
Chapter 6: Visual Attention. Scanning a Scene Visual scanning – looking from place to place –Fixation –Saccadic eye movement Overt attention involves.
黃文中 Preview 2 3 The Saliency Map is a topographically arranged map that represents visual saliency of a corresponding visual scene. 4.
A saliency map model explains the effects of random variations along irrelevant dimensions in texture segmentation and visual search Li Zhaoping, University.
Features and Objects in Visual Processing
Visual Search: finding a single item in a cluttered visual scene.
Experiments for Extra Credit Still available Go to to sign upwww.tatalab.ca.
Upcoming Stuff: Finish attention lectures this week No class Tuesday next week – What should you do instead? Start memory Thursday next week – Read Oliver.
Pre-attentive Visual Processing: What does Attention do? What don’t we need it for?
Organizational Notes no study guide no review session not sufficient to just read book and glance at lecture material midterm/final is considered hard.
Genetic Algorithms  An example real-world application.
Feature Level Processing Lessons from low-level vision Applications in Highlighting Icon (symbol) design Glyph design.
Visual Attention More information in visual field than we can process at a given moment Solutions Shifts of Visual Attention related to eye movements Some.
Next Week Memory: Articles by Loftus and Sacks (following week article by Vokey)
Tracking multiple independent targets: Evidence for a parallel tracking mechanism Zenon Pylyshyn and Ron Storm presented by Nick Howe.
Upcoming: Read Expt 1 in Brooks for Tuesday Read Loftus and Sacks For Thursday Read Vokey Thursday the 6th Idea Journals Due on the 6th! The textbook Cognition.
Visual search: Who cares?
Attention II Selective Attention & Visual Search.
Features and Object in Visual Processing. The Waterfall Illusion.
Attention II Theories of Attention Visual Search.
Chapter Four The Cognitive Approach I: History, Vision, and Attention.
Features and Object in Visual Processing. The Waterfall Illusion.
Parallel vs. Serial Information Processing Remember - attention is about information processing.
The Cognitive Approach I: History, Vision, and Attention
Change blindness: Past, present, and future Professor: Liu Student: Ruby.
Read article by Anne Treisman. Reflexive Orienting Attention can be automatically “summoned” to a location at which an important event has occurred: –Loud.
Experimental Design Tali Sharot & Christian Kaul With slides taken from presentations by: Tor Wager Christian Ruff.
Attention as a Limited Capacity Resource
Neural mechanisms of Spatial Learning. Spatial Learning Materials covered in previous lectures Historical development –Tolman and cognitive maps the classic.
Manipulating Attention in Computer Games Matthias Bernhard, Le Zhang, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of.
Active Vision Key points: Acting to obtain information Eye movements Depth from motion parallax Extracting motion information from a spatio-temporal pattern.
Psych 216: Movement Attention. What is attention? Covert and overt selection appear to recruit the same areas of the brain.
Text Lecture 2 Schwartz.  The problem for the designer is to ensure all visual queries can be effectively and rapidly served.  Semantically meaningful.
1 Perception and VR MONT 104S, Fall 2008 Session 13 Visual Attention.
How Capacity Limits of Attention Influence Information Visualization Effectiveness IEEE INFOVIS 2012 Steve Haroz, University of California, Davis David.
Feature Integration Theory Project guide: Amitabha Mukerjee Course: SE367 Presented by Harmanjit Singh.
Assessment of Computational Visual Attention Models on Medical Images Varun Jampani 1, Ujjwal 1, Jayanthi Sivaswamy 1 and Vivek Vaidya 2 1 CVIT, IIIT Hyderabad,
Gaze-Controlled Human-Computer Interfaces Marc Pomplun Department of Computer Science University of Massachusetts at Boston Homepage:
Visual Distinctness What is easy to find How to represent quantitity Lessons from low-level vision Applications in Highlighting Icon (symbol) design -
Psych 435 Attention. Issues Capacity –We can’t respond to everything in the environment –Too many pieces of information –we can only actively respond.
TEMPORAL DISPLACEMENTS: NEW EVIDENCE ON THE BINDING PROBLEM Ekaterina Pechenkova Moscow State University.
Control of Attention in schizophrenia 1.Advance understanding of schizophrenia. Move from description of deficits to explanation of specific mechanisms.
Lecture 4 – Attention 1 Three questions: What is attention? Are there different types of attention? What can we do with attention that we cannot do without.
Human Factors In Visualization Research Melanie Tory and Torsten Moller Ajith Radhakrishnan Nandu C Nair.
Modeling Visual Search Time for Soft Keyboards Lecture #14.
Binding problems and feature integration theory. Feature detectors Neurons that fire to specific features of a stimulus Pathway away from retina shows.
Visual Search Deficits in Williams Buren Syndrome Montfoort, I., Frens, M.A., Lagers- Van Haselen, G.C., & van der Geest, J.N.
Spatial Smoothing and Multiple Comparisons Correction for Dummies Alexa Morcom, Matthew Brett Acknowledgements.
Psych 335 Attention. Issues Capacity –We can’t respond to everything in the environment –Too many pieces of information –we can only actively respond.
“Nothing average ever stood as a monument to progress. When progress is looking for a partner it doesn't turn to those who believe they are only average.
Optimal Eye Movement Strategies In Visual Search.
Goals for Today’s Class
Eye Movements and Working Memory Marc Pomplun Department of Computer Science University of Massachusetts at Boston Homepage:
CORNELL UNIVERSITY CS 764 Seminar in Computer Vision Attention in visual tasks.
Melanie Boysen & Gwendolyn Walton
Assist. Prof. Dr. Ilmiye Seçer Fall
Kimron Shapiro & Frances Garrad-Cole The University of Wales, Bangor
Cross-cultural differences on object perception
CSc4730/6730 Scientific Visualization
Chapter 7 - Visual Attention
Human vision: function
Presentation transcript:

Studying Visual Attention with the Visual Search Paradigm Marc Pomplun Department of Computer Science University of Massachusetts at Boston Homepage:

Studying Visual Attention with the Visual Search Paradigm Overview: The Feature Integration Theory Visual Search The Guided Search Theory The Area Activation Model

The Binding Problem Different features of the visual scene are coded by separate systems –e.g., direction of motion, location, color and orientation How do we know this? –Anatomical & neurophysiological evidence –Brain Imaging (fMRI & PET) So how do we experience a coherent world?

Feature Integration Theory (Treisman et al) Attention is used to bind features together Code one object at a time on the basis of its location Bind together whatever features are attended at that location

Feature Integration Theory Sensory “features” (color, size, orientation, etc) are coded in parallel by specialized modules Modules form two kinds of “maps” –Feature maps (e.g., color maps, orientation maps etc.) –A master map of locations

Feature Integration Theory Feature maps contain two kinds of information: - presence of a feature anywhere in the field (“there’s something red out there”) - implicit spatial information about the feature Activity in the feature maps can tell us which features are contained in the visual scene. It cannot tell us which other features the “green blob” has. The master map codes the location of features.

Feature Integration Theory The basic idea of the FIT is that visual attention is used for Locating features Binding appropriate features together There are two stages of object perception: Preattentive stage: Individual features are extracted in parallel across the whole visual scene. Attentive stage: When attention is directed to a location, the local features are combined to form a whole.

Feature Integration Theory Attention moves within the location map Focus of attention selects whatever features are linked to that location Features of other objects are excluded Attended features are then entered into the current temporary object representation

Feature Integration Theory Empirical evidence for the FIT has been obtained through Visual search tasks Illusory conjunctions We will focus on the paradigm of visual search.

Visual Search

Feature Search Is there a red T in the display? T T T T T T T T T T T Target defined by a single feature According to FIT, this should not demand attention Target should “pop out”

Conjunction Search Is there a red T in the display? X T T T X T T X T TX X T T Target is now defined by its shape and color This involves binding features and so should demand attention Need to attend to each item until target is found

Feature Search Changing the number of distractors: T T T T T T T T T T T T T T T T T T T T T T T T TT T T T T T TT T T T T T T T T

Conjunction Search Changing the number of distractors: X T T T TX X T X X T T T X T T X T TX X T T X X X T T T T T T T T T T X X X X X X X X X X

Visual Search Experiments Record time taken to determine whether target is present or not Vary the number of distractors Search for features should be independent of the number of distractors Conjunction search should get slower with more distractors

Visual Search Conjunction targets demand serial search  significant slope Feature targets pop out  flat display size function

Problem with FIT: Pop-Out of Conjunction Targets A moving X pops out of a display of moving O’s and static X’s O O O O O X X X X X Target is defined by a conjunction of movement and form At least some conjunctions do not require focal attention

Guided Search Theory The Guided Search Theory (GST) is similar to the FIT in that it also assumes two subsequent stages of visual search performance: a preattentive, parallel stage an attentive, serial stage However, the main difference to FIT is that GST assumes the preattentive stage to obtain spatial saliency information that is used to guide attention in the serial stage.

Guided Search Theory According to GST, saliency is encoded in an additional map, called the saliency map. The saliency map is created during the preattentive stage and can combine multiple features if necessary. In the subsequent serial search process, attention is first directed to the highest “peak” in the saliency map, then to the second-highest, and so on. This visual guidance allows efficient search even for some conjunction targets.

Guided Search Theory Support for the GST comes from eye-movement research. Eye-movement recording allows researchers to determine the items that a subject looks at during visual search.

Guided Search Theory

In the previous example, 80% of fixations were closest to an item sharing color with the target, 20% of fixations were closest to an item sharing orientation with the target. It seems that the color dimension is guiding the subject’s visual search process. Of course, due to imprecision of eye movements and their measurement, better statistics are necessary to determine the guiding dimension.

Guided Search Theory In visual search tasks, subjects are usually guided by one target feature or a combination of target features. This supports the idea of GST that preattentively derived information from multiple dimensions guides and thereby facilitates the subsequent serial search process.

Guided Search Theory There are two problems with GST: According to GST, grouping the guiding distractors should result in reduced guidance (less bottom-up activation). However, the opposite happens. There is no quantitative implementation of a Guided Search model that could predict guidance, i.e., saccadic selectivity for a given search task. To overcome these problems, we proposed the Area Activation Model of saccadic selectivity in visual search tasks.

Area Activation Assumptions: Processing resources during a fixation are distributed like a two-dimensional Gaussian function centered at fixation. Fixation positions are chosen to allow a maximum of information processing according to the assumed processing resources. Scan paths are chosen in such a way that they connect the optimal fixation positions with minimal eye-movement cost (path length).

Area Activation - Strong Guidance

Area Activation - Weak Guidance

Area Activation - Empirical Results

Area Activation Problems with the Area Activation Model: Empirical number of fixations per trial needs to be known in advance. Only very basic factors influencing visual search have been implemented so far. Nevertheless, Area Activation can be considered a very first step towards a quantitative model of visual search.

Conclusions We have discussed how the visual search paradigm can be employed to investigate the mechanisms of visual attention. Various models of attention have been developed and evaluated with visual search tasks; in more recent studies, this was done based on eye-movement data. In the next lecture, we will look at slightly different paradigms, which are aimed at identifying factors that determine visual scan paths. See you then!