Michael Arbib & Laurent Itti: CS664 – Spring 2002. Lecture 5: Visual Attention (bottom-up) 1 CS 664, USC Spring 2002 Lecture 5. Visual Attention (bottom-up)

Slides:

Advertisements

Similar presentations

Visual Saliency: the signal from V1 to capture attention Li Zhaoping Head, Laboratory of natural intelligence Department of Psychology University College.

Advertisements

A Neural Model for Detecting and Labeling Motion Patterns in Image Sequences Marc Pomplun 1 Julio Martinez-Trujillo 2 Yueju Liu 2 Evgueni Simine 2 John.

Object recognition and scene “understanding”

Sparse Coding in Sparse Winner networks Janusz A. Starzyk 1, Yinyin Liu 1, David Vogel 2 1 School of Electrical Engineering & Computer Science Ohio University,

Human (ERP and imaging) and monkey (cell recording) data together 1. Modality specific extrastriate cortex is modulated by attention (V4, IT, MT). 2. V1.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti: CS564 - Brain Theory and Artificial Intelligence Lecture.

HMAX Models Architecture Jim Mutch March 31, 2010.

Attention - Overview Definition Theories of Attention Neural Correlates of Attention Human neurophysiology and neuroimaging Change Blindness Deficits of.

Attention I Attention Wolfe et al Ch 7. Dana said that most vision is agenda-driven. He introduced the slide where the people attended to the many weird.

Covert Attention Mariel Velez What is attention? Attention is the ability to select objects of interest from the surrounding environment Involuntary.

Attention Wolfe et al Ch 7, Werner & Chalupa Ch 75, 78.

Attention, Awareness, and the Computational Theory of Surprise Research Qualifying Exam August 30 th, 2006.

Some concepts from Cognitive Psychology to review: Shadowing Visual Search Cue-target Paradigm Hint: you’ll find these in Chapter 12.

September 7, 2010Neural Networks Lecture 1: Motivation & History 1 Welcome to CS 672 – Neural Networks Fall 2010 Instructor: Marc Pomplun Instructor: Marc.

Exam 1 week from today in class assortment of question types including written answers.

Michigan State University1 Visual Attention and Recognition Through Neuromorphic Modeling of “Where” and “What” Pathways Zhengping Ji Embodied Intelligence.

How does the visual system represent visual information? How does the visual system represent features of scenes? Vision is analytical - the system breaks.

CS564 – Lecture 21 Visual Attention Laurent Itti USC.

Organizational Notes no study guide no review session not sufficient to just read book and glance at lecture material midterm/final is considered hard.

Neural Networks Basic concepts ArchitectureOperation.

Pattern Recognition using Hebbian Learning and Floating-Gates Certain pattern recognition problems have been shown to be easily solved by Artificial neural.

Visual Attention More information in visual field than we can process at a given moment Solutions Shifts of Visual Attention related to eye movements Some.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 4: Introduction to Vision 1 Computational Architectures in Biological.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 6: Low-level features 1 Computational Architectures in Biological.

December 1, 2009Introduction to Cognitive Science Lecture 22: Neural Models of Mental Processes 1 Some YouTube movies: The Neocognitron Part I:

CS 664, Session 19 1 General architecture. CS 664, Session 19 2 Minimal Subscene Working definition: The smallest set of objects, actors and actions in.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 1 CS 664, USC Spring 2002 Lecture 7. Visual Attention (top-down)

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 12: Visual Attention 1 Computational Architectures in Biological Vision,

What is Cognitive Science? … is the interdisciplinary study of mind and intelligence, embracing philosophy, psychology, artificial intelligence, neuroscience,

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 1: Overview & Introduction 1 Computational Architectures in Biological.

Michigan State University 1 “Saliency-Based Visual Attention” “Computational Modeling of Visual Attention”, Itti, Koch, (Nature Reviews – Neuroscience.

From attention to goal-oriented scene understanding Laurent Itti – University of Southern California.

Overview 1.The Structure of the Visual Cortex 2.Using Selective Tuning to Model Visual Attention 3.The Motion Hierarchy Model 4.Simulation Results 5.Conclusions.

Michael Arbib & Laurent Itti: CS664 – USC, spring Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:

What is Cognitive Science? … is the interdisciplinary study of mind and intelligence, embracing philosophy, psychology, artificial intelligence, neuroscience,

Laurent Itti: CS564 – Brain theory and artificial intelligence. Scene Perception 1 Brain theory and artificial intelligence Lecture 23. Scene Perception.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 4: Introduction to Vision 1 Computational Architectures in Biological.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 5: Introduction to Vision 2 1 Computational Architectures in.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC. Lecture 13: Scene Perception 1 Computational Architectures in Biological Vision,

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 13: Scene Perception 1 Computational Architectures in Biological.

Studying Visual Attention with the Visual Search Paradigm Marc Pomplun Department of Computer Science University of Massachusetts at Boston

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis Laurent Itti, Christof Koch, and Ernst Niebur IEEE PAMI, 1998.

Manipulating Attention in Computer Games Matthias Bernhard, Le Zhang, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of.

Psych 216: Movement Attention. What is attention? Covert and overt selection appear to recruit the same areas of the brain.

Text Lecture 2 Schwartz.  The problem for the designer is to ensure all visual queries can be effectively and rapidly served.  Semantically meaningful.

Visual Attention Derek Hoiem March 14, 2007 Misc Reading Group.

黃文中 Introduction The Model Results Conclusion 2.

Assessment of Computational Visual Attention Models on Medical Images Varun Jampani 1, Ujjwal 1, Jayanthi Sivaswamy 1 and Vivek Vaidya 2 1 CVIT, IIIT Hyderabad,

Department of Psychology & The Human Computer Interaction Program Vision Sciences Society’s Annual Meeting, Sarasota, FL May 13, 2007 Jeremiah D. Still,

Street Smarts: Visual Attention on the Go Alexander Patrikalakis May 13, XXX.

Computer Science Readings: Reinforcement Learning Presentation by: Arif OZGELEN.

Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 12: Visual Attention 1 Computational Architectures in Biological.

Neural Models of Visual Attention John K. Tsotsos Center for Vision Research York University, Toronto, Canada Marc Pomplun Department of Computer Science.

Interneuron diversity and the cortical circuit for attention

A bottom up visual saliency map in the primary visual cortex --- theory and its experimental tests. Li Zhaoping University College London Invited presentation.

Spatio-temporal saliency model to predict eye movements in video free viewing Gipsa-lab, Grenoble Département Images et Signal CNRS, UMR 5216 S. Marat,

A Model of Saliency-Based Visual Attention for Rapid Scene Analysis

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern.

Ahissar, Hochstein (1997) Nature Task difficulty and specificity of perceptual learning 1 st third 2nd third Final session Task difficulty Stimulus-to-mask.

March 31, 2016Introduction to Artificial Intelligence Lecture 16: Neural Network Paradigms I 1 … let us move on to… Artificial Neural Networks.

Attention to Orientation Results in an Inhibitory Surround in Orientation Space Acknowledgements Funding for this project was provided to MT through a.

A Neurodynamical Cortical Model of Visual Attention and Invariant Object Recognition Gustavo Deco Edmund T. Rolls Vision Research, 2004.

A bottom up visual saliency map in the primary visual cortex

Implementation of a Visual Attention Model

Computer Vision Lecture 2: Vision, Attention, and Eye Movements

Lecture 22. Saccades 2 Reading Assignments: Reprint

Artificial Neural Networks

Heterogeneous convolutional neural networks for visual recognition

Reading Assignments: Lecture 16. Saccades 2 The NSL Book

Presentation transcript:

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 1 CS 664, USC Spring 2002 Lecture 5. Visual Attention (bottom-up) Reading Assignments: None

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 2

3

4 Several Forms of Attention Attention and eye movements: - overt attention (with eye movements) - covert attention (without eye movements) Bottom-up and top-down control: - bottom-up control based on image features very fast (up to 20 shifts/s) involuntary / automatic - top-down control may target inconspicuous locations in visual scene slower (5 shifts/s or fewer; like eye movements) volitional Control and modulation: - direct attention towards specific visual locations - attention modulates early visual processing at attended location

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 5 What is attention then? Attention is often described as an information processing bottleneck. Controls access to higher levels of processing, short-term memory and consciousness. Hence, the strategy nature has developed to cope with information overload is to break down the problem of analyzing a visual scene: from a massively parallel approach to a rapid sequence of circumscribed recognitions.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 6

7

8

9

10

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 11

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 12

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 13

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 14

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 15

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 16 First Computational Model Didday & Arbib, 1975 introduced a “two visual systems” framework Koch & Ullman, Hum. Neurobiol., 1985 Introduce concept of a single topographic saliency map. Most salient location selected by a winner-take-all network.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 17

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 18 Shifter Circuits Anderson & van Essen, PNAS, 1987 Information dynamically routed through cortical hierarchy. Yields rotation- and scale-independent representation.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 19 Shifter Circuits (cont.) Olshausen et al., J Neurosci, 1993 Implemented shifter circuits and demonstrated proof of concept. Control neurons in the pulvinar send the (attention-based) control signals that will determine the “passing” region of the circuit, through a modulation of intracortical connection weights. Perform recognition using associative memory at top level.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 20 only attended item reaches output layer

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 21 Selective Tuning Model Tsotsos et al., Artificial Intelligence, attention modulates neurons to earliest levels; wherever there is a many-to-one mapping many-to-one mapping - signal interference controlled by surround inhibition throughout processing network throughout processing network -task knowledge biases computations throughout processing network - attentional control is local, distributed and internal - competition is based on WTA (different form than previous models) (different form than previous models) - pyramid representation with reciprocal convergence and divergence neuron ‘sees’ this receptive field subject ‘attends’ to single item

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 22 The basic idea (BBS 1990)

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 23 Selective Tuning Model processing pyramid inhibited pathways pass pathways unit of interest at top input Caputo & Guerra 1998 Bahcall & Kowler 1999 Vanduffel, Tootell, Orban 2000 Smith et al Kastner, De Weerd, Desimone, Ungerleider, 1998

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 24

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 25 Guided Search Wolfe, Psychonomic Bull. & Rev., 1994 How can we combine information from several modalities? Use top-down (task-dependent) weighting.

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 26

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 27

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 28

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 29

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 30

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 31

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 32

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 33

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 34

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 35

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 36

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 37

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 38

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 39

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 40

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 41

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 42 Evaluation of Advertising

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 43

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 44

Michael Arbib & Laurent Itti: CS664 – Spring Lecture 5: Visual Attention (bottom-up) 45 Brefczynski & DeYoe, Nature Neuroscience 1999