Bayesian inference accounts for the filling-in and suppression of visual perception of bars by context Li Zhaoping 1 & Li Jingling 2 1 University College.

Slides:



Advertisements
Similar presentations
Visual Saliency: the signal from V1 to capture attention Li Zhaoping Head, Laboratory of natural intelligence Department of Psychology University College.
Advertisements

Chapter 3: Neural Processing and Perception. Lateral Inhibition and Perception Experiments with eye of Limulus –Ommatidia allow recordings from a single.
Visual Attention Attention is the ability to select objects of interest from the surrounding environment A reliable measure of attention is eye movement.
Neural Correlates of Visual Awareness. A Hard Problem Are all organisms conscious?
A saliency map model explains the effects of random variations along irrelevant dimensions in texture segmentation and visual search Li Zhaoping, University.
Lesions of Retinostriate Pathway Lesions (usually due to stroke) cause a region of blindness called a scotoma Identified using perimetry note macular sparing.
Outline Sensation, Perception, Behavior Process of sensation Perceived vs. “real” world Properties of perceptual processes - Adaptation, pattern coding.
Test on Friday!. Lesions of Retinostriate Pathway Lesions (usually due to stroke) cause a region of blindness called a scotoma Identified using perimetry.
Visual Hemifields and Perceptual Grouping Sarah Theobald & Nestor Matthews Department of Psychology, Denison University, Granville OH USA The human.
Chapter 6 The Visual System
Sensation Overview How is perception different from sensation? What is psychophysics? What do sense organs do? How does vision work? How does this compare.
Writing Workshop Find the relevant literature –Use the review journals as a first approach e.g. Nature Reviews Neuroscience Trends in Neuroscience Trends.
Searching for the NCC We can measure all sorts of neural correlates of these processes…so we can see the neural correlates of consciousness right? So what’s.
Post-test review session Tuesday Nov in TH241.
Decision making as a model 7. Visual perception as Bayesian decision making.
Pattern Recognition Pattern - complex composition of sensory stimuli that the human observer may recognize as being a member of a class of objects Issue.
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 6: Low-level features 1 Computational Architectures in Biological.
Visual Cognition I basic processes. What is perception good for? We often receive incomplete information through our senses. Information can be highly.
Baysian Approaches Kun Guo, PhD Reader in Cognitive Neuroscience School of Psychology University of Lincoln Quantitative Methods 2011.
Effects of Warning Validity and Proximity on Responses to Warnings Joachim Meyer, Israel HUMAN FACTORS, Vol. 43, No. 4 (2001)
1 / 41 Inference and Computation with Population Codes 13 November 2012 Inference and Computation with Population Codes Alexandre Pouget, Peter Dayan,
1 Computational Vision CSCI 363, Fall 2012 Lecture 31 Heading Models.
Lecture 2b Readings: Kandell Schwartz et al Ch 27 Wolfe et al Chs 3 and 4.
Graphical Models in Vision. Alan L. Yuille. UCLA. Dept. Statistics.
Chapter 3: Neural Processing and Perception. Neural Processing and Perception Neural processing is the interaction of signals in many neurons.
Dynamic Causal Modelling Will Penny Wellcome Department of Imaging Neuroscience, University College London, UK FMRIB, Oxford, May
BCS547 Neural Decoding.
1 Perception and VR MONT 104S, Fall 2008 Lecture 4 Lightness, Brightness and Edges.
Binding problems and feature integration theory. Feature detectors Neurons that fire to specific features of a stimulus Pathway away from retina shows.
Ch.9 Bayesian Models of Sensory Cue Integration (Mon) Summarized and Presented by J.W. Ha 1.
Temporary suppression of visual processing in an RSVP task: an attention blink? By Raymond, Shapiro, & Arnell JEP:HPP.
Optimal Eye Movement Strategies In Visual Search.
Signal detection Psychophysics.
Chapter 1: Introduction to Perception. Why Study Perception? Future careers –Graduate school work in perception Medical applications –Devices to assist.
Chapter 9: Perceiving Color. Figure 9-1 p200 Figure 9-2 p201.
Objectives Chapter 1, Part 1 Illustrate and describe the perceptual cycle Explain the difference between top-down and bottom-up processing Compare and.
1 Computational Vision CSCI 363, Fall 2012 Lecture 32 Biological Heading, Color.
Does the brain compute confidence estimates about decisions?
Contrast and Inferences
A Bayesian account of context-induced orientation illusions
Perception Unit How can we so easily perceive the world? Objects may be moving, partially hidden, varying in orientation, and projected as a 2D image.
Early Processing in Biological Vision
Brain States: Top-Down Influences in Sensory Processing
Bayesian inference Li Zhaoping1 & Li Jingling2 accounts for the
Dynamic Causal Modelling
Jason A. Cromer, Jefferson E. Roy, Earl K. Miller  Neuron 
A Vestibular Sensation: Probabilistic Approaches to Spatial Perception
Minami Ito, Gerald Westheimer, Charles D Gilbert  Neuron 
Braden A. Purcell, Roozbeh Kiani  Neuron 
Stereopsis Current Biology
The Neural Basis of Perceptual Learning
Bayesian Methods in Brain Imaging
Confidence as Bayesian Probability: From Neural Origins to Behavior
A Switching Observer for Human Perceptual Estimation
Attentional Modulations Related to Spatial Gating but Not to Allocation of Limited Resources in Primate V1  Yuzhi Chen, Eyal Seidemann  Neuron  Volume.
Ralf M. Haefner, Pietro Berkes, József Fiser  Neuron 
Attention Increases Sensitivity of V4 Neurons
Brain States: Top-Down Influences in Sensory Processing
A Switching Observer for Human Perceptual Estimation
Fangtu T. Qiu, Rüdiger von der Heydt  Neuron 
Ryo Sasaki, Takanori Uka  Neuron  Volume 62, Issue 1, Pages (April 2009)
Bayesian vision Nisheeth 14th February 2019.
Perceptual learning Nisheeth 15th February 2019.
Stephen V. David, Benjamin Y. Hayden, James A. Mazer, Jack L. Gallant 
The Normalization Model of Attention
Experiencing the World
Jason A. Cromer, Jefferson E. Roy, Earl K. Miller  Neuron 
by Kenneth W. Latimer, Jacob L. Yates, Miriam L. R
Top-Down Modulation of Lateral Interactions in Early Vision
Outline Human Visual Information Processing – cont.
Presentation transcript:

Bayesian inference accounts for the filling-in and suppression of visual perception of bars by context Li Zhaoping 1 & Li Jingling 2 1 University College London, 2 China Medical School Based on a publication Zhaoping & Jingling (2008) PLoS Comput. Biol. Ask me for a copy, or download from

Low response from V1 cells to a low contrast bar Higher responses from V1 cells when there is a colinear flanker Test bar within the receptive field of a V1 cell Contextual bar outside the receptive field Some previously known contextual influences in vision Colinear facilitation in V1 Nelson & Frost 1985, Kapadia et al 1995, Li, Piech, & Gilbert 2006 Human sensitivity to detect target bar enhanced by colinear flankers. Polat & Sagi 1993, Morgan & Dresp 1995, Yu & Levi 2000, Huang et al 2006 etc Location for the target bar to appear if it does. Contextual bar 1 st presentation interval 2 st presentation interval Humans are better at saying which interval (the 2AFC task ) contains the target with the colinear contextual bars

A demo of the perception in this study: presence or absence of a vertical target in four different contexts Bar seems present Bar seems absent Bar present? Bar absent? These three combined are contrary to what one may expect from the colinear facilitation in V1 cells and human sensitivity for bar detection WHY???

Hints from another example of how context influence visual perception The same input of a white patch on retina Different perceptions inside the brain Perception fills in the occluded part of the square

Focus of this study: Contextual influence in perceptual bias, not in input sensitivity, in visual object inference not in input image representation Methods in the study: Psychophysical: rather than the 2AFC method, we used one-interval method (observers answered after each one-interval stimulus presentation whether the target bar was present) to probe perceptual bias (rather than input sensitivity). Whether the perception is veridical is not an issue, since we study inference rather than input representation. Computational: build a Bayesian inference model to understand the psychophysical data, showing that the model fits the data with fewer parameters than needed by phenomenological (e.g., logistic) models of the psychometric functions.

without context Psychophysical investigation Ask an observer (with one interval presentation): Is the vertical bar present? Answer: yes or no? Target contrast C t P(yes| C t ) 1 Targets without context contextual effect Yes rate, or psychometric function P(yes|C t ): Probability an observer answering “yes” given contrast C t of the target bar in image 0 With context Targets with context

Observations in an experiment: Perception of the target bar more likely when contextual contrast C c is low Contextual contrast C c is lower, Yes rate P(yes|C t ) is larger Contextual contrast C c is higher, Yes rate P(yes|C t ) is smaller Observations contrary to expectation, since in V1 (primary visual cortex), a neuron’s response is facilitated by the presence of colinear contextual bars. So stronger colinear context should facilitate more. Experiment randomly interleaved trials of different target contrast C t and contextual contrast C c, including C c =0 for the no context condition.

No target in image Seeing ghost? target with contrast C t Not seen without context seen without context Seeing less Perceptual suppression Data Perceptual filling-in P(yes|C t ) In dim vague context with low C c Strong context with high C c No context In medium context with CcCc

Two contextual configuations: colinear and orthogonal Target Less visible Target Less visible Colinear context Orthogonal context

Two contextual configurations: colinear and orthogonal Bright context C c = 0.4 Dim context C c = 0.01 Colinear context Orthogonal context Data suppresses perception regardless of contextual configuration Filling-in only in co-linear contextual configuration colinear orthogonal No context c olinear orthogonal No context Dim context Bright context Again, trials of all contextual and target variations were randomly interleaved

Received visual signal: C t : e.g.: neural activities in response to the target bar or noise. Making decision on: yes or no, the bar is there or not Prior believed probabilities: P(yes), P(no) = 1-P(yes) of visual events “yes” or “no”. Conditional Probability: P(C t |yes), likelihood or evidence of likely contrast C t for target present. Decision probability: P(yes|C t ) = P(C t |yes)P(yes)/P(C t ) Understanding by Bayesian inference : Decision: P(yes | C t ) = P(C t |yes) P(yes) P(C t |yes) P(yes) + (1-P(yes)) P(C t | no) Note: the model above is derived from a neural level model, when input contrasts evoke neural responses, when the brain has an internal model of the likely neural responses to input contrast, and how neural responses to contextual inputs influences the priors and the “likelihood” models. Details of the derivations can be found in the published paper Zhaoping & Jingling 2008.

A Bayesian account: the priors P(yes) and the evidence (likelihood) P(C t |yes) Evidence P(C t |yes) Larger Smaller Larger prior P(yes) in aligned context Smaller prior P(yes) in non-aligned context Prior P(yes) Larger Smaller When these different conditions are interleaved within the same experimental session, different priors manifest themselves in different trials ---- rapidly switching between priors!!!

Bayesian model formulation: Context influences decision in two ways: (1)Contextual configuration determines the prior prob. parameter P(yes) (2) Contextual contrast C c determines likelihood P(C t |yes) P(C t |yes) ~ exp [ -|C t –C c |/ (k C c ) ] favouring targets that resemble context in contrast 3 Parameters: k, σ n, P(yes) can completely model a given contextual configuration to give P(yes|C t ) for all C t and C c P(C t |yes) P(yes) P(C t |yes) P(yes) + (1-P(yes)) P(C t | no) Decision: P(yes | C t ) = Additionally: P(C t |no) ~ exp(-C t / σ n ) a model of noise contrast Note: P(C t |yes) is not the probability of the experimenter presenting a contrast C t for the target, nor is P(yes) the prob. of experimenter presenting a target. Both P(C t |yes) and P(yes) are internal models in the observer’s brain only, and “yes” and “no” refer the brain’s perceptions and assumptions rather than the external stimulus, see paper for more details.

CtCt C t = C c P(C t |no) P(C t |yes) Evidence P(C t |no) for non-target is higher when target contrast C t is close to zero Evidence P(C t |yes) for target is higher for target contrast C t resembling the contextual contrast C c. The decision P(yes|C t ) results from weighing the evidences P(C t |yes) and P(C t |no), for and against the target, weighted by the priors P(yes) and P(no)=1-P(yes) Decision: P(yes | C t ) = P(C t |yes) P(yes) P(C t |yes) P(yes) + (1-P(yes)) P(C t | no) Bayesian decision by combining input evidences with prior beliefs

Effect of contextual contrast C c Weaker C c --- larger P(C t | yes) Stronger C c --- smaller P(C t |yes) Effect of prior P(yes) Higher P(yes) Lower P(yes) Weaker contextual contrast C c and/no higher prior P(yes) bias the response to “yes” P(yes|C t )

The Bayesian model can fit the data well No co-linear facilitation mechanism necessary for explaining the data!!!! If a 4 th parameter for colinear facilitation is fitted, it returns a zero facilitation magnitude Fitting data from a colinear context --- a total of 3 model parameters ( k, P(yes), σ n ) to fit all 3 psychometric curves Typically, 6 parameters would be needed to adequately fit 3 psychometric curves. Using only 3 parameters to fit the data, the Bayesian model demonstrates its adequacy. Solid curves are the results of the Bayesian fit

Bayesian model fitting data from the exp. including both colinear and orthogonal contexts Fitted Priors P(yes)Four model parameters: k, σ n, P(yes) colinear, P(yes) orthogonal used to fit all 4 psychometric curves P(yes) colinear and P(yes) orthogonal are both quite big, reflecting an additional response bias by the subjects to respond roughly 50% “yes” in total. Colinear context, data and fitsOrthogonal context, data and fits

Comparing Bayesian and logistic fitting results: Dashed curves: Logistic fits --- using 8 parameters Solid curves: Bayesian fits --- using 4 parameters Mean Fitting Error in units of error bar size = 0.83 for logistic Mean Fitting Error in units of error bar size = 1.01 for Bayesian Weak contextStrong context

Another example: more subtle difference in context or P(yes)

Data fitting for 3 different contexts, 3 different Contextual contrasts C c = 0.01, 0.05, Bayesian parameters, 18 Logistic parameters. Mean Fitting Error in units of error bar size = 0.54 for logistic fits (dashed curves) Mean Fitting Error in units of error bar size = 1.07 for Bayesian fits (solid curves) Fitted Priors P(yes)

Summary: Studied contextual influence in perceptual bias --- filling-in & suppression Study uses simple stimuli, more easily controlled and modelled one interval tasks used to study bias rather than sensitivities. Found context influences perception by (1) affecting prior expectation of perceptions (2) affecting likelihood model of sensory inputs Findings (1) accountable by a Bayesian inference model (2) unexpected from colinear facilitation in V1, suggest mechanisms beyond V1

Other related works and issues: Contextual influences in object recognition and attentional guidance Contextual effects on mid-level vision assimilation and induction in the perception of motion, orientation, color, and lightness etc. Effects of the input signal-to-noise on input encoding and perception. Perceptual ambiguity Relationship/difference between object inference and image representation Bayesian inference in vision in many previous works, often with more complex stimuli (which can be difficult to manipulate and model) Statistics of the natural scenes and adaptation Decision making, and internal beliefs unchanged by input samples. Etc, etc. … see detailed discussions in the published papers.

2AFC tasks remove the effects of the priors: Visual Signal received: x1, x2, for time interval 1 and 2. Making decision on: y Prior expectation: P(y) Conditional Probability: P(x1|y), P(x2|y) Decision based on: P(y|x1) > ? < P(y|x2) P(x1|y)P(y)/P(x1) > ? < P(x2|y)P(y)P(x2)