1. Decision making as a model 7. Visual perception as Bayesian decision making

2. Given some stimulus, an ideal decision maker/observer would maximize expected utility: Σp i U i “Gain/loss” Posterior probability Likelihood Prior probability (pay off matrix) (“bias”) (Signal and noise distributions) Bayesian decision making Bayes’ rule (Proportion of signals) Cf. signal detection theory:

3. Why could it be a good idea to use a normative model for decision making in modeling vision? - The visual system may not be perfect, but it is quite good - Long evolutionary history in probabilistic environment - Evolved ultimately for doing things (preferably the “right” things) - Humans often deal better with uncertainty when perceiving than when thinking

4. State of world (distal stimulus) Distributions of light on retina (proximal stimulus) produces underdetermined Proximal stimulus is not enough But the visual system usually does make the inference, mostly quite well ! inference inference The problem of vision: very general

5. Example

6. Mamassian, 2001

7. A visual example Guo, Nevado, Robertson, Pulgarin, Thiel & Young (2004) Task (more or less- in reality much faster):

8. You will be presented with several small bars in quick succession. Your task is to indicate whether the last bar (at the blue dot) is collinear with the others. Throughout the experiment fixate the blue dot.

11. “Normal sequence”:4 predictors 200 ms each sequence Variations:-random order sequence -random duration sequence -probability of collinear stimuli (0-50%)

12. Orientation perception is influenced by prior stimuli. In Bayesian terms: by prior 75% “not collinear”

13. Model: orientation difference θ prior distribution normal (mean 0 o ; sd σ p ) “Real” θ leads to a noisy representation θ r in the brain: distribution normal (mean θ; sd σ l ) Perceived orientation: maximum (mean) of posterior distribution of p(θ|θ r )  p(θ r |θ) p(θ) : σ p 2 θ r θ pc = ------------ ( m p = 0) σ p 2 + σ l 2 θ pc is proportional to θ r (σ p 2 /(σ p 2 + σ l 2 ) ) θ r is distributed normally (mean θ; sd σ l ) and θ pc so θ pc is also distributed normally with proportional mean and sd From previous lecture: σ D 2 m p + σ p 2 D σ p 2 σ D 2 mean: ------------------- and variance: -------------- σ D 2 + σ p 2 σ D 2 + σ p 2

14. If perceived orientation difference exceeds threshold θ t, s indicates “not collinear” So for every θ there is a normal pdf of θ pc p(θ pc | θ) σ p 2 θ σ p 2 σ l with mean ------------ and sd --------- σ p 2 + σ l 2 σ p 2 + σ l 2 σ p 2 factor --------------- increases with σ p 2 σ p 2 + σ l 2

15. pdf(θ pc |θ) Fraction “collinear” Larger σ p : larger mean larger variance of p(θ pc |θ) (less influence of prior more influence of real θ) -θ t 0 θ t Orientation difference (θ) θ = 0 o θ = 4 o σ p 2 θ σ p 2 σ l mean ------------, sd --------- σ p 2 + σ l 2 σ p 2 + σ l 2

16. Probability can be computed from σ l, σ p, and θ t Model with three parameters can be fitted to data: within ss σ l θ t are kept constant

17. Fit for one subject: Same values for θ t and σ l, value of σ p varies over conditions

18. Explanation offered: horizontal connections in V1

19. Weiss, Simoncelli &Adelson, 2002 Bayesian explanation of visual illusion This time concentrating on variance of likelihood (σ l )

20. start

34. start

39. An ambiguous stimulus

40. vxvxvxvx vyvyvyvy Velocity space vxvxvxvx vyvyvyvy Prior: preference for low velocities likelihood

41. Another ambiguous stimulus

42. Not an ambiguous stimulus?

43. vxvxvxvx vyvyvyvy vxvxvxvx vyvyvyvy Average? Feature (e.g. intersection)? Intersection of constraints? Velocity space vxvxvxvx vyvyvyvy

44. vxvxvxvx vyvyvyvy vxvxvxvx vyvyvyvy vxvxvxvx vyvyvyvy prior Likelihood low contrast Likelihood high contrast

49. Vision as Bayesian decision making: Likelihood function: likelihood of proximal simulus given various states of the world (“what sensory systems can tell you about the world”) Noise contributes to the variance of the likelihood function

50. Prior pdf: expectation or information about what might be the case in the world, built in or gleaned from earlier experience Informative priors (“strong” expectations) have a small variance Vision as Bayesian decision making:

51. Posterior pdf: the strength of observer’s “convictions” about the possible scenes underlying the image What the observer sees corresponds to some representative value (what is “best” is task-dependent) Vision as Bayesian decision making:

52. Gain/Loss: task-related influence of possible consequences on response In perception often a loss function: -quadratic (  percept is mean of distribution) -narrow peak (  percept is mode of distribution: MAP) S dec - S ℓ(S dec,S) Vision as Bayesian decision making: