1. dopamine DA
serotonin 5-HT
noradrenaline NA
acetylchol. ACh

2. I(S,R)=Σs,rP(s,r)ln2[P(s,r)/P(s)P(r)]
H(S) = - ΣsP(s)ln2P(s)
H(R)
if P(s1,s2)=P(s1)P(s2) then H(s1,s2)=H(s1)+H(s2)
I(S,R)=Σs,rP(s,r)ln2[P(s,r)/P(s)P(r)]

3. If r is binary, e.g. P(r=1)=a P(r=0)=1-a 
H(R) = a ln2 (1/a) + (1-a) ln2 [1/(1-a)]

4. I(S,R) is further limited by the s  r mapping precisionθ θ s
ROC curves
False alarms
Hits
But note: ROC curves are
symmetrical for ‘normal’
signals

5. ‘High-threshold’ processes lead to asymmetrical ROCsθ s
ROC curves
False alarms
Hits
(remember this when we
discuss hippocampus and
neocortex..)

6. If r is binary, e.g. P(r=1)=a P(r=0)=1-a 
H(R) = a ln2 (1/a) + (1-a) ln2 [1/(1-a)]
I(S,R) is further limited by the s  r mapping precision
If r is linear, e.g r = k (s + δ) (Gaussian σs, σδ) 
I(S,R) = ½ ln2 (1+ω2) with ω = σs / σδ (signal-to-noise)

7. a threshold-linear unit is limited both by its response
sparsity (a) and by its signal-to-noise (ω)

9. Walsh patterns
Use a basis for all possible
stimuli to characterize
fully neuronal responses

10. Try then an information theoretic description
How?

11. Extract principal components

12. much more info in the
temporal waveform
T012 >> Ts !

13. Was it just
an artifact?
Finite size bias 
need to correct for it

15. Distributed Representations (rat CA1 place cells, from simultaneous
recordings by Wilson & McNaughton)

16. I(S,R)=Σs,rP(s,r)ln2[P(s,r)/P(s)P(r)]
H(S) = - ΣsP(s)ln2P(s)
H(R)
I(S,R)=Σs,rP(s,r)ln2[P(s,r)/P(s)P(r)]
I(S,R) < H(S) I(S,R) < H(R)
What if {r} is complex, or just high-dimensional?

17. I(S,S’) < I(S,R) (if decoding is honest)
Neural code
Decoding
One possible approach
I(S,S’) < I(S,R) (if decoding is honest)
Pro: reduced complexity H(R)  H(S)
Con: dependence on decoding algorithm

18. A Simplified History of Neural Complexity
Symbolic
 ({ri})
(Noam Chomsky)
106
107
108
109
yrs
Memory
ri (x,t)
(David Marr)
Spatial
r (x,t)
(e.g. Joseph Atick)
Chemical
r (t)
(e.g. Peter Dayan)

19.  ({ri}) ri (x,t) r (x,t) r (t)
A Simplified History of Neural Complexity
Symbolic
 ({ri})
(Noam Chomsky) 3
106
107
108
109
yrs
        mammalian species        
echidna
CA1
CA3 DG
platypus
Memory
ri (x,t)
(David Marr)
lizard 2 1
Spatial
r (x,t)
(e.g. Joseph Atick)
Chemical
r (t)
(e.g. Peter Dayan)

20. Metric

23. The goal: + colour… to account for data on contrast sensitivity
in single
neurons
+ colour…

24. Decorrelation in the absence of noise:
Spatial autocorrelation in the inputs
In Fourier space, for natural images

25. Decorrelation in the presence of noise:
(a simplified treatment; the full one in Atick and Redlich, 1990)

26. goldfish
primates
double opponency single opponency !!

29. Juergen Haag and Alexander Borst
The Journal of Neuroscience, April 15, 2002, 22(8): Dendro-Dendritic Interactions between Motion-Sensitive Large-Field Neurons in the Fly
Juergen Haag and Alexander Borst
For visual course control, flies rely on a set of motion-sensitive neurons called lobula plate tangential cells
(LPTCs). Among these cells, the so-called CH (centrifugal horizontal) cells shape by their inhibitory action
the receptive field properties of other LPTCs called FD (figure detection) cells specialized for figure-ground
discrimination based on relative motion. Studying the ipsilateral input circuitry of CH cells by means of
dual-electrode and combined electrical-optical recordings, we find that CH cells receive graded input from
HS (large-field horizontal system) cells via dendro-dendritic electrical synapses. This particular wiring scheme
leads to a spatial blur of the motion image on the CH cell dendrite, and, after inhibiting FD cells, to an
enhancement of motion contrast. This could be crucial for enabling FD cells to discriminate object from self motion.