1. Cracking the Population Code Dario Ringach University of California, Los Angeles

2. Two basic questions in cortical computation: The Questions How is information represented? How is information processed?

3. How is information encoded in populations of neurons? Representation by Neuronal Populations

4. How is information encoded in populations of neurons? 1.Quantities are encoded as rate codes in ensembles of 50- 100 neurons (eg, Shadlen and Newsome, 1998). Representation by Neuronal Populations

5. How is information encoded in populations of neurons? 1.Quantities are encoded as rate codes in ensembles of 50- 100 neurons (eg, Shadlen and Newsome, 1998). 2.Quantities are encoded as precise temporal patterns of spiking across a population of cells (e.g, Abeles, 1991). Representation by Neuronal Populations

6. How is information encoded in populations of neurons? 1.Quantities are encoded as rate codes in ensembles of 50- 100 neurons (eg, Shadlen and Newsome, 1998). 2.Quantities are encoded as precise temporal patterns of spiking across a population of cells (e.g, Abeles, 1991). 3.Quantities might be encoded as the variance of responses across ensembles of neurons (Shamir & Sompolinsky, 2001; Abbott & Dayan, 1999) Representation by Neuronal Populations

7. Coding by Mean and Covariance Neuron #1 Neuron #2 Averbeck et al, Nat Rev Neurosci, 2006 Mean only B A Responses of two neurons to the repeated presentation of two stimuli:

8. Coding by Mean and Covariance Neuron #1 Neuron #2 Averbeck et al, Nat Rev Neurosci, 2006 Neuron #1 Mean onlyCovariance only B AA B Responses of two neurons to the repeated presentation of two stimuli:

9. Coding by Mean and Covariance Neuron #1 Neuron #2 Averbeck et al, Nat Rev Neurosci, 2006 Neuron #1 Mean onlyCovariance only Neuron #1 Both B AA B B A Responses of two neurons to the repeated presentation of two stimuli:

10. Macaque Primary Visual Cortex

11. Orientation Tuning Receptive field

12. Orientation Columns

13. Primary Visual Cortex 4mm V1 surface and vasculature under green illumination

14. Orientation Columns and Array Recordings 1mm Optical imaging of intrinsic signals under 700nm light

15. Alignment of Orientation Map and Array 0.0 0.4 Find the optimal translation and rotation of the array on the cortex that maximizes the agreement between the electrical and optical measurements of preferred orientation. (3 parameters and 96 data points!) Error surfaces:

16. Micro-machined Electrode Arrays

17. Array Insertion Sequence 12 34

18. Input Output Basic Experiment We record single unit activity (12-50 cells), multi-unit activity (50-80 sites) and local field potentials (96 sites). What can we say about:

19. Dynamics of Mean States

20. Dynamics of Mean Responses Multidimensional scaling to d=3 (for visualization only)

21. Dynamics of Mean Responses Multidimensional scaling to d=3 (for visualization only)

22. Stimulus Triggered Covariance

23. Covariance matrices are low-dimensional Average spectrum for co-variance matrices in two experiments

24. Covariance matrices are low-dimensional (!) Two Examples

25. Bhattacharyya Distance and Error Bounds Differences in meanDifferences in co-variance Bhattacharyya distance:

26. Information in Covariance Information in Mean

27. Bayes’ Decision Boundaries – N-category classification Hyperquadratic decision surfaces Where:

28. Confusion Matrix and Probability of Classification

30. Stimulus-Triggered Responses 150ms n=41 channels ordered according their preferred orientation Channel # (orientation) 0.0 2.0

31. Stimulus-Triggered Responses 150ms n=32 channels ordered according their preferred orientation Channel # (orientation) 0.0 2.0

32. Mean Population Responses

34. Population Mean and Variance Tuning

40. Bandwidth of Mean and Variance Signals

41. Estimates of Mean and Variance in Single Trials Population of independent Poisson spiking cells:

42. Estimating Mean and Variances Trial-to-Trial mean variance Noise correlation = 0.0

43. Estimating Mean and Variances Trial-to-Trial mean variance Noise correlation = 0.1

44. Estimating Mean and Variances Trial-to-Trial mean variance Noise correlation = 0.2

45. Tiling the Stimulus Space and Response Heterogeneity Dimension #1 Dimension #2 Orientation

46. Tiling the Stimulus Space and Response Heterogeneity Dimension #1 Dimension #2 Orientation Population response to the same stimulus

47. Tiling the Stimulus Space and Response Heterogeneity Dimension #1 Dimension #2 Orientation Population response to the same stimulus

48. Tiling the Stimulus Space and Response Heterogeneity Dimension #1 Dimension #2 Orientation Population response from independent single cell measurements

49. Tiling the Stimulus Space and Response Heterogeneity Dimension #1 Dimension #2 Orientation Population response from independent single cell measurements

50. Silberberg et al, J Neurophysiol., 2004 Can single cells respond to input variance?

51. Silberberg et al, J Neurophysiol., 2004

52. Summary Heterogeneity leads to population variance as a natural coding signal in the cortex. Response variance has as smaller bandwidth than the mean response. For small values of noise correlation variance is already a more reliable signal than the mean.

53. In a two-category classification problem the variance signal carries about 95% of the total information (carried by mean and variance together.) The covariance of the class-conditional population responses is low dimensional, with the first eigenvector most likely indicating fluctuations in cortical excitability (or gain). Cells may be perfectly capable of decoding the variance across their inputs (Silberberg et al, 2004) In prostheses, the use of linear decoding based on population rates may be sub-optimal. Quadratic models may work better. Summary

54. Acknowledgements V1 imaging/electrophysiology (NIH/NEI) Brian Malone Andy Henrie Ian Nauhaus Topological Data Analysis (DARPA) Gunnar Carlsson (Stanford) Guillermo Sapiro (UMN) Tigran Ishakov (Stanford) Facundo Memoli (Stanford) Bayesian Analysis of Motion in MT (NSF/ONR) Alan Yuille (UCLA) HongJing Lu (Hong Kong) Neovision phase 2 (DARPA) Frank Werblin (Berkeley) Volkan Ozguz (Irvine Sensors) Suresh Subramanian (Irvine Sensors) James DiCarlo (MIT) Bob Desimone (MIT) Tommy Poggio (MIT) Dean Scribner (Naval Research Labs)