1. Scaling-up Cortical Representations David W. McLaughlin Courant Institute & Center for Neural Science New York University http://www.cims.nyu.edu/faculty/dmac/ NJIT-- May ‘04

2. In collaboration with:   David Cai Louis Tao Michael Shelley Aaditya Rangan

5. Lateral Connections and Orientation -- Tree Shrew Bosking, Zhang, Schofield & Fitzpatrick J. Neuroscience, 1997

8. Coarse-Grained Asymptotic Representations Needed for “Scale-up”

10. Coarse-Grained Reductions for V1 Average firing rate models (Cowan & Wilson; ….; Shelley & McLaughlin) m  (x,t),  = E,I

11. But realistic networks have a very noisy dynamics Strong temporal fluctuations On synaptic timescale Fluctuation driven spiking

12. Fluctuation-driven spiking Solid: average ( over 72 cycles) Dashed: 10 temporal trajectories (very noisy dynamics, on the synaptic time scale)

13. Experiment Observation Fluctuations in Orientation Tuning (Cat data from Ferster’s Lab) Ref: Anderson, Lampl, Gillespie, Ferster Science, 1968-72 (2000) threshold (-65 mV)

14. To accurately and efficiently describe these fluctuations, the scale-up method will require pdf representations –   (v,g; x,t),  = E,I To “benchmark” these, we will numerically simulate I&F neurons within one CG “patch”

15. Coarse-Grained Reductions for V1 PDF representations (Knight & Sirovich; Tranchina, Nykamp & Haskell; Cai, Tao, Shelley & McLaughlin)   (v,g; x,t),  = E,I

16. Coarse-Grained Reductions for V1 PDF representations (Knight & Sirovich; Tranchina, Nykamp & Haskell; Cai, Tao, Shelley & McLaughlin)   (v,g; x,t),  = E,I Sub-network of embedded point neurons -- in a coarse- grained, dynamical background (Cai,Tao & McLaughlin)

18. We’ll replace the 200 neurons in this CG cell by an effective pdf representation For convenience of presentation, I’ll sketch the derivation of the reduction for 200 excitatory Integrate & Fire neurons Later, I’ll show results with inhibition included – as well as “simple” & “complex” cells.

19. N excitatory neurons (within one CG cell) All-to-all coupling; AMPA synapses (with time scale  )   t v i = -(v – V R ) – g i (v-V E )   t g i = - g i +  l f  (t – t l ) + (Sa/N)  l,k  (t – t l k )  (g,v,t)  N -1  i=1,N E{  [v – v i (t)]  [g – g i (t)]}, Expectation “E” over Poisson spike train

20.   t v i = -(v – V R ) – g i (v-V E )   t g i = - g i +  l f  (t – t l ) + (Sa/N)  l,k  (t – t l k ) Evolution of pdf --  (g,v,t):  t  =  -1  v {[(v – V R ) + g (v-V E )]  } +  g {(g/  )  } + 0 (t) [  (v, g-f/ , t) -  (v,g,t)] + N m(t) [  (v, g-Sa/N , t) -  (v,g,t)], where 0 (t) = modulated rate of incoming Poisson spike train; m(t) = average firing rate of the neurons in the CG cell =  J (v) (v,g;  )| (v= 1) dg where J (v) (v,g;  ) = -{[(v – V R ) + g (v-V E )]  }

21.  t  =  -1  v {[(v – V R ) + g (v-V E )]  } +  g {(g/  )  } + 0 (t) [  (v, g-f/ , t) -  (v,g,t)] + N m(t) [  (v, g-Sa/N , t) -  (v,g,t)], N>>1; f << 1; 0 f = O(1);  t  =  -1  v {[(v – V R ) + g (v-V E )]  } +  g {[g – G(t)]/  )  } +  g 2 /   gg  + … where  g 2 = 0 (t) f 2 /(2  ) + m(t) (Sa) 2 /(2N  ) G(t) = 0 (t) f + m(t) Sa

22. Moments  (g,v,t)  (g) (g,t) =   (g,v,t) dv  (v) (v,t) =   (g,v,t) dg  1 (v) (v,t) =  g  (g,t  v) dg where  (g,v,t) =  (g,t  v)  (v) (v,t) Integrating  (g,v,t) eq over v yields:   t  (g) =  g {[g – G(t)])  (g) } +  g 2  gg  (g)

23.  t  =  -1  v {[(v – V R ) + g (v-V E )]  } +  g {[g – G(t)]/  )  } +  g 2 /   gg  + …

24. Moments  (g,v,t)  (g) (g,t) =   (g,v,t) dv  (v) (v,t) =   (g,v,t) dg  1 (v) (v,t) =  g  (g,t  v) dg where  (g,v,t) =  (g,t  v)  (v) (v,t) Integrating  (g,v,t) eq over v yields:   t  (g) =  g {[g – G(t)])  (g) } +  g 2  gg  (g)

25. Fluctuations in g are Gaussian   t  (g) =  g {[g – G(t)])  (g) } +  g 2  gg  (g)

26. Integrating  (g,v,t) eq over g yields:  t  (v) =  -1  v [(v – V R )  (v) +  1 (v) (v-V E )  (v) ] Integrating [g  (g,v,t)] eq over g yields an equation for  1 (v) (v,t) =  g  (g,t  v) dg, where  (g,v,t) =  (g,t  v)  (v) (v,t)

27.  t  =  -1  v {[(v – V R ) + g (v-V E )]  } +  g {[g – G(t)]/  )  } +  g 2 /   gg  + …

28.  t  1 (v) = -  -1 [  1 (v) – G(t)] +  -1 {[(v – V R ) +  1 (v) (v-V E )]  v  1 (v) } +  2 (v)/ (  (v) )  v [(v-V E )  (v) ] +  -1 (v-V E )  v  2 (v) where  2 (v) =  2 (v) – (  1 (v) ) 2. Closure: (i)  v  2 (v) = 0; (ii)  2 (v) =  g 2

29. Thus, eqs for  (v) (v,t) &  1 (v) (v,t):  t  (v) =  -1  v [(v – V R )  (v) +  1 (v) (v-V E )  (v) ]  t  1 (v) = -  -1 [  1 (v) – G(t)] +  -1 {[(v – V R ) +  1 (v) (v-V E )]  v  1 (v) } +  g 2 / (  (v) )  v [(v-V E )  (v) ] Together with a diffusion eq for  (g) (g,t):   t  (g) =  g {[g – G(t)])  (g) } +  g 2  gg  (g)

30. But we can go further for AMPA (   0):  t  1 (v) = - [  1 (v) – G(t)] +  -1 {[(v – V R ) +  1 (v) (v-V E )]  v  1 (v) } +  g 2 / (  (v) )  v [(v-V E )  (v) ] Recall  g 2 = f 2 /(2  ) 0 (t) + m(t) (Sa) 2 /(2N  ); Thus, as   0,  g 2 = O(1). Let   0: Algebraically solve for  1 (v) :  1 (v) = G(t) +  g 2 / (  (v) )  v [(v-V E )  (v) ]

31. Result: A Fokker-Planck eq for  (v) (v,t):   t  (v) =  v {[ (1 + G(t) +  g 2 /  ) v – ( V R + V E (G(t) +  g 2 /  ) ) ]  (v) +  g 2 /  (v- V E ) 2  v  (v) }  g 2 /  -- Fluctuations in g

32. Remarks: (i) Boundary Conditions; (ii) Inhibition, spatial coupling of CG cells, simple & complex cells have been added; (iii) N   yields “mean field” representation.

33. New Pdf Representation  (g,v,t) -- (i) Evolution eq, with jumps from incoming spikes; (ii) Jumps smoothed to diffusion in g by a “large N expansion”  (g) (g,t) =   (g,v,t) dv -- diffuses as a Gaussian  (v) (v,t) =   (g,v,t) dg ;  1 (v) (v,t) =  g  (g,t  v) dg Coupled (moment) eqs for  (v) (v,t) &  1 (v) (v,t), which are not closed [but depend upon  2 (v) (v,t) ] Closure -- (i)  v  2 (v) = 0; (ii)  2 (v) =  g 2, where  2 (v) =  2 (v) – (  1 (v) ) 2.   0  eq for  1 (v) (v,t) solved alge…y in terms of  (v) (v,t), resulting in a Fokker-Planck eq for  (v) (v,t)

34. Local temporal asynchony enhanced by synaptic failure – permitting better amplification

35. 1 - p: Synaptic Failure rate

38. Under ASSUMPTIONS: 1) 2) Summed intra-cortical low rate spike events become Poisson : 2) Summed intra-cortical low rate spike events become Poisson :

40. Closures:

42. PDF of v Theory→ ←I&F (solid) Fokker-Planck→ Theory→ ←I&F ←Mean-driven limit ( ): Hard thresholding Fluctuation-Driven Dynamics N=75 σ=5msec S=0.05 f=0.01 firing rate (Hz)

43. PDF of v Theory→ ←I&F (solid) Fokker-Planck→ Theory→ ←I&F ←Mean-driven limit ( ): Hard thresholding Fluctuation-Driven Dynamics N=75 σ=5msec S=0.05 f=0.01 Experiment firing rate (Hz)

44. Mean­Driven: Bistability and Hysteresis   Network of Simple, Excitatory only Fluctuation­Driven: N=16 Relatively Strong Cortical Coupling: N=16!

45. Mean­Driven: N=16! Bistability and Hysteresis   Network of Simple, Excitatory only Relatively Strong Cortical Coupling:

47. Simple & Complex Cells; Multiple interacting CG Patches

48. Recall:

53. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory

54. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Complex Excitatory Cells Mean-Driven

55. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Complex Excitatory Cells Mean-Driven

56. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Complex Excitatory Cells Mean-Driven

57. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Complex Excitatory Cells Mean-Driven

58. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Simple Excitatory Cells Mean-Driven

59. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Simple Excitatory Cells Mean-Driven

60. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Simple Excitatory Cells Mean-Driven

61. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Simple Excitatory Cells Mean-Driven

62. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Simple Excitatory Cells Mean-Driven

63. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Complex Excitatory Cells Fluctuation-Driven

64. Incorporation of Inhibitory Cells 4 Population Dynamics Simple:   Excitatory   Inhibitory Complex:   Excitatory   Inhibitory Simple Excitatory Cells Fluctuation-Driven

65. Three Dynamic Regimes of Cortical Amplification: 1) Weak Cortical Amplification No Bistability/Hysteresis 2) Near Critical Cortical Amplification 3) Strong Cortical Amplification Bistability/Hysteresis (2) (1) (3) I&F Excitatory Complex Cells Shown (2) (1)

66. Simple & Complex Cells; Multiple interacting CG Patches

67. Fluctuation­Driven Tuning Dynamics Near Critical Amplification vs. Weak Cortical Amplification Sensitivity to Contrast Ring Model A less-tuned complex cell Ring Model — far field A well-tuned complex cell Large V1 Model A complex cell in the far-field Ring Model of Orientation Tuning: Near Pinwheel Far Field A Cell in Large V1 Model

68. Computational Efficiency For statistical accuracy in these CG patch settings, Kinetic Theory is 10 3 -- 10 5 more efficient than I&F;

69. Conclusions Kinetic Theory is a numerically efficient, and remarkably accurate, method for “scale-up”. Kinetic Theory introduces no new free parameters into the model, and has a large dynamic range from the rapid firing “mean-driven” regime to a fluctuation driven regime. Kinetic Theory does not capture detailed “spike-timing” Sub-networks of point neurons can be embedded within kinetic theory to capture spike timing, with a range from test neurons to fully interacting sub-networks.

70. Embedded Point Neurons Firing rate codes Vs Spike timing codes

72. Embedded Point Neurons For “scale-up” – computer efficiency Yet maintaining firing properties of individual neurons -- for spike coding, coincidence detection, etc. Model relevant for biologically distinguished sparse, strong sub- networks – perhaps such as long-range connections Point neurons -- embedded in, and fully interacting with, coarse- grained kinetic theory, Or, when kinetic theory accurate by itself, embedded as “test neurons”

75. I&F vs. Embedded Network Spike Rasters a) I&F Network: 50 “Simple” cells, 50 “Complex” cells. “Simple” cells driven at 10 Hz b)-d) Embedded I&F Networks: b) 25 “Complex” cells replaced by single kinetic equation; c) 25 “Simple” cells replaced by single kinetic equation; d) 25 “Simple” and 25 “Complex” cells replaced by kinetic equations. In all panels, cells 1-50 are “Simple” and cells 51-100 are “Complex”. Rasters shown for 5 stimulus periods.

77. I&F vs. Embedded Network Spike Rasters a) I&F Network: 40 “Simple” cells, 40 “Complex” cells. “Simple” cells driven at 10 Hz; b)-d) Embedded I&F Networks: b) 20 “Complex” cells replaced by log-form rate equation; c) 20 “Simple” cells replaced by log-form rate equation; d) 20 “Simple” and 20 “Complex” cells replaced by log-form rate equations. In all panels, cells 1-40 are “Simple” and cells 41-80 are “Complex”. Rasters shown for 5 stimulus periods.

78. a) I&F Network: 50 Simple cells, 50 Complex”cells. “Simple”cells driven at 10 Hz; b) Embedded I&F Networks, with Complex I&F neurons receiving input but not outputing to the network. 25 Complex”cells replaced by kinetic equations. In all panels, cells 1-50 are Simple and cells 51-75 are Complex”. Rasters shown for 5 stimulus periods.

79. Dynamic Firing Rates: (Shown: Exc Simple Pop in a 4 Pop Model) Forcing--time-dependent Poisson process, sinusoidal driving at 10 Hz. In the embedded models, excitatory neurons are I&F and inhibitory neurons are replaced by Kinetic Theory.

80. Cycle-averaged Firing Rate Curves [Shown: Exc Cmplx Pop in a 4 population model): Full I&F network (solid), Full I&F + KT (dotted); Full I&F coupled to Full KT but with mean only coupling (dashed).] In both embedded cases (where the I&F units are coupled to KT), half the simple cells are represented by Kinetic Theory The Importance of Fluctuations

81. Steady State Firing Rate Curves (for the excitatory population): I&F, Exc. & Inh. (Magenta), Kinetic Theory, Exc. & Inh. (Red), Embedded Network (Blue), Log-form (Green). In the Embedded model, Excitatory neurons are I&F and Inhibitory neurons are modelled by Kinetic Theory.

82. Steady State Firing Rate Curves for Embedded Sub-networks: (Shown, Exc Cmplx Pop): In the embedded models, half of the simple cells are replaced by Kinetic Theory. However, in the “I&F + KT (mean only), they are coupled back to the I&F neurons as mean conductance drives.

83. (From left to right) Rasters, Cross-correlation and ISI distributions for two simulations: (Upper panels) Kinetic Theory of a neuronal patch driving test neurons, which are not coupled to each other; (Lower panels) KT of a neuronal patch driving strongly coupled neurons. In both cases, the CGed patch is being driven at 1 Hz. Neurons 1-6 are excitatory; Neurons 7-8 are inhibitory; the S-matrix is (S ee, S ei, S ie, S ii ) = (0.3, 0.4, 0.6, 0.4). EPSP time constant 3 ms; IPSP time constant 10 ms.

84. (From left to right) Rasters, Cross-correlation and ISI distributions for two simulations: (Upper panels) Kinetic Theory of a neuronal patch driving test neurons, which are not coupled to each other; (Lower panels) KT of a neuronal patch driving strongly coupled neurons. In both cases, the CGed patch is being driven “asynchronously,” and are firing at a fixed rate. I&F Neurons 1-6 are excitatory; Neurons 7-8 are inhibitory; the S-matrix is (S ee, S ei, S ie, S ii ) = (0.4, 0.4, 0.8, 0.4). EPSP time constant 3 ms; IPSP time constant 10 ms.

85. Raster Plots, Cross-correlation and ISI distributions. (Upper panels) KT of a neuronal patch with strongly coupled embedded neurons; (Lower panels) Full I&F Network. Shown is the sub-network, with neurons 1-6 excitatory; neurons 7-8 inhibitory; EPSP time constant 3 ms; IPSP time constant 10 ms. Embedded Network Full I & F Network

86. ISI distributions for two simulations: (Left) Test Neuron driven by a CG neuronal patch; (Right) Sample Neuron in the I&F Network. “Test neuron” within a CG Kinetic Theory

87. Reverse Time Correlations Correlates spikes against driving signal Triggered by spiking neuron Frequently used experimental technique to get a handle on one description of the system P( ,  ) – probability of a grating of orientation , at a time  before a spike -- or an estimate of the system’s linear response kernel as a function of ( ,  )

88. Time → Reverse-Time Correlation (RTC)   System analysis   Probing network dynamics

89. Reverse Correlation Left: I&F Network of 128 “Simple” and 128 “Complex” cells at pinwheel center. RTC P(  ) for single Simple cell. Below: Embedded Network of 128 “Simple” cells, with 128 “Complex” cells replaced by single kinetic equation. RTC P(  ) for single Simple cell.

90. Computational Efficiency For statistical accuracy in these CG patch settings, Kinetic Theory is 10 3 -- 10 5 more efficient than I&F; The efficiency of the embedded sub-network scales as N 2, where N = # of embedded point neurons; (i.e. 100  20 yields 10,000  400)

93. Conclusions and Directions Constructing ideal network models to discern and extract possible principles of neuronal computation and functions Mathematical methods for analytical understanding Search for signatures of identified mechanisms Mean-driven vs. fluctuation-driven kinetic theories New closure, Fluctuation and correlation effects Excellent agreement with the full numerical simulations Large-scale numerical simulations of structured networks constrained by anatomy and other physiological observations to compare with experiments Structural understanding vs. data modeling New numerical methods for scale-up --- Kinetic theory

94. Three Dynamic Regimes of Cortical Amplification: 1) Weak Cortical Amplification No Bistability/Hysteresis 2) Near Critical Cortical Amplification 3) Strong Cortical Amplification Bistability/Hysteresis (2) (1) (3) I&F Excitatory Cells Shown  Possible Mechanism for Orientation Tuning of Complex Cells Regime 2 for far-field/well-tuned Complex Cells Regime 1 for near-pinwheel/less-tuned Summed Effects (2) (1)

95. Summary & Conclusion

96. Summary Points for Coarse-Grained Reductions needed for Scale-up 1.Neuronal networks are very noisy, with fluctuation driven effects. 2.Temporal scale-separation emerges from network activity. 3.Local temporal asynchony needed for the asymptotic reduction, and it results from synaptic failure. 4.Cortical maps -- both spatially regular and spatially random -- tile the cortex; asymptotic reductions must handle both. 5.Embedded neuron representations may be needed to capture spike-timing codes and coincidence detection. 6.PDF representations may be needed to capture synchronized fluctuations.

97. Scale-up & Dynamical Issues for Cortical Modeling of V1 Temporal emergence of visual perception Role of spatial & temporal feedback -- within and between cortical layers and regions Synchrony & asynchrony Presence (or absence) and role of oscillations Spike-timing vs firing rate codes Very noisy, fluctuation driven system Emergence of an activity dependent, separation of time scales But often no (or little) temporal scale separation

98. Under ASSUMPTIONS: 1) 2) Summed intra-cortical low rate spike events become Poisson : 2) Summed intra-cortical low rate spike events become Poisson :

100. Closures:

108. Kinetic Theory for Population Dynamics Population of interacting neurons: 1-p: Synaptic Failure rate

109. Kinetic Equation: Under ASSUMPTIONS: 1) 2) Summed intra-cortical low rate spike events become Poisson : 2) Summed intra-cortical low rate spike events become Poisson :

110. Fluctuation-Driven Dynamics Physical Intuition: Fluctuation-driven/Correlation between g and V Hierarchy of Conditional Moments

111. Closure Assumptions: Closed Equations — Reduced Kinetic Equations Closed Equations — Reduced Kinetic Equations :

112.   Fluctuation Effects   Correlation Effects Fokker-Planck Equation: Flux: Determination of Firing Rate: For a steady state, m can be determined implicitly Coarse-Graining in Time: