1. Tahereh Toosi IPM

2. Recap 2 [Churchland and Abbott, 2012]

3. Outline Some general problems in Neuroscience Information Theory IT in Neuroscience Entropy Mutual Information How to use Mutual Information? Summary 3

4. Some general problems Coding scheme : What is being encoded? How is it being encoded? With what precision? With what limitations? Multi trial or single trial? Goodness of our neural decoding model 4 [Borst and Theunissen, 1999]

5. Stimulus space 5 Stimulus parameter S 1 Stimulus parameter S 2 Distribution of stimuli P[S|{t i }] given spike train Estimate of stimulus given spike train accessible region for all stimuli in P[S] [Spikes : Exploring the neural code]

6. Statistical significance of how neural responses vary with different stimuli 6 [Borst and Theunissen, 1999]

7. Shannon helps… 7 [Borst and Theunissen, 1999]

8. Information Theory 8 information is reported in units of ‘bits’, with :  The minus sign makes h a decreasing function of its argument as required.  Note that information is really a dimensionless number. [Abbott and Dayan, 2001]

9. Encoding numbers in a digital code! 9 [Spikes : Exploring the neural code] one hand can carry log 2 (6) ≈22.58 bits

10. Channel capacity of a neuron 10 [Borst and Theunissen, 1999]

11. Information loss 11

12. Entropy 12 quantifies the surprise or unpredictability associated with a particular response Shannon’s entropy is just this measure averaged over all responses: [Abbott and Dayan, 2001] H =−(1− P[r + ]) log 2 (1− P[r + ])− P[r + ] log 2 P[r + ] The entropy of a binary code

13. To convey information about a set of stimuli, neural responses must be different for different stimuli. Entropy is a measure of response variability Mutual Information idea : comparing the responses obtained using a different stimulus on every trial with those measured in trials involving repeated presentations of the same stimulus The mutual information is the difference between the total response entropy and the average response entropy on trials that involve repetitive presentation of the same stimulus Mutual Information idea 13

14. Mutual Information 14 The entropy of the responses to a given stimulus : The noise entropy: This is the entropy associated with that part of the response variability that is not due to changes in the stimulus, but arises from other sources. Mutual Information

15. How to use Information Theory: 1.Show your system stimuli. 2.Measure neural responses. 3.Estimate: P( neural response | stimulus presented ) 4.From that, Estimate: P( neural repsones ) 5.Compute: H(neural response) and H(neural response | stimulus presented) 6. Calculate: I(response ; stimulus) 15

16. Entropy : spike count 16 11001010 4 01000110 3 00110111 5 11000000 2 Stimuli responses spike count

17. How to screw it up: Choose stimuli which are not representative. Measure the “wrong” aspect of the response. Don’t take enough data to estimate P( ) well. Use a crappy method of computing H( ). Calculate I( ) and report it without comparing it to anything... 17 [Math for Neuroscience, Stanford]

18. Summary Strength of IT in Neuroscience: Coding efficiency : Comparing the overall information transfer to maximum spike train entropy Calculating the absolute amount of information transmitted Identifying system nonlinearities and validating any nonlinear system Neural code precision : The measure of Mutual Information Goodness of our neuronal decoding model : comparison of upper/lower estimate to direct estimate 18

19. THANK YOU! 19

20. 20

21. 21