1. Neuronal Ensembles in Motor Cortex
Encoding & Decoding of
Neuronal Ensembles in Motor Cortex
Nicholas Hatsopoulos
Dept. of Organismal Biology & Anatomy
Committees on Computational Neuroscience & Neurobiology
University of Chicago
Co-founder & board member of Cyberkinetics, Inc.

2. Encoding Problem Decoding Problem Multi-trial averaging Behavior
Single-trial prediction

3. “The motor cortex appears to be par excellence a synthetic organ for motor
acts… the motor cortex seems to possess, or to be in touch with, the small
localized movements as separable units, and to supply great numbers of
connecting processes between these, so as to associate them together in
extremely varied combinations. The acquirement of skilled movements,
though certainly a process involving far wider areas (cf. V. Monakow) of the
cortex than the excitable zone itself, may be presumed to find in the motor
cortex an organ whose synthetic properties are part of the physiological basis
that renders that acquirement possible.”
Leyton & Sherrington (1917)

4. The two components of language
Words or elementary primitives of meaning
Rules or grammar by which the primitives
are combined
Pinker (1999)

5. The language of motor action in motor cortex
Motor primitives: position, velocity,
direction, trajectory
Motor grammar: addition

6. Center-Out Task

7. Directional Tuning 150 frequency (Hz) NEXT SLIDE: Outline -0.5 1.09 1 3 5 4 5
180° 2 4 2 2 5 3 1 5
In this example, the neuron fires maximally for movements to the left and forward (i.e. 135 degrees) and minimally in the opposite direction. We could fit the tuning curve to a cosine function whose maximum value occurs at the so-called preferred direction of the cell. 80% of neurons we recorded from were well fit with a cosine function.
NEXT SLIDE: Outline 2 7
150
frequency (Hz)
-0.5
1.0
time (s)

8. Behavioral Apparatus

9. Random-walk task A B

10. Chronic Multi-electrode array
We have been using the Utah Intracortical electrode array developed by Dick Normann at the University of Utah.
The array is silicon-based and is composed of 100 micro-electrodes arranged in a 10 by 10 matrix, each of which is separated from each other by 400 microns.
So far, we have performed 24 implants in 12 animals recording from primary,dorsal premotor, and supplementary motor cortices. And we recorded from up to 50 neurons simultaneously.
The figure shows an array implanted in the arm area of MI.
Next Slide: Waveforms
Utah/Bionic Technologies Probe
Richard Normann U Utah

11. Primary Motor Cortex (MI)
Leg
Arm MI
We have been using the Utah Intracortical electrode array developed by Dick Normann at the University of Utah.
The array is silicon-based and is composed of 100 micro-electrodes arranged in a 10 by 10 matrix, each of which is separated from each other by 400 microns.
So far, we have performed 24 implants in 12 animals recording from primary,dorsal premotor, and supplementary motor cortices. And we recorded from up to 50 neurons simultaneously.
The figure shows an array implanted in the arm area of MI.
Next Slide: Waveforms
Face
5 mm

13. Long-term Reliability & Stability
0.8
0.7
0.6
0.5
ARRAY YIELD
0.4
0.3
0.2
subject 1 (tom)
subject 2 (coco)
0.1
subject 3 (buddy)
subject 4 (radley)
100
200
300
400
500
600
700
800
900
1000
Many Neurons Every Day
(19 tests over 110 days)
Blue - no recording
Red - best recordings
DAYS POSTIMPLANT

14. Why view motor cortical encoding as
time-dependent?
Trajectory-selective activity in motor cortex
(Hocherman & Wise, 1990, 1991)
Preferred directions shift in time (Mason et al., 1998;
Sergio et al., 2005; Sergio & Kalaska, 1998)

15. Center-out task
Shifts versus time

16. Center-out task Random-walk task Shifts versus time
Shifts versus lead/lag time
lag
lead
movement onset

18. Temporal tuning (information theory)
Lead/lag (s)

19. The Encoding Model
A class of general linear models (e.g. logistic regression) that
estimates the probability of a spike given a particular
movement trajectory:
= preferred velocity trajectory
integrated
= preferred trajectory and path (“pathlet”)

21. model input =
probability of a spike
model input
lead/lag (ms)

22. Temporal stability of pathlet representation

23. ROC analysis to quantify the performance of the encoding model

24. Encoding Performance as a function of trajectory length

25. Decoding Performance as a function of trajectory length

26. Map of Pathlets

28. Horizontal connectivity in motor cortex
1 mm
Huntley & Jones (1991)

29. Rule for combining pathlets:
Additive rule
Assuming conditional independence,

33. Potential violations of conditional independence
Case #1
Case #2
neuron 39 vs. neuron 51
-0.4
-0.2
0.2
0.4 20 40 60 80
100
neuron 40 vs. neuron 41
-0.4
-0.2
0.2
0.4 20 60
100
140
180
220
260
Counts/bin
1 ms bin

34. Spike Jitter Method (reference) (target) neuron 1 neuron 2
2 parameters:
+/-w
w, time resolution
of synchrony
J, the jitter window
neuron 1
(reference)
neuron 2
(target) +J -J
2 spike
jittered
trains
This method evolved from a thought experiment that Elie Bienenstock had. Imagine …
Would this magical drug destroy the apparent synchrony that we observe and if it did, what that disrupt thinking in the animal.

35. 10% of all cell pairs (N=1431) show significant synchrony at a
Case #1
Case #2
1000 jitters
10% of all cell pairs (N=1431) show significant synchrony at a
resolution of +/-5 ms, p<0.05

36. Potential violations of conditional independence
Case #1
Case #2

38. When conditional independence
appears to be violated
Conclusion: Synchronization preserves
additive rule but increases the sensitivity
of the tuning function by increasing the gain

39. Neuro-motor prosthetic system
2) Decoding of neural signals
1) Multi-electrode array implant
Computer
3) Output interface
I want to talk to you today about our efforts at Brown University with John Donoghue to try to develop a neuro-motor prosthetic system. The problem of using cortical signals to drive an external device can be broken up into three components. First, an chronic electrode implant is required. In our case, we have used a multi-electrode array to record from large numbers of single units. Second, these signals need to be decoded or mapped to a set of movement parameters. These parameters could include discrete behavioral states or continuous parameters such as position of the hand in 2 or 3 D space. Finally, these parameters are sent to a variety of output devices such a computer’s mouse or keyboard, or some sort of assistive robot, or even interface with biological tissue such as the muscles, peripheral nerves, or spinal chord gray matter.
Personal computer:
• mouse
• keyboard
Assistive Robotics:
• robotic arm
• mechanized prosthetic arm
Biological interface:
• muscles
• peripheral nerve
• spinal cord

40. BrainGateTM Pilot Device
Cable
Cart
Sensor

41. BrainGateTM Sensor Implantation and Post-Op Recovery as Planned
Surgery as planned
Post-op recovery unremarkable
Wound healing around pedestal complete
Array
on Cortex
2 months post implant
Insertion

42. Binary Modulation-Imagined Opening/Closing of Hand

43. BrainGate Signal Detection and Analysis; SCI Able to Modulate Neural Output
This slide shows the ability of patient #1 to modulate his neural output (from the motor cortex). When the operator says “left” you can immediately see activity in the top row of the display. This corresponds to one neuron. This neuron shows no activity in response to the “right” command.

44. ( ) 2. Algorithmic level: Optimal Linear Filter Reconstruction Rf X(t)
Response of neural
ensemble in time
Estimated position of hand
in time Rf
X(t) = ˆ
filter coefficients ( ) X R f T 1 - =
Essentially, this was a linear regression approach where we estimated the hand position, X, based on the product of the ensemble firing rates over time multiplied by a set of filter coefficients. This allowed us to come up with a least squares estimate for the filter coefficients. We then tested the regression model on data that had not been used to build the model.
NEXT SILDE: RESULTS OF RECONSTRUCTION
Warland et al. (1997)

45. Two Dimensional Cursor control

47. Hatsopoulos Lab Qingqing Xu Wei Wu, PhD Sunday Francis Zach Haga
Jignesh Joshi
John O’Leary
Dawn Paulsen
Jake Reimer
Jonathan Ko
Joana Pellerano
Richard Penn, MD
Matt Fellows
Yali Amit
Cyberkinetics, Inc.

48. Cross-Validated Performance as a function of pathlet length

49. Cross-validated Pathlet Population Vector Decoding