Biological motor behaviour (cont.) Central Pattern Generating Neural Networks (CPGs): Small, relatively simple neural systems with well-defined units, well-defined circuitry, and well-defined function Brain control Central Pattern Generators Muscles reflexes control modulation feedback Such central pattern generators are believed to be responsible for practically all known muscle behaviour.

simple models for pattern generating networks Wilson‘s model of the locust flight CPG (1961) Delcomyn (1998)

Spinal Circuits for Generating Rhythm

simple models for pattern generating networks Postinhibitory rebound: tendency of a neuron to fire a brief series of APs upon being released from inhibition, without any excitatory input. Delcomyn (1998)

Rhythm generation in CPG circuits Figure with permission: A. Ayali Understanding CPG circuits: Models of biological neural circuits generating self-sustained out-of- phase (or anti-phase) oscillations.

How do neuromodulators and neuromodulatory neurons reconfigure circuits so that the same group of neurons can produce a variety of behaviorally relevant outputs? The output pattern of the STG can be modulated The big question:

Alteration of synaptic strength by neuromodulators presynaptic facilitationpresynaptic inhibitiondiffuse presynaptic modulation diffuse postsynaptic modulation There are probably only few synapses which are not subject of some kind of modulation. Marder, Thirumalai (2002)

Development of circuits that generate simple rhythmic behaviors in vertebrates Martyn Goulding1 and Samuel L Pfaff2

Mathematical Modeling

Mathematical Analysis and Simulations of the Neural Circuit for Locomotion in Lampreys Li Zhaoping,1 Alex Lewis,1 and Silvia Scarpetta2 1University College, London, United Kingdom 2INFM & Department of Physics, University of Salerno, Italy

Experimental data Spontaneous oscillations in decapitated sections with a minimum of 2-4 segments, from anywhere along the body. Three types of neurons: E (excitatory), C (inhibitory), and L (inhibitory). Connections as in diagram E,C neurons: shorter range connections (a few segments), L: longer range Head-to-tail (rostral-to-caudal) descending connections stronger E and L oscillate in phase, C phase leads. L To motor neurons L L L E E E E C CC C Two segments in CPG

ELLLCLELLLCLL () d/dt= - Neurons modeled as leaky integrators Membrane potentials + external inputs ELLLCLELLLCLL () Decay (leaky) term E C L E C L C L E C L E + Firing rates g(E L ) ( ) Connection strengths () JWQJWQ 0 0 -H g(L L ) g(C R ) -K -A -B Contra-lateral connections from C neurons Left-right symmetry in connections

ELLLCLELLLCLL () d/dt= - Neurons modeled as leaky integrators Membrane potentials + external inputs ELLLCLELLLCLL () E C L E C L C L E C L E + g(E L ) ( ) Connection strengths () JWQJWQ 0 0 -H g(L L ) g(C R ) -K -A -B E,L,C are vectors: E L = E1LE1L E2LE2L E3LE3L :. J 11 J 12 J 13 J,K,etc are matrices: J=J= J 21 J 22 J

Left and right sides are coupled Swimming mode always dominant d/dt= - ELLLCLELLLCLL () + external inputs ++ ( g(E L ) ) g(L L ) g(C R ) ELLLCLELLLCLL () + () JWQJWQ 0 0 -H -K -A -B The connections scaled by the gain g’(.) in g(.), controlled by external inputs. d/dt= - E+L+C+E+L+C+ () + external inputs ++ ( E+E+ ) L+ L+ C+ C+ E+L+C+E+L+C+ () + () JWQJWQ 0 0 -H -K -A -B d/dt= - E-L-C-E-L-C- () ++ ( E-E- ) L- L- C- C- E-L-C-E-L-C- () + () JWQJWQ 0 0 -H +K +A +B EECC LL LeftRight E ± L ± C ± () ERLRCRERLRCR () ELLLCLELLLCLL () = ± Linear approximation leads to decoupling E C L E C L “ + ” mode “-” mode Swimming mode C- becomes excitatory.

E-L-C-E-L-C- () d/dt =- E-L-C-E-L-C- () + ( E-E- ) L- L- C- C- 0 () JWQJWQ 0 -H K A B Experimental data show E &L synchronize, C phase leads E=L, J=W, K=A, simplification d/dt = E-C-E-C- () ( E-E- ) C- C- () JQJQ -H K B Prediction 1: H>Q needed for oscillations! (E, L) - C-C- excite inhibit The swimming mode Oscillator equation: d 2 /dt 2 E + (2-J-B) d/dt E + [(1-J)(1-B) +K(H-Q)] E =0 Damping Restoration force

Single segment: J ii + B ii < 2 Oscillator equation: d 2 /dt 2 E + (2-J-B) d/dt E + [(1-J)(1-B) +K(H-Q)] E =0 Segmt. j Segmt. i F ji F ij An isolated segment does not oscillate (unlike previous models) Self excitation does not overcome damping Inter-segment interaction: Coupling: F ij = (J ij + B ij ) d/dt E j +[B+J] ij E j -[BJ+K(H-Q)] ij E j When driving forces feed “energy” from one oscillator to another, global spontaneous oscillation emerges. i th damped oscillator segment of frequency w o d 2 /dt 2 E i + a d/dt E i + w o 2 E i = Σ j F ij Driving force from other segments.

Feeds energy when E i & E j in phase Feeds energy when E i lags E j Feeds energy when E i leads E j Coupling: F ij = (J ij + B ij ) d/dt E j +[B+J] ij E j -[BJ+K(H-Q)] ij E j Segmt. j Segmt. i F ji F ij Given F ji > F ij, ( descending connections dominate ) B+J > BJ+K(H-Q) Forward swimming (head phase leads tail) B+J < BJ+K(H-Q) Backward swimming (head phase lags tail) Controlling swimming directions Prediction 2: swimming direction could be controlled by scaling connections H, (or Q,K, B, J), e.g., through external inputs

d/dt = E-C-E-C- () ( E-E- ) C- C- () JQJQ -H K B More rigorously: Its dominant eigenvector E(x) ~ e t+ikx ~ e -i(  t-kx) determines the global phase gradient (wave number) k Eg. Rostral-to-caudal B tends to increase the head-to-tail phase lag (k>0); while Rostral-to-caudal H tends to reduce or reverse it (k<0). So, increasing H (e.g, via input to L neurons) Backward Swimming For small k, Re(λ) ≈ const + k · function of (4K(H-Q) –(B-J) 2 ) +ve k forward swimming -ve k backward swimming

Forward swimmingBackward swimming Simulation results: Increase H,Q

Turning Amplitude of oscillations is increased on one side of the body. Achieved by increasing the tonic input to one side only (see also Kozlov et al., Biol. Cybern. 2002) E L,R Time Turn Left Right Simulation Forward swim

Summary Control of swimming speed (oscillation frequency) over a larger range Include synaptic temporal complexities in model Further work: Analytical study of a CPG model of suitable complexity gives new insights into How coupling can enable global oscillation from damped oscillators How each connection type affects phase relationships How and which connections enable swimming direction control.

Renshaw Cell

Renshaw Cells : Spinal Inhibitory Interneurons Motor neurons send collaterals to small interneurons Excitability of motor neurons was reduced following electrical stimulation of its peripheral axon

Renshaw Cells : Spinal Inhibitory Interneurons Renshaw cells produce ‘Recurrent Inhibition’. Increase motor neuron firing causes increased inhibition via Renshaw Cell (also to inh n of synergist muscles) Acts to prevent large transient changes in motor neuron firing

Renshaw Cells : Spinal Inhibitory Interneurons Renshaw cells ‘Dysinhibit’ antagonistic muscles Recurrent inhibition of muscle & dysinhibition of antagonist acts to prevent large sudden movements Descending Cortical Pathways modulate Renshaw Cells

Reflexes Recurrent inhibition: Renshaw cells control the degree of efferent signal sent to the muscle Presynaptic inhibtion Interneurons mediate the afferent signal before it gets to motor neuron, influencing the signal sent back to the muscle

Evolutionary Training of a Biologically Realistic Spino-neuromuscular System Stanley Gotshall

Results With Renshaw Interconnections Behavior with network with Renshaw interconnections Fitness =

Results No Renshaw Cells Behavior with no Renshaw cells Fitness =

Results No Renshaw Interconnections Behavior with no Renshaw interconnections Fitness =

Results Trainability Average Best Fitness v. GA Iterations 75,000 GA Iterations

Observations GA is successful in training controlled behavior. Removing Renshaw cell connections makes training difficult. GA takes advantage of more complex model.

Second Paper Using Genetic Algorithms to Train Biologically Realistic Spino-neuromuscular Behaviors Introduction Experiments Results Conclusions

Introduction Uses more complex model –Train model with six muscle units and proxy afferent neurons. –Allow GA to evolve the strengths of each muscle unit. Hypotheses –The GA will find better solutions when training muscle unit strengths (force multipliers) –Adding a proxy afferent neuron will result in more realistic alpha-MN firing patterns.

New Network Model Subnet Input Synfire Chain Alpha-MN Gamma-MN Proxy Afferent Neuron Renshaw Cell

Experiments Renshaw cells vs. no Renshaw Cells Training with Muscle Force Multipliers With and Without Renshaw Cells Training With Proxy Afferent Neurons Vary Forearm Mass With and Without Afferents

Renshaw cells vs. no Renshaw cells Fixed force force multiplier (0.8) Upward frequency 1 in 6 Downward frequency 1 in 140 Purpose –(confirm that results are consistent with first paper)

Results – Renshaw cells vs. no Renshaw cells Trends from first paper hold. This is a good thing!

Trained Muscle Force Multipliers With and Without Renshaw Cells 2 Configurations –Same parameters as first experiment Purpose –In biology, strengths of muscle units vary. So we add this flexibility to the model. No longer only training synaptic weights. Now we also train the strengths of the muscle units.

Results – Trained Force Multipliers With and Without Renshaw Cells Both cases show improvement when force multipliers are trained.

Results – Trained Force Multipliers With and Without Renshaw Cells Renshaw cells become less relevant because GA uses force multipliers to compensate

Results – Trained Force Multipliers With and Without Renshaw Cells

Trained with Proxy Afferent Neuron Add proxy afferent neuron and gamma motoneuron 6 motor units up – 3 motor units down Upward frequency 1 in 6 (same) Downward frequency 1 in 12 –Potentially smoother downward motion –The GA can train a given motion with different frequencies. Purpose (Observe effect of active feedback)

Results – Trained with Proxy Afferent Neurons and Force Multipliers

0.2 Second Delay

Vary Forearm Mass With and Without Afferents Train 4 configurations –0.55 kg / no afferents –0.55 and 0.65 kg / no afferents –0.55 kg / with afferents –0.55 kg and 0.65 kg / with afferents Purpose –To test networks on parameters they weren’t explicitly trained on

Results – Vary Forearm Mass With and Without Afferents Training with multiple masses results in more robust solutions

Conclusions Training with a GA is a reasonable method to model biologically realistic behaviors in neuromuscular systems. –GA is an effective training tool for the system –GA finds better solutions as the model gets more complex (Renshaw cells, variable muscle strenghts) –Active feedback from the muscle results in more biologically realistic behavior. Model has promise for testing hypotheses regarding SNMS pathways and components.

Future Work Make so anyone can use Expand Model –Model intrafusal fiber –Triceps –Multiple joints –More spinal connections

The End

Group 1a Inhibitory Interneurons 1a Inhibitory interneuron produces reciprocal inhibition of antagonistic muscle Homonymous muscle synergist

Group 1a Inhibitory Interneurons Descending Cortical Pathways can counteract Reciprocal Innervation to produce Co-Contraction (stiffening of a joint). Allows fine control of joint stiffness

1b Inhibitory Interneuron 1b Inhibitory Interneurons receive convergent input from Cutaneous and Golgi Tendon Organs 1b Inhibitory Interneurons inhibit homonymous muscle and excite antagonistic muscles This is opposite to the innervation of 1a Inhibitory Interneurons

1b Inhibitory Interneuron Cutaneous input reduces firing rate of motor neuron Provides a spinal mechanism for fine control during exploratory movements Descending inputs can modulate sensitivity of 1b afferent input

Summary The STNS of crustaceans - preliminary the STG -, serves as a model system to study the properties and the modulation of central pattern generators (CPGs) A CPG consists of a network of neurons and produces a repetitive output pattern, generally needed for stereotype movements (e.g. stomach movements, walking, flying, breathing etc.) The properties of a CPG depend on intrinsic properties of the individual neurons and their connectivity. Both, intrinsic properties and connectivity can be subject of modulation – which eventually allows flexibility of the output pattern. Activity of the STG circuits can be modulated by a large amount of neuromodulators, brought by projection neurons from other areas of the NS or hormonally. Co-transmission plays an important role in neuronal network modulation. Co-transmitters can act on different or on the same targets within cells.