1. Functions of Distributed Plasticity in a Biologically-Inspired Adaptive Control Algorithm: From Electrophysiology to Robotics University of Edinburgh University of Sheffield University of the West of England

2. Slide No 2AwayDay 2005 1.Background to project

3. Slide No 3AwayDay 2005 In some respects animal movements better than robot movement Could in part be due to characteristics of biological control algorithms Which region of the brain particularly concerned with skilled movement?

4. Slide No 4AwayDay 2005 Cerebellum located at base of brain (here a human brain) looks like a small version of overlying cerebral cortex? cerebellum = ‘little brain’

5. Slide No 5AwayDay 2005 Cerebellar Function Clinical and experimental observations of cerebellar damage Does not cause paralysis, but makes many movements inaccurate, slow and uncoordinated Similar to effects of alcohol: tests for intoxication may resemble clinical test for cerebellar impairment

6. Slide No 6AwayDay 2005 Conclusion: cerebellum is particularly associated with those features of movements that distinguish animals from robots Framework of project: to investigate whether there are features of cerebellar control that are likely to be of interest to robotics

7. Slide No 7AwayDay 2005 1.Framework: is cerebellar control of interest to robotics? 2.Problem

8. Slide No 8AwayDay 2005 Cerebellar Cortex Adjacent to and connected with the the brainstem Has its own cortex (= rind)

9. Slide No 9AwayDay 2005 Cerebellar Cortex Small number of cell types in cerebellar cortex Connected to form a distinctive microcircuit

10. Slide No 10AwayDay 2005 Cerebellar Microcircuitry Classic work published in 1967 Investigated anatomy and electrophysiology of microcircuit Same basic circuit repeated many times (hence “neuronal machine”) Important: half the cells in the entire brain are in the cerebellum

11. Slide No 11AwayDay 2005 Idea of Cerebellar ‘Chip’ Structure of cerebellar cortex is very uniform over its entire surface Different regions have different inputs and outputs, (microzones) but same basic organisation Gives rise to idea of cerebellar chip: ~5000, each with its own particular connections. Mossy Fibres

12. Slide No 12AwayDay 2005 Choose Your Task Consequence of this arrangement: all motor tasks using the cerebellum employ the same basic cerebellar algorithm The investigator can therefore choose the most ‘appropriate’ motor task In our case, control of the vestibulo-ocular reflex (VOR)

13. Slide No 13AwayDay 2005 Vestibulo-Ocular Reflex (VOR) Vision is degraded if the image moves (‘slips’) too much across the retina Retinal slip would be produced by movements of the head, such as occur in locomotion The VOR acts to counter- rotate the eyes to prevent retinal slip, i.e. to maintain stable gaze Usually not aware when we use it

14. Slide No 14AwayDay 2005 VOR Control: Basic Circuit Input from vestibular position, senses head movement Passed to interneurons in vestibular nuclei (secondary vestibular neurons) Thence to motor neurons that control the eye muscles This circuit in brainstem (just below cerebellum) Semicircular canals Primary Vestibular Neurons Ocular Motor Neurons Extraocular Muscles Secondary Vestibular Neurons

15. Slide No 15AwayDay 2005 VOR Control: Cerebellum Cerebellar flocculus receives information about –Head velocity –Eye movement commands –Retinal slip Projects back to brainstem motoneuron eye firing Eye Muscles Orbital Tissue velocity Brainstem head velocity Flocculus Retinal slip

16. Slide No 16AwayDay 2005 VOR Control: Generalised Version motoneuron eye firing Eye Muscles Orbital Tissue velocity Flocculus and Brainstem head velocity command output u(t) Plant y(t) Controller reference r(t)

17. Slide No 17AwayDay 2005 Not Feedback Control Retinal slip signal is delayed by 100 ms (visual processing) Feedback control would become unstable at ~ 2.5 Hz, yet VOR operates up to ~25 Hz Feedback control not suitable command output u(t) Plant y(t) Controller reference r(t) Sensor X

18. Slide No 18AwayDay 2005 Control Method: Open-Loop If feedback not available, then open-loop control must be used If reference signal is desired output, then the controller becomes an inverse model of the plant (‘plant compensation’) command output u(t) Plant P y(t) Inverse Plant Model P -1 reference r(t)

19. Slide No 19AwayDay 2005 Adaptive Control How can we be sure the inverse plant model is accurate? Requires constant calibration – ‘adaptive control’ Use information about system output for learning, rather than on- line control command output u(t) Plant P y(t) Inverse Plant Model P -1 desired output r(t) Sensor training signal

20. Slide No 20AwayDay 2005 VOR Equivalent Available training signal is retinal slip, known to be sent to the flocculus Consistent with flocculus being the adaptive part of the controller Consistent with e.g. lesion evidence that VOR adaptation is lost after floccular inactivation motoneuron eye firing Eye Muscles Orbital Tissue velocity Brainstem head velocity retinal slip Flocculus

21. Slide No 21AwayDay 2005 Why VOR Calibration? 1.Well-defined adaptive control problem 2.Eye movements are relatively simple –single joint instead of up to ~6 joints in finger movements –constant load 3.Great deal known about underlying circuitry 4.Well established cerebellar involvement

22. Slide No 22AwayDay 2005 1.Framework: is cerebellar control of interest to robotics? 2.Problem: adaptive calibration of VOR 3.Approach: multidisciplinary

23. Slide No 23AwayDay 2005 Multidisciplinary Approach Modelling –(theoretical neuroscience, Sheffield) Electrophysiology –(experimental neuroscience, Edinburgh) Robotics –(University of the West of England, Bristol)

24. Slide No 24AwayDay 2005 General Modelling Task Devise a working algorithm that connects the microcircuit to the behavioural competence Obeys known anatomical and physiological constraints

25. Slide No 25AwayDay 2005 Cerebellar Modelling Cerebellar microcircuit has been extensively modelled, starting with classic work of Marr (1969) and Albus (1971) Here in more modern form of the adaptive filter

26. Slide No 26AwayDay 2005 Specific Modelling Problem Extensive experimental work shows that in VOR calibration there are TWO sites of plasticity 1.In cerebellar cortex, as predicted by adaptive filter models 2.In the brainstem What are the computational advantages of this distributed plasticity?

27. Slide No 27AwayDay 2005 Electrophysiology: Problem What are the learning rules underlying brainstem plasticity? Existence known for ~20 years, rules yet to be identified Critical for understanding computation significance Mayank B Dutia Centre for Integrative Physiology University of Edinburgh

28. Slide No 28AwayDay 2005 Electrophysiology: Technique Record from neurons in slices through brainstem Look for neurons that receive input for the flocculus (flocculus target neurons, FTNs) Medial Vestibular Nucleus Midline Rostral Caudal Rat Brainstem Slice

29. Slide No 29AwayDay 2005 Robotics Does Algorithm Work in Real World? Tony Pipe, Chris Melhuish, UWE Bristol Camera stabilisation How does algorithm compare with control engineering alternatives?

30. Slide No 30AwayDay 2005 Multidisciplinary Approach 1.Framework: is cerebellar control of interest to robotics? 2.Problem: adaptive calibration of VOR 3.Approach: multidisciplinary Modelling: plausible candidate algorithm Electrophysiology: biological underpin Robotics: real world application

31. Slide No 31AwayDay 2005