ONLY CONNECT David Willshaw Institute for Adaptive & Neural Computation School of Informatics University of Edinburgh

ONLY CONNECT Computational thought  Hamming Seminars Bell Labs  Radar  Family history Bell Labs  Information Theory  My research

ONLY CONNECT Only connect ! That was all her sermon. Only connect the pride and the passion and both will be exalted, and human love will be seen at its height. Live in fragments no longer. Only connect..... From Howard’s End by E M Forster

This is my research area: COMPUTATIONAL MODELLING OF THE DEVELOPMENT OF NEURAL CONNECTIVITY

Why is this an important area? Why is this an important area now?

Without the correct specific connectivity patterns between our neurons we cannot function correctly We don’t yet know the mechanisms for how the brain is wired up Computational modelling is used to explore particular hypotheses and suggest experiments to try to understand the underlying mechanisms New technologies are giving us much better data about connectivity

Human Brain – MRI scan (Wellcome Images; Mark Lythgoe, Chloe Hutton) 7 Cerebellum

The cerebellar cortex contains nerve cells of several different types 8 Cajal(1905)

Purkinje cell (Wellcome Images; David Becker) 9

Shows the Purkinje cells lined up and the parallel fibres (Cunningham, 1913) 10

Purkinje cells and parallel fibres (Wellcome Images; Spike Walker) 11

Vertebrate retina 13

Vertebrate retina

Visual pathways in mammals 15

Ocular dominance columns The binocular projection from retina to cortex in mammals Zebra stripes? Reminiscent of Turing Patterns postulated to be formed in morphogenesis by mechanisms of reaction- diffusion AM Turing, Phil. Trans. Roy. Soc. B, 237, 37-52, 1952

Computational modelling in neuroscience 17

Molecules Synapses Neurons Networks Systems CNS 1 cm 100  m 10 cm m  m A Maps 1 cm

19 Modelling at the nerve cell level (Wellcome Images; Benedict Campbell)

1952: The first computational neuroscience model A quantitative description of membrane current and its application to conduction and excitation in nerve. Hodgkin & Huxley, J Physiology (1952)

The Hodgkin-Huxley model Impulse propagation caused by flow of K+ and Na+ currents through separate channels in the membrane Permeability to ion flow in these channels is dependent on the potential difference across the membrane

Modelling a segment of the axon as an electrical circuit where the resistances are voltage dependent

HH equations account for all the data

24 A model at the network level

25 Learning and Memory: Hebb’s rule ‘When an axon of cell A is near enough to excite cell B or repeatedly or consistently take part in firing it, some growth or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.’ Hebb (1949)

26 Hebb's rule and associative memory Distributed Memory: The Associative Net (Willshaw, Buneman & Longuet- Higgins, Nature, 1969) Clipped Hebbian rule

Modelling of the development of nerve connections 27

J.F. Tello Polyneuronal innervation in foetal human muscle (1917)

Connections between neonatal nerve and muscle (Wellcome Images, Ribchester & Gillingwater)

Visual pathways in mammals 30

FROG BRAIN CAT BRAIN

Xenopus tadpoles 32

Frogs and toads 33 Xenopus

Frog visual system 34

What is the mechanism for the formation of ordered maps of nerve connections? Both flexibility and rigidity in connection pattern are seen - probably more than one mechanism act together? 35 From Jacobson (1967)

The main theories 36 1.Chemoaffinity – molecular cues guide each axon to its target cell or cells (usually associated with rigidity of connection) 2.Electrical signalling - e g, nearby cells that fire together may be more active than more distant cells and so can signal neighbour relations to the cells to which they are connected – usually associated with flexibility of connections.

So what is the link with Informatics? 37

“Informatics” means different things to different people? 38 “When I use a word” Humpty Dumpty said rather in a scornful tone “It means what I choose it to mean –neither more nor less” Alice Through the Looking Glass, Lewis Carroll

39 Neuroinformatics  INCF information-processing in the nervous system Computational Models inspire new hardware and software methods Neural Engineering collect, analyze, archive, share, simulate and visualize data and models Software Systems

40 Neuroinformatics  INCF INCF – International Neuroinformatics Coordinating Facility ( An international organisation subscribed to by 15 governments Dedicated to the coordination of neuroinformatics world wide. Each country has its own local organisation; I am the UK Coordinator and scientific representative at INCF

41 Mike Fourman’s formulation: “Informatics is the study of how natural and artificial systems store, process and communicate information”

42 The School of Informatics at Edinburgh is inclusive rather than exclusive. ++: Aren’t we lucky to be not constrained!

43 The School of Informatics at Edinburgh is inclusive rather than exclusive. ++: Cross-fertilisation --: Because of the breadth there is a danger that individuals have a lack of understanding of other fields of research practised in Informatics

44 A snapshot, which I prepared for Mike Fourman, of the interactions between academic and research staff in the three departmental groupings in 1997, prior to the formation of the School of Informatics AI/AIAI: Artificial Intelligence/AI Applications Institute DCS: Department of Computer Science CCS/HCRC: Centre for Cognitive Science/Human Communications Research Centre

45 The importance of technology in the computational modelling of the nervous system

Julius Bernstein ( ), after whom the Bernstein Centres for Computational Neuroscience in Germany are named. His membrane theory of the propagation of the nerve impulse (1902) was almost right.

But his equipment for measuring the properties of the nerve impulse was inadequate

Alan Hodgkin Andrew Huxley Once Hodgkin had been to Chicago (50 years later) to learn how to build an amplifier, he and Huxley could collect, analyse and model the required data, leading to a Nobel Prize for them

49 My current research problem

What is the mechanism for the formation of ordered maps of nerve connections? Both flexibility and rigidity in connection pattern are seen - probably more than one mechanism act together? 50 From Jacobson (1967)

The main theories 51 1.Chemoaffinity – molecular cues guide each axon to its target cell or cells (usually associated with rigidity of connection) 2.Electrical signalling - e g, nearby cells that fire together may be more active than more distant cells and so can signal neighbour relations to the cells to which they are connected – usually associated with flexibility of connections.

52 An example of mechanism 2: nearest neighbour interactions through correlated neural activity According to the neural activity model, spontaneous electrical activity drives the process. By a Hebbian-type mechanism, connections between neighbouring retinal cells and neighbouring tectal cells are strengthened; those between non-neighbours are weakened (Willshaw and von der Malsburg 1976).

53 New technology furnishes higher quality data: Mouse superior colliculus maps (Cang et al, J. Neurosci, 2008) The colour-coded noisy X, Y coordinates of the receptive fields of each small part of a 2 mm square brain area including colliculus X Y

54 Distribute ‘recording’ positions regularly over the colliculus 100  m

55 Then join up nearest neighbours to form a lattice

56 Translating colours into field positions, plot out the receptive field position for each recording point, averaging over nearby collicular points Then project the collicular lattice into the field Colliculus Field Wild type (normal) 20 recording points, or nodes

57 Projection of collicular lattice to field 50  m separation between nodes Projection of field lattice to colliculus Wild type (#006, 170 nodes)

Theories for the formation of nerve connections can be tested in mice for which the genome is known Genes that are thought to be determining developmental mechanisms can be manipulated Their effects on connectivity can be observed and compared with the model predictions 58

Beta2 knockout Knockout of the Beta2 component of the acetylcholine receptor is thought to diminish the strength of the correlated firing activity in the retina And hence the precision of the map? 59

60 Remove correlated activity in a Beta2 knockout Colliculus to field Orientation: 19+/-17 degrees The largest connected ordered submap – covers 138/145 nodes (95%)

Modern evidence for molecular guidance cues from McLaughlin, Hindges and O’Leary (2003) 61

Are these molecules used in map-making? If Ephs and ephrins are the labels of chemoaffinity, then changing them should result in abnormal patterns of connectivity 62

63 If all these ephrins are knocked out will the mapping along the rostrocaudal axis be destroyed?? EphrinA triple knock out Three ephrinAs, ephrinA2, ephrinA3 and ephrinA5, are thought to label the rostrocaudal axis of the colliculus.

64 Remove molecular cues: Heterozygote triple EphA knockout [A2-/-A3+/-A5- Largest ordered submap covering 90% of the nodes

Normal maps have quite high precision Removing genes controlling activity cues and molecular cues still does not destroy the order New models are needed! 65 In summary:

Collaborations are fun 66

67 At the 1 st Connectionist Summer School, Pittsburgh (1986)

68

69 Shows the fledgling algorithm applied to a 30-city problem (Hopfield & Tank, 1985). As presented to the 1986 Connectionists summer school

70 Application of the Elastic Net to a 100-City Problem (Durbin & Willshaw, Nature, 1988)

Connections with the Bell Labs of Hamming’s era Scientists at Bell Labs prided themselves in researching fundamental problems whereas in reality they were employed by a company to develop products Is there a duality here? In this university or in Informatics many of us are employed to do fundamental research yet there is increasing pressure to go after the WOW Factor 71

But there has to be something behind the WOW 72 WOW!

But there has to be something behind the WOW 73 WOW!

Connections with Hamming’s aphorisms from his Bell Communications Research Seminar “You and your research” 74

1. “You have to learn to sell yourself, to write clearly” 75

1. “You have to learn to sell yourself, to write clearly” 2. “The closed door is symbolic of a closed mind” 76

1. “You have to learn to sell yourself, to write clearly” 2. “The closed door is symbolic of a closed mind” 3. Against working single-handedly with total control: “If you learn to work with the system, you can go as far as the system will support you 77

1. “You have to learn to sell yourself, to write clearly” 2. “The closed door is symbolic of a closed mind” 3. Against working single-handedly with total control: “If you learn to work with the system, you can go as far as the system will support you 4 “If you do some good work you will find yourself on all kinds of committee and unable to do any more work.” 78

1. “You have to learn to sell yourself, to write clearly” 2. “The closed door is symbolic of a closed mind” 3. Against working single-handedly with total control: “If you learn to work with the system, you can go as far as the system will support you 4 “If you do some good work you will find yourself on all kinds of committee and unable to do any more work.” 5 “Plant the little acorns from which the mighty acorns grow” 79

If you want to find out more about computational modelling in neuroscience, you could consult our forthcoming Cambridge University Press book: Principles of Computational Modelling in Neuroscience by David Sterratt Bruce Graham Andrew Gillies David Willshaw PDF of almost-final version available on request! 80