Optimizing with synapses Sebastian Seung Howard Hughes Medical Institute and Brain & Cog. Sci. Dept., MIT

Practice makes perfect Birdsong learned from male tutor Stored template Zebra finch: up to 100,000 iterations Known anatomy and physiology

Hahnloser, Kozhevnikov, Fee (2002)

Supervisory signals in the brain Global broadcast of reward signal E.g. dopaminergic system

Neural basis of learning Global signal (Reward, motor error, etc.) Local signals (Voltage, calcium, etc.) Synaptic plasticity The interaction between global and local signals is largely uncharacterized.

Noise injection hypothesis HVC vs. LMAN –lesion –neural activity RA HVC LMAN motor neurons Doya and Sejnowski

Trial and error learning Generation of variability Reinforcement of favorable variations

Synaptic learning rule 1 2 reward 3 regular synapses noise synapses regular noise reward Fiete and Seung

Optimization in biology Evolution –Search in genotype space –Random genetic variation Learning –Search in synapse space? –Unreliable synapses –Noise injection

Stochastic gradient learning Systems-level models of learning –Birdsong –Oculomotor system Synaptic plasticity in vitro –Microisland cultures Training neural circuits in vitro –Silicon stimulation –Pattern culture –Intrinsic imaging by interferometry

Reward-driven plasticity in vitro Jen Wang Naveen Agnihotri

Patterned culture by inkjet printing Sawyer Fuller and Neville Sanjana

In vitro models of learning Goal: study how synaptic plasticity affects the dynamics of neural circuits Technical challenge: electrical and chemical control of neurons Conceptual challenge: training neural circuits