1 Complejidad Dia 7 Ecología Biologí a Psicologia Meteorología MacroEconomía Geofisica UBA, Junio 19, 2012.

2 “Learning as a collective“ Chialvo and Bak, Neuroscience (1998) Bak and Chialvo, Phys. Rev. E (2001). Wakeling J. Physica A, 2003) Wakeling and Bak, Phys.Rev. E (2001). Hoy:

3 Learning is never smooth

What Is the Problem? The current emphasis is in … To understand how billions of neurons learn, remember and forget on a self-organized way. To find a relationship between neuronal long-term potentiation, (so called “LTP”) of synapses and memory.

Biology is concerned with “Long-Term Potentiation”  If A and B succeed together to fire the neuron (often enough) synapse B will be reinforced

Steps of Long-term Potentiation 1.Rapid stimulation of neurons depolarizes them. 2.Their NMDA receptors open, Ca 2+ ions flows into the cell and bind to calmodulin. 3.This activates calcium-calmodulin- dependent kinase II (CaMKII). 4.CaMKII phosphorylates AMPA receptors making them more permeable to the inflow of Na + ions (i.e., increasing the neuron’ sensitivity to future stimulation. 5.The number of AMPA receptors at the synapse also increases. 6.Increased gene expression (i.e., protein synthesis - perhaps of AMPA receptors) and additional synapses form.

What Is Wrong With the emphasis on “LTP”? Nothing but there is no evidence linking memory and LTP and LTP is not the solution of how memory works

How difficult would be for a neuronal network to learn? The idea was not to invent another “learning algorithm” but to play with the simplest, still biologically realistic, one. Chialvo and Bak, Neuroscience (1999) Bak and Chialvo, Phys. Rev. E (2001). Wakeling J. Physica A, 2003) Wakeling and Bak, Phys.Rev. E (2001).

Self-organized Learning: Toy Model 1) Neuron “I*” fires 2) Neuron “j*” with largest W*(j*,I*) fires and son on neuron with largest W*(k*,j*) fires… 3) If firing leads to success: Do nothing  Do nothing otherwise otherwise   decrease W* by  That is all

How It Works on a Simple Task Connect one (or more) input neurons with a given output neuron. Chialvo and Bak, Neuroscience (1999)

A simple gizmo a)left right b)10% “blind” c)10% “stroke” d)40% “stroke” Chialvo and Bak, Neuroscience (1999)

How performance scales with “brain” size More neurons -> faster learning. It makes sense! The only model where larger is better Chialvo and Bak, Neuroscience (1999)

How It Scales With Problem Size (on the Parity Problem) A) Mean error vs Time for various problem’ sizes (i.e., N=2 m bit strings) B) Rescaled Mean error (with k=1.4) Chialvo and Bak, Neuroscience (1999)

Order-Disorder Transition Learning time is optimized for  > 1

Order-Disorder Transition At  = 1 the network is critical

Synaptic landscape remains rough Elimination of the least-fit connections Activity propagates through the best-fit ones At all times the synaptic landscape is rough Fast re-learning Chialvo and Bak, Neuroscience (1999)

17

18 If you make a mistake, next do something different H. Ohta, Y.P. Gunji / Neural Networks 19 (2006) 1106–1119

19 By “inhibiting” the past states H. Ohta, Y.P. Gunji / Neural Networks 19 (2006) 1106–1119

20 H. Ohta, Y.P. Gunji / Neural Networks 19 (2006) 1106–1119 So you can learn new thing without deleting the old ones

Solid ‐ State Atomic Switch o “Mermistors” nanoresistores con memoria (o “electroquimica seca” o “electrolitos solidos”)

Tsuyoshi Hasegawa et al, Learning Abilities Achieved by a Single Solid ‐ State Atomic Switch Advanced Materials, 22, , 2010

Tsuyoshi Hasegawa et al, Learning Abilities Achieved by a Single Solid ‐ State Atomic Switch Advanced Materials, 22, , 2010 Experimental result of a gradual increase in the current

Memory is in the spatial configuration of the Ag cations. A collective memory… nanogap Ag 2 S Electrodo Ag Electrodo metal Ag atomic bridge Tsuyoshi Hasegawa et al, Learning Abilities Achieved by a Single Solid ‐ State Atomic Switch Advanced Materials, 22, , 2010

a, Schematics of a Ag2S inorganic synapse and the signal transmission of a biological synapse. b,c, Change in the conductance of the inorganic synapse when the input pulses were applied with intervals of T=20 s (b) and 2 s (c). Inorganic synapse showing STP and LTP, depending on input-pulse repetition time. “Short-term plasticity and long- term potentiation mimicked in single inorganic synapses” Takeo Ohno et al. Nature Materials 10, 591–595 (2011)

Emergent Criticality in Complex Turing B‐Type Atomic Switch Networks “Emergent Criticality in Complex Turing B ‐ Type Atomic Switch Networks” Advanced Materials Stieg et al, 24, , Fabrication scheme for complex, electronic networks

Emergent Criticality in Complex Turing B‐Type Atomic Switch Networks (a) Experimental I–V curve demonstrating hysteresis (b) Ultrasensitive IR image of a distributed device conductance. (c,e) Representative experimental network current response to a 2 V pulse showing switching between discrete, metastable conductance states. (d,f) Metastable states residence times for (d) single 10 ms pulse and (f) over 2.5 s during extended periods of pulsed stimulation. “Emergent Criticality in Complex Turing B ‐ Type Atomic Switch Networks” Advanced Materials Stieg et al, 24, , 2011.

Desafío: 1) Modelar eficientemente la física del collectivo de mermistores. Es decir: modelos numéricos eficientes de una red arbitraria de mermistores ( probable punto de partida: random fuse model) 2) Modelar aprendizaje en esa red: Es decir: Encontrar algoritmos de aprendizaje auto-organizables implementables in silico.