2. Workshop on Mathematical Models of Cognitive Architectures December 5-9, 2011 CIRM, Marseille Workshop on Mathematical Models of Cognitive Architectures December 5-9, 2011 CIRM, Marseille Abstract This presentation will look at action, perception and cognition as emergent phenomena under a unifying perspective: This Helmholtzian perspective regards the brain as a (generative) model of its environment. The imperative for any brain is then to optimize a free energy bound on the (Bayesian) evidence for its model of the world. We will see that this is not just mandated for the brain but for any self-organizing system that resists a natural tendency to disorder in a changing environment. More specifically, maximizing Bayesian evidence leads in a fairly straightforward way to an understanding of action as active inference, and perception in terms of predictive coding. I hope to illustrate these points using simulations of perceptual categorization and action observation. Active inference, free energy and the Bayesian brain Karl Friston University College London

3. “Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism” - Hermann Ludwig Ferdinand von Helmholtz Thomas Bayes Geoffrey Hinton Richard Feynman From the Helmholtz machine to the Bayesian brain and self-organization Hermann Haken Richard Gregory Gerry Edelman Stephen Grossberg

4. Overview Ensemble dynamics Entropy and equilibria Free-energy and surprise Free-energy principle Action and perception Hierarchies and generative models Perception Birdsong and categorization Simulated lesions Action Active inference Action observation

5. temperature What is the difference between a snowflake and a bird? Phase-boundary …a bird can move (to avoid surprises)

6. What is the difference between snowfall and a flock of birds? Ensemble dynamics, clumping and swarming …birds (biological agents) stay in the same place They resist the second law of thermodynamics, which says that their entropy should increase

7. This means biological agents must self-organize to minimize surprise - to ensure they occupy a limited number of states (cf homeostasis). But what is the entropy? …entropy is just average surprise Low surprise (we are usually here) High surprise (I am never here)

8. But there is a small problem… agents cannot measure their surprise But they can measure their free-energy, which is always bigger than surprise This means agents should minimize their free-energy ?

9. Change sensory input sensations – predictions Prediction error Change predictions Action Perception action and perception to suppress prediction errors and minimise surprise What is free-energy? …free-energy is basically prediction error

10. Action to minimise a bound on surprisePerception to optimise the bound Action External states in the world Internal states of the agent ( m ) Sensations More formally,

11. Free-energy is a function of sensations and a proposal density over hidden causes and can be evaluated, given a generative model comprising a likelihood and prior: So what models might the brain use? Action External states in the world Internal states of the agent ( m ) Sensations

12. Backward (modulatory) Forward (driving) lateral Hierarchal models in the brain And their hidden states, causes and parameters

13. Synaptic gain Synaptic activity Synaptic efficacy Activity-dependent plasticity Functional specialization Attentional gain Enabling of plasticity Perception and inference Learning and memory The proposal density and its sufficient statistics Laplace approximation: Attention and salience

14. Synaptic activity Synaptic plasticitySynaptic gain cf Hebb's Lawcf Rescorla-Wagner cf Bayesian filtering or Predictive coding Laplace code assumption Free energy minimisation Generative model

15. Backward predictions Forward prediction error Synaptic activity and message-passing David Mumford Predictive coding

16. Adjust hypotheses sensory input Backward connections return predictions …by hierarchical message passing in the brain prediction Forward connections convey feedback Perceptual inference hierarchical message passing Prediction errors Predictions

17. Summary Biological agents resist the second law of thermodynamics They must minimize their average surprise (entropy) They minimize surprise by suppressing prediction error (free-energy) Prediction error can be reduced by changing predictions (perception) Prediction error can be reduced by changing sensations (action) Perception entails recurrent message passing in the brain to optimise predictions Action makes predictions come true (and minimises surprise)

18. Overview Ensemble dynamics Entropy and equilibria Free-energy and surprise Free-energy principle Action and perception Hierarchies and generative models Perception Birdsong and categorization Simulated lesions Action Active inference Action observation

19. Generating bird songs with attractors Syrinx HVC time (sec) Frequency Sonogram 0.511.5 causal states hidden states

20. 102030405060 -5 0 5 10 15 20 prediction and error 102030405060 -5 0 5 10 15 20 hidden states Backward predictions Forward prediction error 102030405060 -10 -5 0 5 10 15 20 causal states Perception and message passing stimulus 0.20.40.60.8 2000 2500 3000 3500 4000 4500 5000 time (seconds)

21. Perceptual categorization Frequency (Hz) Song a time (seconds) Song bSong c

22. Hierarchical (deep) birdsong: sequences of sequences Syrinx Neuronal hierarchy Time (sec) Frequency (KHz) sonogram 0.511.5 Christoph von der Malsburg

23. Frequency (Hz) percept Frequency (Hz) no top-down messages time (seconds) Frequency (Hz) no lateral messages 0.511.5 -40 -20 0 20 40 60 LFP (micro-volts) LFP -60 -40 -20 0 20 40 60 LFP (micro-volts) LFP 0500100015002000 -60 -40 -20 0 20 40 60 peristimulus time (ms) LFP (micro-volts) LFP Simulated lesions and false inference no structural priors no dynamical priors

24. Overview Ensemble dynamics Entropy and equilibria Free-energy and surprise Free-energy principle Action and perception Hierarchies and generative models Perception Birdsong and categorization Simulated lesions Action Active inference Action observation

25. predictions Reflexes to action action dorsal root ventral horn sensory error Active inference Action can only suppress (sensory) prediction error. This means action fulfils our (sensory) predictions

26. Descending proprioceptive predictions visual input proprioceptive input Action, predictions and priors Exteroceptive predictions

27. Autonomous behavior and action-observation 00.20.40.60.811.21.4 0.4 0.6 0.8 1 1.2 1.4 action position (x) position (y) 00.20.40.60.811.21.4 observation position (x) Descending predictions hidden attractor states (Lotka-Volterra)

28. Thank you And thanks to collaborators: Rick Adams Sven Bestmann Jean Daunizeau Harriet Brown Lee Harrison Stefan Kiebel James Kilner Jérémie Mattout Klaas Stephan And colleagues: Peter Dayan Jörn Diedrichsen Paul Verschure Florentin Wörgötter And many others

29. Perception and Action: The optimisation of neuronal and neuromuscular activity to suppress prediction errors (or free- energy) based on generative models of sensory data. Learning and attention: The optimisation of synaptic gain and efficacy over seconds to hours, to encode the precisions of prediction errors and causal structure in the sensorium. This entails suppression of free-energy over time. Neurodevelopment: Model optimisation through activity- dependent pruning and maintenance of neuronal connections that are specified epigenetically Evolution: Optimisation of the average free-energy (free-fitness) over time and individuals of a given class (e.g., conspecifics) by selective pressure on the epigenetic specification of their generative models. Time-scale Free-energy minimisation leading to…