You wanted to know what the Matrix is? Matthew Casey

2 The Matrix (1999), © 1999 Warner Bros.

3 Modelling the Action of the Brain You wanted to know what the Matrix is? –‘What you can feel […] smell […] taste and see […] are simply electrical signals interpreted by your brain’ (The Matrix 1999) –The outlook may be dystopian, but in artificial intelligence, the dream is still to build ‘intelligent machines’ (even if they do take over the world) Even Alan Turing was similarly motivated, for example, in 1947, when he wrote: –"I am more interested in the possibility of producing models of the action of the brain than in the practical applications to computing.“ (Hodges 1992:363) Yet modelling “the action of the brain” –Requires knowledge of the actions to be modelled –Requires a robust understanding of the tools used –Has practical applications as well Hodges, A. (1992). Alan Turing: The Enigma. London: Vintage, Random House.

4 Knowledge of the Actions Why? –To understand ‘the brain’ better –To use knowledge of the brain to build more ‘intelligent machines’ Do we know enough about the brain? –We have a developing understanding (e.g. Carter 2000) –…and detailed models of specific aspects (e.g. Feigenson et al 2004) –…but our knowledge appears quite poor (e.g. Olshausen 2005) Olshausen, B.A. & Field, D.J. (2005). How Close are we to Understanding V1? Neural Computation, vol. 17, pp Carter, R. (2000). Mapping the Mind. London, UK: Phoenix. Feigenson, L., Dehaene, S. & Spelke, E. (2004). Core systems of number. Trends in Cognitive Sciences, vol. 8(7), pp

5 Knowledge of the Tools Why? –To explore new architectures (theory and application) –To understand how biological systems can give rise to behaviour Which architectures and algorithms? –What types of neuron, network or algorithm? –Our focus is on combining neural networks (Sharkey 1999) –Limited theory, but wide application, not just computational neuroscience (e.g. Kittler at al 1998) But we need a sufficiently robust understanding of these –To understand model behaviour –To generalise and apply elsewhere –So far, this robust understanding exists only for a small number of multi- net architectures (negative correlation learning and mixture-of-experts) Sharkey, A.J.C. (1999). Multi-Net Systems. In Sharkey, A. J. C. (Ed), Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pp London: Springer-Verlag. Kittler, J., Hatef, M., Duin, R.P.W. & Matas, J. (1998). On Combining Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20(3), pp

6 Practical Applications The grand aim of artificial intelligence? –To build ‘intelligent machines’? Grand Challenges (Hoare & Milner 2004) –Architecture of Brain and Mind –‘Bottom-up specification […] of computational models’ –‘Top-down development of a new kind of theory’ But –Knowledge spread across different disciplines –Foresight Cognitive Systems (Sharpe 2003): we need to develop an inter-disciplinary understanding Hoare, T & Milner, R. (2004). Grand Challenges in Computing: Research. UK Computing Research Committee (UKCRC). Sharpe, B. (2003). Foresight Cognitive Systems Project: Applications and Impact. London: Department of Trade and Industry, Office of Science and Technology.

7 Solutions? Modelling the ‘action of the brain’ –Use available tools to prototype aspects of cognition: neural networks and signal processing, … –Use inter-disciplinary knowledge: psychophysics and neurobiology Grand Challenge suggests: –Build increasingly more complex models –Combine theory and empirical results to build better models Most of the brain is dedicated to some form of sensory processing –A good place to start… –…building upon the wealth of computational work that has already been done, but hasn’t quite solved the problem

8 The Matrix (1999), © 1999 Warner Bros.

9 Multi-sensory Processing Could you understand what was being said in the film clip? –You should be able to, even without the sound –Your other senses, memory, emotions, etc. work together But, can we get a machine to do the same thing? –Computer vision, speech recognition, etc. are hard tasks –The brain does it very well – but how? Uni-modal or multi-modal processing? –Typically, only single modalities have been modelled –Yet evidence suggests that there is some low-level cross-sensory processing (Thesen et al 2004) –Can computational models benefit from a similar approach? Aim –To explore multi-sensory processing to see if it can help us build models/machines that are closer to being intelligent –We therefore need to build ever more complex models of the brain that can process different sensory inputs in an integrated way Thesen, T., Vibell, J.F., Calvert, G.A. & Österbauer, R.A. (2004). Neuroimaging of Multisensory Processing in Vision, Audition, Touch, and Olfaction. Cognitive Processing, vol. 5(2), pp

10 Multi-modal Processing Multi-sensory integration leads to a multi-modal understanding: the whole ‘picture’ Numerical cognition –Exploring a multi-modal understanding in numeracy –Integrated cognitive abilities for manipulating numbers: subitization, counting, addition and number representation (Casey et al, Casey 2004, Ahmad et al 2002) Multi-sensory processing and tools –Combines visual and linguistic inputs to produce a single output –Combining supervised and unsupervised learning in parallel and in sequence –But better (and less specific) models of vision, audition, etc. needed Ahmad, K., Casey, M.C. & Bale, T. (2002). Connectionist Simulation of Quantification Skills. Connection Science, vol. 14(3), pp Casey, M.C. & Ahmad, K. (accepted). A Competitive Neural Model of Small Number Detection. Neural Networks. Casey, M.C. (2004). Integrated Learning in Multi-net Systems. Unpublished doctoral thesis. Guildford, UK: University of Surrey.

11 Low-level Vision Modelling low-level human vision (with Sowden) –Category learning task (Notman et al 2005) –Task dependence tunes low-level processing –Categorical perception effect: measurable difference in ‘within class’ versus ‘between class’ discrimination –What causes this CP Effect? Well-established area of investigation and computational modelling –However, new understanding of sensory processing: low-level vision is not static –Dynamic changes to how we process low-level visual input depending upon task (visual and task inputs) –Linked to low-level cross-modal processing Notman, L.A., Sowden, P.T. & Özgen, E. (2005). The Nature of Learned Categorical Perception Effects: A Psychophysical Approach. Cognition, vol. 95(2), pp. B1-B14.

12 Categorical Perception Do these belong to the same or a different category?

13 Category Learning Category B Category A Notman et al o0o 45 o 90 o 135 o 180 o 315 o 270 o 225 o Distance changes through learning 3f phase angle Images combine an f and 3f grating

14 Receptive Field Modelling Modelling low-level vision: –2-D Gabor filtering: frequency, phase and orientation (cf. Itti & Koch 2001) –Split into receptive fields –Neuron per field, fed into discrimination model –Task driven: discrimination/categorisation –Meant to learn how to combine receptive field values But… –Grappling with ‘plausible’ models of vision –MLP only: needs to model ‘templates’ and lateral inhibition (competitive learning?) –Assumes a model of vision that may be wrong (cf. Olshausen 2005) –Relies upon simplistic grating patterns (as used in human tests) Despite the problems –This simple model is starting to show that the CP Effect can be reproduced because of the process of learning categories Itti, L. & Koch, C. (2001). Computational Modelling of Visual Attention. Nature Reviews Neuroscience, vol. 2(3), pp Olshausen, B.A. & Field, D.J. (2005). How Close are we to Understanding V1? Neural Computation, vol. 17, pp

15 What Next? Work so far has: –Built multi-net adaptive models of specific cognitive abilities –Focussed on single vision, linguistic and task inputs Need to: –Build better model of vision and other senses –Need to combine these models –Perhaps demonstrate via simple robotics But… –We still lack a robust understanding of the tools –Despite exploring novel architectures and algorithms: sequential and parallel systems (Casey et al 2004) Casey, M.C. & Ahmad, K. (2004). In-situ Learning in Multi-net Systems. In Yang, Z.R., Everson, R. & Yin, H. (Ed), Proceedings of the 5th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL 2004), Lecture Notes in Computer Science 3177, pp Heidelberg: Springer-Verlag.

16 Work in Progress Limited theory on combined systems: –Ensemble (NCL) and modular systems (mixture-of-experts) Individual networks are well understood –Are multi-nets just single networks? –Are neural network ensembles just partially connected feedforward systems (cf. Brown 2004)? We need a better understanding of useful architectures (without getting lost in the detail) –Ensembles: game theory approach (Zanibbi, Casey & Brown) –Ensembles: application to classification (Zhang & Casey) –Recurrence and single-nets (Taskaya-Temizel & Casey) …of generic architectures: –Set theoretic (Shields & Casey) –Infer properties of combined system from components …and to think about other aspects of the brain: –Emotion (Pavlou & Casey) Brown, G. (2004). Diversity in Neural Network Ensembles. Unpublished doctoral thesis. Birmingham, UK: University of Birmingham.

17 Coming Soon… Workshop on Biologically Inspired Information Fusion UniS 22nd and 23rd August 2006 Matthew Casey, Paul Sowden, Tony Browne, Hujun Yin Bringing together computer scientists, engineers, psychologists and biologists to discuss multi-sensory processing

Thank you Questions?

19 Numeracy and the Brain Dehaene, S. (2000). The Cognitive Neuroscience of Numeracy: Exploring the Cerebral Substrate, the Development, and the Pathologies of Number Sense. In Fitzpatrick, S.M. & Bruer, J.T. (Eds), Carving Our Destiny: Scientific Research faces a New Millennium, pp Washington: Joseph Henry Press.