Neural Networks Si Wu Dept. of Informatics PEV III 5c7 Spring 2008

Today’s Topics:  Course Organization  The origin of neural networks

Teaching Methods Lectures -- 2 hours per week, Thursday, 9-11am, PEV 1 1A06 Reading week: Week 5 Seminar– 1-2 hour per week (Mr. Thomas Baker) 3 rd : Thursday, 11-12pm, Chichester 1 CI003 Msc: Monday, 14-16pm, Chichester 1 CI204/5 Office hour: Thursday 14-16pm, PEV III 5c7 Lecture notes will be available online:

The course covers the fundamental theory of artificial neural networks (ANN), and will present basic models and learning algorithms of ANN. Seminars will be supervised Mr. Thomas Baker, and will be used to answer concerns related to the course and courseworks. Coursework will give you chance to practice ANN. Course Summary

Topics to be covered 1.Basics of the neural networks method 2.Single layer perceptrons 3.Multilayer networks 4.Radial basis function networks 5.Principle component analysis 6.Bayesian inference 7.Support Vector Machines

Assessment Only coursework Coursework consists of two parts: –The first one, for both 3 rd and Msc, counts 40%, due in Thursday 4pm, week 5 –The second one 60% For 3 rd year, due in Thursday 4pm, week 10 For Msc, due in Monday 12:00 noon, week 1 of Summer Semester

Recommended Textbooks 1.Haykin S (1999). Neural networks. Prentice Hall International. Excellent but quite heavily mathematical 2.Bishop C (1995). Neural networks for pattern recognition. Oxford: Clarendon Press (good but a bit statistical, not enough dynamical theory) 3.Pattern Classification, John Wiley, 2001 R.O. Duda and P.E. Hart and D.G. Stork 4.Hertz J., Krogh A., and Palmer R.G. Introduction to the theory of neural computation (nice, but somewhat out of date) 5.Pattern Recognition and Neural Networks by Brian D. Ripley. Cambridge University Press. Jan ISBN Neural Networks. An Introduction, Springer-Verlag Berlin, 1991 B. Mueller and J. Reinhardt Find the one that best suits your background.

What is Neural Networks? Inspired from real neural systems Having a network structure, consisting of nodes (artificial neurons) and weights (neuronal connection) A general methodology for function approximation

How neural systems look like?

The structure of neural systems Neuron: the fundamental singaling/computational units Synapses: the connections between neurons Layer: neurons are organized into layers Extremely complex: around neurons in the brain, each with 10 3 connections

Why do we learn from neural systems? The brain is still superior than modern computers in many aspects A different style of computation: parallel distributed processing Adaptive and can learn new knowledge An universal computational architecture: the same structure carries out many different functions

The function of a single neuron Hillock input output

Cell  Membrane potential The state of neuron Membrane potential: the voltage difference between the cell body and the surrounding.

spike Neuron as a computational unit Neuron fires when its input is larger than a threshold Two states: on & off

The Idea of Artificial Neural Networks A single neuron’s function is simple, the specialty of brain functions is on the network structure ANN is to mimic the network structure of neural systems The constitution of ANN – Nodes: artificial neurons, performing a fixed linear or non- linear mapping –Network Weights: interactions between nodes –Layers: nodes are organized into layers – Connection style: feed-forward, feed-back or recurrent ANN is more and more engineering-driven nowadays, its biological root is gradually losing. The key of ANN is on the design and training of suitable network structures

An example of one-layer feed-forward neural network x w y b

xnxn x1x1 x2x2 Input Output An Example of Three-Layer Feed-forward Networks Hidden layers