Soft computing Lecture 6 Introduction to neural networks

Disadvantages of fuzzy systems Difficulties of formalized of fuzzy sets and linguistic variables –Its description is subjective, –Its description depend on context, features of situation and inference, Fuzzy systems are keeping knowledge base systems with its main disadvantage – orientation on formalizing of knowledge by anybody (expert) and unable to learn Way out is to use neural networks instead of its or together with its

visual cortex motor cortex association cortex to motor output

neurons 3 km wires 1mm Signal: action potential (spike) action potential

Formal neuron by MacCallock-Pitts x i – binary signal

Non-binary inputs or

Geometric interpretation of TLU action

Training of TLU repeat for each training vector pair (v, t) evaluate the output y when v is input to the TLU if y ≠ t then form a new weight vector w ’ according to formulas above else do nothing end if end for until y = t for all vectors w’ i = w i + ∆w i

Perceptron of Rosenblatt The A-units can be assigned any arbitrary Boolean functionality but are fixed - they do not learn.

Classification

1. The four classes may separated by 2-hyperplanes 2. (A,B) was linearly separable from (C,D) and (A,D) was linearly separable from (B,C).

Rule of training of Widrow-Hoff w ij (t+1)=w ij (t)+x j (d i -y i ) Task of minimization of function: It was implemented in ADALINE (Adaptive Linear Elements)

Classification of models of neural networks Tutoring –Supervised learning –Unsupervised learning –Reinforcement learning Structure –Forward or recurrent networks –With regular or not links –Describes by full-links graph or no –Static or dynamic (constructive learning) Signals (inputs or outputs, hidden) –Binary –Analog Time –Discrete –Continuous Kind of giving of inputs and getting of outputs –State of synapses –State of neurons –Weights of synapses

Tasks solved by neural networks Classification –In diagnostic systems –In monitoring systems –In recognition systems of robots –In speech recognition –In security systems for authentication Clusterization –In Data Mining for extracting of knowledge –In search systems for indexing of documents Prediction –In financial analyzing –In control systems of mobile robots Approximation –In control systems of technological processes

Comparison of Computing Approaches

Comparisons of Expert Systems and Neural Networks

Advantages of neural computing Clearly the style of processing is completely dierent - it is more akin to signal processing than symbol processing. The combining of signals and producing new ones is to be contrasted with the execution of instructions stored in a memory. Information is stored in a set of weights rather than a program. The weights are supposed to adapt when the net is shown examples from a training set. Nets are robust in the presence of noise: small changes in an input signal will not drastically aect a node's output. Nets are robust in the presence of hardware failure: a change in a weight may only aect the output for a few of the possible input patterns. High level concepts will be represented as a pattern of activity across many nodes rather than as the contents of a small portion of computer memory. The net can deal with `unseen' patterns and generalise from the training set. Nets are good at `perceptual' tasks and associative recall. These are just the tasks that the symbolic approach has diculties with.