4.3 Feedforward Net. Applications MLP: Signum - Classification Sigmoid - Mapping/Classification Some weights may be fixed (at zero for partially connected) or shared. Neural computing can offer relatively simple solutions to complex pattern classification problems. ‘Black Box' solution (1) M-class Classification(with sigmoidal nonlinearity) Train with {x, ek} if x  ωk ek = unit vector along kth dimension = (0 … 0 1 0 … 0) Test : If yk is MAX then x  ωk For large training set No local minimum trap Enough Plasticity Neural Network is a nonparametric probability density estimator with Perfect training. - [Ref. Rumelhart PDP Vol.1 Ch.8] k x N y MLP classifier yk  P (ωkx)  A posteriori class probability

(2) Parity (XOR / n-bit Parity) (3) Encoder-Decoder (Autoassociator / Autoencoder) Compression Expansion 1 O O 1 Ex. 4 - 2 - 4 5 - 3 - 5 N - M - N ( M < N) O O O O O O O O (4) ’87 Cottrell : Image Compression with limited channel capacity [too small to allow transmission of color and intensity ] (HDTV) − redundancy elimination via self-supervised BP. Train on random overlapping patches, test on a complete set of non-overlapping patches of the same image or even of very different images. * PCA Network with linear activation function can compress better. patch 8 Input Image 64 16 Output Compression Expansion (Encoding) (Decoding)

(5) T-C problem (6) Engineering T h i s i s t h e input (7-letter window slides over text) 80 Hidden units (6) Engineering a. Speech Synthesis - NETtalk by Sejnowski - trained with 1024 words – intelligible after 10 epochs - 95% accuracy after 50 epochs - like a child learning to talk - 78% generalization after complete training (still intelligible) - NN is easy to construct, can be used even when a problem is not fully understood. Adding noise to connections or removing units only degrades performance gracefully. cf. Commercial DECTALK ( 10 years of analysis by many linguists, rule-based ) b. Signal Processing : Ref. NN for SP, Kosko, 91.

c. Signal Prediction a time T into the future – Time Series Analysis ) ( t x 1 - z T + Neural Network current future t t +T Past input Benchmark Problem : <Mackey-Glass Differential Delay Eq.> - Dynamic system that is more chaotic if  is bigger. - NN models the dynamics of this system. It predicts better than traditional methods.

Question: IV must rapidly adapt to external changing Env. How ? d. Carnegie Mellon University Autonomous Land Vehicle ** To be shown on Video Question: IV must rapidly adapt to external changing Env. How ? CMU Navlab and the NN Based Autonomous Driving

Some Real World Data that Can be Used for NN Design

Students’ Questions from 2005 I think that learning in Speech Recognition also needs a desired output. While in English some sound follows some other sound, in Korean there will be far more combinations of consonants, vowels, and undersymbols. Do we need to compare all of them ? With that amount of data, learning time will explode. If data gets reduced in face recognition, then also the size of the face DB for comparison diminish ? In HDTV, why use 8x8 patches ? How about using 16x16 or more ? Do they use NN in practice for HDTV ? In the HDTV compression, even under no redundancy, some important features such as in face recognition might be extracted. Do you foresee any problem in this ? What happens to the indirect control when multiple inverses exist, i.e., when the next desired state differs even for the same current state ?