Neural Nets Applications Introduction

Outline(1/2) 1. What is a Neural Network? 2. Benefit of Neural Networks 3. Structural Levels of Organization in the Brain 4. Models of a Neuron 5. Network Architectures 6. Artificial Intelligence and Neural Networks

Outline(2/2) 7. Existing Applications 8. Possible Applications 9. Experiment I 10. Experiment II 11. Other names for Neural Networks 12. Who are the key player?

What is a Neural Networks(1/5)  Neural networks technology is not trying to produce biological machine  but is trying to mimic nature’s approach in order to mimic some of nature’s capabilities.

What is a Neural Networks (2/5)  Definition: A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use.

What is a Neural Networks (3/5)  It resembles the brain in two respects: 1. Knowledge is acquired by the network through a learning process. 2. Interneuron connection strengths known as synaptic weight are used to store the knowledge.

What is a Neural Networks (4/5)  The Human Brain:  Five to six orders of magnitude slower than silicon logic gates  With 60 trillion synapses or connections  A highly complex, nonlinear, and parallel computer.  Figure 1.1

What is a Neural Networks (5/5)

Benefits of Neural Networks (1/2) 1. Nonlinearity 2. Input-Output Mapping 3. Adaptivity 4. Evidential Response 5. Contextual Information

Benefits of Neural Networks (2/2) 6. Fault Tolerance 7. Implementability 8. Uniformity of Analysis and Design 9. Neurobiological Analogy

Structural Levels of Organization in the Brain (1/3) 1. Figure Figure 1.3

Structural Levels of Organization in the Brain (2/3)

Structural Levels of Organization in the Brain (3/3)

Models of a Neuron (1/6) 1. Figure Three basic elements of the neuron model:  A set of synapses or connecting links, each of which is characterized by a weight or strength of its own.  An adder for summing the input signals, weighted by the respective synapses of the neuron; the operations described here constitute a linear combiner.  An activation function for limiting the amplitude of the output of a neuron.

Models of a Neuron (2/6)

Models of a Neuron (3/6) 3. Mathematical terms: where: x j : input signals w kj : synaptic weights u k : linear combiner output θ k: : threshold f() : activation function y k : output signal

Models of a Neuron (4/6) 4. Types of activation function: a.Threshold function

Models of a Neuron (5/6) 4. Types of activation function: b.Piecewise-linear function

Models of a Neuron (6/6) 4. Types of activation function: c.Sigmoid Function

Network Architecture (1/5) 1.single-layer feedforward network

Network Architecture (2/5) 2.Multilayer feedforward network (fully connected

Network Architecture (3/5) 2.Multilayer feedforward network (partially connected

Network Architecture (4/5) 3.Recurrent networks

Network Architecture (5/5) 4.Lattice Structures

Artificial Intelligence and Neural Networks (1/5) AI system

Artificial Intelligence and Neural Networks (2/5) a.Representation - use a language of symbol structures to represent both general knowledge about a problem domain of interest and specific knowledge about the solution to the problem.

Artificial Intelligence and Neural Networks (3/5) b.Reasoning - the ability to solve the problems - be able to express and solve a broad range of problems and problem types. - be able to make explicit and inplicit information known to it - have a control mechanism that determines which operations to apply to a particular problem.

Artificial Intelligence and Neural Networks (4/5) c.Learning - Fig Inductive, rules are from raw data and experience - Deductive, rules are used to determine specific facts

Artificial Intelligence and Neural Networks (5/5)

Existing Applications(1/4) 1. Long distance echo adaptive fitter  adaptive noise canceling -- ADALINE 2. Mortgage risk evaluator 3. Bomb sniffer -- SNOOPE -- JFK airport

Existing Applications(2/4) 4. Process Monitor -- GTE used in fluorescent bulb plant. -- To determine optimum manufacturing condition. -- To indicate what controls need to be adjusted, and potentially to even shut down the line. -- Statistics could provide same result but with huge data.

Existing Applications(3/4) 5. Word Recognizer --Intel used single speaker on limited vocabulary. 6. Blower Motor Checker --Siemens used to check Blower motor noise is heater. 7. Medical events

Existing Applications(4/4) 8. US postal office for hand written 9. Airline marketing tactician.

Possible Applications(1/6) 1. Biological --Learning more about the brain and other systems --Modeling retina, cochlea 2. Environmental --Analyzing trends and patterns --Forecasting weather

Possible Applications(2/6) 3. Business --Evaluating probability of oil in geological formations --Identifying corporate candidates for special positions --Mining corporate databases --Optimizing airline seating and fee schedules --Recognizing handwritten characters, such as Kanji

Possible Applications(3/6) 4. Financial --Assessing credit risk --Identifying forgeries --Interpreting handwritten forms --Rating investments and analyzing portfolios

Possible Applications(4/6) 5. Manufacturing --Automating robots and control system (with machine vision and sensors for pressure. temperature, gas, etc.) --Controlling production line processes --Inspecting for quality --Selecting parts on an assembly line

Possible Applications(5/6) 6. Medical --Analyzing speech in hearing aids for the profoundly deaf --Diagnosing/prescribing treatments from symptoms --Monitoring surgery --Predicting adverse drug reactions --Reading X-rays --Understanding cause of epileptic seizures

Possible Applications(6/6) 7. Military --Classifying radar signals --Creating smart weapons --Doing reconnaissance --Optimizing use of scarce resources --Recognizing and tracking targets

Experiment I 1. to understand a sentence are character a time is much larger than one word a time 2. conventional computer processes its input one of a time, working sequentially 3. our eyes look at the whole sentence 4. vowels are missing 5. three different groupings

Experiment II(1/2) 1. Toss a chalk to another one -- it is hard in dynamics -- estimate the speed, the trajectory, the weight -- in real time -- computer must be faster

Experiment II(2/2) But -- our brain is lower than computer -- our brain still better than computer Why?  parallel processing

Other Names for Artificial Neural Networks Parallel/distributed processing models Connectivist/connectionism models adaptive systems self-organizing systems Neurocomputing Neuromorphic systems Self-learning systems

Who Are the Key Players? (1/2) 1. Medical and theoretical neurobiologists --Neurophysiology, drug chemistry, molecular biology 2. Computer and information scientists --Information theory 3. Adaptive control theorists/psychologists --Merging learning and control theory

Who Are the Key Players? (2/2) 4. Adaptive systems -- researchers/biologists --Self-organization of living species 5. AI researchers --Machine learning mechanisms