1. Ch1: Introduction to ANS Technology
Human brain: the most complex computing
device that we have ever known.
Conventional computers: are good in
Scientific and mathematical computations
Creation and manipulation of databases
Word processing, Graphics,
Desktop publishing, Control functions
How do computers: learn, analyze, adapt, organize,
comprehend, associate, recognize, plan, decide

2. 。 Many problems are not suitable to be solved
by sequential machines and algorithms.
Solving these problems should involve a
number of processes, such as analyzing,
associating, organizing, comprehending,
and recognizing processes
e.g., Perceptual-Organizing Problem
Visual Pattern Recognition
Subjective inference

3. Example 1: Perceptual-Organization Problem
25 line segments

4. Is local evidence so little use ?
26 line segments

8. Perceptual organization is well processed in
a parallel manner.

9. Example 2: Visual Pattern Recognition

10. Conventional processing:
Thresholding
Noise removal
Group ─ form significant regions
Cluster ─ associate with objects
Segment ─ locate primary areas
Edge detection , Edge following
Recognition
Input image
Thresholding
Edge detection
Edge following
Grouping,
Clustering
Segmentation

11. Example 3: Subjective inference (Reasoning)

12. Parallel-processing architectures may be good tools for solving difficult-to-solve, or unsolved problems 12 12

13. 1.1 Neurophysiology ◎ Three major components constructing the human
nervous system: 1. brain, 2. spinal cord, 3. periphery

14. Brain Cortex Activities: sensation, perception, cognition,
Cerebral cortex
Cortex
Size: 1
Thick: mm
Layers: 6
Activities: sensation, perception, cognition,
recognition, imagination, dream,
consciousness
result in
functions: thinking, feeling, willing

15. Spinal Nerves

16. Periphery 16 16

17. Single-Neurons Physiology
Three types
of neurons:
1. unipolar
2. bipolar
3. multipolar
Terminal
末梢神經
感覺器官
Pathway
連絡神經 脊髓
Central
中樞神經 腦

18. ○ Structure
The major structure of a nerve cell include dendrites, the cell body,
and a single axon. The axon is surrounded by a membrane called
the myelin sheath. Nodes of Ranvier interrupt the myelin sheath
periodically along the length of the axon. Synapses connect the
axons of one neuron to various parts of other neurons.

19. Nerve impulses from other neurons can result in changes in the
potential of the neuron. The input potentials can be excitatory
(decreasing the polarization of the cell) or inhibitory (increasing
the polarization of the cell). They are summed at the axon hillock.
If the summed depolarization is sufficient, an action potential is
generated; it travels down the axon away from the main cell body.

20. ○ Synaptic Function

21. Signal: frequency of pulses
Learning: adjusting synaptic gaps
potentials
firing rates
Memory: synaptic connections
strength of connections
Knowledge is acquired through a learning process.
Acquired knowledge is stored in interneuron links
(i.e., synaptic connections) in terms of strengths.

22. Nervous systems
Distinguished Characteristics
。Adaptation, 。Self-organizing
。Parallel processing, 。Nonlinear function
。Associative memory

23. 1.2 Artificial neural systems
-- A collection of parallel processing units connected
together
-- Various neural systems with different functions and
characteristics result from
i) Functions of neurons
ii) Ways of connections
iii) Flows of information

24. Characteristics:
nonlinearity, non-locality, non-algorithm,
dynamics, adaptivity, fault-tolerance,
input-output mapping, evidential response,
self-organization,
Basic functions:
learn, analyze, organize, comprehend,
associate, recognize, plan, decide

25. ○ Example: Optical Character Recognition (OCR)
10 decimal characters
10 output units each corresponding to one character

26. Represent a character image (10 by 8) as a vector containing binary elements
。 Two different phases of network operation
1) Training phase – encoding information
2) Production phase – recalling information

27. Abilities of ANS： noise, distortion, incompletion

28. ○ McCullock-Pitts Theory (1943)
1.3 Neural Circuits .
○ McCullock-Pitts Theory (1943)
-- the first theory for treating the brain as
a computational organism
。 Assumptions：
All-or-none (on-or-off) process
Synapses excited ===> neuron active
Synapses delay
Synapses inhibited ===> neuron inactive
Fixed interconnection

29. number of basic neural circuits
○ A complex neural circuit is composed of a
number of basic neural circuits
Basic Neural Circuits – characterized by
convergence, divergence, feedback, excitatory
and inhibitory connections

30. 。 Propositional logic Statement Assertion T, F Proposition P
Predicate P(x)
Quantifiers
Operators , , If ….. then
Propositional expression
: neuron i fires at time t
: not fire

31. Graphic representation of propositional expressions
(1) Excitatory:
(2) Inhibitory:
(3) Fire:
(4) Activate:
(5) Precession:
Neuron 2 activates after neuron 1 fires

32. (6) Disjunction:
(7) Conjunction:
(8) Conjoined negation:

33. 。Example : Combination of simple networks
Input nodes:
Hidden nodes:
Output nodes:
Similarly,

34. 1.4 Processing Element (PE) and Neuron

35. Types of activation function :
1. Threshold function (Heaviside function)
2. Sigmoid function
Logistic function
Hyperbolic function

36. 1.5 Hebbian Learning
○ Learning process –
Modifies the network to incorporate new
information
○ Hebb’s Learn Theory --
When cell A is near enough to excite cell B
and repeatedly or persistently takes part in
firing it, some metabolic process takes place
in one or both cells such that A’s efficiency
is increased

37. Example: (Pavlov experiment)37 37

38. ◎ Dynamic system: a system evolves over time
Example :
i for a sufficiently long time
reaches an equilibrium value, i.e.,
ii, Remove input,
◎ Learning: modification of synaptic strengths
(i.e., weight values)
where : learning law

39. 1.6 Example Neural Models ◎ Photoperceptron The Human Eye Two kinds of
photoreceptors:
rods, cones

40. Receptive field, Excitatory connection,
Inhibitory connection, Feedback

41. Each R unit inhibits the A units in the complement to its own source set.
Each R unit inhibits the other.
The above aid in the establishment of a winning
R unit. (competition)
Application
Classify patterns into categories
Distinguish pattern classes

42. The following NM can differentiate patterns only if
the patterns are linearly separable. (Minsky & Papert
1969)
Let : a set of inputs,
Let be a corresponding set of
weights

43. XOR problem (cannot be solved by a perceptron)
Output function:
Activation:
Question: select , and such that each pair of
input values results in the proper output value.

44. This task cannot be done
Reason:
Let
A line in the plane

45. One solution:
The hidden layer provides two lines that can separate the
plane into three regions. The two regions containing (0,0)
and (1,1) are associated with a network output of 0. The
central region is associated with a network output of 1.

46. Homework Write the propositional expression for
(b) Construct McCulloch-Pitts networks for the
following expressions 1. 2.