1. Hopfield Neural Networks for Optimization
虞台文
大同大學資工所
智慧型多媒體研究室

2. Content Introduction A Simple Example  Race Traffic Problem
Example  A/D Converter
Example  Traveling Salesperson Problem

3. Hopfield Neural Networks for Optimization
Introduction
大同大學資工所
智慧型多媒體研究室

4. Energy Function of a Hopfield NN
Interaction btw neurons
Interaction
to the external
constant
Running a Hopfield NN asynchronously, its energy is monotonically non-increasing.

5. Solving Optimization Problems Using Hopfield NNs
Reformulating the cost of a problem in the form of energy function of a Hopfield NN.
Build a Hopfield NN based on such an energy function.
Running the NN asynchronously until the NN settles down.
Read the answer reported by the NN.

6. Hopfield Neural Networks for Optimization
A Simple Example
Race Traffic Problem
大同大學資工所
智慧型多媒體研究室

7. A Simple Hopfield NN 1 2 I1 I2

8. The Race Traffic Problem+1 1 v1 v2

9. The Race Traffic Problem1 2 1

10. The Race Traffic Problem1 2 1 1 1 1
Stable State

11. The Race Traffic Problem1 2 1 1 1 1
Stable State

12. The Race Traffic Problem1 2 1 1 1
How about if to run synchronously?

13. Hopfield Neural Networks for Optimization
Example
A/D Converter
大同大學資工所
智慧型多媒體研究室

14. Reference
Tank, D.W., and Hopfield, J.J., “Simple "neural" optimization networks: An A/D converter, signal decision circuit and a linear programming circuit,” IEEE Transactions on Circuits and Systems, Vol. CAS-33 (1986)

15. I A/D Converter Using Unipolar Neurons v0 A/D v1 Analog v2 v3 20 21 2223 I
Analog
Using Unipolar Neurons

16. A/D Converter
Using Unipolar Neurons

17. A/D Converter 1 2 3 v0 v1 v2 v3 I0 I1 I2 I3

18. Hopfield Neural Networks for Optimization
Example
Traveling Salesperson Problem
大同大學資工所
智慧型多媒體研究室

19. Reference
J. J. Hopfield and D. W. Tank, “Neural” computation of decisions in optimization problems, ”
Biological Cybernetics, Vol. 52, pp , 1985.

20. Traveling Salesperson Problem

21. Traveling Salesperson Problem
Given n cities with distances dij, what is the shortest tour?

22. Traveling Salesperson Problem2 3 4 1 5 9 6 11 8 10 7

23. Traveling Salesperson Problem
Distance Matrix
Find a minimum cost Hamiltonian Cycle.

24. Search Space Find a minimum cost Hamiltonian Cycle.
Assume we are given a fully connection graph with n vertices and symmetric costs (dij=dji).
The size of search space is
Find a minimum cost Hamiltonian Cycle.

25. Problem Representation Using NNs
Time 1 2 3 4 5 2 1 1 2 3 4 5 4
City 3 5

26. Problem Representation Using NNs
The salesperson reaches city 5 at time 3.
Time 1 2 3 4 5 2 1 1 2 3 4 5 4
City 3 5

27. Problem Representation Using NNs
Goal: Find a minimum cost
Hamiltonian Cycle.
Problem Representation Using NNs
Time 1 2 3 4 5 2 1 1 2 3 4 5 4
City 3 5

28. The Hamiltonian Constraint
Goal: Find a minimum cost
Hamiltonian Cycle.
The Hamiltonian Constraint
Time 1 2 3 4 5
Each row and column can have only one neuron “on”.
For a n-city problem, n neurons will be on. 2 1 1 2 3 4 5 4
City 3 5

29. Cost Minimization Goal: Find a minimum cost Hamiltonian Cycle. Time 12 3 4 5
The total distance of the valid tour have to be very low. 2 1 1 2 3 4 5
d35
d54
d42
d25
d51 4
City
The summation of these dij’s is very low. 3 5

30. Indices of Neurons i
Time
vxi 1 2 3 4 5 1 2 3 4 5
City x

31. Hamiltonian-Cycle Satisfaction
Energy Function
Hamiltonian-Cycle Satisfaction
Cost Minimization

32. Energy Function n neurons ‘on’ Each row one or zero neuron ‘on’
Each column
one or zero
neuron ‘on’

33. Total distance of the tour
Energy Function
Total distance of the tour

34. Energy Function

35. Build NN for TSP
Mapping
Energy function of a
2-D neural network

36. Analog Hopfield NN for 10-City TSP

37. Analog Hopfield NN for 10-City TSP
The shortest path

38. Analog Hopfield NN for 10-City TSP
The shortest path

39. Analog Hopfield NN for 30-City TSP

40. Analog Hopfield NN for 30-City TSP