1. Hopfield Neural Networks for Optimization 虞台文 大同大學資工所 智慧型多媒體研究室

2. Content Introduction A Simple Example  Race Traffic Problem Example  A/D Converter Example  Traveling Salesperson Problem

3. Hopfield Neural Networks for Optimization Introduction 大同大學資工所 智慧型多媒體研究室

4. Energy Function of a Hopfield NN Interaction btw neurons Interaction to the external constant Running a Hopfield NN asynchronously, its energy is monotonically non-increasing.

5. Solving Optimization Problems Using Hopfield NNs Reformulating the cost of a problem in the form of energy function of a Hopfield NN. Build a Hopfield NN based on such an energy function. Running the NN asynchronously until the NN settles down. Read the answer reported by the NN.

6. Hopfield Neural Networks for Optimization A Simple Example Race Traffic Problem 大同大學資工所 智慧型多媒體研究室

7. A Simple Hopfield NN 11 11 22 22 I1I1 I2I2

8. The Race Traffic Problem +1 11 11 v1v1 v2v2

9. The Race Traffic Problem 11 11 22 22 00 11

10. 11 11 22 22 00 11 11 11 1 Stable State

11. The Race Traffic Problem 11 11 22 22 00 11 11 11 1 Stable State

12. The Race Traffic Problem 11 11 22 22 00 11 11 11 How about if to run synchronously?

13. Hopfield Neural Networks for Optimization Example A/D Converter 大同大學資工所 智慧型多媒體研究室

14. Reference Tank, D.W., and Hopfield, J.J., “Simple "neural" optimization networks: An A/D converter, signal decision circuit and a linear programming circuit,” IEEE Transactions on Circuits and Systems, Vol. CAS-33 (1986) 533-541.

15. Analog A/D Converter A/D v0v0 v1v1 v2v2 v3v3 2020 2121 2 2323 I Using Unipolar Neurons

16. A/D Converter Using Unipolar Neurons

17. A/D Converter 0 0 1 1 2 2 3 3 v0v0 v1v1 v2v2 v3v3 I0I0 I1I1 I2I2 I3I3

18. Hopfield Neural Networks for Optimization Example Traveling Salesperson Problem 大同大學資工所 智慧型多媒體研究室

19. Reference J. J. Hopfield and D. W. Tank, “Neural” computation of decisions in optimization problems, ” Biological Cybernetics, Vol. 52, pp.141-152, 1985.

20. Traveling Salesperson Problem

21. Given n cities with distances d ij, what is the shortest tour?

22. Traveling Salesperson Problem 1 2 3 4 5 6 7 8 9 10 11

23. Traveling Salesperson Problem Distance Matrix Find a minimum cost Hamiltonian Cycle.

24. Search Space Find a minimum cost Hamiltonian Cycle. Assume we are given a fully connection graph with n vertices and symmetric costs ( d ij =d ji ). The size of search space is

25. Problem Representation Using NNs 1 2 4 3 5 12345 1 2 3 4 5 Time City

26. Problem Representation Using NNs 1 2 4 3 5 12345 1 2 3 4 5 Time City The salesperson reaches city 5 at time 3.

27. Problem Representation Using NNs 1 2 4 3 5 12345 1 2 3 4 5 Time City Goal: Find a minimum cost Hamiltonian Cycle.

28. The Hamiltonian Constraint 1 2 4 3 5 12345 1 2 3 4 5 Time City Goal: Find a minimum cost Hamiltonian Cycle. Each row and column can have only one neuron “on”. For a n -city problem, n neurons will be on. Each row and column can have only one neuron “on”. For a n -city problem, n neurons will be on.

29. Cost Minimization 1 2 4 3 5 12345 1 2 3 4 5 Time City Goal: Find a minimum cost Hamiltonian Cycle. The total distance of the valid tour have to be very low. d 35 d 54 d 42 d 25 d 51 The summation of these d ij ’s is very low.

30. Indices of Neurons 12345 1 2 3 4 5 Time City v xi x i

31. Energy Function Hamiltonian-Cycle Satisfaction Cost Minimization

32. Energy Function Each row one or zero neuron ‘on’ Each column one or zero neuron ‘on’ n neurons ‘on’

33. Energy Function Total distance of the tour

34. Energy Function

35. Build NN for TSP Energy function of a 2-D neural network Mapping

36. Analog Hopfield NN for 10-City TSP

37. The shortest path

38. Analog Hopfield NN for 10-City TSP The shortest path

39. Analog Hopfield NN for 30-City TSP