Hopfield Neural Networks for Optimization 虞台文 大同大學資工所 智慧型多媒體研究室

Content Introduction A Simple Example  Race Traffic Problem Example  A/D Converter Example  Traveling Salesperson Problem

Hopfield Neural Networks for Optimization Introduction 大同大學資工所 智慧型多媒體研究室

Energy Function of a Hopfield NN Interaction btw neurons Interaction to the external constant Running a Hopfield NN asynchronously, its energy is monotonically non-increasing.

Solving Optimization Problems Using Hopfield NNs Reformulating the cost of a problem in the form of energy function of a Hopfield NN. Build a Hopfield NN based on such an energy function. Running the NN asynchronously until the NN settles down. Read the answer reported by the NN.

Hopfield Neural Networks for Optimization A Simple Example Race Traffic Problem 大同大學資工所 智慧型多媒體研究室

A Simple Hopfield NN 11 11 22 22 I1I1 I2I2

The Race Traffic Problem +1 11 11 v1v1 v2v2

The Race Traffic Problem 11 11 22 22 00 11

11 11 22 22 00 11 11 11 1 Stable State

The Race Traffic Problem 11 11 22 22 00 11 11 11 1 Stable State

The Race Traffic Problem 11 11 22 22 00 11 11 11 How about if to run synchronously?

Hopfield Neural Networks for Optimization Example A/D Converter 大同大學資工所 智慧型多媒體研究室

Reference Tank, D.W., and Hopfield, J.J., “Simple "neural" optimization networks: An A/D converter, signal decision circuit and a linear programming circuit,” IEEE Transactions on Circuits and Systems, Vol. CAS-33 (1986)

Analog A/D Converter A/D v0v0 v1v1 v2v2 v3v I Using Unipolar Neurons

A/D Converter Using Unipolar Neurons

A/D Converter v0v0 v1v1 v2v2 v3v3 I0I0 I1I1 I2I2 I3I3

Hopfield Neural Networks for Optimization Example Traveling Salesperson Problem 大同大學資工所 智慧型多媒體研究室

Reference J. J. Hopfield and D. W. Tank, “Neural” computation of decisions in optimization problems, ” Biological Cybernetics, Vol. 52, pp , 1985.

Traveling Salesperson Problem

Given n cities with distances d ij, what is the shortest tour?

Traveling Salesperson Problem

Traveling Salesperson Problem Distance Matrix Find a minimum cost Hamiltonian Cycle.

Search Space Find a minimum cost Hamiltonian Cycle. Assume we are given a fully connection graph with n vertices and symmetric costs ( d ij =d ji ). The size of search space is

Problem Representation Using NNs Time City

Problem Representation Using NNs Time City The salesperson reaches city 5 at time 3.

Problem Representation Using NNs Time City Goal: Find a minimum cost Hamiltonian Cycle.

The Hamiltonian Constraint Time City Goal: Find a minimum cost Hamiltonian Cycle. Each row and column can have only one neuron “on”. For a n -city problem, n neurons will be on. Each row and column can have only one neuron “on”. For a n -city problem, n neurons will be on.

Cost Minimization Time City Goal: Find a minimum cost Hamiltonian Cycle. The total distance of the valid tour have to be very low. d 35 d 54 d 42 d 25 d 51 The summation of these d ij ’s is very low.

Indices of Neurons Time City v xi x i

Energy Function Hamiltonian-Cycle Satisfaction Cost Minimization

Energy Function Each row one or zero neuron ‘on’ Each column one or zero neuron ‘on’ n neurons ‘on’

Energy Function Total distance of the tour

Energy Function

Build NN for TSP Energy function of a 2-D neural network Mapping

Analog Hopfield NN for 10-City TSP

The shortest path

Analog Hopfield NN for 10-City TSP The shortest path

Analog Hopfield NN for 30-City TSP