1. Landscapes of the brain and mind
3rd MPSGC
KUALA LUMPUR 2007
Landscapes of the brain and mind
Wan Ahmad Tajuddin Wan Abdullah*
Complex Systems Group
Department of Physics
Universiti Malaya
50603 Kuala Lumpur *

2. 3rd MPSGC The Little-Hopfield Neural Network Model
KUALA LUMPUR 2007
The Little-Hopfield Neural Network Model
McCulloch-Pitts neurons:
Binary: Vi = 0,1
Summed inputs: hi = Σj TijVj
Theshold: Vi → H(hi-Ui)

3. 3rd MPSGC The Little-Hopfield Neural Network Model
KUALA LUMPUR 2007
The Little-Hopfield Neural Network Model
Network: N coupled nonlinear equations –
Vi(t+τ) = H(Σj TijVj(t)-Ui)
solve simultaneously!

4. 3rd MPSGC The Little-Hopfield Neural Network Model If
KUALA LUMPUR 2007
The Little-Hopfield Neural Network Model If
exchange symmetry in synaptic strengths Tij = Tji
no self-interactions Tii = 0
dynamics understood in terms of a Lyapunov function
E = - ½ Σi Σj TijViVj + Σi UiVi
Look:
ΔE = - hi ΔVi
monotone decreasing wrt neuron dynamics
cf conservative forces  potential function
cf spin systems (bipolar neurons, Si = 1)

5. 3rd MPSGC The Little-Hopfield Neural Network Model gradient descent
KUALA LUMPUR 2007
The Little-Hopfield Neural Network Model
'energy' landscape
gradient descent
energy minimum ≡ stable configurations
Energy
Configuration
Energy
Configuration

6. 3rd MPSGC The Little-Hopfield Neural Network Model
KUALA LUMPUR 2007
The Little-Hopfield Neural Network Model
Combinatorial optimization –
map combinatorial choices to neuron configuration
map cost function to energy function
obtain synaptic weights
let network relax to minimum energy configuration

7. 3rd MPSGC The Little-Hopfield Neural Network Model
KUALA LUMPUR 2007
The Little-Hopfield Neural Network Model
Associative memory –
optimize which ‘image’ stored nearest to input ‘image’
(initial configuration)
Tij := Σ(r) (2Vi (r) - 1) (2Vj (r) - 1) Ui = 0
Cooper-Hopfield prescription
Check: E = - ½ Σi Σj Σ(r) (2Vi (r) - 1) (2Vj (r) - 1)ViVj
minimum when Vi ~ Vi (r)
spurious memories – local minima
forgetting
temperature – simulated annealing
basins of attraction

8. 3rd MPSGC Energy landscapes Minima = stable states / solutions
KUALA LUMPUR 2007
Energy landscapes
Minima = stable states / solutions
Global minima = good solutions
Local minima = spurious states / solutions
Ruggedness (measured by e.g. correlations) = difficulty in finding solution

9. 3rd MPSGC Logic Programming Bird(x)Have_feathers(x),Fly(x).
KUALA LUMPUR 2007
Logic Programming
Bird(x)Have_feathers(x),Fly(x).
x Bird if x Have_feathers and x Fly.
Fly(Tweety).
Tweety Fly.
Have_feathers(Tweety).
Tweety Have_feather.
Have_fur(Sylvester).
Sylvester Have_fur. 
Bird(Tweety)
Horn clauses – at most 1 logical atom in consequent

10. 3rd MPSGC Logic Programming on Little-Hopfield networks
KUALA LUMPUR 2007
Logic Programming on Little-Hopfield networks
Logic programming ~ minimization of “logical inconsistency”
A ← B, C. A v ¬B v ¬C
D ← B. D v ¬B
C ←. C
EP = ⅛(1 - SA) (1 + SB) (1 + SC)
+ ¼ (1 - SD) (1 + SB)
+ ½ (1 - SC)
3rd order bipolar neural network
E = - ⅓ Σi Σj Σk Jijk(3)SiSjSk - ½ Σi Σj Jij(2)SiSj - Σi Ji(1)Si
Si := sign(Σj Σk Jijk(3)SjSk + Σj Jij(2)Sj + Ji(1) )

11. translate clauses in the logic program
3rd MPSGC
KUALA LUMPUR 2007
translate clauses in the logic program
Logic Programming on Little-Hopfield networks
Boolean algebraic form.
Derive a cost function that is associated with the negation of all the clauses
Obtain the values of connection strengths by comparing the cost function with the energy function
Let the neural networks evolve until minimum energy is reached.
The neural states provide a solution interpretation for the logic program, and the truth of a ground atom in this interpretation may be checked .

12. translate clauses in the logic program
3rd MPSGC
KUALA LUMPUR 2007
Logic Programming on Little-Hopfield networks
translate clauses in the logic program
how rugged is landscape from logic programming?
Computer simulations:… [SS & WATWA]
– flat energy landscape  no satisfiability problem
(all clauses can be satisfied i.e. solution always guaranteed)

13. 3rd MPSGC Satisfiability
KUALA LUMPUR 2007
Satisfiability
In general, general CNF clauses (conjuctions of disjunctions) not necessarily satisfiable
- depends on number of atoms in disjunctions
number of disjunctions
number of distinct atoms
Exist phase transition from easily soluble problems to difficult problems
[Zecchina & Monasson]

14. 3rd MPSGC Can 'knowledge' may be associated with the energy landscape?
KUALA LUMPUR 2007
Can 'knowledge' may be associated with the energy landscape?
What is knowledge?
? How ingrained are logical rules in neural network –
“ingrainedness”:
GX←Y,Z = < E({Si} satisfying X←Y,Z ) - E({Si} not satisfying X←Y,Z ) >
Can this be related to landscape ruggedness e.g. correlations?

15. 3rd MPSGC
KUALA LUMPUR 2007
Terima kasih