1. 1 COMP305. Part I. Artificial neural networks.

2. 2 The McCulloch-Pitts Neuron (1943). McCulloch and Pitts demonstrated that “…because of the all-or-none character of nervous activity, neural events and the relations among them can be treated by means of the propositional logic”.

3. 3 The McCulloch-Pitts Neuron – Thus, the McCulloch-Pitts neuron operates on a discrete time scale, t = 0,1,2,3,… discrete time machine.

4. 4 The McCulloch-Pitts Neuron – The input values a i t from the i-th presynaptic neuron at any instant t may be equal either to 0 or 1 only discrete time machine. binary unit.

5. 5 The McCulloch-Pitts Neuron – The weights of connections w i are +1 for excitatory type connection and -1 for inhibitory type connection. discrete time machine.

6. 6 The McCulloch-Pitts Neuron – There is an excitation threshold  associated with the neuron. discrete time machine.

7. 7 The McCulloch-Pitts Neuron. Output X t+1 of the neuron at the following instant t+1 is defined according to the rule:

8. 8 The McCulloch-Pitts Neuron. In the MP neuron, we shall call the instant total input S t – instant state of the neuron.

9. 9 The McCulloch-Pitts Neuron. The statement “ “ means that activity of a single inhibitory input, i.e. input via a connection with negative weight w i = -1, would absolutely prevents excitation of the neuron at that instant.

10. 10 Activation function. The output X t+1 is function of the state S t of the neuron, therefore it also may be written as function of discrete time where g(S t ) is the threshold activation function

11. 11 MP-neuron example.

12. 12 MP-neuron example. Input 1. 1) a 1 =0, a 2 =0, a 3 =1 2)All inhibitory connections are silent 3) S = 0×(-1) + 0×1 + 1×1 = 1 < θ 4) S X = 0

13. 13 MP-neuron example. Input 2. 1) a 1 =0, a 2 =1, a 3 =1 2)All inhibitory connections are silent 3) S = 0×(-1) + 1×1 + 1×1 = 2 = θ 4) S = θ => X = 1

14. 14 MP-neuron example. Input 3. 1) a 1 =1, a 2 =1, a 3 =1 2)There is an inhibitory connection activated 3)X = 0

15. 15 MP-neuron as a binary unit. Simple logical functions can be implemented directly with a single McCulloch-Pitts unit. The output value 1 can be associated with the logical value true and 0 with the logical value false. Now, let us demonstrate how weights and thresholds can be set to yield neurons which realise the logical functions AND, OR and NOT.

16. 16 MP-neuron logic. “AND” - the output fires if a 1 and a 2 both fire. a1a1 a2a2 “AND” 111 010 100 000 = “AND”

17. 17 MP-neuron logic. “OR” - the output fires if a 1 or a 2 or both fire. a1a1 a2a2 “OR” 111 011 101 000 = “OR”

18. 18 MP-neuron logic. “NOT”: the output fires if a 1 does NOT fire and vice versa. = “NOT” a1a1 “ NOT” 10 01