1 COMP305. Part I. Artificial neural networks.

2 The McCulloch-Pitts Neuron (1943). McCulloch and Pitts demonstrated that “…because of the all-or-none character of nervous activity, neural events and the relations among them can be treated by means of the propositional logic”.

3 The McCulloch-Pitts Neuron – Thus, the McCulloch-Pitts neuron operates on a discrete time scale, t = 0,1,2,3,… discrete time machine.

4 The McCulloch-Pitts Neuron – The input values a i t from the i-th presynaptic neuron at any instant t may be equal either to 0 or 1 only discrete time machine. binary unit.

5 The McCulloch-Pitts Neuron – The weights of connections w i are +1 for excitatory type connection and -1 for inhibitory type connection. discrete time machine.

6 The McCulloch-Pitts Neuron – There is an excitation threshold  associated with the neuron. discrete time machine.

7 The McCulloch-Pitts Neuron. Output X t+1 of the neuron at the following instant t+1 is defined according to the rule:

8 The McCulloch-Pitts Neuron. In the MP neuron, we shall call the instant total input S t – instant state of the neuron.

9 The McCulloch-Pitts Neuron. The statement “ “ means that activity of a single inhibitory input, i.e. input via a connection with negative weight w i = -1, would absolutely prevents excitation of the neuron at that instant.

10 Activation function. The output X t+1 is function of the state S t of the neuron, therefore it also may be written as function of discrete time where g(S t ) is the threshold activation function

11 MP-neuron example.

12 MP-neuron example. Input 1. 1) a 1 =0, a 2 =0, a 3 =1 2)All inhibitory connections are silent 3) S = 0×(-1) + 0×1 + 1×1 = 1 < θ 4) S X = 0

13 MP-neuron example. Input 2. 1) a 1 =0, a 2 =1, a 3 =1 2)All inhibitory connections are silent 3) S = 0×(-1) + 1×1 + 1×1 = 2 = θ 4) S = θ => X = 1

14 MP-neuron example. Input 3. 1) a 1 =1, a 2 =1, a 3 =1 2)There is an inhibitory connection activated 3)X = 0

15 MP-neuron as a binary unit. Simple logical functions can be implemented directly with a single McCulloch-Pitts unit. The output value 1 can be associated with the logical value true and 0 with the logical value false. Now, let us demonstrate how weights and thresholds can be set to yield neurons which realise the logical functions AND, OR and NOT.

16 MP-neuron logic. “AND” - the output fires if a 1 and a 2 both fire. a1a1 a2a2 “AND” = “AND”

17 MP-neuron logic. “OR” - the output fires if a 1 or a 2 or both fire. a1a1 a2a2 “OR” = “OR”

18 MP-neuron logic. “NOT”: the output fires if a 1 does NOT fire and vice versa. = “NOT” a1a1 “ NOT” 10 01