An Associative Memory based on a Mixed-Signal Cellular Neural Network Michael Flynn, Daniel Weyer

 Design of an Associative Memory  Implementation with a Mixed-Signal Neural Network Motivation: Weight Lookup Memory neuron ID bus neuron ID weight …… retrieved weight 2

Associative memory can recover corrupted bits Neuron states represent bits of stored bit patterns  bit patterns are composed of address bits and data bits Treat data bits as corrupted bits  provide address bits and let neural network dynamics retrieve correct data bits Data Retrieval Concept amam...a3a3 a2a2 a1a1 dndn d2d2 d1d1 address bits (neuron ID) data bits (associated weight) 3

Architectural approach: Cellular Neural Network  every neuron is connected to neighboring neurons only  limited interconnectivity = good for VLSI implementation “Data neurons” encircled by “address neurons” Neural Network Architecture “address bit neuron” “data bit neuron” 4

Example: 12-bit pattern, 8 address bits + 4 data bits Bit Mapping on the Neuron States neuron network

Example: 12-bit pattern, 8 address bits + 4 data bits Data Retrieval Process initialize states of “address neurons” read out final states of “data neurons” update network state (synchronous updates) ① ② ③ 6

Mixed-signal implementation of the neuron transfer function Circuit Implementation comparator weight multiplier … x1x1 x2x2 xnxn w1w1 w2w2 wnwn I1I1 I2I2 InIn  Ii Ii … xnxn sgn( w n ) |wn||wn| InIn weight multiplier neuron 7

Bit patterns are held in the connection weights of the network Repetitive application of Hebb’s rule to store multiple patterns Storage of k-th bit pattern:  w ij (k) = w ij (k-1) + 1 if bits i (k) and j (k) are equal  w ij (k) = w ij (k-1) - 1 if bits i (k) and j (k) have opposite values Contributions to w ij are likely to cancel each other if stored bit patterns are disparate, i.e. weakly correlated  store bit patterns only if certain level of correlation is given Programming of the Memory i j w ij 8

Storage of 12-bit patterns in a 25-neuron network Simulation Setup 9 mixed-signal neuron multiplexer for node initialization digital blocks for weight programming

Example: Stored 12-bit patterns 0xC54, 0x835, 0xE8A, 0x0E6 Simulation Results 10 clock init signal applied address data bit 4 data bit 3 data bit 2 data bit 1 programming phase initialization phases update & retrieval phases

Example: 12-bit patterns, 8 address bits + 4 data bits  increased storage capacity if address patterns differ in about half the number of bit positions Memory Capacity d:Hamming distance between address bit patterns no restriction 1  d  7 2  d  6 3  d  5 d = 4 11

“Filter” bit patterns to ensure desired pattern distances  optimize overall storage capacity Outlook: Modular Memory Composition pattern filter 12