1. An Associative Memory based on a Mixed-Signal Cellular Neural Network Michael Flynn, Daniel Weyer

2.  Design of an Associative Memory  Implementation with a Mixed-Signal Neural Network Motivation: Weight Lookup Memory neuron ID bus neuron ID weight …… retrieved weight 2

3. Associative memory can recover corrupted bits Neuron states represent bits of stored bit patterns  bit patterns are composed of address bits and data bits Treat data bits as corrupted bits  provide address bits and let neural network dynamics retrieve correct data bits Data Retrieval Concept amam...a3a3 a2a2 a1a1 dndn d2d2 d1d1 address bits (neuron ID) data bits (associated weight) 3

4. Architectural approach: Cellular Neural Network  every neuron is connected to neighboring neurons only  limited interconnectivity = good for VLSI implementation “Data neurons” encircled by “address neurons” Neural Network Architecture “address bit neuron” “data bit neuron” 4

5. Example: 12-bit pattern, 8 address bits + 4 data bits Bit Mapping on the Neuron States 876543214321 1 2312 4 1 524 6 7867 43 4 54 12 1 32 5 25-neuron network

6. Example: 12-bit pattern, 8 address bits + 4 data bits Data Retrieval Process 8765 1 2312 4 1 524 6 7867 43 4 54 12 1 32 4321 4321 initialize states of “address neurons” read out final states of “data neurons” update network state (synchronous updates) ① ② ③ 6

7. Mixed-signal implementation of the neuron transfer function Circuit Implementation comparator weight multiplier … x1x1 x2x2 xnxn w1w1 w2w2 wnwn I1I1 I2I2 InIn  Ii Ii … xnxn sgn( w n ) |wn||wn| InIn weight multiplier neuron 7

8. Bit patterns are held in the connection weights of the network Repetitive application of Hebb’s rule to store multiple patterns Storage of k-th bit pattern:  w ij (k) = w ij (k-1) + 1 if bits i (k) and j (k) are equal  w ij (k) = w ij (k-1) - 1 if bits i (k) and j (k) have opposite values Contributions to w ij are likely to cancel each other if stored bit patterns are disparate, i.e. weakly correlated  store bit patterns only if certain level of correlation is given Programming of the Memory i j w ij 8

9. Storage of 12-bit patterns in a 25-neuron network Simulation Setup 9 mixed-signal neuron multiplexer for node initialization digital blocks for weight programming

10. Example: Stored 12-bit patterns 0xC54, 0x835, 0xE8A, 0x0E6 Simulation Results 10 clock init signal applied address data bit 4 data bit 3 data bit 2 data bit 1 programming phase initialization phases update & retrieval phases 0 1 0 01 0 0 10 1 1 0 0 1 1 0

11. Example: 12-bit patterns, 8 address bits + 4 data bits  increased storage capacity if address patterns differ in about half the number of bit positions Memory Capacity d:Hamming distance between address bit patterns no restriction 1  d  7 2  d  6 3  d  5 d = 4 11

12. “Filter” bit patterns to ensure desired pattern distances  optimize overall storage capacity Outlook: Modular Memory Composition pattern filter 12