Associative Pattern Memory (APM) Larry Werth July 14, 2007

Introduction and Background of APM Human Associative Pattern Memory Computer Implemented APM Basis for Two Successful Startup Companies Six Patents Granted and Others Pending Successful Implementation of NKS

Objective of My Presentation Describe the APM Concept & Implementation Describe its Advantages / Features Identify Types of Applications Describe its Current Status and Future Goals

Origin of Concept Randomly Connected Neural Network Models States Sequence Terminates in a Cycle Randomly Map Each State to an Input Pattern Sampled Pattern Value & Current State Determine Next State The Ultimate Cycle Represents the Input Pattern Cycles Form the Basis of the APM

Cycle Properties Randomly Connected DFA’s Expected# Expected# Expected# Fraction Terminal Number Transition Terminal Total States (N) States(S) Cycles(C) States(T) States(F) 100 12 3 7 .12 1,000 40 4 20 .040 10,000 125 5 63 .0125 100,000 396 6 198 .00396 1,000,000 1,253 8 627 .001253 10,000,000 3,963 9 1982 .000396 100,000,000 12,533 10 6267 .0001253 1,000,000,000 39,632 11 19817 .0000396

Conceptual Implementation of APM Train Pattern (Write to Cycle Addresses) Input Pattern Array State Array Pattern Address Current State Address Next State Address Response Array Next State Array (Value = 0) Respond to Pattern (Read From Cycle Addresses) Pattern Value Next State Array (Value = 1) State Array: Filled with Random Pattern Addresses Next State Arrays : Filled with Random State Addresses Response Array: Assigned Responses to Patterns

Solution to Multiple Cycles Introduce a Refractory Period A State Can Not Occur Again Until After a Specified Number of Steps Establishes a Minimum Cycle Length Assures One Cycle Per Input Pattern Independent of Initial State Input Pattern is Represented By a Single Sequence of Random Addresses in Memory

Minimum Cycle Length Example Number of States: 1,000,000 Minimum Cycle Length: 3,700 Probability of a Second Cycle of 3,700 in Length: 1 in 1,000,000 Based on the probability of not picking one of 3700 in 1,000,000 after 3700 tries.

Response/Recognition Capacity During Training Desired Responses are Written to Cycle Addresses in Response Memory Problem: Response Memory Fills UP Quickly Any Cycle Address has Memory of Previous Input Sample Values Do Not Need to Use All Cycle Addresses Solution: Vertical Sensors

Vertical Sensor Cycle Detection Upper Memory Plane forms New Input Pattern Based on Sensor Status Vertical Sensors Detect Presence Absence of Cycle State Addresses Plane With Cycle

Vertical Sensor Implementation Number of States: 1,000,000 Minimum Cycle Length: 3,700 One of 270 Addresses are in Cycles Vertical Sensor Field Size: 135 Probability Field Contains Cycle Address: .5 Vertical Sensor Determines Bit Status of Hash Values that Addresses Response Memory

Fuzzy Hash Similar Input Patterns Produce Similar Cycles Similar Input Patterns Generate the same or Similar Hash Codes Multiple Independent Hash Codes are Generated By One Cycle (One Input Pattern) A Voting Mode For Response Identification Contributes to Fuzzy Recognition

Advantages of Using Cycles Creates a Fuzzy Hash Simple and Fast Implementation Common Language for Different Pattern Types Spatial and Temporal Integration to Form New Higher Level Input Patterns Automatic Segmentation of Time Varying Patterns

Applications Actual Applications: Hand Printed Character Recognition, Machine Vision, Video Compression, Financial Pattern Forecasting Signal Processing – Vector Quantization Video Surveillance – Smart Cameras Video Object Tracking Stereo Vision

Current Status and Objectives Software Library Written in C/C++ Objective: General Purpose Tool for Pattern Recognition Development Looking for a Business Partner Software Will be Available on Our Web Site www.netwerth.net