SPATIAL POOLER BY GIL SHOTAN

PRIMARY GOALS AND OBJECTIVES Move towards pure C++ implementation of CLA Easier to create bindings with other environments Boost performance Learn about the algorithm

SECONDARY GOALS AND OBJECTIVES In redesigning the spatial pooler, I had the following guiding principles in mind: Simplicity - Reduce barrier to Open Source Community Use terminology of CLA white paper Testability Maintenance of dual Python/C++ implementations Decouple “Grok-Specific” functionality into separate sub class called “Flat Spatial Pooler”

FILES $NUPIC/py/nupic/research/spatial_pooler.py $NUPIC/py/nupic/research/flat_spatial_pooler.py $NUPIC/nta/algorithms/spatial_pooler.[hc]pp $NUPIC/nta/algorithms/flat_spatial_pooler.[hc]pp $NUPIC/py/nupic/research/FDRCSpatial2.py << OLD

USAGE To switch between the implementations, change the value of the newly added model parameter “spatialImp”, which can be cound under “spParams”. model_params.py MODEL_PARAMS = { ‘spParams’ : { ‘spatialImp’ : “cpp” / “py”

BACKGROUND Basically, learning in the spatial pooler consists of changing the permanence values between columns and input bits Columns Input Bits Permanences

OLD ARCHITECTURE The following slides will present the main architecture changes in the spatial pooler module.

OLD ARCHITECTURE The old architecture, which was designed for vision, had permanence matrix for each column (which represented a receptive field for a v1 neuron in the brain), which was overlaid on top of a 2D input grid. Inputs Permanence Values For a Column (Receptive Field)

OLD ARCHITECTURE The permanence matrices of the columns were mapped onto the grid in such a way that certain parts of the image had overlapped. Inputs Permanence Values For a Column (Receptive Field)

OLD ARCHITECTURE In addition, some columns, mostly those columns mapped to the edge of the input region, had only a subset of the Values of the permanence matrix valid. Inputs

OLD ARCHITECTURE This leads to two additional data structures: one representing the subset of inputs in the input grid that are relevant for each column (red), and the other is the subset of the entries in the permanence matrix that are valid for the column (yellow) Inputs

OLD ARCHITECTURE Therefore even such a basic operation as accessing the permanence values for a column requires a lookup in 3 different places! Inputs

NEW ARCHITECTURE The new architecture tries to address this by simply maintaining one big matrix. The rows of the matrix represent cortical columns whereas the columns of the matrix represent input bits. Access is now fast and simple The new design is agnostic to any topological structure inherent in the data and allows the developer to implement any topology he or she wishes to with the modification of 1 method: mapPotential Illustration is on the next page

NEW ARCHITECTURE Inputs Columns Arbitrary Topology: Initialization + Neighbors(5) -> [3,7,8] numpy.reshape(-1) Permanence Matrix: The permanence between column 5 and input 7 is 0.2

NEW ARCHITECTURE There is an additional data structure called the potentialPool matrix which stores the ‘receptive field’ of a column.

NEW ARCHITECTURE Inputs Columns Potential Pool Matrix: column 3 is connected to inputs 6,7,8,9, whereas column 5 is connected to inputs bits 7,8,9,

TESTS Substantial unit tests for both C++ and python implementations Tests to ensure both implementations are identical Tests to ensure both implementations conform to the same public API $NUPIC/tests/unit/py2/nupic/research/flat_spatial_pooler_compatability_test.py $NUPIC/tests/unit/py2/nupic/research/flat_spatial_pooler_cpp_api_test.py $NUPIC/tests/unit/py2/nupic/research/flat_spatial_pooler_py_api_test.py $NUPIC/tests/unit/py2/nupic/research/flat_spatial_pooler_unit_test.py $NUPIC/tests/unit/py2/nupic/research/spatial_pooler_compatability_test.py $NUPIC/tests/unit/py2/nupic/research/spatital_pooler_cpp_api_test.py $NUPIC/tests/unit/py2/nupic/research/spatial_pooler_py_api_test.py $NUPIC/tests/unit/py2/nupic/research/spatial_pooler_unit_test.py $NUPIC/nta/algorithms/unittest/SpatialPoolerTest.[hc]pp

TESTS Running Hotgym with 3 different spatial poolers:

RESULTS Python Implementation Cleaner, commented, tested (100% coverage), readable code 2x Slower (Spatial Pooler alone) 2x Slower (Whole algorithm) 15% reduction in memory usage 15x Reduction in size on disk (Spatial Pooler alone) C++ Implementation Cleaner, commented, tested (100% coverage), readable code 10-20x Faster* (Spatial Pooler alone) 40% Faster* (Whole algorithm) 15% reduction in memory usage 15x Reduction in size on disk (Spatial Pooler alone) (*)doesn’t apply to anomaly models (which use a random spatial pooler)