Soft Competitive Learning without Fixed Network Dimensionality Jacob Chakareski and Sergey Makarov Rice University, Worcester Polytechnic Institute
Algorithms Neural Gas Competitive Hebbian Learning Neural Gas + Competitive Hebbian Learning Growing Neural Gas
Neural Gas Sorts the network units based on their distance from the input signal Adapts a certain number of units, based on this “rank order” The number of adapted units and the adaptation strength are decreased according to a fixed schedule
The algorithm Initialize a set A with N units c i Sort the network units Adapt the network units
Simulation Results
Competitive Hebbian Learning Usually not used on its own, but in conjunction with other methods It does not change reference vectors w j at all It only generates a number of neighborhood edges between the units of the network
The algorithm Initialize a set A with N units c i and the connection set C Determine units s 1 and s 2 Create a connection between s 1 and s 2
Simulation Results
Neural Gas + CHL A superposition of NG and CHL Sometimes denoted as “topology- representing networks” A local edge aging mechanism implemented to remove edges which are not valid anymore
The algorithm Set the age of the connection between s 1 and s 2 to zero (“refresh” the edge) Increment the age of all edges emanating from s 1 Remove edges with an age larger than the current age T(t)
Simulation Results
Growing Neural Gas Number of units changes (mostly increases) during the self-organization process Starting with very few units new units are added successively Local error measures are gathered to determine where to insert new units Each new unit is inserted near the unit with the largest accumulated error
The algorithm Add the squared distance between the input signal and the winner to a local error variable Adapt the winner and its neighbors If the number of input signals generated so far is a multiple integer of a parameter, insert a new unit :
Determine the unit with the max Err Determine the neighbor of q with the max Err Add a new unit r to the network Insert edges connecting r with q and f, and remove the original edge between q and f Decrease the error variables of q and f
Interpolate the error variable of r from q and f Decrease the error variables of all units
Simulation Results
Applications: Web/Database Maps