Model-based learning: Theory and an application to sequence learning P.O. Box 49, 1525, Budapest, Hungary Zoltán Somogyvári and Péter Érdi Hungarian Academy of Science, Research Institute for Particle and Nuclear Physics, Department of Biophysics

Model-based learning: A new framework 1. Background 2. Theory 3. Algorithm 4. Application to sequence learning: 4.1 Evaluation of convergence speed 4.2 How to avoid sequence ambiguity 4.3 Storage of multiple sequences 5. Outlook

Model-based learning: Background I. Learning algorithms Supervised Unsupervised Learning by linking neurons with existing and fixed receptive fields. Hopfield-network Attractor network Symbol linking Symbol generation Learning by receptive field generation Topographical projection generation Ocular dominance formation Kohonen-map

Model-based learning: Background II. In many (if not all) symbol generator learning algorithms a built-in connection structure determines the formation of receptive fields.  Lateral inhibition in wide variety of learning algorithms.  `Mexican hat' lateral interaction in the topographic map formation algorithms and in ocular dominance generation.  Most explicitly in Kohonen's self-organizing map.

A symbol generator learning: Self-organizing maps Input layer, the `external world' Internal connection structure Connections between internal and external Learning: Modification of connections between neurons of external and internal layer. Changes in the receptive fields. A 2 dimensional grid of neurons Samples from an N dimensional vector space

Self-organizing maps II. Stages of the learning: the internal net stretches out to wrap the input space

Self-organizing maps II. The result of learning: the neural grid is fitted to the input space. The result of the learning is stored in the internal-external connections, in the locations of receptive fields. Each of the neurons, in the internal layer represents a part of external world. Map formation.

Model-based learning principle: Encounter between internal and external structures From this unusual point of view, it is an evident generalization, to extend the set of applicable internal connection structures, and using them as built-in models or scheme. In this way, the learning procedure is become an encounter between an internal model and the structures in the signals coming from the `external world.' The result of the learning is a correspondence between neurons of the internal layer and elements of input space.

Model-based learning: Internal models Any connection structure can be used to be fitted to the signal, and the same input can be represented many ways, even parallel. The models may represent different reference frames, hierarchical structures, periodical processions...

Model-based learning: Application to sequence learning One of the most important internal model structure type, is a linear chain of neurons connected with directed connections. A directed linear chain of neurons is able to represent a temporal sequence. The question: An instantaneous position in the state space. The answer: The prediction of the following state: Or even the prediction of the whole sequence: If the system is able to addresses, theoretically it can accessed to any of the following states in one step, or even to the preceding states.

Model-based learning: Basic algorithm L cells in a chain N dimensional input LN connections to modify Internal dynamics: Learn when internally activated: Decreasing learning rate:

Model-based learning: Simple sequence learning task Steps of learning without noise, from the random initial distribution of receptive fields. T=100, the number of iteration steps. During one iteration the whole sequence is presented, thus it requires NL weight modification. A Lissajous-curve applied as input. N=5, L=12 The final distribution of receptive fields

Model-based learning: Sequence with noise The same input with additive noise  The steps of learning. The result of the learning is very good, but (of course) less precise.

Model-based learning: Noise dependence of convergence Err Iterations The time evolution of error, in case of different noise amplitude.  =0.5  =0.3  =0.1  =0.05  =0

Noise dependence of speed Iterations  The required number of iterations to reach a given precision is slightly increases with the noise amplitude.

Sequence length dependence of speed The required number of iterations to reach a given precision does not depend on the length of the sequence. Length of sequence, L Iterations

Input dimension dependence of speed The required number of iterations to reach a given precision does not depend on the dimension of the input. Input dimension, N Iterations

Model based learning: evaluation of learning speed Since the algorithm does LN operations during an iteration, and the required number of iterations to reach a given precision does not depend either on the length of the sequence (L), either on the dimension of the input (N), the whole learning procedure works with O(LN) operations.

Model based learning avoids sequence ambiguity The task is to learn a self-crossing sequence. The sequence is noisy The result of the learning The usual way of solving the problem is the extension of state-space with the recent past of the system.

Model based learning avoids sequence ambiguity II. Two portion of the sequence are overlap. Of course sequence is noisy The result of the learning. This problem can be solved if the state-space become extended with the derivative of the signal.

Model based learning avoids sequence ambiguity III. Two portion of the sequence are overlap and their directions are the same. The noisy signal. This type of problem is hard to solve with traditional methods, because of the length of the overlapping parts are not known previously. The well-trained connections

Model-based learning: Multiple sequences Learning of multiple sequences needs:  A set of built-in neuron chains as models of sequences.  An organizer algorithm to conduct this orchestra. Different strategies can exist, but the most important functions of it: The initiation of a models' activity. The termination of them. To harmonize the predictions of different models with each other and with the external world.

Model-based learning: Outlook