Ferdinando A. Mussa-Ivaldi, “Modular features of motor control and learning,” Current opinion in Neurobiology, Vol. 9, 1999, pp The focus on complex dynamical behaviors has been a feature of recent progress in the control of artificial systems. Advances have been recently brought about in neurobiology by dealing with the complexity of ‘real’ motor behaviors. Among these advances is: The formulation of new hypotheses on the building blocks, or “modules” that the central nervous system (CNS) may use to form useful representations for planning and control of actions. Internal Models

The acquisition of complex skills is facilitated by internal models of dynamics. To achieve its typical competence, the motor system must take advantage of experience for going beyond experience itself !!!!!!!!! Internal Model allow us to generate new behaviors and to handle situations that have not yet been encountered.

An Example: The task of balancing an inverted pendulum on the hand of a robotic arm:  The balancing task can be accomplished by humans without much effort => copying every movement made by a human expert (Imitation)

Imitation approach has been failed:  Even if the hand movements are carefully copied by the robot, the smallest amount of uncertainty associated with sensing and acting resulted in failure.  This does not mean that imitation of successful behavior is not a valid strategy.  The robot learned to act successfully when the data from the observed demonstrations were used not to mimic the expert’s movements but to build an internal representation of the dynamics associated with the task.  Imitation is a useful strategy when it provides the learner with a good starting point for the subsequent development of an internal model by a search procedure.

Multiple Internal Models Why a modular structure?  Many situations are derived from a combination of previously experienced contexts.  The use of modular system allows individual modules to participate in motor learning without affecting the motor behaviors already learned by other modules: Adapting a new module (Learning) & De-adaptation  Multiple internal models can be regarded conceptually as motor primitives or building blocks used to construct intricate motor behaviors with an enormous vocabulary!

Mixture of Experts (Jordan & Jacobs, 1991) X X X X + Expert 1 Expert n p1p1 pnpn Desired Movement (X D ) Desired Motor Command u1u1 unun p1p1 pnpn Gater Stochastic Decision Making Model 1 Model n - - Predicted Motor Command 1.Gater 2.Role of Forward model 3.Switching 4.Supervised learning Hierarchical structure (HME)

Multiple Paired Forward-Inverse Model

M. Kawato, D. Wolpert, “Internal models for motor control,” Sensory Guidance of Movement, John Wiley & Sons, 1998, pp Multiple Paired Forward-Inverse Model

D. Wolpert, M. Kawato, “Multiple paired forward and inverse models for motor control,” Neural Network, 11, 1998, pp

Feed-forward networks do not incorporate any dynamics in their structure Recurrent networks incorporate system dynamics in their structure They are capable of reach dynamics, and exhibiting memory and context sensitivity Past inputs and targets Neural Networks as Constructive Instruments for Internal Models

Standard Global Recurrent Network (GRN) GRN with embedded memory or order 2 Standard Locally Recurrent Network (LRN) LRN with embedded memory or order 2