Brain Regions Involved in USCBP Reaching Models A High Level Overview.

Slides:



Advertisements
Similar presentations
Giacomo Rizzolatti and Corrado Sinigaglia. Basic knowledge Mirror mechanism Unifies perception and action Its functional role depends on its anatomical.
Advertisements

Attention and neglect.
1 Motor Control Chris Rorden Ataxia Apraxia Motor Neurons Coordination and Timing.
Kinematic Synthesis of Robotic Manipulators from Task Descriptions June 2003 By: Tarek Sobh, Daniel Toundykov.
Biological Arm Motion through Reinforcement Learning by Jun Izawa, Toshiyuki Kondo, Koji Ito Presented by Helmut Hauser.
NATURE REVIEWS | NEUROSCIENCE SEP 01
Human (ERP and imaging) and monkey (cell recording) data together 1. Modality specific extrastriate cortex is modulated by attention (V4, IT, MT). 2. V1.
Cerebellar Spiking Engine: Towards Object Model Abstraction in Manipulation UGR with input from PAVIA and other partners  Motivation 1.Abstract corrective.
The Motor Cortex April 24 th 2008 Bijan Pesaran. Nature Reviews Neuroscience 5, (2004) Multiple motor systems PosturalVoluntary.
Cross-cortical Coherence during Effector Decision Making Chess Stetson Andersen Laboratory Caltech Sloan-Swartz Meeting 2009/07/28.
1.Exams due 9am 16 th. (grades due 10am 19 th ) 2.Describe the organization of visual signals in extra-striate visual cortex and the specialization of.
SA-1 Robotic Self-Perception and Body Scheme Learning Jürgen Sturm Christian Plagemann Wolfram Burgard University of Freiburg Germany.
Covert Attention Mariel Velez What is attention? Attention is the ability to select objects of interest from the surrounding environment Involuntary.
Attention Wolfe et al Ch 7, Werner & Chalupa Ch 75, 78.
Computational Analysis of Motor Learning. Three paradigms Force field adaptation Visuomotor transformations Sequence learning Does one term (motor learning)
Copyright © 2006 by Allyn and Bacon Chapter 8 The Sensorimotor System How You Do What You Do This multimedia product and its contents are protected under.
Quantifying Generalization from Trial-by-Trial Behavior in Reaching Movement Dan Liu Natural Computation Group Cognitive Science Department, UCSD March,
From Perception to Action And what’s in between?.
Physiology and Psychophysics of Eye Movements 1.Muscles and (cranial) nerves 2. Classes of eye movements/oculomotor behaviors 3. Saccadic Eye Movements,
Control of Movement. Patterns of Connections Made by Local Circuit Neurons in the Intermediate Zone of the Spinal Cord Gray Matter Long distance interneurons.
Spatial representation and coordinate frames in the brain.
Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme Neural Robot Control Cornelius Weber Hybrid Intelligent.
Motor System
A brain-machine interface instructed by direct intracortical microstimulation Joseph E. O’Doherty, Mikhail A. Lebedev, Timothy L. Hanson, Nathan A. Fitzsimmons.
PY202 Overview. Meta issue How do we internalise the world to enable recognition judgements to be made, visual thinking, and actions to be executed.
Sensory-motor integration Bijan Pesaran May 1 st, 2008.
Biological motor control Andrew Richardson McGovern Institute for Brain Research March 14, 2006.
From T. McMillen & P. Holmes, J. Math. Psych. 50: 30-57, MURI Center for Human and Robot Decision Dynamics, Sept 13, Phil Holmes, Jonathan.
Cortical motor structures. Hierarchical Organization of Motor System.
Neural circuits for bias and sensitivity in decision-making Jan Lauwereyns Associate Professor, Victoria University of Wellington, New Zealand Long-term.
Voluntary Movement II. Cortical representation of movements and parameters. Claude Ghez, M.D.
Adaptive, behaviorally gated, persistent encoding of task-relevant auditory information in ferret frontal cortex.
STUDY, MODEL & INTERFACE WITH MOTOR CORTEX Presented by - Waseem Khatri.
1 Computational Vision CSCI 363, Fall 2012 Lecture 31 Heading Models.
Sensorimotor systems Chapters 8.
T. Bajd, M. Mihelj, J. Lenarčič, A. Stanovnik, M. Munih, Robotics, Springer, 2010 ROBOT CONTROL T. Bajd and M. Mihelj.
CS344 : Introduction to Artificial Intelligence Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 26- Reinforcement Learning for Robots; Brain Evidence.
Robotica Lecture 3. 2 Robot Control Robot control is the mean by which the sensing and action of a robot are coordinated The infinitely many possible.
Optimal Therapy After Stroke: Insights from a Computational Model Cheol Han June 12, 2007.
REQUIRED READING: Kandel text, Chapters 33 & 38
Operant Conditioning of Cortical Activity E Fetz, 1969.
Chapter 16. Basal Ganglia Models for Autonomous Behavior Learning in Creating Brain-Like Intelligence, Sendhoff et al. Course: Robots Learning from Humans.
Chapter 50 The Prefrontal Cortex and Executive Brain Functions Copyright © 2014 Elsevier Inc. All rights reserved.
Pathways for Motor Control and Learning. Spinal Cord: The stretch reflex Maintain stability.
MSc CogNeuro Lecture 1-2: Intro + Primary Motor Cortex (MI)
The Reward Factor in the Control of Action: A Neurophysiological Theory Johan Lauwereyns Victoria University of Wellington, New Zealand.
Motor Control. Beyond babbling Three problems with motor babbling: –Random exploration is slow –Error-based learning algorithms are faster but error signals.
September 2, 2009 Kamini Krishnan Tandra Toon. Article Focus Review of literature that combines use of functional or structural MRI and microelectrode.
Introduction to the Motor Systems John H. Martin, Ph.D. Center for Neurobiology and Behavior.
Abstract This presentation questions the need for reinforcement learning and related paradigms from machine-learning, when trying to optimise the behavior.
Intro. ANN & Fuzzy Systems Lecture 14. MLP (VI): Model Selection.
Voluntary Movement I. Psychophysical principles & Neural control of reaching and grasping Claude Ghez, M.D.
Direct visuomotor transformations for reaching (Buneo et al.) 협동과정 뇌과학 김은영.
Goal Directed Reaching with the Motor Cortex Model Cheol Han Feb 20, 2007.
Motor learning through the combination of primitives. Mussa-Ivaldi & Bizzi Phil.Trans. R. Soc. Lond. B 355:
Bonaiuto, – November 2009 The Infant Learning Grasping and Affordances (ILGA) Model: Brief Overview and Progress.
2/11/20071 ACQ and the Basal Ganglia Jimmy Bonaiuto USC Brain Project 2/12/2007.
Neural Circuitry underlying generation of saccades and pursuit Lab # 1 - Catching a ball - What to expect/ think about.
Group 2 Youngjin Kang Anthony Correa Stephanie Regan.
Ali Ghadirzadeh, Atsuto Maki, Mårten Björkman Sept 28- Oct Hamburg Germany Presented by Jen-Fang Chang 1.
Cortical Control of Movement
The Motor Systems.
Using a Compound Gain Field to Compute a Reach Plan
Mapping Behavioral Repertoire onto the Cortex
Computational Mechanisms of Sensorimotor Control
Inner space: Reference frames
Posterior parietal cortex
Neural and Computational Mechanisms of Action Processing: Interaction between Visual and Motor Representations  Martin A. Giese, Giacomo Rizzolatti  Neuron 
Chapter 4 . Trajectory planning and Inverse kinematics
Richard A. Andersen, He Cui  Neuron 
Presentation transcript:

Brain Regions Involved in USCBP Reaching Models A High Level Overview

Brain Regions Cheol’s Models –Motor cortex (M1) –Spinal cord –Basal Ganglia (BG) –Dorsal Premotor (PMd, providing input) Jimmy’s Models –Parieto-occipital area (V6a) –Lateral intraparietal area (LIP) –BG –PMd (specifically F2)

Issues In Model Integration Unified View of M1 Interactions between PMd and M1 Role of the BG Involvement of the Cerebellum

M1 Modeling Cheol –Top-down model – directional tuning with supervised and unsupervised learning –Bottom-up model – input and output maps with controlling muscle synergies Jimmy –Robotic control model – trajectory generator, inverse kinematics, PD controllers (probably not all in M1)

Cheol’s Top-Down M1 Model Directional tuning of M1 neurons tuned using supervised learning and unsupervised learning Arm choice learned with reinforcement learning –Jimmy: Equivalent to noisy WTA based on executability –Cheol: connecting to unified view of motor learning

Possible motor procedures in the motor cortex Inverse dynamics and muscle models learned using temporal difference learning in an actor- critic architecture The actor may correspond to the motor cortex. Trajectory Generator Inverse Dynamics Inverse Muscle Model Motoneurons (spinal cord) Arm Evaluator Of Mvmt Joint static Level Planning Joint “force” Level Planning Muscle Level Planning ACTORCRITIC TD error

Cheol’s Bottom-up M1 model (based on feedback signal) Motor Cortex Model (map) Target location (premotor) Muscle Synergy ISM Feedback signal (premotor) IDM: mapping the error direction to muscle synergy (directly related to directional tuning) (with optimal feedback controller) Forward model Pesaran et al. (2006) indicated that PMd neurons encoded both target location and feedback signal.

ILGA Motor Controller Input - reach target in wrist-centered coordinates Dynamic Motor Primitives – generate reach trajectory Inverse Kinematics – pseudo-inverse of Jacobian matrix PD controllers – one for each DOF

Interactions Between PMd and M1 Our views of the role of PMd are very similar Jimmy –PMd (F2) provides M1 with target location in wrist-centered coordinates Cheol –Supra-motor-cortex coding in PMd may be feedback error (target location in hand- centered reference frame) and/or target location in the fixation point coordinates.

ILGA: F2 Integrates Bottom-Up and Top-Down Reach Target Signals Rostral F2 performs target selection based on parietal and prefrontal input Caudal F2 encodes selected target and initiates reach –F6 detects go signal and disinhibits via BG Tanne et al (1995)

Reconciliation with FARS view of PMd FARS implicated F2 in conditional action selection and F4 in reach target selection However many studies show F2 to contain directionally tuned neurons that discharge prior to reaching F4 contains bimodal (visual / somatosensory) neurons that respond when objects approach their somatosensory receptive field on the arm or hand

F2 vs. F4: Experimental Data Neurons in F2 are broadly tuned to multidimensional direction in a reaching task (Caminiti, 1991; Fu et al., 1993) Pesaran, Nelson & Andersen (2006) – PMd neurons encode relative positions of eye, hand, and target –PMd contains combined signals. –MIP contains more (target-eye) coding – fixation point coordinate F4 bimodal visual-tactile neurons have very large visual and somatosensory receptive fields and visual field is anchored to somatosensory field –But most don’t fire for stimuli farther than 25cm away (Graziano et al., 1997) - Not suitable for encoding reach target! –May be involved in feedback control of reach-grasp coordination – tactile RFs may contribute to transition from visual- to haptic-based control

Role of the BG Cheol –Adaptive critic in actor-critic architecture Jimmy –Adaptive critic gated by internal state –Action disinhibition Role in previous USCBP models –DA / DAJ – action disinhibition –ILGM – reward signal –Extended TD – adaptive critic –Bischoff BG model – next-state prediction

BG Disinhibition of Action ILGA’s use of the basal ganglia to disinhibit actions is largely consistent with its role in the Dominey-Arbib and Dominey-Arbib- Joseph Models The cortical target of context-dependent biases are different

BG as an Adaptive Critic The basal ganglia’s role as an adaptive critic is not very controversial However, each of our models uses it to learn different parameters –Cheol’s top-down model – to modify arm selection –Cheol’s bottom-up model – to learn inverse models –ILGA – kinematic parameters and contextual bias –ACQ – executability and internal state-dependent desirability Does this imply several actor/critic combinations (1:1, N:1, 1:N, N:N)? –Cheol’s top-down model – actor / critic –Cheol’s bottom-up model – actor / critic –ILGA – actor / critic –ACQ – actor / multiple critics

M1 & BG roles in Cheol’s unified view representation of the actuator Target related signal Critic (motor-task-related) Any motor actuators It represent the current maximum capability of the motor actuators. So, if the motor actuators are based on muscles, it will be the muscle synergies and the limitation of muscle-based actuator. If there is a stroke on it, the maximum capability is changed and the limitation of the world increases. If there is a rehabilitation, the maximum capability is changed again and the limitation decreases. The reinforcement learning framework will replace “optimization of a task- related cost function” with “maximization of a task-related reward function” which also accounts for actuators’ limitation The critic encodes the current task-related reward function. The reward or an action value is defined only when we have an “objective”. So, the critic will try to encode which action might be the best action in terms of reward (action value) to achieve a certain objective. It will monitor that the current movement’s performance. If the performance is changed, the critic will give the information of the next best action. And it will facilitate changing the actor accordingly. If there are multiple tasks, there should be multiple critics. What is now the critic’s role? It will encode the objective function and provide the “teaching” signal to the actor through TD error: if TD error is zero, we don’t need to change the actor, and so on. This arrow is the actor. TD error. Send limitation of the actuators via unsupervised learning Send limitation of the actuators via TD error Visual signal (world representation) Critic X (vision-task-related) Action-oriented perception ?

M1 & BG roles in Cheol’s unified view Critic actor Representation of the actuators Because of the stroke on a motor cortex, we have a change in limitation (performance change) of the corresponding actuator. The action choice module will encode which arm is better in a certain direction. So when the performance of the affected arm decreased, it will say that the best action is using the unaffected arm. (i.e. behavioral compensation). Can we connect these ideas with the words executability and desirability? In general, the objective function contains both concepts I think. Reaching module Grasping module Coordination manager Critic PLoS model Jool’s variability data In this coordination problem, we may have an objective of the coordination. As an example, we can weigh more on faster movement, or on the accurate movement, or accurate grasping. So based on the different objective, we may have variability in coordination. However, this coordination is not free from the actuators. First, if there is a signal dependent noise, we cannot have too fast movement. (This limitation is already in the Hoff-Arbib model). Second, too large initial aperture can assure the more accurate grasping but will give a limitation of the reaching module (slower reaching). Hierarchical Optimal Feedback Controller Todorov et al (2005) found a similar idea on hierarchical optimization of the plants. But the reinforcement learning framework will provide the more general framework of the motor system learning and may be more applicable Maybe separated obtaining of those two modules (early learning) Motor cortex model Kambara et al. (2008) showed the possibility and I also would implement it with map reorganization!

Involvement of the Cerebellum Schweighofer’s Modeling – corrects for nonlinearities in arm control Cheol – what about learning projections from cerebellum to M1?

References Caminiti, R., Johnson, P.B., Galli, C., Ferraina, S., Burnod, Y. (1991) Making Arm Movements within Different Parts of Space: The Premotor and Motor Cortical Representation of a Coordinate System for Reaching to Visual Targets. The Journal of Neuroscience, 11(5): Fu, Q.G, Suarez, J.I., Ebner, T.J. (1993) Neuronal Specification of Direction and Distance During Reaching Movements in the Superior Precentral Premotor Area and Primary Motor Cortex of Monkeys. Journal of Neurophysiology, 70(5): Graziano, M.S.A., Hu, X.T., Gross, C.G. (1997) Visuospatial Properties of Ventral Premotor Cortex. Journal of Neurophysiology, 77: Tanne, J., Boussaoud, D., Boyer-Zeller, N., Roiuller, E.M. (1995) Direct visual pathways for reaching movements in the macaque monkey. NeuroReport, 7: Pesaran, B., Nelson, MJ., Andersen, RA. (2006) Dorsal premotor neurons encode the relative position of the hand, eye, and goal during reach planning. Neuron 51, Buneo, CA., Jarvis, MR., Batista, AP., Andersen RA, (2002) Direct visuo-motor transformation for reaching, Nature 416, Todorov, E., Li, W., Pan X., (2005) From task parameters to motor synergies: A hierarchical framework for approximately optimal control of redundant manipulator, J Robot Syst. 22(11), Kambara, H., Kim, K., Shin, D., Sato, M., Koike, Y., (2006) Motor control-learning model for reaching movements, IJCNN2006