1. MindRACES, First Review Meeting, Lund, 11/01/2006 1 Anticipatory Behavior for Object Recognition and Robot Arm Control Modular and Hierarchical Systems, & Anticipatory Behavior and Control Department of Cognitive Psychology University of Würzburg, Germany Martin V. Butz, Oliver Herbort, Joachim Hoffmann, Andrea Kiesel

2. MindRACES, First Review Meeting, Lund, 11/01/2006 2 Related Publications DateJournal/con ference TitleAuthor 09/2005CogWiss 2005 Towards the Advantages of Hierarchical Anticipatory Behavioral Control Oliver Herbort, Martin V. Butz, & Joachim Hoffmann 11/2005AAAI Fall Symposium Towards an Adaptive Hierarchical Anticipatory Behavioral Control System Oliver Herbort, Martin V. Butz, & Joachim Hoffmann 09/2005In Book: Foundations of Learning Classifier Systems Computational Complexity of the XCS Classifier System Matin V. Butz, David E. Goldberg, & Pier Luca Lanzi (in press)Evolutionary Computation Journal (ECJ) Automated Global Structure Extraction For Effective Local Building Block Processing in XCS Matin V. Butz, Martin Pelikan, Xavier Llor à, & David E. Goldberg DateJournal/con ference TitleAuthor 11/2005IEEE Transactions on Evolutionary Computation Gradient Descent Methods in Learning Classifier Systems: Improving XCS Performance in Multistep Problems Martin V. Butz, David E. Goldberg, & Pier Luca Lanzi 07/2005GECCO 2005 (best paper nomination) Extracted Global Structure Makes Local Building Block Processing Effective in XCS Martin V. Butz, Martin Pelikan, Xavier Llora, David E. Goldberg 07/2005GECCO 2005 (best paper nomination) Kernel-based, Ellipsoidal Conditions in the Real- Valued XCS Classifier System Martin V. Butz 11/2005BookRule-based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design Martin V. Butz

3. MindRACES, First Review Meeting, Lund, 11/01/2006 3 Overview Anticipatory Behavioral Control Scenario involvement Modular systems Targeted system integrations  Learning of environment dynamics  Object recognition, symbol grounding  Hierarchical anticipatory arm control

4. MindRACES, First Review Meeting, Lund, 11/01/2006 4 Anticipatory Behavior Control (Hoffmann, 1993, 2003) effect A effect B action effect C situation Actions are selected, initiated and controlled by anticipating the desired sensory effects. Goal

5. MindRACES, First Review Meeting, Lund, 11/01/2006 5 The Big Challenge

6. MindRACES, First Review Meeting, Lund, 11/01/2006 6 Scenario Involvement Watching a scene, learning existence and behavior of objects (Scenario 2)  Continuous movement  Blocking of movement  Object permanence Control and manipulation of objects (Scenario 1)  Cognitive, anticipatory arm control  Interactive object manipulation Finding objects (Scenario 1)  Search of particular objects (with certain properties)  Search in room or house Behavior triggered by motivations (and possibly emotions) (Scenario 1)

7. MindRACES, First Review Meeting, Lund, 11/01/2006 7 Simple Object Recognition Scenario 2:  Watching a scene  Predicting object behavior / movement  Tracking multiple objects  Learning object permanence Scenario 1:  Manipulating objects (with robot arm or directly)  Anticipatory control with inverse models (IM)

8. MindRACES, First Review Meeting, Lund, 11/01/2006 8 Multiple Objects Scenario 1:  Searching objects  Searching objects of certain properties  Partial observability (fovea, multiple rooms)  Multiple motivations for multiple objects

9. MindRACES, First Review Meeting, Lund, 11/01/2006 9 Learning Modules XCS predictive modules  State prediction  RL prediction The ALCS framework  ACS2 & XACS  Predictive module  RL module AIS for rule-linkage (OFAI) Neural network modules  Hebbian-learning  LSTM units (IDSIA)  Rao-Ballard networks Kalman filtering techniques Context processing (LUCS)

10. MindRACES, First Review Meeting, Lund, 11/01/2006 10 Integration of Modules Learning environment dynamics  AIS-based sequences (OFAI)  Context information for sequences (LUCS)  Top-down, bottom-up (Kalman filtering-based) combination of information Combination with LSTM-based mechanisms (IDSIA)  For object permanence  Object location out of sight (fovea region)

11. MindRACES, First Review Meeting, Lund, 11/01/2006 11 A Hierarchical Control Model Body / Environment interneurons processing (visual, …) motorsignals proprioception IM desired effects IM descending signals exteroception hand coordinates joint angles muscle length muscle tension

12. MindRACES, First Review Meeting, Lund, 11/01/2006 12 IM motor torque joint angle arm configuration hand coordinates IM Current Cognitive Arm Model

13. MindRACES, First Review Meeting, Lund, 11/01/2006 13 Results: Arm IM

14. MindRACES, First Review Meeting, Lund, 11/01/2006 14 Summary Simple simulations  For object recognition  Object manipulation  Development of interactive control structures Modular system combinations  LSTM integration into XCS / ACS  Context processing integration into XCS / ACS  Integration of Kalman filtering techniques  Rule-linkage with AIS principles  Hierarchical combinations Anticipatory, developmental arm control models  Learning to control an arm  Learning the existence of objects  Object recognition  Object behavior  Object persistence