ROBOT BEHAVIOUR CONTROL SUCCESSFUL TRIAL OF MARKERLESS MOTION CAPTURE TECHNOLOGY Student E.E. Shelomentsev Group 8Е00 Scientific supervisor Т.V. Alexandrova Language supervisor T.I.Butakova

Plan Introduction Methodology Markerless Motion Capture HAMMER architecture Results Conclusion

Current State of Robotics Industrial roboticsSocial robotics

What will we do? The main goals of our research: - to develop and try a new method of human motions recognizing - to create software for the robot which will build an appropriate model of the robot’s behavior with using the new method of human motions recognizing

Motion Capture Marker TechnologyMechanical Technology

Markerless Motion Capture RGB-D SensorHumanObtained Data

Hierarchical Attentive Multiple Models for Execution and Recognition (HAMMER) Purposes of use: To determine the intentions of the human To form the robot reactions to various actions HAMMER World State Inverse Models Forward Models Action Signals Confidence Evaluation Function

HAMMER architecture

Results Robot simulates the motions of the operator Robot teaches children to dance

Conclusion What have we done? Robot Reflex System Problem of motion recognizing Application of the Markerless Motion Capture technology Problem of robot reactions building Implementation of the HAMMER algorithm

References 1.S. Schaal, The New Robotics-towards human-centered machines, HFSP journal, vol. 1, no. 2, pp. 115–26, Y. Demiris, Prediction of intent in robotics and multi-agent systems, Cognitive processing, vol. 8, no. 3, pp. 151–158, Arnaud Ramey, Víctor González-Pacheco, Miguel A Salichs. Integration of a Low-Cost RGB-D Sensor in a Social Robot for Gesture Recognition. 6th international conference on Humanrobot interaction HRI 11, Miguel Sarabia, Raquel Ros, Yiannis Demiris. Towards an open-source social middleware for humanoid robots, 11th IEEE-RAS International Conference on Humanoid Robots, Y. Demiris and B. Khadhouri, Hierarchical Attentive Multiple Models for Execution and Recognition (HAMMER), Robotics and Autonomous Systems, vol. 54, no. 5, pp. 361–369, Abstraction in Recognition to Solve the Correspondence Problem for Robot Imitation, in Proc. of the Conf. Towards Autonomous Robotics Systems, 2004, pp. 63–70. 8.M. F. Martins and Y. Demiris, Learning multirobot joint action plans from simultaneous task execution demonstrations, in Proc. of the Intl. Conf. on Autonomous Agents and Multiagent Systems, vol. 1, 2010, pp. 931– S. Butler and Y. Demiris, Partial Observability During Predictions of the Opponent’s Movements in an RTS Game, in Proc. of the Conf. on Computational Intelligence and Games, 2010, pp. 46– A. Karniel, Three creatures named ‘forward model’, Neural Networks, vol. 15, no. 3, pp. 305–7, Y. Wu, Y. Demiris, Learning Dynamical Representations of Tools for Tool-Use Recognition, IEEE International Conference on Robotics and Biomimetics, 2011

ROBOT BEHAVIOUR CONTROL SUCCESSFUL TRIAL OF MARKERLESS MOTION CAPTURE TECHNOLOGY Student E.E. Shelomentsev Group 8Е00 Scientific supervisor Т.V. Alexandrova Language supervisor T.I.Butakova Mission Completed! Next research can be found here: