Synchronization of movements of a real humanoid robot with music Amstutz Patrick BIRG, EPFL

Goals of the project Study trajectories generation system Find an approach for adding sound capabilities for Webots videos Implement collision detection for visual feedback of drumming task Synchronize the robot with music: investigate possible solutions for extracting tempo information from music

System overview (1)

System overview (2)

Mathematical Aspect of CPG i Discrete Part: Rhythmic Part:

Matlab Simulation

Process schema

Tempo detection Matthew, Davies and Plumbley Aubio Library

Webots –Professional mobile robot simulation

Collision detection ODE Physic Plugin – Message to supervisor –Texture change –Sound info file

Matlab Music Generation Generation function: –generation(left, right, sound1, sound2, sound3) Parameters: –Two matrices of simulation information –Three sounds corresponding to each instrument Output: –Wav file

Merging Video and Sound VirtualDub –Xvid –Mp3 Script Generation Output avi files

References

Thank you Sarah Degallier, for your help Yvan Bourquin, for Webots help Alessandro Crespi, for VirtualDub help Pr. Auke Ijspeert for allowing me to perfom this project at BIRG