1. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 1/24 Outline Introduction Language Description Examples Liburbi Soft Devices Conclusion Outline Introduction Language Description Examples Liburbi Soft Devices Conclusion Jean-Christophe Baillie Electronics & Computer Engineering Dpt. Ecole Nationale Supérieure des Techniques Avancées Paris, France

2. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 2/24 URBI is an Interface Language to control robots Key features: 1.Simplicity: easy to understand, but with high level capabilities 2.Flexibility: independant of the robot, OS, plateform, interfaced with many languages (C++, Java, Matlab…) 3.Modularity: soft devices to extend the language 4.Parallelism: Parallel processing of commands, concurrent variable access policies, event based programming,… URBI is an Interface Language to control robots Key features: 1.Simplicity: easy to understand, but with high level capabilities 2.Flexibility: independant of the robot, OS, plateform, interfaced with many languages (C++, Java, Matlab…) 3.Modularity: soft devices to extend the language 4.Parallelism: Parallel processing of commands, concurrent variable access policies, event based programming,… Introduction URBI server URBI client(s) TCP/IP IPC Open-R …

3. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 3/24 Devices Every sensor, motor, camera or physical hardware in the robot is a device. camera grip wheelR legRF3 headPan micro A device is similar to a C++ object: it has methods and properties. The val property is related to the device value (telnet session, port 54000, with Aibo ERS7) headPan = 15; // or headPan.val = 15 headPan; [136901543:notag] 15.1030265089 accelX; [136901543:notag] 0.002938829104 (telnet session, port 54000, with Aibo ERS7) headPan = 15; // or headPan.val = 15 headPan; [136901543:notag] 15.1030265089 accelX; [136901543:notag] 0.002938829104 camera; [145879854:notag] BIN 5347 jpeg 208 160 ############# 5347 bytes ########### speaker = bin 54112 wav 2 16000 16; ####### 54112 bytes ##### camera; [145879854:notag] BIN 5347 jpeg 208 160 ############# 5347 bytes ########### speaker = bin 54112 wav 2 16000 16; ####### 54112 bytes #####

4. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 4/24 Messages headTilt; [136901543:notag] 15.1030265089 headTilt; [136901543:notag] 15.1030265089 Time stamp (ms)Command tagMessage content mytag: headTilt; [136901543:mytag] 15.1030265089 mytag: headTilt; [136901543:mytag] 15.1030265089 custom tag: useful to know who is sending what All messages from the server are composed with the same standard structure: the command to the server: the message from the server:

5. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 5/24 Advanced Tagging mytag: { commands...}; stop mytag; // stops the commands block mytag; // kills any new command with tag "mytag« unblock mytag; freeze mytag; // freeze any running or new command unfreeze mytag; mytag: { commands...}; stop mytag; // stops the commands block mytag; // kills any new command with tag "mytag« unblock mytag; freeze mytag; // freeze any running or new command unfreeze mytag; stop / block / freeze Flags: tag modifiers +error: command; // reports errors +begin: command; // reports beginning of the command +end: command; // reports ending of the command +error: command; // reports errors +begin: command; // reports beginning of the command +end: command; // reports ending of the command

6. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 6/24 Complex Assignments Numerical assignments can be specified via “modifiers” Simple assignment: headPan = -2; headPan = 15 time:5s; 5000ms -2 15 headPan = 15 speed:0.34; speed = 0.34 unit/s -2 15 headPan = 15 accel:0.02; accel = 0.02 unit/s² -2 15 headPan = -2 sin:1s ampli:3; 1000ms -2 6 units This command never terminates Any function can be assigned as time parameterized trajectory with the function modifier (v1.0): headPan = function(t):sqr(t)+sin(3*t+pi);

7. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 7/24 Time Operators Commands can be executed in serial or parallel mode: Set headPan to 15 and neck to 30 at the same time Set headPan to 15 and after, set neck to 30. AB A B AB A B A | B B.Start == A.end A & B B.Start == A.start A ; B B.Start >= A.end A, B B.Start >= A.start Operators, and ; are also available and have a semantics identical to & and | except that they have looser constraints: gap headPan = 15 & neck = 30; headPan = 15 | neck = 30; NB: Brackets can be used to group commands, like in C: { headPan = 15 | headTilt = 23 time:1000 } & neck = 10;

8. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 8/24 Variable Properties & Extensions Variables can be assigned several properties, using the -> indirection: myvar->rangemin = x; Stores the min range for the variable myvar myvar->rangemax = 145; device.val->speedmin = 24.4; Speedmin is the minimal speed value (units/s) device->speedmax = 112; Speedmax is a hard limit on the variable derivative. neck.val->unit = "deg"; Variable unit (for information only) neck.val->blend = normal; Variable blend mode (next slide) Stores the max range for the variable myvar neck.val->delta = 12; Variable tolerance used in adapative mode Variables evaluation can be specified using “extensions” : myvar’; device.x’’; First/Second order derivative, from assignments device.val’d; device.val’dd; neck’n; Normalized value (using rangemin/rangemax) neck’e; Assigned – measured error (approx. torque measure). First/Second order derivative of device.val, from sensors Variables in URBI are not simple containers like in C or similar languages: they are dynamic structures evolving over time. Variables in URBI are not simple containers like in C or similar languages: they are dynamic structures evolving over time.

9. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 9/24 Blending modes Conflicting assignments can occur from several clients or inside the same program. How to handle it? NB: this is also true for sound devices => simple multiplexer Blending modes neck->blend = add; Each assignment occurs at the same time and is added to the others => Used for walk sequences neck->blend = mix; Like add, but they are averaged instead of added neck->blend = queue; Each assignement occurs only when the others are finished neck->blend = discard; Each conflicting assignment is ignored neck->blend = cancel; Each new assignment terminate any other pending assignment neck->blend = normal; The latest assignment has the focus, be the others run in background

10. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 10/24 Device Grouping Devices can be grouped into virtual devices to form a hierarchie: group legLF {lefLF1, legLF2, legLF3}; group legs {legLF, legLH, legRF, legRH}; group legLF {lefLF1, legLF2, legLF3}; group legs {legLF, legLH, legRF, legRH}; legs legLFlegLHlegRFlegRH legLF1legLF2legLF3 Commands go down the hierarchy, allowing useful factorizations: legs.PGain = 0; // will affect the P Gain of all sub devices

11. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 11/24 C-like Features Function definition Functions can be defined on devices or virtual devices: Control structures: Standard control structures are available and some more specific to URBI: def robot.walk (x,y) { … /* walk code*/ }; def add (x,y) { return x+y }; def fibo(n) { if (n<2) return 1 else { a = fibo(n-1); b = fibo(n-2); return a+b } }; def robot.walk (x,y) { … /* walk code*/ }; def add (x,y) { return x+y }; def fibo(n) { if (n<2) return 1 else { a = fibo(n-1); b = fibo(n-2); return a+b } }; usage: robot.walk (14,255); myresult = fibo(10); robot.process_mystring("bonjour"); robot.walk (14,255); myresult = fibo(10); robot.process_mystring("bonjour"); no semicolon // the classical for loop for (i=0;i<10;i++) echo i; // soft tests : must be true for 3ms while (headsensor > 0) { instructions… } // loop 10 times loopn (10) legLF1 = legRF1; // Funny average calculation with for& avg = 0; avg->blend = mix; for& (i=0;i<10;i++) avg = tab[i]; // the classical for loop for (i=0;i<10;i++) echo i; // soft tests : must be true for 3ms while (headsensor > 0) { instructions… } // loop 10 times loopn (10) legLF1 = legRF1; // Funny average calculation with for& avg = 0; avg->blend = mix; for& (i=0;i<10;i++) avg = tab[i];

12. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 12/24 Event catching Several event catching mechanisms are available: at (test) {at (test) { instructionsA; instructionsA; };} onleave { instructionsB; }; at (test) {at (test) { instructionsA; instructionsA; };} onleave { instructionsB; }; A will be executed once when test becomes true. Then, as soon as test becomes false, B is executed and the server waits for test to be true again. test can be set with an hysteresis on the number of successful test checks or time. whenever (test) {whenever (test) { instructionsA; instructionsA; };} else { instructionsB; }; whenever (test) {whenever (test) { instructionsA; instructionsA; };} else { instructionsB; }; When test becomes true, A is executed. When A is finished, it is executed again if test is still true, otherwise, B is executed. waituntil (test); Terminates only when test becomes true. => If given a number, the wait command pauses for this nb of ms. usage: waituntil (test) | instructions…

13. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 13/24 Event catching (2) You can control the lifespan of a command: timeout (time) command; //example timeout(10s) robot.walk(); timeout (time) command; //example timeout(10s) robot.walk(); command will be executed until time is over. This command runs in the background. stopif (test) command; //example stopif(headSensor) legRF1 = legLF1; stopif (test) command; //example stopif(headSensor) legRF1 = legLF1; command will be executed until the test becomes true. This command runs in the background. freezeif (test) command; //example freezeif(!ball.visible) balltracking(); freezeif (test) command; //example freezeif(!ball.visible) balltracking(); command will be executed until the test becomes true, then it is freezed. When test becomes false again, it is unfreezed. Also runs in the background.

14. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 14/24 Event catching(2) You can emit your own events with the emit function: emit myevent; emit myevent (1,"hello"); emit myevent; emit myevent (1,"hello"); This creates an spiking event with or without parameters. at (myevent)... whenever (myevent (x,y)) echo x+y; at (myevent (1,s)) echo s;... at (myevent)... whenever (myevent (x,y)) echo x+y; at (myevent (1,s)) echo s;... You can catch events with a simple test. If there are parameters, you get them together with the event and you can filter on the base of those parameters value. emit(2s) myevent; emit() myevent (1,"hello"); emit(2s) myevent; emit() myevent (1,"hello"); This will add a duration to then event. Possibly, no time limit.

15. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 15/24 Advanced Features Many other advanced features: Interconnection commands, variables by name, aliases, load/save files, def checking, debugging options,… None of these are required to move a joint! joint.val = 30; URBI.INI is executed at start and usualy contains useful device groupings, aliases and an anim sequence. CLIENT.INI is executed when a new client connects to the robot and usualy contains a welcome message or a battery monitor. Important files: URBI.INI CLIENT.INI

16. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 16/24 Examples with Aibo // The ball tracking head program: whenever (ball.visible) { headPan = headPan + camera.xfov * ball.x & headTilt = headTilt + camera.yfov * ball.y }; // The ball tracking head program: whenever (ball.visible) { headPan = headPan + camera.xfov * ball.x & headTilt = headTilt + camera.yfov * ball.y }; // Stand up (leg1, leg2 and leg3 are virtual devices defined elsewhere) getup: { { leg2 = 90 time:2s & leg3 = 0 time:2s } | leg1 = 90 time:1s | leg2 = 10 time:1s | { leg1 = -10 time:2s & leg3 = 90 time:2s } }; // Stand up (leg1, leg2 and leg3 are virtual devices defined elsewhere) getup: { { leg2 = 90 time:2s & leg3 = 0 time:2s } | leg1 = 90 time:1s | leg2 = 10 time:1s | { leg1 = -10 time:2s & leg3 = 90 time:2s } }; // record and play a sound for 10s: sound = bin 0; // an empty buffer to store the sound timeout(10s) loop sound = sound + micro; // record the sound... speaker = sound; //...play it // record and play a sound for 10s: sound = bin 0; // an empty buffer to store the sound timeout(10s) loop sound = sound + micro; // record the sound... speaker = sound; //...play it

17. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 17/24 def robot.walk(duration) { echo "go for " + string(duration) + " secs"; direction = 1; if (duration <0) { duration = - duration, direction = -1 }; walk: timeout(duration) { for &(x=1;x<=2;x++) for &(y=1;y<=2;y++) for &(j=1;j<=3;j++) for &(d=1;d<=2;d++) robot.leg[x][y][j] = walk.mean[x][j] sin:walk.speed*walk.coef[d] ampli:walk.amp[x][j][d]*4 phase:direction*walk.phase[x][y][j][d]+pi*(direction-1)/2 } }; def robot.walk(duration) { echo "go for " + string(duration) + " secs"; direction = 1; if (duration <0) { duration = - duration, direction = -1 }; walk: timeout(duration) { for &(x=1;x<=2;x++) for &(y=1;y<=2;y++) for &(j=1;j<=3;j++) for &(d=1;d<=2;d++) robot.leg[x][y][j] = walk.mean[x][j] sin:walk.speed*walk.coef[d] ampli:walk.amp[x][j][d]*4 phase:direction*walk.phase[x][y][j][d]+pi*(direction-1)/2 } }; Walk Sequence Simple walk: Using an original walk from the robocup, we extracted the two main Fourier coefficients for each joints. With blend = add mode, these two components are added on each joint to reproduce the original periodic oscillation:

18. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 18/24 // Tracking state def tracking() { whenever (ball.visible) { headPan = headPan + camera.xfov * ball.x & headTilt = headTilt + camera.yfov * ball.y } }; // Searching state def searching() { period = 10s; { headPan’n = 0.5 smooth:1s & headTilt’n = 1 smooth:1s } | { headPan’n = 0.5 sin:period ampli:0.5 & headTilt’n = 0.5 cos:period ampli:0.5 } }; // Tracking state def tracking() { whenever (ball.visible) { headPan = headPan + camera.xfov * ball.x & headTilt = headTilt + camera.yfov * ball.y } }; // Searching state def searching() { period = 10s; { headPan’n = 0.5 smooth:1s & headTilt’n = 1 smooth:1s } | { headPan’n = 0.5 sin:period ampli:0.5 & headTilt’n = 0.5 cos:period ampli:0.5 } }; Behavior example This example shows how to write behavior graphs with URBI: Track ball Search ball ball.visible == false ball.visible == true speaker = found; speaker = lost; // Transitions track_transition: at (ball.visible ~ 400ms) { stop search; speaker = found; track: tracking(); }; search_transition: at (!ball.visible ~ 400ms) { stop track; speaker = lost; search: searching(); }; // Transitions track_transition: at (ball.visible ~ 400ms) { stop search; speaker = found; track: tracking(); }; search_transition: at (!ball.visible ~ 400ms) { stop track; speaker = lost; search: searching(); };

19. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 19/24 Telnet is of course too limited => liburbi for C++ programming. Exists also for JAVA, Matlab and OPENR (soon: Python, C) The URBI library (liburbi) give simple methods to send commands to and receive messages from the URBI server. Liburbi: quick review client->send(" headPan = %d", x); client->syncGetDevice("neck",&neckVal); // exists also for binary client->setCallback(&receiveMyImage,"imgtag"); client->send("loop imgtag:camera.val;"); UCallbackAction receiveMyImage(const UMessage &msg) { … /* handles the image contained in UMessage */ } UCallbackAction receiveMyImage(const UMessage &msg) { … /* handles the image contained in UMessage */ } UClient * client = new UClient("myrobot.ensta.fr"); C++ example: synchronous asynchronous

20. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 20/24 How to use URBI? URBI Server // C++ code with liburbi C++ main() { UClient * client = new UClient("myrobot.ensta.fr"); int pos; pos = complex_calculation(x,y); client->send(“neck.val = %d;”,pos); } C++ client URBI.INI onboard scripts URBI.INI // C++ code with liburbi OPEN-R main() { UClient * client = new UClient("localhost"); int pos; pos = complex_calculation(x,y); client->send(“neck.val = %d;”,pos); urbi::execute(); } Onboard client // Java code with liburbi Java import liburbi.UClient; robotC = new UClient(robotname); robotC.send("motoron;"); robotC.setCallback(image, "cam"); Java client headPan.val = 15; headPan.val; [136901543:notag] 15.1030265089... telnet or urbilab client other integrated clients (matlab, python,...) simple commands simple commands functions definition functions definition complex scripts complex scripts

21. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 21/24 Making reusable software modules URBI Server Compute some complex function, algorithm or signal processing task. Example: a ball tracking algorithm. Activity script: loop modulename:camera; Controlled by: block modulename; stop modulename; External client module Sensors or variable values Ex: camera.val Algorithm results as URBI variables Ex: ball.x, ball.y, ball.size Make use of the URBI variables constantly updated by the client module. client program or URBI script

22. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 22/24 Soft Devices The creation of “soft” devices extends the devices available in the robot. This is done by creating a subclass of USoftDevice. Soft devices can monitor variables, redirect function calls (C++/URBI binding) and events relative to them. Example: ball => ball.x, ball.y voice => voice.say(“hello”), voice.hear(x) URBI Server Extern Soft Device OffboardOffboard URBI Server Extern Soft Device Onboard

23. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 23/24 Soft Devices (2) URBI Server Extern Soft Device Proxy Server Proxy Kernel plugins URBI Server Integrated Soft Device Integrated With the URBI Module Library: the same C++ code for all integration possibilities

24. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 24/24 Short demo video: Demo

25. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 25/24 URBI Server for Aibo & liburbi are freely available for non commercial use at: www.urbiforge.com Next steps: Support more robots and develop partnership/contracts with robot manufacturers: Currently: Aibo, HRP-2, Webots5 (robot simulator, Cyberbotics), ENSTAR, Aldebaran Robotics, Pioneer robots. Next: Sony, Philips iCat? Increase the number of languages for the liburbi. Currently: C++, Java, Matlab, C++/OPENR. Next: Python, C, perl Continue to add important features in the kernel: client functions, debugging facilities, multi tagging, kernel 2 with multicore processors support and real-time scheduling. Develop URBI modules to enhance the language and stimulate the community (GPL) Develop useful associated software & tutorials: URBI Lab, URBI center, URBI Dev Conclusion

26. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 26/24 at (talk.finished == true) echo « Thank you for » + « your attention »; at (talk.finished == true) echo « Thank you for » + « your attention »;

27. URBI – Universal Robotic Body Interface Jean-Christophe Baillie Introduction to URBI 27/24 Performances TCP/IP Latency: 1.5ms ± 0.5 Bandwidth: ~500Ko/s OPEN-R Latency: 600 µs Bandwidth: ~2Mo/s Jpeg compress (90): 10ms for 208x160 3ms for 104x80 ~30 motor commands per second per motor 208x160 jpeg92 = 30fps Test on a wifi 802.11B wireless connections, with a URBI server for ERS7, a linux C++ client and tests with both the client and the server on the robot, and OPEN-R based connection. Good performances in general. For highly demanding low level action/perception loops, the best is to run some URBI Clients on the robot and leave the high level slow control architecture offboard. Performances