1. The MU Mites Robot Team Marjorie Skubic Derek Anderson Srikanth Kavirayani Mohammed Khalilia Benjamin Shelton Computational Intelligence Research Lab University of Missouri-Columbia

2. Outline A sketch interface for one robot (previous) A new sketch interface for controlling a robot team Robot design Robot formations Sketch understanding methods Concluding Remarks

3. A Sketch Interface for One Robot The user sketches a route map for the robot to follow.

4. A Sketch Interface for One Robot chair crate table files trash boxes cabinet desk And the robot follows the sketch. How can we use a sketch-based interface to control a team of robots? Guinness

5. A Sketch Interface for a Robot Team Sketch a robot formation

6. Robot Design Sensors: IR receiver for commands CMUcam (color) 3 front IR sensors 1 rear IR sensor Use knob setting for ID# Actuators: 2 motors for differential control with wheel encoders and a passive front wheel Servo motor for the camera

7. Robot Formations Follow the leader –Leader moves straight –Followers track & follow orange

8. Robot Formations Follow the leader in a circle –Leader moves clockwise or counter-clockwise as drawn –Followers track & follow orange

9. Robot Formations Follow the leader in a square –Leader moves clockwise or counter-clockwise as drawn –Followers move in squares of a different size

10. Robot Formations March side by side –Cameras aimed at the side –Track green & move forward

11. Sketch Pad Reference Frame User can set the reference frame Default: Assume initial robot direction is away from the user 1. Go forward 2. Turn right, go forward 3. Turn left, go forward 4. Turn around, go forward

12. Sketch Understanding Methods Symbol recognition via trained Hidden Markov Models – clockwise and counter-clockwise –Arrow –Line –Rectangle –Ellipse –Triangle From AAAI Fall Symposium 2004 paper (Anderson, Bailey & Skubic)

13. Sketch Understanding Methods HMM features –Local angle –Independent of orientation Global feature –Open vs. closed Filter out false alarms using local features Feature extraction Feature Distribution for Triangle (left) and Circle (right)

14. Sketch Understanding Methods Typical HMM Results Correctly recognized Correctly not recognized

15. Sketch Understanding Methods Squiggle for a robot icon –Displayed as a solid black circle Delete robot icon –2 short lines that intersect –Shown in green Move robot icon –Grab and drag to new location

16. PDA to Robot Commands Control commands –Forward –Reverse –Right turn –Left turn –Turn around –Stop Initialization –Reset ID –Set ID from sensors Formations –March 2 –March 3 –Follow 2-Line –Follow 3-Line –Left Circle –Right Circle –Left Square –Right Square

17. PDA-only Functions Clear the sketch Set reference frame Reference frame instruction Train IR commands Toggle IR on/off Toggle sound on/off

18. Challenges Recognizing sketched symbols –Complicated by multiple PDA platforms Interpreting a configuration of symbols Editing commands on the PDA IR communication from PDA to robot Defining a reference frame for directional commands Multi-robot formations based on local sensing only

19. Future Work Sketch understanding –Explore different features for improved recognition –Extend the recognizer to multi-stroke symbols –Explore hierarchical representations for symbol configurations Robot control –Scale up to support communication, synchronization, and negotiation between robots –Investigate more complex tasks Supported in part by the Naval Research Lab Publications at www.cecs.missouri.edu/~skubic/Paperswww.cecs.missouri.edu/~skubic/Papers

20. The Team Derek Srikanth Mohammed Ben