A Multi-Touch Display for Robotic Team Control Andrew Ng Stanford University Students: Morgan Quigley, Tony Ricciardi, Ashley Wellman.

How can a small number of human operators control a large swarm of robots?

Motivation rescue and search missions dangerous environments unmanned battlefield units hazardous materials

The Hardware

The Interface similar to real-time strategy video games incorporates multi-touch gestures

Touch Gestures two-finger zoom and pan path-drawing

Potential Problems correcting paths that go through obstacles coordinating paths for multiple robots avoiding information overload recognizing when a command is unsafe and should be ignored or modified

Paths Through Obstacles (portions at 2x speed)

Paths Through Obstacles uses a specialized cost function to find a valid path that captures the user’s intent

Coordinating Paths Current algorithm uses combination of heuristics. e.g., If there is a room nearby, go there and wait for other robot to pass. Other possible approaches: Modify cost function to take into account paths of nearby robots. Treat group of robots as one robot with many degrees of freedom, and find roadmap for composite robot.

Information Overload Can’t display all sensor data for all robots at once. Still want operator to have situational awareness.

Short-Term Goals Explore different path-coordination algorithms. Add context menu for accessing sensor data and issuing special commands. Get system working with iRobot hardware.

Long-Term Goals Scale up to environments with many robots. Support more advanced robot hardware. Extend gestures to use different parts of hand. Extend interface to support collaboration between multiple operators.