See3PO - A Visually Capable Path Finding Robot See3PO Frank Marino, Nick Wang, Jacky Yu, Hao Wu and Debarati Basu Department of Computer Science University of Massachusetts at Boston Tyler Garaas and Srikumar Ramalingam Mitsubishi Electric Research Laboratories (MERL)

See3PO - A Visually Capable Path Finding Robot See3PO –Our robot, equipped with two cameras, will use vision to navigate its environment Host –An application that runs on a Host PC, allowing the operator to set up and direct the robot. Overview

See3PO - A Visually Capable Path Finding Robot GUI Local Brain Floor Plan Processing Locator Host PathFind er Host PC Floor Plan Position Destination Path FP Image Floor Plan FP Image Destinatio n Position Floor Plan Robot Images Command s Robot Host

See3PO - A Visually Capable Path Finding Robot Host – An overview The Host class manages all Host-Side processes 1.The User selects a Floor Plan Image from the GUI 2.Host sends this image to FloorPlanProcessing, which returns a FloorPlan. 3.This FloorPlan is sent to PathFinder. It is also sent back to GUI for display.

See3PO - A Visually Capable Path Finding Robot Host – An overview 4.Either the User (GUI) or the Locator sets the robot’s current Position, then the User (GUI) selects a Destination. If using the Locator, the Host requests images from LocalBrain, and sends these to the Locator. 5.These points are sent to the PathFinder, which returns a Path of (X,Y) coordinates. 6.Host converts this Path to distinct Left/Right/Duration MoveComand objects, and sends them to the LocalBrain, one at a time.

See3PO - A Visually Capable Path Finding Robot FloorTile – represents one square foot on a floor. –Walkability - We can hold a range of values, but currently, we only use: Floor = 1 Wall = 0 –End/Start – booleans showing whether this is this tile is the start or endpoint of the graph. –Neighbors – the four neighboring FloorTiles FloorPlan –A graph of Floor Tiles –Each tile is connected to each of its four neighbors if that neighbor’s walkability is not zero. –Start/End – pointers to the start and end tiles Floor Plan Processing

See3PO - A Visually Capable Path Finding Robot The getPath() method takes a start point and a destination point and returns the shortest path between these. –Currently uses Dijkstra’s algorithm for shortest path in order to account for varying “Walkability” of tiles. –Returns a list of FloorTiles, which contain XY coordinates. Pathfinder

See3PO - A Visually Capable Path Finding Robot Science Building – 3F (image) Graph wall walkable s t s t Floor Plan Processing Generate Move Command Position Finder Confirm Position s t

See3PO - A Visually Capable Path Finding Robot GUI

See3PO - A Visually Capable Path Finding Robot GUI Live Panel Displays a live view from the Robot Status Screen Displays detailed messages from the Host Floor Panel The Main Interface: Displays The Floor Plan Allows the user to set: Scale Current Position Destination Displays the best Path During execution, updates the Path and Position. Connection Status Instruction s

See3PO - A Visually Capable Path Finding Robot Locator  Returns the current Position of the robot: Location(X,Y) Facing (0 -359)  This position is used by PathFinder to create a Path, and to monitor and correct the robot’s progress while moving.  The only external inputs required are: A Floor Plan Image Robot Images from the robot’s two cameras contd..

See3PO - A Visually Capable Path Finding Robot Locator Advantages:  Precise  Works Indoors  Self-correcting  Recognizes changes  open/closed door  obstacles

See3PO - A Visually Capable Path Finding Robot Testing  Unit tests – are already in place  System tests – test plans are being developed  System testing had been done partially

See3PO - A Visually Capable Path Finding Robot Robot Main parts of the robot Local Brain –Receive commands from Host –Send commands to ATOM and Servos –Send image back to Host ATOM –Control wheels Servos –Control cameras Cameras Wheels

See3PO - A Visually Capable Path Finding Robot Cameras Local Brain Servos Atom Wheels The Anatomy of a See3PO

See3PO - A Visually Capable Path Finding Robot Communication Protocol Command sent from Host to local brain (TCP) Command sent from local brain to ATOM (RS-232) Byte x010x100x11LWS_HLWS_LRWS_HRWS_LTM_HTM_L0xEF Byte x010x100x11 LWS_HLWS_LRWS_HRWS_L

See3PO - A Visually Capable Path Finding Robot What we need to know pulsout pin12, 8 –duration 8*0.5µs =4µs The longer the pulse, the faster the speed. Syntax pulsout pin, time pin: the I/O pin to use. time: the duration of the pulse in 0.5 microseconds increments. time

See3PO - A Visually Capable Path Finding Robot High Packet Loss Rate-ATOM Original Code main: “send speed to wheel every 20ms for i = 1 to 5 pulsout p12, rightSpeed pulsout p13, leftSpeed pause 20 next “read packet through COM port SERIN S_IN,i9600,… … goto main Why???? If the packet come in during the loop…

See3PO - A Visually Capable Path Finding Robot Our Solution New Code ONINTERRUPT TIMERAINT,drive ENABLE TIMERAINT main: “read packet in from COM port SERIN S_IN,i9600,… … goto main drive: pulsout p12, rightSpeed pulsout p13, leftSpeed resume What did we implement? Interrupted by a timer every 20ms

See3PO - A Visually Capable Path Finding Robot Robot cannot move straight Original Code leftSpeed = 3000 rightSpeed = 3000 main: gosub drive gosub receive_command goto main drive: if initialized = 1 then for i = 1 to 5 pulsout p12, rightSpeed pulsout p13, leftSpeed pause 20 next endif return

See3PO - A Visually Capable Path Finding Robot How do we solve that Send different speed to the wheels –Take the move forwards for example – leftspeed=3179 rightspeed=3150 Note: these data comes from experimental testing

See3PO - A Visually Capable Path Finding Robot Local Brain UI Debug mode –show commands Movement control –forward –left –stop –.. Setting left and right speed as well as driving time

See3PO - A Visually Capable Path Finding Robot What can we expect for version 1.0? A working servo system The robot can send pictures and communicate with the host from anywhere within wireless range. Based on these images, the Host can return a location. After giving a destination, the host can find the shortest path between the robot and any destination. Using this information, the robot will successfully navigate to the destination.

See3PO - A Visually Capable Path Finding Robot Possible extras Allow the user to change destination while the robot is moving. Allow optional manual control over the robot. Account for doors and obstacles. Use new wireless system (we use the wireless card, so it can just cover about 10 square meters) –Use local internet (like the UMass wireless) to make the robot move in the whole campus –Use other types of connections, like a tethered iPhone, extending the range further.

See3PO - A Visually Capable Path Finding Robot Questions? Please feel free to ask questions.