1. Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin

2. Outline  Introduction  Hardware  Project Requirements  Project Progress  Questions

3. Introduction Human Computer Interaction - HCI Detecting human faces Detecting interested people  Face detection  Face orientation and face rotation Keeping people interested by interacting with them Getting closer when a certain distance is exceeded

4. Hardware Robotino: a mobile robot platform from Festo To satisfy project's requirements, we are planning some extensions on Robotino: Cylindirical body, total height: 1.60m including camera A laptop computer, attached to body at 1.20m Second color camera on top of the laptop computer dedicated to HCI 9 infrared sensors 3 omnidirectional drive units Bump sensor Color camera Control Unit: a PC with a 300 Mhz Processor and 128 MB SDRam

5. Hardware con’t Sketch of hardware modifications on the robot base Robotino 1 additional laptop computer, 2 additional color cameras, skeleton with steel sticks and not yet sketched cylindirical cover for better appearance

6. Project Requirements A tour guide robot should: be able to navigate autonomously and purposefully inside the building operate in a dynamic environment attract visitors' attention by interacting with them be able to monitor its charge level and move to its charging dock on its own be able to dynamically arrange tours by combining selected destinations deliver information in forms of visual, audio and text messages be able to synchronize with the CMPE web site, in order to get up to date information.

7. Project Progress What we did?  Architecture  Pseudocode  Examples What we did not? What we encountered? What we achieved?

8. What we did? Combining Robotino API and OpenCV Modifying the open source code to satisfy our needs Running the modified algorithm on the mobile robot Testing the behavior

9. Architecture Robotino RobotinoAPI ApplicationOpenCV RobotinoAPI Watchdog Signal Action Requests Camera Requests Camera Data Application Action Requests Camera Data OpenCV Image Data Face Information Wireless

10. Pseudocode Connect to Robotino over Wireless While bumpsensorvalue = false do Retrieve camera image from RobotinoAPI Process image. OpenCV face detection using haarcascades, frontal and profile faces. Compute approximate distances of faces and their angle to the robot Decide action; rotate, move forward, attract attention, stop Act end While

11. Examples

12. Examples cont’d

13. What we did not? Pose estimation Tracking Face recognition Completion of hardware modifications

14. What we encountered? Major challenge : Erroneous detection of faces Processing power Minor problems :  Relatively low (10 fps) performance when frontal and profile face detection algorithms are combined  Position of the camera

15. What we achieved? Experiments held in laboratory Moving towards the human face until the distance is reduced to 1 meter approximately Turning with respect to the weighted average of areas of detected faces

16. Questions???

17. Thank you!