1. San Diego State University College of Engineering A Web-Based Mobile Robotic System for Control and Sensor Fusion Studies Christopher Paolini 1, Gerold Huber 2, Quentin Collier 3 and Gordon K. Lee 1 1 Dept of Elect &Comp Engr 5500 Campanile Drive San Diego State University San Diego, CA 92182 2 Management Center Innsbruck University of Applied Sciences Universitätsstraße 15 6020 Innsbruck, Austria 3 IUT de Bethune Networks & Telecomm Dept. 62408 Bethune, France

2. San Diego State University College of Engineering Outline of Presentation:  The Mobile Robotic System Overview  The ANFIS Algorithm  Sensor Integration  Graphics User-Interface  Results  Conclusions and Future Work

3. San Diego State University College of Engineering Goal: Develop a mobile robotic testbed to investigate several control algorithms and sensor fusion techniques Approach: Use web-based video streaming and embedded control architecture for flexibility and robustness

4. San Diego State University College of Engineering

5. iRobot Create® Platform Single Board Computer (SBC) Linux Voyage Unibrain Fire-i™ Digital Camera Linksys Wireless-G PC adapter card iRobot Create® platform Single board computer (SBC) Voyage Linux (Debian) Unibrain Fire-i™ digital camera (IEEE 1394) Proxima 802.11g PCMCIA adapter card with external 5db gain antenna

6. San Diego State University College of Engineering iRobot Create® with Sensor Arrays

7. iRobot Create® based robot designed with several sensors Streaming video Web teleoperation ANFIS automation Experimental iRobot developed at San Diego State University Unibrain Fire-i™ Digital Camera 802.11g PCMCIA Transceiver 360° Ultrasonic Sensor Array Arduino Mega MCU 2.5dBi gain indoor omni- directional antenna Thermal Sensor Migrus C787 DCF-P single board computer with a 1.2GHz Eden ULV Processor IR Sensor 9DOF Inertial Measurement Unit

8. San Diego State University College of Engineering Sensors 3-Axis Rate Gyroscope3-Axis Accelerometer 3-Axis Magnetometer Inertial Measurement Unit Controller

9. San Diego State University College of Engineering Ultrasonic sensor array using an Arduino Mega 2560 microcontroller

10. San Diego State University College of Engineering MaxBotix LV-EZ1 Ultrasonic Sensor An array of 10 MaxBotix LV-EZ1 sensors suspended on two circular plates. Each MaxBotix sensor provides a 36 degree FOV. Ultrasonic Sensor Array

11. San Diego State University College of Engineering Thermal Sensor

12. How the iRobot Adjusts Its Heading  The iRobot changes its azimuth by sending a 16 bit signed value in the range [-2000, 2000] mm that defines a turning radius  The turning radius is a ray from the center of the turning circle to the center of the robot  r > 0  robot turns left  r < 0  robot turns right  Special cases: r = 32768 or 32767 (0x8000 or 0x7FFF) causes robot to move straight  r = 0xFFFF  robot turns in place clockwise  r = 0x0001  robot turns in place counter-clockwise r2 r1 Large radius  small curvature Small radius  large curvature

13. How the Web GUI Determines the Turning Radius  (x,y) In quadrant I and IV for  >5  In quadrant II and III for  <-5  III III IV Assume r varies linearly with 

14. Robot has an onboard 9DOF IMU Incorporates four sensors: LY530ALH single-axis gyro, LPR530AL dual-axis gyro, ADXL345 triple- axis accelerometer, and a HMC5843 triple-axis magnetometer Gives nine degrees of inertial measurement LY530ALH: STMicroelectronics ±300 °/s analog yaw-rate gyroscope HMC5843: Honeywell HMC5843, a 3-axis digital magnetometer outputs Euler X,Y,Z orientation vectors and roll (  ) and pitch (  ) angles (tilt sensor) with 12- bit ADC at 10 Hz MCU computes azimuth or “yaw” with an accuracy of 1-2 º Inertial Measurement Unit (IMU)

15. We want the absolute bearing defined by the remote user with the GUI to equal the absolute bearing reported by the IMU How to compensate for unknown system dynamics? Can define a neural network to model robot (plant) dynamics and use training data to tune network parameters Define a set of 4-tuple training data: which are the i th desired (from GUI) azimuth, azimuth rate, actual (from IMU) azimuth and azimuth rate, respectfully. How to Model Robot Dynamics while Turning? From the LY530ALH yaw rate sensor From the HMC5843 digital magnetometer

16. San Diego State University College of Engineering ADAPTIVE AJAX-BASED STREAMING VIDEO SYSTEM AJAX based Web interface for telerobotic control of the iRobot Create Encoding bit rate as a function of fps

17. Virtual Force Field Approach for Obstacle Avoidance

18. The Certainty Grid 2-Dimensional array of cells Each cell contains a Certainty Value (CV) CV indicates the measure of confidence that an object exist within a cell Instantaneous map for obstacle representation dx = dy = 15 cm 21 by 21 square cells represent the Certainty Grid Front

19. Method for Updating Certainty Values Each sensor corresponds to a particular angle Ө, based on its position on the sensor assembly At a given time, a sensor returns a distance d Eq. 1 and 2 transform (d, Ө) → (x’,y’) (1) (2)

20. Obstacle Avoidance for Path Planning Task Port Side Front

21. San Diego State University College of Engineering Cells Located on the Acoustic Axis

22. San Diego State University College of Engineering (a) Histogram Grid ; (b) Snapshot of Video Camera

23. San Diego State University College of Engineering If x is A i and y is B j, then The ANFIS Architecture

24. San Diego State University College of Engineering Forward pass: consequent parameters Off-line Training Backwards pass: premise parameters

25. San Diego State University College of Engineering Forward pass: consequent parameters On-line Learning Backwards pass: premise parameters

26. San Diego State University College of Engineering

27. San Diego State University College of Engineering Simulation Results

28. System Integration through an Arduino MCU  ANFIS controller is being implemented on an Arduino Mega that uses the magnetometer and accelerometer output for path tracking.  Initial simulation studies have been conducted using a MISO controller to test this proof of concept.  Then performed actual experimentation using one and two inputs to the ANFIS controller Multiple sensors: thermal (person/fixture differentiation), ultrasonic (collision avoidance and path planning), IR (automatic docking), video (telerobotic control), and magnetometer and accelerometer (orientation and position) have been tested and integrated into the overall system architecture.

29. Initial Web Browser Interface

30. San Diego State University College of Engineering

31. San Diego State University College of Engineering Experimental Results Bearing Scenario

32. San Diego State University College of Engineering Bearing Scenario

33. San Diego State University College of Engineering Constant Turn

34. San Diego State University College of Engineering Following Scenario

35. San Diego State University College of Engineering Learning Scenario

36. San Diego State University College of Engineering ANFIS Responding Time Response Time InputsMembership-functionsANFIS Computation Time 123-416-18 37-820-23 49-1123-26 513-1526-29 617-1930-32 722-2336-38 2212-1426-28 330-3144-46 480-8294-96

37. San Diego State University College of Engineering Conclusions and Future Work A MIMO ANFIS controller has been designed and tested through simulation and experimental studies The desired controller can adaptively adjusting to system variations through supervised and un-supervised learning. Future tasks include extending the MIMO design to a multiple inputs, two output structure and evaluate the performance of this MIMO implementation using Player/Stage and experimentation. We will add on-line learning functionality to our embedded MIMO ANFIS that will effectively tune the parameters computed from off-line training data.

38. San Diego State University College of Engineering

39. San Diego State University College of Engineering