San Diego State University College of Engineering A Web-Based Mobile Robotic System for Control and Sensor Fusion Studies Christopher Paolini 1, Gerold Huber 2, Quentin Collier 3 and Gordon K. Lee 1 1 Dept of Elect &Comp Engr 5500 Campanile Drive San Diego State University San Diego, CA Management Center Innsbruck University of Applied Sciences Universitätsstraße Innsbruck, Austria 3 IUT de Bethune Networks & Telecomm Dept Bethune, France

San Diego State University College of Engineering Outline of Presentation:  The Mobile Robotic System Overview  The ANFIS Algorithm  Sensor Integration  Graphics User-Interface  Results  Conclusions and Future Work

San Diego State University College of Engineering Goal: Develop a mobile robotic testbed to investigate several control algorithms and sensor fusion techniques Approach: Use web-based video streaming and embedded control architecture for flexibility and robustness

San Diego State University College of Engineering

iRobot Create® Platform Single Board Computer (SBC) Linux Voyage Unibrain Fire-i™ Digital Camera Linksys Wireless-G PC adapter card iRobot Create® platform Single board computer (SBC) Voyage Linux (Debian) Unibrain Fire-i™ digital camera (IEEE 1394) Proxima g PCMCIA adapter card with external 5db gain antenna

San Diego State University College of Engineering iRobot Create® with Sensor Arrays

iRobot Create® based robot designed with several sensors Streaming video Web teleoperation ANFIS automation Experimental iRobot developed at San Diego State University Unibrain Fire-i™ Digital Camera g PCMCIA Transceiver 360° Ultrasonic Sensor Array Arduino Mega MCU 2.5dBi gain indoor omni- directional antenna Thermal Sensor Migrus C787 DCF-P single board computer with a 1.2GHz Eden ULV Processor IR Sensor 9DOF Inertial Measurement Unit

San Diego State University College of Engineering Sensors 3-Axis Rate Gyroscope3-Axis Accelerometer 3-Axis Magnetometer Inertial Measurement Unit Controller

San Diego State University College of Engineering Ultrasonic sensor array using an Arduino Mega 2560 microcontroller

San Diego State University College of Engineering MaxBotix LV-EZ1 Ultrasonic Sensor An array of 10 MaxBotix LV-EZ1 sensors suspended on two circular plates. Each MaxBotix sensor provides a 36 degree FOV. Ultrasonic Sensor Array

San Diego State University College of Engineering Thermal Sensor

How the iRobot Adjusts Its Heading  The iRobot changes its azimuth by sending a 16 bit signed value in the range [-2000, 2000] mm that defines a turning radius  The turning radius is a ray from the center of the turning circle to the center of the robot  r > 0  robot turns left  r < 0  robot turns right  Special cases: r = or (0x8000 or 0x7FFF) causes robot to move straight  r = 0xFFFF  robot turns in place clockwise  r = 0x0001  robot turns in place counter-clockwise r2 r1 Large radius  small curvature Small radius  large curvature

How the Web GUI Determines the Turning Radius  (x,y) In quadrant I and IV for  >5  In quadrant II and III for  <-5  III III IV Assume r varies linearly with 

Robot has an onboard 9DOF IMU Incorporates four sensors: LY530ALH single-axis gyro, LPR530AL dual-axis gyro, ADXL345 triple- axis accelerometer, and a HMC5843 triple-axis magnetometer Gives nine degrees of inertial measurement LY530ALH: STMicroelectronics ±300 °/s analog yaw-rate gyroscope HMC5843: Honeywell HMC5843, a 3-axis digital magnetometer outputs Euler X,Y,Z orientation vectors and roll (  ) and pitch (  ) angles (tilt sensor) with 12- bit ADC at 10 Hz MCU computes azimuth or “yaw” with an accuracy of 1-2 º Inertial Measurement Unit (IMU)

We want the absolute bearing defined by the remote user with the GUI to equal the absolute bearing reported by the IMU How to compensate for unknown system dynamics? Can define a neural network to model robot (plant) dynamics and use training data to tune network parameters Define a set of 4-tuple training data: which are the i th desired (from GUI) azimuth, azimuth rate, actual (from IMU) azimuth and azimuth rate, respectfully. How to Model Robot Dynamics while Turning? From the LY530ALH yaw rate sensor From the HMC5843 digital magnetometer

San Diego State University College of Engineering ADAPTIVE AJAX-BASED STREAMING VIDEO SYSTEM AJAX based Web interface for telerobotic control of the iRobot Create Encoding bit rate as a function of fps

Virtual Force Field Approach for Obstacle Avoidance

The Certainty Grid 2-Dimensional array of cells Each cell contains a Certainty Value (CV) CV indicates the measure of confidence that an object exist within a cell Instantaneous map for obstacle representation dx = dy = 15 cm 21 by 21 square cells represent the Certainty Grid Front

Method for Updating Certainty Values Each sensor corresponds to a particular angle Ө, based on its position on the sensor assembly At a given time, a sensor returns a distance d Eq. 1 and 2 transform (d, Ө) → (x’,y’) (1) (2)

Obstacle Avoidance for Path Planning Task Port Side Front

San Diego State University College of Engineering Cells Located on the Acoustic Axis

San Diego State University College of Engineering (a) Histogram Grid ; (b) Snapshot of Video Camera

San Diego State University College of Engineering If x is A i and y is B j, then The ANFIS Architecture

San Diego State University College of Engineering Forward pass: consequent parameters Off-line Training Backwards pass: premise parameters

San Diego State University College of Engineering Forward pass: consequent parameters On-line Learning Backwards pass: premise parameters

San Diego State University College of Engineering

San Diego State University College of Engineering Simulation Results

System Integration through an Arduino MCU  ANFIS controller is being implemented on an Arduino Mega that uses the magnetometer and accelerometer output for path tracking.  Initial simulation studies have been conducted using a MISO controller to test this proof of concept.  Then performed actual experimentation using one and two inputs to the ANFIS controller Multiple sensors: thermal (person/fixture differentiation), ultrasonic (collision avoidance and path planning), IR (automatic docking), video (telerobotic control), and magnetometer and accelerometer (orientation and position) have been tested and integrated into the overall system architecture.

Initial Web Browser Interface

San Diego State University College of Engineering

San Diego State University College of Engineering Experimental Results Bearing Scenario

San Diego State University College of Engineering Bearing Scenario

San Diego State University College of Engineering Constant Turn

San Diego State University College of Engineering Following Scenario

San Diego State University College of Engineering Learning Scenario

San Diego State University College of Engineering ANFIS Responding Time Response Time InputsMembership-functionsANFIS Computation Time

San Diego State University College of Engineering Conclusions and Future Work A MIMO ANFIS controller has been designed and tested through simulation and experimental studies The desired controller can adaptively adjusting to system variations through supervised and un-supervised learning. Future tasks include extending the MIMO design to a multiple inputs, two output structure and evaluate the performance of this MIMO implementation using Player/Stage and experimentation. We will add on-line learning functionality to our embedded MIMO ANFIS that will effectively tune the parameters computed from off-line training data.

San Diego State University College of Engineering

San Diego State University College of Engineering