Simultaneous Localization and Map Building System for Prototype Mars Rover CECS 398 Capstone Design I October 24, 2001

Project Mentors: Dr. Jeffrey Uhlmann Dr. Marjorie Skubic Project Members: Karen Casey Kenneth Estes Laura Heffernan

Overview Problem Definition Problem Definition Background Background Goals and Objectives Goals and Objectives

Overview Requirements Analysis Requirements Analysis System Components System Components Constraints Constraints Requirements Requirements Alternative Approaches Alternative Approaches Testing Methods Testing Methods Scheduling Scheduling

Background Covariance Intersection Research and Development Naval Research Laboratory Naval Research Laboratory Decentralized data fusion problems Decentralized data fusion problems Dynamic map building and localization Dynamic map building and localization NASA Mars Rover NASA Mars Rover Onboard data fusion system Onboard data fusion system Simultaneous Localization and Map Building System (SLAM) Simultaneous Localization and Map Building System (SLAM)

Background Covariance Intersection Research and Development Sojourner rover on Mars taken by the Mars Pathfinder Lander

Background Covariance Intersection Research and Development What is data fusion? Demands a method of combining information from multiple sources Demands a method of combining information from multiple sources Covariance Intersection (CI) consistently provides a conservatively fused estimate of the input.

Background Kalman Filter vs. CI Filter Kalman Filter Represents information about estimated or measured quantities in terms of a mean and a covariance matrix Represents information about estimated or measured quantities in terms of a mean and a covariance matrix Can combine estimates with a known degree of independence Can combine estimates with a known degree of independence

Background Kalman Filter vs. CI Filter CI Filter Does not make any assumptions about the degree of independence between information it fuses Does not make any assumptions about the degree of independence between information it fuses Exhibits considerably more stable behavior than Kalman filters Exhibits considerably more stable behavior than Kalman filters

Background Kalman Filter vs. CI Filter Kalman Filter: CI Filter: Equations for combining estimates {a,A} and {b,B} Equations only differ by the parameter ω.

Background Kalman Filter vs. CI Filter Principle advantage of CI: Permits filtering and data fusion without the need to know the degree of correlation between the estimates being fused Permits filtering and data fusion without the need to know the degree of correlation between the estimates being fused Applications include simultaneous map building and localization for autonomous vehicles.

Goals To prove that CI is a reliable solution to the SLAM problem To prove that CI is a reliable solution to the SLAM problem Use a test robot to identify predefined beacons Use a test robot to identify predefined beacons Given the information gathered, use CI to create a simultaneous and localization map Given the information gathered, use CI to create a simultaneous and localization map

Objectives Research and obtain a robot that will be able to move in two dimensions. We will also need to be able to get speed information from the robot in order to dependably update our map. Research and obtain a robot that will be able to move in two dimensions. We will also need to be able to get speed information from the robot in order to dependably update our map. Construct a small controlled test bed environment with identifiable beacons. Construct a small controlled test bed environment with identifiable beacons.

Objectives Develop software that will take the image and extract the relative beacon position we will need in order to build a map. Develop software that will take the image and extract the relative beacon position we will need in order to build a map. Develop software that will control the robot's movements and navigate it around its environment so it can map the beacons in the test bed. Develop software that will control the robot's movements and navigate it around its environment so it can map the beacons in the test bed. Use Covariance Intersection to estimate the beacon locations so that the relative map can be updated. Use Covariance Intersection to estimate the beacon locations so that the relative map can be updated.

System Components Palm Pilot Robot Palm Pilot Robot Handspring Visor Prism Handspring Visor Prism Eyemodule2 camera Eyemodule2 camera CI software CI software

System Components Palm Pilot Robot constructed from the Palm Pilot Robot Kit constructed from the Palm Pilot Robot Kit uses a Palm Pilot to move around and sense the nearby environment uses a Palm Pilot to move around and sense the nearby environment

System Components Handspring Visor Prism Used for: Robot motion control Robot motion control Object avoidance Object avoidance Image capture and processing system Image capture and processing system CI program CI program Localization Localization Map building Map building

System Components Eyemodule2 Camera Serves as integrated image capture device Serves as integrated image capture device Connects directly to Visor Prism Connects directly to Visor Prism Produces color images Produces color images

System Components CI Software Modular program will include: Robot movement and motor control Robot movement and motor control Image processing Image processing Coordinate triangulation Coordinate triangulation Covariance matrix calculations Covariance matrix calculations

Constraints Vision system Vision system Environment Environment Beacon size and height Beacon size and height Walls and floor Walls and floor Color recognition Color recognition Hardware interface connections Hardware interface connections Maintenance Maintenance

Constraints Vision System Environment: Beacons of same size and height Beacons of same size and height White walls White walls White level floor White level floor Color Recognition: Beacons of different colors for individual beacon identification with largest threshold difference between them Beacons of different colors for individual beacon identification with largest threshold difference between them Objective is to minimize noise in vision system.

Constraints Hardware Interface Connections Need serial connection between Visor and robot Need serial connection between Visor and robot Need communication between development software and Visor to download code Need communication between development software and Visor to download code

Constraints Maintenance Minimal maintenance: Connections and wiring Connections and wiring C program for Palm OS C program for Palm OS OS version OS version Visor model Visor model Camera model Camera model NASA testing and modifications NASA testing and modifications

Requirements Cost Requirements Cost Requirements Time: Time: Research the hardware/software components Research the hardware/software components Implement the robot and software. Implement the robot and software. Meet with project mentors. Meet with project mentors. Resources: Resources: Mentors will serve as our basic reference tool for background and implementation information. Mentors will serve as our basic reference tool for background and implementation information.

Requirements Mentors as a Resource Dr. Marjorie Skubic, Assistant Professor Specialties: Sensory perception Pattern Recognition Intelligent control Robotics

Requirements Mentors as a Resource Dr. Jeffery Uhlmann, Assistant Professor Specialties: Kalman Filtering Statistical Algorithms Autonomous Vehicles and Robotics Large Scale Simulation

Requirements Cost Requirements Cost Requirements Resources: Resources: Reference books (CodeWarrior) Reference books (CodeWarrior) Visor developer support for code to access the structure that contains pixel information. Visor developer support for code to access the structure that contains pixel information. Covariance Intersection software libraries Covariance Intersection software libraries Facilities: Facilities: EBW Lab 222 with Artemis. EBW Lab 222 with Artemis.

Requirements Cost Requirements Cost Requirements Money: Money: Purchase serial connection converter to handle communication needs between the robot and Visor. Purchase serial connection converter to handle communication needs between the robot and Visor. Materials for the testing environment. Materials for the testing environment. Performance Requirements Performance Requirements No required time limit for system response. No required time limit for system response.

Alternative Solutions for Finding Beacon Distance Infrared LEDs and sensors Infrared LEDs and sensors Pros Pros Inexpensive Inexpensive Have been proven to work Have been proven to work Cons Cons Need filters Need filters Occupies too much space Occupies too much space Adds complexity Adds complexity

Alternative Solutions for Finding Beacon Distance Sonar emitter and sensors Sonar emitter and sensors Pros Pros More information about the environment More information about the environment Cons Cons Too much information Too much information Adds complexity Adds complexity

Testing Navigation and Control Test limitations of robot control Test limitations of robot control Successful if the robot is able to move about the environment and move within visual range of the beacons for identification Successful if the robot is able to move about the environment and move within visual range of the beacons for identification

Testing Identification of Beacons Test the ability of our software to distinguish between the background and a beacon Test the ability of our software to distinguish between the background and a beacon Need to be able to identify the size of the beacon and the position of the center of the beacon Need to be able to identify the size of the beacon and the position of the center of the beacon

Testing Identification of Beacons Use trial and error to create a hash table associating the size of the beacon to the distance to the beacon Use trial and error to create a hash table associating the size of the beacon to the distance to the beacon Successful if the software can identify beacons, determine the distance to the beacons, and reduce the information to a specific XYZ coordinate for a beacon Successful if the software can identify beacons, determine the distance to the beacons, and reduce the information to a specific XYZ coordinate for a beacon

Testing CI Solution Hardware is being used in order to test our CI solution Hardware is being used in order to test our CI solution Successful when the robot is able to identify beacons and move around while maintaining an acceptably accurate map Successful when the robot is able to identify beacons and move around while maintaining an acceptably accurate map

Schedule

Conclusion Further refine scope of the project Further refine scope of the project Familiarize ourselves with the operation of the system components Familiarize ourselves with the operation of the system components Design the algorithm for image processing Design the algorithm for image processing Continue research into CI Continue research into CI