Kiyoshi Irie, Tomoaki Yoshida, and Masahiro Tomono 2011 IEEE International Conference on Robotics and Automation Shanghai International Conference Center May 9-13, 2011, Shanghai, China A High Dynamic Range Vision Approach to Outdoor Localization Andy

2 Intelligent Systems Lab. Problem setting 9am4pm9am4pm Auto exposureHDR Main goal: design algorithm for localization robust to changing in illumination conditions based on SIFT

3 Intelligent Systems Lab. HDR Keypoint set - differently exposed images ordered by shutter speed - sets of SIFT keypoints detected at corresponding images - all detected keypoints - duplicated keypoints - HDR keypoint set

4 Intelligent Systems Lab. Detecting duplicated keypoints 1.Only keypoints of neighboring images are compared together 2.Every keypoint in K j (j=0,…,n-1) is compared with every keypoint in K j+1 3.Keypoints with smallest euclidean distance of their feature vectors are selected and stored as M j 4.Check each matched pair in M j to satisfy the following condition: Essential matrix Rotation matrixTranslation matrix Computed from odometry

5 Intelligent Systems Lab. Detecting keypoints example Keypoints detected by SIFTMatched pairs of keypointsFalse matching removed by constraint

6 Intelligent Systems Lab. Localization Basic assumptions: - Robot assumed to navigate on flat 2D surface - 2D pose of robot defined as - odometry information is known. Errors in odometry assumed to have normal distribution. Localization is based on Monte-Carlo method

7 Intelligent Systems Lab. Localization Main task in robot localization: estimate robot state at time-step k, given knowledge about the initial step and all measurements Z k ={z 1,z 2,…,z k } In terms of probability we need to construct posterior density At each step we know all previous measurements z 1,…,z k-1 and control input u k-1 Typical localization process Update phase Prediction phase motion modelprevious state

8 Intelligent Systems Lab. Monte-Carlo Localization method Main idea: represent as a set of N random samples Prediction phaseUpdate phase Apply motion model to each particleWeight each sample taking measurement into account Resample according to weight

9 Intelligent Systems Lab. 1. In the prediction step for each particle a new set of particles is generated: 2. Set of HDR keypoints H t is computed 3. Particles are updated by weighting each particle using the likelihood of H t given map M and particles Localization For each particle i the HDR keypoint set that is close to particle position is choosen Detected keypoints are matched with keypoints in map forming a set of pairs: Particles are scored by counting the number of matched pairs. Relative camera pose is calculated using robot pose of and the pose of the particle at time t

10 Intelligent Systems Lab. Robot Resolution: 2448 x 2048 Framerate: 15 FPS Angles: 185° x 185°

11 Intelligent Systems Lab. Conclusions

12 Intelligent Systems Lab. Accuracy of localization

13 Intelligent Systems Lab. Computational time

14 Intelligent Systems Lab. Accuracy of localization

15 Intelligent Systems Lab. Position tracking results

16 Intelligent Systems Lab. Conclusions Advantages : Localization method using High Dynamic Range vision is proposed Localization based on Monte-Carlo method Using HDR images increases number of key-points HDR key-point set improves localization in terms of accuracy and computational cost Future work: How to determine optimal number of exposures? Shadows could effect matching

17 Intelligent Systems Lab. Images captured by robot in 3 different points