1. SLAM (Simultaneously Localization and Mapping)
Presenter : Jeongkyun Lee

2. Contents What is SLAM SfM-based SLAM Filter-based SLAM Comparison
Other SLAMs
Research topics

3. What is SLAM Simultaneously Localization and Mapping
Unknown Environment
Given only images
Unknown Pose

4. What is SLAM How to localize & map Structure-from-Motion based
Filtering based
Pay attention to :
Initialization
Measures ( Matching features )
Localization & Mapping

5. Fundamentals Geometry Projection matrix Calibration matrix
Rotation matrix
Translation vector
3D homogeneous vector
2D image point
Normalized point
Fundamental matrix
Essential matrix
where *
* Multiple View Geometry in Computer Vision, R. Hartley and A. Zisserman, Cambridge, University Press, 2000

6. Fundamentals 5-point algorithm1)
SfM-based SLAM
Fundamentals
5-point algorithm1)
Rotation matrix : 3 DoF (Rodrigues’ formula)
Translation vector : 3 DoF
Thus, is 5 DoF. ( – 1, 1 DoF for scaling factor )
Given 5 pairs of points on the image planes,
We can obtain .
PnP problem (Perspective-n-Point problem)
Given n 3D-to-2D point correspondences
We can obtain .
Grunert’s algorithm2) (P3P)
EPnP3)
Robust PnP4)
...
Known Environment
Unknown pose
Corresponding image points
1) D. Nister, An Efficient Solution to the Five-Point Relative Pose Problem, IEEE Conference on Computer Vision and Pattern Recognition, Volume 2, pp , 2003.
2) R. M. Haralick, C. N. Lee, K. Ottenberg and M. Nolle, Review and Analysis of Solutions of the Three Point Perspective Pose Estimation Problem, International Journal of Computer Vision, 1994.
3) F. Moreno-Noguer, V. Lepetit and P. Fua , Accurate Non-Iterative O(n) Solution to the PnP Problem, IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, October 2007.
4) S. Li, C. Xu, M. Xie, A Robust O(n) Solution to the Perspective-n-Point Problem, IEEE Transactions on Pattern Analysis and Machine Intelligence (2012) Volume 34, Issue 7, pp. 

7. SfM-based SLAM Visual Odometry1) … Feature Detection : Harris corners
Feature Matching : Normalized Corss Correlation (NCC)
Only matches between detected features within a fixed distance.
Procedure
5-point algorithm3),4) & RANSAC3)
Given 3 frames
P3P algorithm & RANSAC3)
Triangulation
Re-triangulation
using first & last observations
Given 1 frames …
P3P 5P
1) D. Nister, O. Naroditsky, J. Bergen, Visual odometry, Computer Vision and Pattern Recognition, July 2004.
2) D. Nister, Preemptive RANSAC for Live Structure and Motion Estimation, IEEE International Conference on Computer Vision, 2003.
3) D. Nister, An Efficient Solution to the Five-Point Relative Pose Problem, IEEE Conference on Computer Vision and Pattern Recognition, Volume 2, pp , 2003.
4) D. Nister, An Efficient Solution to the Five-Point Relative Pose Problem, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004.

8. SfM-based SLAM Real Time Localization and 3D Reconstruction1)
LBA2) (Levenberg-Marquardt algorithm (LM))
Minimize
where
Visual odometry
Local bundle adjustment (LBA) +
: Extrinsic parameters
: Projection matrix
: The square of Euclidean distance
: Estimated projection of point through the camera
: Observation
1) E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, P. Sayd, Real Time Localization and 3D Reconstruction, Computer Vision and Pattern Recognition, 2006.
2) B. Triggs, P. F. McLauchlan, R. I. Hartley & A. W. Fitzibbon, Bundle adjustment – A modern synthesis, in Vision Algorithms: Theory and Practice, LNCS, pp , Springer Verlag, 2000.

9. SfM-based SLAM Real Time Localization and 3D Reconstruction
n : number of optimized camera poses
N : number of cameras used for reprojection criterion minimization

10. SfM-based SLAM Real Time Localization and 3D Reconstruction
Key frame selection
Number of matched points
Uncertainty of camera pose ( Obtained from the hessian inverse )
Complexity :
Experiments
512 x 384 pixes, 75 fps, 94 key frames from a series of 445.

11. Measurements Acquisition
Filter-based SLAM
MonoSLAM1)
EKF-based
Filter initialization
Map management
( Generate & delete features )
Prediction
Measurements acquisition
Data association
Update
To easily explain….
Prediction
Measurements Acquisition
Update
1) A. J. Davison, I. D. Reid, N. D. Molton, O. Stasse, MonoSLAM: Real-Time Single Camera SLAM, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 6, June 2007.

12. Measurements Acquisition
Filter-based SLAM
MonoSLAM
Prediction
Measurements Acquisition
Update
State
Prediction
Dynamic System Model
(Constant Velocity Model)
: 3D position vector
: orientation quaternion
: linear velocity vector
: angular velocity vector
: landmark position vector

13. Measurements Acquisition
Filter-based SLAM
MonoSLAM
Prediction
Measurements Acquisition
Update
For
Matching the patch by NCC at
Max NCC value at > threshold
Measurement
Prediction of measurements
Find measurements
Active search1),2)
: a covariance matrix for the 2D position of i th landmark
1) A. J. Davison, Active Search for Real-Time Vision, International Conference Computer Vision, 2005.
2) M. Chli, A. J. Davison, Active Matching for Visual Tracking, Robotics and Autonomous Systems, 57(12): , 2009.

14. Measurements Acquisition
Filter-based SLAM
MonoSLAM
Prediction
Measurements Acquisition
Update
Update
: a Kalman gain at time t
1) A. J. Davison, Active Search for Real-Time Vision, International Conference Computer Vision, 2005.
2) M. Chli, A. J. Davison, Active Matching for Visual Tracking, Robotics and Autonomous Systems, 57(12): , 2009.

15. Filter-based SLAM MonoSLAM Initialization of features Data association
Delayed : SfM
Undelayed : Inverse depth parameterization1)
Data association
1-point RANSAC2)
Joint Compatibility Branch and Bound3) (JCBB)
Experiment
1.6GHz Pentium M processor
1) J. Civera, A. J. Davison, J. M. M. Montieal, Inverse Depth Parametrization for Monocular SLAM, IEEE Transactions on Robotics 24(5): , 2008.
2) J. Civera, O. G. Grasa, A. J. Davison, J. M. M. Montiel, 1-Point RANSAC for EKF Filtering. Application to Real-Time Structure from Motion and Visual Odometry , Journal of Field Robotics, 2010
3) J. Neira, J. D. Tardos, Data association in stochastic mapping using the joint compatibility test, IEEE Transactions on Robotics and Automation, 17(6): , Dec 2001.

16. Comparison SfM-based Filter-based Initialization 5-point algorithm
Delayed : SfM
Undelayed : Inverse depth parameterization
Measurement
NCC matching (from extracted feature points)
KLT tracker
Active search (prediction & templete matching)
Estimation technique
LBA (after p3p algorithm)
Kalman filtering (prediction & update)
Tracking 3~400 points in a frame
Working in real time within 100 landmarks.

17. Searching a large number (1000) of the re-projected features
Other SLAMs
PTAM (Parallel Tracking and Mapping)1)
Separate Tracking / Mapping
Redundancy : use only key frames.
Accuracy : available to optimization.
Tracking
Mapping
Pose estimate &
Map point projection
Searching a small number (50) of the coarsest-scale features by pyramid
Pose update
Searching a large number (1000) of the re-projected features
Final pose estimation
1) G. Klein, D. Murray, Parallel Tracking and Mapping for Small AR Workspaces, ACM International Symposium on Mixed and Augmented Reality, 2007.

18. Other SLAMs PTAM (Parallel Tracking and Mapping) Features Experiments
Matching : zero mean SSD
Key frame
> 20 frames from the last key frame.
Minimum distance away from the nearest key point.
Point initialization
Epipolar search
Data association refinement
Create new features in older keyframes.
Re-measure outlier measurements.
Experiments
Intel Core 2 Duo 2.66GHz, 600x480 pixels
6000 point, 150 keyframes.

19. Topics Filter-based… Divergence Error accumulation Data association
Beyond Spatial Pyramids: A New Feature Extraction Framework with Dense Spatial Sampling for Image Classification
Topics
Filter-based…
Divergence
Relocalization
Multiple model
Resilience
Other filtering techniques
Error accumulation
Loop-closing
Combining LBA, visual odometry
Data association
1-point RANSAC
ICNN, SCNN, JCBB
Dynamic environment
SLAMMOT (SLAM and Moving Object Tracking)
Multi-view, Sensor fusion
Application
Dense 3d reconstruction AR
Deblurring

20. Thank you!