1. Vision Reading Group, 2nd of May 2018 Martin Rünz
CodeSLAM — Learning a Compact, Optimisable Representation for Dense Visual SLAM
Vision Reading Group, 2nd of May 2018
Martin Rünz

2. An alternative representation to depth maps is presented
In a nutshell
An alternative representation to depth maps is presented

3. Background – Common Map Representations
Surfels
Mesh
Pose-Graph + Depth-Maps
TSDF

4. Ideas
Depth-maps are not random

5. Ideas
Depth-maps are not random (especially in man-made environments)

6. Ideas Depth-maps are not random
A compact representation – code – can be learned by encoder
Decoders are differentiable → code can be optimised (w.r.t. photometric loss...)
This is useful in SfM or SLAM scenarios
Reconstruct high-frequency details from color

7. Depth Reconstruction Depth-from-mono Depth-from-mono + code
Given this differentiable function, warping constraints can be used to optimise c and pose

8. Architecture Laplace distribution
Color feature extractor + uncertainty predictor
Laplace distribution
Only for training,
ground-truth
depth
Decoder:
CODE
Variational autoencoder, to increase smoothness of mapping between code and depth
Small changes in code → small changes in depth

9. Video

10. Inverse depth parametrization
average
Error behaves more Gaussian
original depth

11. Warping Both functions differentiable to inputs Transform → B 3D
Photometric error:
Both functions differentiable to inputs
Expensive (convolutions)
Pre-computed if decoder is linear!

12. Application SfM Cost function:
Initialise poses and code with zero-vector
Use residuals + Jacobians in Gauss-Newton style optimisation
Cost function:
Functionality of :
Mask invalid correspondences
Relative weighting
Huber weighting
Down-weight slanted surfaces
Down-weight occluded pixels

13. Experiments
SfM

14. Experiments
SfM

15. Experiments
SLAM

16. Experiments
Setups

17. Experiments
Setups

18. Experiments
Influence code entries

19. Video

20. Thanks for listening!