Download presentation
Presentation is loading. Please wait.
Published byYanti Rachman Modified over 5 years ago
1
3D Point Capsule Networks Lifting Capsule Networks to Raw 3D Data
Yongheng Zhao, Tolga Birdal, Haowen Deng, Federico Tombari CVPR Tutorial Sunday, June 16, 2019
2
representing 3d data octree graphs implicit surfaces
algebraic surfaces
3
NxD matrix of attributes
point clouds NxD matrix of attributes …... Raw data: Efficient Sparse: Memory friendly Generic Arbitrary accuracy x, y, z r,g,b
4
why are point clouds hard?
unstructured geometry cannot be projected on a single plane (different manifolds exist) basic representation is permutation dependent sparse input : dense convolutions are wasteful varying data density
5
consuming point clouds in networks: point-net
Nx3 point set X MLP local feature global feature Qi, Charles R., et al. "Pointnet: Deep learning on point sets for 3d classification and segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
6
point-net pointnet
7
a new 3D auto-encoder D E Input Shape Latent Capsules Latent Feature
Reconstruction
8
"Learn to Fold a Napkin into Almost Any 3D Shape, Deeply"
upsample vs deform latent code deform (MLP) concatanate Fixed Grid Template "Learn to Fold a Napkin into Almost Any 3D Shape, Deeply" Yang, Yaoqing, et al. "Foldingnet: Point cloud auto-encoder via deep grid deformation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
9
a quick look at the decoders
Groueix, Thibault, et al. "A papier-mâché approach to learning 3d surface generation." Proceedings of the IEEE conference on computer vision and pattern recognition Yang, Yaoqing, et al. "Foldingnet: Point cloud auto-encoder via deep grid deformation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
10
architecture
11
capsules act locally
12
Input point cloud with part label Part label for the capsule
optionally supervise Reconstruction Latent Capsules capsule-part association Input point cloud with part label Part label for the capsule
13
Part label for the capsule
optionally supervise Latent Capsules Part prediction Conv Cross Entropy Loss Part label for the capsule
14
part segmentation D Latent Capsules Part prediction Conv
15
part segmentation
16
part segmentation
17
a rather new application: part interpolation/replacement
Part replacement A Capsule-Part Association Latent Capsules E Target shape Latent Capsules E Tail Wing Body A Tail Wing Body A Part interpolation Segmentation D Segmentation D Source shape
18
part interpolation/replacement
19
part interpolation / replacement
21
extracting invariant 3D local descriptors
Deng, Haowen, Tolga Birdal, and Slobodan Ilic. "Ppfnet: Global context aware local features for robust 3d point matching." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
22
reconstruction quality
transfer learning part segmentation with limited data
23
https://tinyurl.com/yxq2tmv3
to take home... representation matters and is unsolved. a rich latent space is desirable and can (to a certain extent) be achieved by capsules + dynamic routing. can we make capsules specialize on other 3D shape properties?
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.