Presentation is loading. Please wait.

Presentation is loading. Please wait.

3D Point Capsule Networks Lifting Capsule Networks to Raw 3D Data

Published byYanti Rachman Modified over 5 years ago

Similar presentations

Presentation on theme: "3D Point Capsule Networks Lifting Capsule Networks to Raw 3D Data"— Presentation transcript:

1 3D Point Capsule Networks Lifting Capsule Networks to Raw 3D Data
Yongheng Zhao,  Tolga Birdal,  Haowen Deng,  Federico Tombari CVPR Tutorial Sunday, June 16, 2019

2 representing 3d data octree graphs implicit surfaces
algebraic surfaces

3 NxD matrix of attributes
point clouds  NxD matrix of attributes …... Raw data: Efficient Sparse: Memory friendly Generic Arbitrary accuracy x, y, z r,g,b

4 why are point clouds hard?
 unstructured geometry  cannot be projected on a single plane (different manifolds exist)  basic representation is permutation dependent  sparse input : dense convolutions are wasteful  varying data density

5 consuming point clouds in networks: point-net
Nx3 point set X MLP local feature global feature Qi, Charles R., et al. "Pointnet: Deep learning on point sets for 3d classification and segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

6 point-net pointnet

7 a new 3D auto-encoder D E Input Shape Latent Capsules Latent Feature
Reconstruction

8 "Learn to Fold a Napkin into Almost Any 3D Shape, Deeply"
upsample vs deform latent code deform (MLP) concatanate Fixed Grid Template "Learn to Fold a Napkin into Almost Any 3D Shape, Deeply" Yang, Yaoqing, et al. "Foldingnet: Point cloud auto-encoder via deep grid deformation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

9 a quick look at the decoders
Groueix, Thibault, et al. "A papier-mâché approach to learning 3d surface generation." Proceedings of the IEEE conference on computer vision and pattern recognition Yang, Yaoqing, et al. "Foldingnet: Point cloud auto-encoder via deep grid deformation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

10 architecture

11 capsules act locally

12 Input point cloud with part label Part label for the capsule
optionally supervise Reconstruction Latent Capsules capsule-part association Input point cloud with part label Part label for the capsule

13 Part label for the capsule
optionally supervise Latent Capsules Part prediction Conv Cross Entropy Loss Part label for the capsule

14 part segmentation D Latent Capsules Part prediction Conv

15 part segmentation

16 part segmentation

17 a rather new application: part interpolation/replacement
Part replacement A Capsule-Part Association Latent Capsules E Target shape Latent Capsules E Tail Wing Body A Tail Wing Body A Part interpolation Segmentation D Segmentation D Source shape

18 part interpolation/replacement

19 part interpolation / replacement

20

21 extracting invariant 3D local descriptors
Deng, Haowen, Tolga Birdal, and Slobodan Ilic. "Ppfnet: Global context aware local features for robust 3d point matching." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

22 reconstruction quality
transfer learning part segmentation with limited data

23 https://tinyurl.com/yxq2tmv3
to take home... representation matters and is unsolved. a rich latent space is desirable and can (to a certain extent) be achieved by capsules + dynamic routing. can we make capsules specialize on other 3D shape properties?

Download ppt "3D Point Capsule Networks Lifting Capsule Networks to Raw 3D Data"

Similar presentations

Semi-Supervised Hierarchical Models for 3D Human Pose Reconstruction Atul Kanaujia, CBIM, Rutgers Cristian Sminchisescu, TTI-C Dimitris Metaxas,CBIM, Rutgers.

Semi-Supervised Hierarchical Models for 3D Human Pose Reconstruction Atul Kanaujia, CBIM, Rutgers Cristian Sminchisescu, TTI-C Dimitris Metaxas,CBIM, Rutgers.
Intelligent Systems Lab. Recognizing Human actions from Still Images with Latent Poses Authors: Weilong Yang, Yang Wang, and Greg Mori Simon Fraser University,

Intelligent Systems Lab. Recognizing Human actions from Still Images with Latent Poses Authors: Weilong Yang, Yang Wang, and Greg Mori Simon Fraser University,
Fast and Extensible Building Modeling from Airborne LiDAR Data Qian-Yi Zhou Ulrich Neumann University of Southern California.

Fast and Extensible Building Modeling from Airborne LiDAR Data Qian-Yi Zhou Ulrich Neumann University of Southern California.
GATE-540 1 3D Object Representations (GATE-540) Dr.Çağatay ÜNDEĞER Instructor Middle East Technical University, GameTechnologies & General Manager SimBT.

GATE D Object Representations (GATE-540) Dr.Çağatay ÜNDEĞER Instructor Middle East Technical University, GameTechnologies & General Manager SimBT.
AN ANALYSIS OF SINGLE- LAYER NETWORKS IN UNSUPERVISED FEATURE LEARNING [1] Yani Chen 10/14/2014 1.

AN ANALYSIS OF SINGLE- LAYER NETWORKS IN UNSUPERVISED FEATURE LEARNING [1] Yani Chen 10/14/
Spatial Pyramid Pooling in Deep Convolutional

Spatial Pyramid Pooling in Deep Convolutional
Shape Matching for Model Alignment 3D Scan Matching and Registration, Part I ICCV 2005 Short Course Michael Kazhdan Johns Hopkins University.

Shape Matching for Model Alignment 3D Scan Matching and Registration, Part I ICCV 2005 Short Course Michael Kazhdan Johns Hopkins University.
ALIGNMENT OF 3D ARTICULATE SHAPES. Articulated registration Input: Two or more 3d point clouds (possibly with connectivity information) of an articulated.

ALIGNMENT OF 3D ARTICULATE SHAPES. Articulated registration Input: Two or more 3d point clouds (possibly with connectivity information) of an articulated.
Scale-less Dense Correspondences Tal Hassner The Open University of Israel ICCV’13 Tutorial on Dense Image Correspondences for Computer Vision.

Scale-less Dense Correspondences Tal Hassner The Open University of Israel ICCV’13 Tutorial on Dense Image Correspondences for Computer Vision.
Building high-level features using large-scale unsupervised learning Anh Nguyen, Bay-yuan Hsu CS290D – Data Mining (Spring 2014) University of California,

Building high-level features using large-scale unsupervised learning Anh Nguyen, Bay-yuan Hsu CS290D – Data Mining (Spring 2014) University of California,
Recognizing Deformable Shapes Salvador Ruiz Correa UW Ph.D. Graduate Researcher at Children’s Hospital.

Recognizing Deformable Shapes Salvador Ruiz Correa UW Ph.D. Graduate Researcher at Children’s Hospital.
Fully Convolutional Networks for Semantic Segmentation

Fully Convolutional Networks for Semantic Segmentation
Discussion of Pictorial Structures Pedro Felzenszwalb Daniel Huttenlocher Sicily Workshop September, 2006.

Discussion of Pictorial Structures Pedro Felzenszwalb Daniel Huttenlocher Sicily Workshop September, 2006.
Philipp Gysel ECE Department University of California, Davis

Philipp Gysel ECE Department University of California, Davis
3D ShapeNets: A Deep Representation for Volumetric Shapes Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, Jianxiong Xiao.

3D ShapeNets: A Deep Representation for Volumetric Shapes Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, Jianxiong Xiao.
Age-invariant Face Recognition

Age-invariant Face Recognition
Yann LeCun Other Methods and Applications of Deep Learning Yann Le Cun The Courant Institute of Mathematical Sciences New York University

Yann LeCun Other Methods and Applications of Deep Learning Yann Le Cun The Courant Institute of Mathematical Sciences New York University
Unsupervised Learning of Video Representations using LSTMs

Unsupervised Learning of Video Representations using LSTMs
Automatic Lung Cancer Diagnosis from CT Scans (Week 1)

Automatic Lung Cancer Diagnosis from CT Scans (Week 1)
Convolutional Neural Network

Convolutional Neural Network

Similar presentations

Ads by Google