VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel.

Slides:



Advertisements
Similar presentations
Problem solving with graph search
Advertisements

Object Specific Compressed Sensing by minimizing a weighted L2-norm A. Mahalanobis.
Surface Simplification using Quadric Error Metrics Guowei Wu.
Globally Optimal Estimates for Geometric Reconstruction Problems Tom Gilat, Adi Lakritz Advanced Topics in Computer Vision Seminar Faculty of Mathematics.
CSci 6971: Image Registration Lecture 14 Distances and Least Squares March 2, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware Prof. Chuck Stewart,
Computer vision: models, learning and inference
Two-View Geometry CS Sastry and Yang
Mapping: Scaling Rotation Translation Warp
Presented by: Mingyuan Zhou Duke University, ECE April 3, 2009
Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes Speaker:James Chang.
Xianfeng Gu, Yaling Wang, Tony Chan, Paul Thompson, Shing-Tung Yau
Iterative closest point algorithms
Numerical geometry of non-rigid shapes
Direct Methods for Visual Scene Reconstruction Paper by Richard Szeliski & Sing Bing Kang Presented by Kristin Branson November 7, 2002.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
1 Numerical geometry of non-rigid shapes Spectral Methods Tutorial. Spectral Methods Tutorial 6 © Maks Ovsjanikov tosca.cs.technion.ac.il/book Numerical.
1 Numerical geometry of non-rigid shapes Lecture II – Numerical Tools Numerical geometry of shapes Lecture II – Numerical Tools non-rigid Alex Bronstein.
Visualization and graphics research group CIPIC January 30, 2003Multiresolution (ECS 289L) - Winter MAPS – Multiresolution Adaptive Parameterization.
3D reconstruction of cameras and structure x i = PX i x’ i = P’X i.
Fitting Lines to Data with Outliers and Errors in the Variables Nahum KiryatiFreddy Bruckstein TAU Technion © Copyrighted material. All rights reserved.
Numerical geometry of non-rigid shapes
1 Bronstein, Bronstein, and Kimmel Joint extrinsic and intrinsic similarity of non-rigid shapes Rock, paper, and scissors Joint extrinsic and intrinsic.
1 Numerical geometry of non-rigid shapes A journey to non-rigid world objects Numerical methods non-rigid Alexander Bronstein Michael Bronstein Numerical.
Non-Euclidean Embedding
Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane.
Flattening via Multi- Dimensional Scaling Ron Kimmel Computer Science Department Geometric Image Processing Lab Technion-Israel.
A Global Geometric Framework for Nonlinear Dimensionality Reduction Joshua B. Tenenbaum, Vin de Silva, John C. Langford Presented by Napat Triroj.
1 Bronstein 2 & Kimmel An isometric model for facial animation and beyond AMDO, Puerto de Andratx, 2006 An isometric model for facial animation and beyond.
Atul Singh Junior Undergraduate CSE, IIT Kanpur.  Dimension reduction is a technique which is used to represent a high dimensional data in a more compact.
CSE473/573 – Stereo Correspondence
1 Numerical geometry of non-rigid shapes Non-Euclidean Embedding Non-Euclidean Embedding Lecture 6 © Alexander & Michael Bronstein tosca.cs.technion.ac.il/book.
Camera parameters Extrinisic parameters define location and orientation of camera reference frame with respect to world frame Intrinsic parameters define.
NonLinear Dimensionality Reduction or Unfolding Manifolds Tennenbaum|Silva|Langford [Isomap] Roweis|Saul [Locally Linear Embedding] Presented by Vikas.
1 M. Bronstein Multigrid multidimensional scaling Multigrid Multidimensional Scaling Michael M. Bronstein Department of Computer Science Technion – Israel.
Nonlinear Dimensionality Reduction Approaches. Dimensionality Reduction The goal: The meaningful low-dimensional structures hidden in their high-dimensional.
CSC 589 Lecture 22 Image Alignment and least square methods Bei Xiao American University April 13.
Manifold learning: Locally Linear Embedding Jieping Ye Department of Computer Science and Engineering Arizona State University
CSE554Laplacian DeformationSlide 1 CSE 554 Lecture 8: Laplacian Deformation Fall 2012.
3D Fingertip and Palm Tracking in Depth Image Sequences
1 PDE Methods are Not Necessarily Level Set Methods Allen Tannenbaum Georgia Institute of Technology Emory University.
1 Numerical geometry of non-rigid shapes Shortest path problems Shortest path problems Lecture 2 © Alexander & Michael Bronstein tosca.cs.technion.ac.il/book.
Epipolar geometry The fundamental matrix and the tensor
Course 12 Calibration. 1.Introduction In theoretic discussions, we have assumed: Camera is located at the origin of coordinate system of scene.
Expression-invariant Face Recognition using Geodesic Distance Isometries Kerry Widder A Review of ‘Robust expression-invariant face recognition from partially.
CSIE Dept., National Taiwan Univ., Taiwan
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO. 2, FEBRUARY Leonardo De-Maeztu, Arantxa Villanueva, Member, IEEE, and.
Metrology 1.Perspective distortion. 2.Depth is lost.
Computer Graphics Some slides courtesy of Pierre Alliez and Craig Gotsman Texture mapping and parameterization.
Medical Image Analysis Image Registration Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.
Computer Vision Lab. SNU Young Ki Baik Nonlinear Dimensionality Reduction Approach (ISOMAP, LLE)
Vector Norms and the related Matrix Norms. Properties of a Vector Norm: Euclidean Vector Norm: Riemannian metric:
Manifold learning: MDS and Isomap
Jan Kamenický.  Many features ⇒ many dimensions  Dimensionality reduction ◦ Feature extraction (useful representation) ◦ Classification ◦ Visualization.
A Flexible New Technique for Camera Calibration Zhengyou Zhang Sung Huh CSPS 643 Individual Presentation 1 February 25,
CENG 789 – Digital Geometry Processing 04- Distances, Descriptors and Sampling on Meshes Asst. Prof. Yusuf Sahillioğlu Computer Eng. Dept,, Turkey.
Fast Marching Algorithm & Minimal Paths Vida Movahedi Elder Lab, February 2010.
An Area-Preserving Cortical Flattening Method Yuefeng Lu 1, Moo K. Chung 1,2,3,, Steve Robbins 4, Alan C. Evans 4 1 Department of Statistics, 2 W.M. Keck.
1 Teaching Innovation - Entrepreneurial - Global The Centre for Technology enabled Teaching & Learning, N Y S S, India DTEL DTEL (Department for Technology.
Camera Calibration Course web page: vision.cis.udel.edu/cv March 24, 2003  Lecture 17.
Lecture 16: Image alignment
Spectral Methods for Dimensionality
CENG 789 – Digital Geometry Processing 08- Rigid-Body Alignment
Multiplicative updates for L1-regularized regression
CS4670 / 5670: Computer Vision Kavita Bala Lec 27: Stereo.
Depth from disparity (x´,y´)=(x+D(x,y), y)
Morphing and Shape Processing
Spectral Methods Tutorial 6 1 © Maks Ovsjanikov
Outline Nonlinear Dimension Reduction Brief introduction Isomap LLE
Estimating 2-view relationships
Presentation transcript:

VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel.

SURFACE FLATTENING Some distortion is usually unavoidable Need to define the most important property to preserve (area, angles,...) Use cuts (interruptions) to selectively reduce distortion Transform a surface from 3D space to 2D plane. Peters Projection: area accuarteCuts reduce distortion

Applications: Medical Cortical surface flattening for visualization, registration, etc. Virtual colonoscopy Gray-white matter interface Courtesy of Gabriele Lohmann, MPG Leipzig

Applications: Computer Graphics Flatten the surface Map the 2D pattern onto the flattened surface Map the flattened surface back to 3D Texture Mapping From Texture Mapping: Projecting a planar pattern onto a 3D surface

Some previous works E.L. Schwartz, A. Shaw and E. Wolfson, “A Numerical Solution to the Generalized Mapmaker’s Problem: Flattening Nonconvex Polyhedral Surfaces”, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 11, pp , C. Bennis, J.M. Vezien and G. Iglesias, “Piecewise Surface Flattening for Non-Distorted Texture Mapping”, Computer Graphics, Vol. 25, pp , H.A. Drury, D.C. Van Essen, C.A. Anderson, C.W. Lee, T.A. Coogan and J.W. Lewis, “Computerized Mappings of the Cerebral Cortex: A Multiresolution Flattening Method and Surface-Based Coordinate System”, J. of Cognitive Neuroscience, Vol. 8, pp. 1-28, S. Angenent, S. Haker, A. Tanenbaum and R. Kikinis, “Conformal Geometry and Brain Flattening”, Proc. 2nd Intl. Conf. on Medical Image Computing and Computer Assisted Intervention (MICCAI), Cambridge, England, pp , A.M. Dale, B. Fischl and M.I. Sereno, “Cortical Surface-Based Analysis. II: Inflation, Flattening and a Surface-Based Coordinate System”, NeuroImage, Vol. 9, pp , S. Haker, S. Angenent, A. Tanenbaum, R. Kikinis, G. Sapiro and M. Halle, “Conformal Surface Parameterization for Texture Mapping”, IEEE Trans. on Visualization and Computer Graphics, Vol. 6, No. 2, 2000.

New Method: Unique Features Voxel data From Operates on voxel data: Triangulation not necessary Optimal (global preservation of geodesic distances) Solid mathematical basis. Analytic solution (in principle) Computationally efficient

Our Approach to Surface Flattening Mark N (very many) points on the 3D surface. Estimate the geodesic distances between each point and all others (how?) Optimally map the N points onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding 3D geodesic distances (how?) Regard the N points as control points, and map the rest by interpolation.

Crucial problem #1: Represent the surface as a weighted graph: The weight of an edge depends on the link type Minimal geodesic distance ~ shortest path in graph Graph is sparse: very fast algorithm Kiryati & Szekely, 1993: Estimate the minimal (geodesic) distance between points on a 3D surface. Accurate and efficient solution for surface represented by voxels. direct, minor diagonal, major diagonal Question: what weights should we use?

Crucial problem #1: Represent the surface as a weighted graph: The weight of an edge depends on the link type Minimal geodesic distance ~ shortest path in graph Graph is sparse: very fast algorithm Kiryati & Székely, 1993: Estimate the minimal (geodesic) distance between points on a 3D surface. Accurate and efficient solution for surface represented by voxels. direct, minor diagonal, major diagonal Question: what weights should we use? Wrong answer:

Bias in length estimation (2D example) Unbiased, minimum MSE* length estimation: Continuous curve (length L) Digital curve Digital curve Chain code (8-connected) Estimate length: * Unbiased and minimum MSE for straight lines. Unbiased for lines  wonderful for curves (errors cancel out).

Kiryati & Kübler, 1992: Unbiased, minimum MSE length estimation for 3D curves RMS error for straight lines: 2.88% Straight lines are the worst case (much better with curves) Use the weights (0.9016, 1.289, 1.615) in the algorithm for finding shortest paths (geodesics) and minimal distances on digitized surfaces. Verwer, Beckers & Smeulders

Crucial problem #1: Estimate the minimal distance between points on a digitized 3D surface. Solved (Kiryati & Székely, 1993)

Our Approach to Surface Flattening Mark N (very many) points on the 3D surface. Estimate the geodesic distances between each point and all others (how?) Optimally map the N points onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding 3D geodesic distances (how?) Regard the N points as control points, and map the rest by interpolation. Reminder:

Crucial problem #2: Given N points on a surface in 3D, map them onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding geodesic distances on the 3D surface.

Crucial problem #2: Given N points on a surface in 3D, map them onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding geodesic distances on the 3D surface. Special case of Multi-Dimensional Scaling (MDS)

Goal: Place N points in the 2D plane, such that the sets and are as similar as possible. Objective functions NameDefinitionAlgorithm Stress iterative SStress iterative Strainsomething elseanalytic solution! The analytic solution for minimal strain is called classical scaling (Borg & Groenen, 1997).

Classical Scaling X Coordinate Matrix: (N x 2) matrix with the (x,y) coordinates of N points in the plane 1 (unknown) B Scalar Product Matrix: (N x N) matrix (unknown) d* Squared Distances Matrix: (N x N) matrix containing the squared Euclidean distances between points in the plane (unknown) Suppose that d* is known. It can be shown that: where and 1 is a vector of 1 ’s. Having B, X can be found via eigendecomposition: holds exactly two (positive) non-zero eigenvalues. So where and are (2 x 2) and (N x 2) submatrices. 1 Column centered: coordinate system origin at the point-set centroid.

Classical Scaling (cont’d) But: we don’t have the Squared Distances Matrix d*, since interpoint distances in the plane are unknown. We do have the interpoint geodesic distances on the 3D surface. Algorithm 1.Create D* (Squared Geodesic Distance Matrix) 2.Compute the estimated Scalar Product Matrix: 3.Compute the eigendecomposition of up to rank 2: 4.The flattened Coordinate Matrix is:  Classical scaling minimizes the error metric (strain). Zero for a true Scalar Product Matrix obtained from a true Squared Distance Matrix d* containing squared planar Euclidean distances.  Partial eigendecomposition (up to rank 2) is easy and cheap (power method).

Our Approach to Surface Flattening Mark N (very many) points on the 3D surface. Estimate the geodesic distances between each point and all others. Optimally map the N points onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding 3D geodesic distances. Regard the N points as control points, and map the rest by interpolation. Reminder: We used N ~ 1000 control points, and mapped the rest using radial basis function interpolation. Any reasonable interpolation method can be used.

Curled planar surface (synthetic) Unfolded Euclidean distance (2D) vs. geodesic distance (3D)

Curled planar surface (synthetic) Unfolded Euclidean distance (2D) vs. geodesic distance (3D) Can you figure out why errors over small distances are larger than errors over large distances in this case?

Curled planar surface (synthetic) Unfolded Euclidean distance (2D) vs. geodesic distance (3D) Can you figure out why errors over small distances are larger than errors over large distances in this case? On this surface, many short geodesic paths are straight, or nearly straight. Straight lines are the worst case for the geodesic distance estimator. Over long distances, paths are not straight, and errors cancel out.

A depth image of a human face.Euclidean distance (2D) vs. geodesic distance (3D)

A depth image of a human face.Euclidean distance (2D) vs. geodesic distance (3D) The flattened surface is not shown, because the depth image has no texture.

A depth image of a human face.Euclidean distance (2D) vs. geodesic distance (3D) Errors are larger than in the previous example, because this surface is not developable – it cannot be flattened without some distortion.

Textures

Texture Mapping via Surface Flattening

Highlights New surface flattening algorithm Operates on voxel data. Triangulation is not necessary. Distance measurements on 3D surface + classical scaling. Global distance preservation Essentially analytic solution: Global optimum guaranteed Surface flattening, texture mapping

Appeared

Companion paper Flattening with triangulated data Geodesic distance measurements via Fast Marching on Triangulated Domains + classical scaling State of the art texture mapping quality