Atomic Volumes for Mesh Completion Joshua Podolak Szymon Rusinkiewicz Princeton University

Outline Problem Description Problem Description Background Background Algorithm Algorithm Results Results

Motivation Scanners usually need two unobstructed lines of sight between the cameras and the model Scanners usually need two unobstructed lines of sight between the cameras and the model Even with multiple scans, there are areas containing no depth information that need to be filled Even with multiple scans, there are areas containing no depth information that need to be filled

Holes bounded by ring of half-edges. Filling Holes

Holes bounded by ring of half-edges. In simple cases, it is sufficient to create a patch by triangulating the half-edge loops around the holes. Triangulate?

Challenge Triangulation is not enough.

Challenge

Talk Overview Problem Description Problem Description Background Background Algorithm Algorithm Results Results

Related Work Point cloud reconstruction Point cloud reconstruction [Amenta01, Kolluri04] Half-edge boundary triangulation Half-edge boundary triangulation [Berg97, Liepa03] Implicit hole filling Implicit hole filling [Curless96, Davis02, Masuda04, Ju04, Sharf04] Volumetric hole filling Volumetric hole filling[Murali97]

Related Work Point cloud reconstruction Point cloud reconstruction [Amenta01, Kolluri04] Half-edge boundary triangulation Half-edge boundary triangulation [Berg97, Liepa03] Implicit hole filling Implicit hole filling [Curless96, Davis02, Masuda04, Ju04, Sharf04] Volumetric hole filling Volumetric hole filling[Murali97]

Related Work Point cloud reconstruction Point cloud reconstruction [Amenta01, Kolluri04] Half-edge boundary triangulation Half-edge boundary triangulation [Berg97, Liepa03] Implicit hole filling Implicit hole filling [Curless96, Davis02, Masuda04, Ju04, Sharf04] Volumetric hole filling Volumetric hole filling[Murali97]

Related Work Point cloud reconstruction Point cloud reconstruction [Amenta01, Kolluri04] Half-edge boundary triangulation Half-edge boundary triangulation [Berg97, Liepa03] Implicit hole filling Implicit hole filling [Curless96, Davis02, Masuda04, Ju04, Sharf04] Volumetric hole filling Volumetric hole filling [Murali97]

Talk Overview Problem Description Problem Description Background Background Algorithm Algorithm Results Results

Definition: Atomic Volume A volume is atomic if it cannot be intersected by the surface of the model. A volume is atomic if it cannot be intersected by the surface of the model. An atomic volume must be either completely inside or completely outside the model. 2 atomic volumes

Overview of Approach 1. Divide all of space into atomic volumes. Regions of space that will either be wholly in or wholly out of the reconstructed solid. Regions of space that will either be wholly in or wholly out of the reconstructed solid. 2. For each atomic volume, decide whether it is inside the mesh or outside. 3. The boundary between interior volumes and exterior volumes is the new surface.

Step 1: Spatial Partitioning

Blank Region Blank cubes do not contain any part of the input model. Blank cubes

Step 1: Spatial Partitioning In Region Out Region Blank Region Inside/Outside (IO): Cubes containing elements of the surface away from the holes. IO cubes

Step 1: Spatial Partitioning In Region Out Region Hole Region Blank Region Hole cubes are subdivided until they can be trivially triangulated [Mitchell92]. Hole cubes

Step 2: Label Assignment Most of the volumes can be labeled as inside or outside based on the normals of the input mesh. Most of the volumes can be labeled as inside or outside based on the normals of the input mesh. Filling the hole requires labeling the remaining volumes as inside and outside. Filling the hole requires labeling the remaining volumes as inside and outside. Outside Inside

Step 2: Label Assignment Atomic volumes correspond to nodes in the graph with edges between neighboring volumes. Atomic volumes correspond to nodes in the graph with edges between neighboring volumes.

Step 2: Label Assignment Atomic volumes correspond to nodes in the graph with edges between neighboring volumes. Atomic volumes correspond to nodes in the graph with edges between neighboring volumes. Atomic volumes on either side of a surface are not connected. Atomic volumes on either side of a surface are not connected.

Step 2: Label Assignment Atomic volumes corresponds nodes in the graph with edges between neighboring volumes. Atomic volumes corresponds nodes in the graph with edges between neighboring volumes. Atomic volumes on either side of a surface are not connected. Atomic volumes on either side of a surface are not connected. The mesh graph can be split into two disjoint sub graphs. The mesh graph can be split into two disjoint sub graphs.

Mesh Graph In Region Out Region Hole Region Blank Region Blank cube

Mesh Graph I/O cube In Region Out Region Hole Region Blank Region

Mesh Graph Hole cube In Region Out Region Hole Region Blank Region

In a watertight surface, there can be no path between inside and outside nodes. In a watertight surface, there can be no path between inside and outside nodes. Mesh Graph In Region Out Region Hole Region Blank Region

Step 2: Label Assignment Splitting the graph into two sub graphs is equivalent to labeling each of the atomic volumes. Splitting the graph into two sub graphs is equivalent to labeling each of the atomic volumes. Initially labeled atomic volumes must retain original labeling. Initially labeled atomic volumes must retain original labeling.

Minimum Cut Minimum Cut – Add source and sink nodes. – Constraint edges have a weight of infinity. Labeling Volumes Sink (outside) Source (inside) Constraint edges

Creating Patch Every edge cut in the graph corresponds to a surface added to the model. Every edge cut in the graph corresponds to a surface added to the model. In Region Out Region Hole Region Blank Region

Topological Control A graph allows adding additional constraints based on other sources of information such as Space Carving, Shadow Carving, or direct user input. A graph allows adding additional constraints based on other sources of information such as Space Carving, Shadow Carving, or direct user input. Adding an edge with weight infinity between an atomic volume to the sink node constrains that volume to be outside the model. Adding an edge with weight infinity between an atomic volume to the sink node constrains that volume to be outside the model. Atomic volumes containing points with contradicting constraints may be subdivided. Atomic volumes containing points with contradicting constraints may be subdivided.

Topological Control In the figure on the right, points are added manually to indicate that the space between the two toes is outside. In the figure on the right, points are added manually to indicate that the space between the two toes is outside. Initial mesh Unconstrained solution Constrained solution

Smoothing Big atomic volumes cause faceting of the surface. Big atomic volumes cause faceting of the surface. We introduce a smoothing step to address faceting. We introduce a smoothing step to address faceting. In Region Out Region Hole Region Blank Region

Smoothing Big atomic volumes cause faceting of the surface. Big atomic volumes cause faceting of the surface. We introduce a smoothing step to address faceting. We introduce a smoothing step to address faceting.

Smoothing Big atomic volumes cause faceting of the surface. Big atomic volumes cause faceting of the surface. We introduce a smoothing step to address faceting. We introduce a smoothing step to address faceting. UnsmoothedSmoothed

Smoothing Coarse smoothing will changes the labeling of entire atomic volumes. Fine smoothing keeps the topology of the mesh graph constant, but allows the boundary between adjacent atomic volumes to change. Unsmoothed Smoothed

Smoothing Constrained Laplacian Smoothing. We do not move vertices that cause the mesh graph to change. Result after five iterations

Talk Overview Problem Description Problem Description Background Background – Related Work – Volumetric Representation Algorithm Algorithm Results Results

Results Cross section of spiral torus.Half-torus filled with two different patches.

Results Bunny: 70k faces. 78 sec. Angel: 350k faces. 15 min.

Conclusions Atomic Volumes is a discrete volumetric method for mesh completion. Atomic Volumes is a discrete volumetric method for mesh completion. The use of an adaptive subdivision of space allows the algorithm to focus on target areas. The use of an adaptive subdivision of space allows the algorithm to focus on target areas. Using a mesh-graph provides flexibility in determining the patch. Using a mesh-graph provides flexibility in determining the patch.

Future Work Soft constraints. Soft constraints. Advanced smoothing. Advanced smoothing. – Flexible atomic volumes. – Thin plate energy. – Texture Synthesis. Signal processing. Signal processing. – Localized filtering. – Deformation.

Thank you

Mesh Graph A mesh graph contains a node for each atomic volume. A mesh graph contains a node for each atomic volume. An edge between neighboring atomic volumes represents the strength of the belief that they will be labeled similarly. An edge between neighboring atomic volumes represents the strength of the belief that they will be labeled similarly.