Context-based Surface Completion Andrei Sharf, Marc Alexa, Daniel Cohen-Or

IntroductionIntroduction Holes in models: Imperfect range scanned data: –Complex objects with non-visible regions. –Misalignment of multiple-views depth image scans. –Material reflections. Surface editing operations.

MotivationMotivation Smooth filling is sufficient: Small holes. Smooth surfaces. Smooth filling is insufficient : Surfaces containing fine geometric detail. Topology of hole is more complex than a disk.

Context-based Completion Complete the missing region with patches that conform with its context SmoothContext-based

MethodMethod Import patches with matching context from the surface itself : Analyze surface characteristics. Find best matching patch. Fit imported patch to boundary.

Related Work Example-based image completion: [Drori et al. 2003; Jia and tang 2003; Sun et al. 2003;] Texture synthesis: [Efros and Leung 1999; Efros and Freeman 2001;Wei and Levoy 2000; Ying et al. 2001;] Smooth surface completion: [Curless and Levoy 1996; Davis et al. 2002; Ilic and Fua 2003; Verdera et al. 2003; Liepa 2003;] Model-based surface reconstruction [Savchenko et al. 2002] Curve analogies [Hertzmann et al. 2002] Drori et al. 03 Liepa 03 Wei and Levoi 00 Hertzmann et al. 02

Moving to 3D Surfaces Images have a regular spatial structure domain. Problems in 3D: Topology and geometry of missing region. Fitting a patch to the boundary of the missing region. Definition of similarity of shapes. Definition of a surface patch.

AlgorithmAlgorithm Given an incomplete shape Create initial spatial subdivision For each cell –Compute a local shape representation. –Compute a shape signature. For each empty cell: –Find matching nonempty cell ω’. –Copy patch of ω’ into ω. Subdivide cells and repeat Until completed region matches its neighborhood.

AlgorithmAlgorithm Given an incomplete shape Create initial spatial subdivision For each cell –Compute a local shape representation. –Compute a shape signature. For each empty cell: –Find matching nonempty cell ω’. –Copy patch of ω’ into ω. Subdivide cells and repeat Until completed region matches its neighborhood.

Shape Representation Shape sampled point-set –Range scan output –Easy to merge Octree hierarchy on top of the point set Implicit surface approximation by fitting polynomials [Ohtake et al. 2003] : –General quadric: –Bivariate quadratic: –Edge or corner fitting

Shape Analysis A local signature of a cell consists: Implicit shape characteristics: –Signed distance –Normal variation Detail amount: –Depth in octree

AlgorithmAlgorithm Given an incomplete shape Create initial spatial subdivision For each cell –Compute a local shape representation. –Compute a shape signature. For each empty cell: –Find matching nonempty cell ω’. –Copy patch of ω’ into ω. Subdivide cells and repeat Until completed region matches its neighborhood.

Missing Region Automatic identification: Intersect shape approximation with cells. Empty cell that intersects surface is part of missing region. Empty cell definition can alter due to refinement of missing region.  

MatchingMatching Distance metrics: d c : Signature distance inside cells. d a : Signature distance of adjacent cells. d l : Difference in amount of detail. Find most similar non-empty cell:

Matching-Candidate Set Non-empty cells of the same size Symmetry rotations of cells (  /2, ,...) Rotation of all point-set with some angle (  /4,  /3,...)

Transferring Points ICP: Copy points into empty cell. Find closest point correspondence Align points rigidly Align points non-rigidly using polynomial form.

AlgorithmAlgorithm Given an incomplete shape Create initial spatial subdivision For each cell –Compute a local shape representation. –Compute a shape signature. For each empty cell: –Find matching nonempty cell ω’. –Copy patch of ω’ into ω. Subdivide cells and repeat Until completed region matches its neighborhood.

David’s Hair OriginalDown- sampled Smooth completion Context- based completion

Completion Process Original Initial approximation Final result

Manual Editing of Bunny Model

Manual Editing of Knot Model

Scan of “Youth” Statue (Rear View) OriginalSmoothResult

Scan of Human Bone Original Smooth Result

Limitations: Semantics

Sample Rate vs. Detail Frequency

SummarySummary A fully automatic method to complete a missing region in a surface from its context. Completed patches geometrically conform with neighborhood. Incremental scale-space framework for finer approximation of the unknown region.

Future Work Explore with other spatial hierarchies. Enlarge the search space of examples by building a class based example set. Couple method with image completion methods for texture completion. Rotation and translation invariant signature

Acknowledgments: –David Levin and Olga Sorkine –Israel science foundation; Israeli ministry of science –Digital Michelangelo project 3D model repository; Stanford 3D scanning repository; Darmstadt university of technology; Imager computer graphics laboratory of the university of British Columbia

Thank you