Surface Compression with Geometric Bandelets Gabriel Peyré Stéphane Mallat

Outline Why Discrete Multiscale Geometry? Image-based Surface Processing Geometry in the Wavelet Domain Moving from 2D to 1D The Algorithm in Details Results

Geometry of Surfaces: Creation Clay modeling Low-poly modeling

Geometry of Surfaces: Rendering Stoke rendering taken from [Rössl-Kobbelt 01] Line-Art Rendering of 3D Models. Lines of curvatures and highlights

Geometry of Surfaces: Mesh processing Anisotropic Remeshing [Alliez et Al. 03] Robust Moving Least-squares Fitting with Sharp Features [Fleishman et Al. 05]

Geometry is Discrete continous discrete multiscale geometry acquisition [Digital Michelangelo Project]

Geometry is Multiscale? continous discrete multiscale geometry Surface simplification [Garland & Heckbert 97] Normal meshes [Guskov et Al. 00]

Sharp Geometry is Multiscale! Smoothed Fine Scale Edge extraction is an ill-posed problem. Localization is not needed for compression!

Geometry is not Defined by Sharp Features Edge localization [Ohtake et Al. 04] [Ohtake et Al. 04] Ridge-valley lines on meshes via implicit surface fitting. [DeRose et Al. 98] Semi-sharp features [DeRose et Al. 98] Subdivision Surfaces in Character Animation

Outline Why Discrete Multiscale Geometry? Image-based Surface Processing Geometry in the Wavelet Domain Moving from 2D to 1D The Algorithm in Details Results

[Gu et Al.] Geometry images [Gu et Al.] irregular mesh  2D array of points [r,g,b] = [x,y,z] cutparameterize No connectivity information. Simplify and accelerate hardware rendering. Allows application of image-based compression schemes.

Our Functional Model 3D model 2D GIM (lit) Uniformly regular areas + Sharp features + Smoothed features

Outline Why Discrete Multiscale Geometry? Image-based Surface Processing Geometry in the Wavelet Domain Moving from 2D to 1D The Algorithm in Details Results

Hierarchical Cascad Acquisition (scanner, etc) Wavelet transform Orthogonal dilated filters cascad Proposition: to continue the cascad. Geometric transform ?

What is a wavelet transform? Original surface Geometry image Wavelet transform Decompose an image at dyadic scales. 3 orientations by scales H/V/D. Compact representation: few high coefficients. But… still high coefficients near singularities. HV D

Outline Why Discrete Multiscale Geometry? Image-based Surface Processing Geometry in the Wavelet Domain Moving from 2D to 1D The Algorithm in Details Results

Some insights about bandelets Moto: wavelets transform is cool, re-use it! Goal: remove the remaining high wavelet coefficients. Hope: exploit the anisotropic regularity of the geometry. Tool: 2D anisotropy become isotropic in 1D.

Construction of this Reordering Geometry image 2D Wavelet Transform HV D Choose a direction Project points orthogonally on Report values on 1D axis Resulting 1D signal

Choosing the square and the direction T -T T + threshold T Too big: direction deviates from geometry How to choose: 1D wavelet transform Too much high coefficients!

Choosing the Square and the Direction T -T T Bad direction: direction deviates from geometry Still too much high coefficients!

Choosing the Square and the Direction T -T T Correct direction: direction matches the geometry Nearly no high coefficients!

Outline Why Discrete Multiscale Geometry? Image-based Surface Processing Geometry in the Wavelet Domain Moving from 2D to 1D The Algorithm in Details Results

The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree Original surface Geometry image

The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Tran (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree Geometry image 2D Wavelet Transform HV D

The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivis (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree 2D Wavelet Transform HV D Zoom on D D

The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivis (4) Extract Sub-sq (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree Zoom on D D Sub-square 2D Wavelet Transform HV D

The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometr (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree Sub-squareZoom on D D

Sub-square The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree 1D Signal

1D Wavelet Transform The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Tran (8) Select Geometry (9) Output Coefficients (10) Build Quadtree 1D Signal

vs T 1D Wavelet Transform The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree 1D Signal T vs T

The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficie (10) Build Quadtree 1D Signal 1D Wavelet Transform T T vs T Bandelets coefficients Untransformed coefs (For comparison)

The Algorithm in 10 Steps (1) Geometry Image (2) 2D Wavelet Transf. (3) Dyadic Subdivision (4) Extract Sub-square (5) Sample Geometry (6) Project Points (7) 1D Wavelet Transf. (8) Select Geometry (9) Output Coefficients (10) Build Quadtree Don’t use every dyadic square … Compute an optimal segmentation into squares. Fast pruning algorithm (see paper). 2D Wavelet Transform HV D D Zoom on D

What does bandelets look like? Transform = decomposition on an orthogonal basis. Basis functions are elongated “bandelets”. The transform adapts itself to the geometry.

Transform Coding in a Bandelet Basis Bandelet coefficients are quantized and entropy coded. Quadtree segmentation and geometry is coded. Possibility to use more advanced image coders (e.g. JPEG2000).

Outline Why Discrete Multiscale Geometry? Image-based Surface Processing Geometry in the Wavelet Domain Moving from 2D to 1D The Algorithm in Details Results

Results: Sharp features Original bpv bpv Hausd. PNSR: +2.2dB

Results: More complex features Original bpv bpv Hausd. PNSR: +1.6dB

Blurred Features Hausd. PNSR: +1.3dB bpv bpv Original

Spherical Geometry Images bpv bpv Original Hausd. PNSR: +1.0dB

Spherical Geometry Images bpv bpv Original Hausd. PNSR: +1.25dB

Conclusion Approach: re-use wavelet expansion. Contribution: bring geometry into the multiscale framework. Results: improvement over wavelets even for blurred features. Extension: other maps (normals, BRDF, etc.) and other processings (denoising, deblurring, etc.).