Rate-Distortion Optimization for Geometry Compression of Triangular Meshes Frédéric Payan PhD Thesis Supervisor : Marc Antonini I3S laboratory - CReATIVe Research Group Université de Nice - Sophia Antipolis Sophia Antipolis - FRANCE

Motivations Goal : propose an efficient compression algorithm for highly detailed triangular meshes Objectives : High compression ratio Rate-Quality Optimization Multiresolution approach Fast algorithm

Summary Background Distortion criterion for multiresolution meshes Optimization of the Rate-Distorsion trade-off Experimental results Conclusions and perpectives

Summary Background Triangular Meshes Remeshing I. Background Summary Background Triangular Meshes Remeshing Multiresolution analysis Compression Bit allocation

Triangular Meshes 3D modeling Applications : Medecine CAD Map modeling I. Background Triangular Meshes 3D modeling Applications : Medecine CAD Map modeling Games Cinema Etc.

Irregular meshes valence different of 6 => 2 informations : I. Background Irregular meshes valence different of 6 => 2 informations : Geometry (vertices) Connectivity (edges) 4 neighbors 5 neighbors 9 neighbors

More than 380 millions of triangles => several Gigabytes (Michelangelo Project, 1999) Examples 40,000 triangles => + 0.45 Mb 99,732 triangles => + 1.1 Mb

=> Considered solution : Semi-regular remeshing I. Background Irregular meshes (2) Multiresolution Analysis : Without connectivity modification => wavelet transform for irregular meshes (S.Valette et R.Prost, 2004) A mesh is only one instance of the surface geometry => Remeshing goal : regular and uniform geometry sampling => Considered solution : Semi-regular remeshing

Summary Background Triangular Meshes Remeshing I. Background Summary Background Triangular Meshes Remeshing Multiresolution analysis Compression Bit allocation

Semi-regular remeshing I. Background Semi-regular remeshing Simplification Irregular mesh Coarse mesh Subdivision Semi-regular mesh Coarse mesh Finest semi-regular version Subdivised mesh (1) Original mesh

Semi-regular remesher I. Background Semi-regular remesher MAPS (A. Lee et al. , 1998) Coarse mesh (geometry+connectivity) N sets of 3D details (geometry) => 3 floating numbers Normal Meshes (I. Guskov et al., 2000) N’ sets of 3D details (geometry) => 1 floating number

=> More compact representation I. Background Normal Meshes Known direction: normal at the surface Surface to remesh => More compact representation

Summary Background Triangular Meshes Remeshing I. Background Summary Background Triangular Meshes Remeshing Multiresolution analysis Compression Bit allocation

Multiresolution analysis I. Background Multiresolution analysis … Details Details Details Details Multiresolution Representation: Low frequency (LF) mesh (geometry + topology) N sets of wavelet coefficients (3D vectors) (geometry)

Summary Background Triangular Meshes Remeshing I. Background Summary Background Triangular Meshes Remeshing Multiresolution analysis Compression Bit allocation

I. Background Compression Objective : reduce the information quantity useful for representing numerical data 2 approachs : Lossy or lossless compression High compression ratii => Lossy compression

Target bitrate or distortion I. Background Compression scheme Semi-regular Wavelet coefficients Entropy Coding Q 1010… Transform Bit Allocation Target bitrate or distortion Remeshing Optimize the Rate-Distortion (RD) tradeoff Preprocessing

Summary Background Triangular Meshes Remeshing I. Background Summary Background Triangular Meshes Remeshing Multiresolution analysis Compression Bit allocation

I. Background Bit allocation: goal Optimization of the tradeoff between bitstream size and reconstruction quality: minimize D(R) or minimize R(D) D R

Bit allocation and meshes I. Background Bit allocation and meshes Related Works (geometry compression): Zerotree coding PGC : Progressive Geometry Compression (A. Khodakovsky et al., 2OOO) NMC : Normal Mesh Compression ( A. Khodakovsky et I. Guskov, 2002). => Stop coding when bitstream given size is reached. Estimation-quantization (EQ) coding MSEC : Geometry Compression of Normal Meshes Using Rate-Distortion Algorithms (S. Lavu et al., 2003) => Local RD optimization.

Proposed bit allocation I. Background Proposed bit allocation Low computational complexity Improve the quantization process Maximize the quality of the reconstructed mesh according to a given target bitrate => Which distortion criterion for evaluating the losses?

Summary Background Distortion criterion for multiresolution meshes Optimization of the Rate-Distorsion trade-off Experimental results Conclusions and perpectives

Target bitrate or distortion II. Distortion criterion for multiresolution meshes Coding/Decoding Semi-regular Entropy coding Q 1010… Transform Remeshing Bit Allocation Target bitrate or distortion Preprocessing Inverse transform Entropy Decoding Q* Quantized semi-regular

Considered distorsion criterion II. Distortion criterion for multiresolution meshes Considered distorsion criterion MSE due to quantization of the semi-regular mesh Number of vertices semi-regular vertices quantized semi-regular vertices MSE for one subband Wavelet => ?

II. Distortion criterion for multiresolution meshes Related works K.Park and R.Haddad (1995) M-channel scheme quantization model : “noise plus gain” B.Usevitch (1996) quantization model : “additive noise” N decomposition levels Sampled on square grids Filter bank Problem : - non adapted for lifting scheme ! - usable for any sampling grid ?

Lifting scheme for meshes II. Distortion criterion for multiresolution meshes Lifting scheme for meshes 3 prédiction operators P => wavelet coefficients 3 update operators U => LF mesh Triangular grid => 4 channels

Triangulaire sampling II. Distortion criterion for multiresolution meshes Triangulaire sampling 1 triangular grid => 4 cosets n1 n2 2 3 LF subband (0) 1 HF subband 1 HF subband 2 HF subband 3

4-channel lifting scheme: analysis II. Distortion criterion for multiresolution meshes 4-channel lifting scheme: analysis + + + LF -P1 U1 + HF 1 split -P2 U2 + HF 2 Semi-regular mesh -P3 U3 + HF 3

4-channel lifting scheme: synthesis II. Distortion criterion for multiresolution meshes 4-channel lifting scheme: synthesis LF + + + P -U HF 1 + -U P Merge HF 2 + Semi-regular mesh -U P HF 3 + => Derivation of the MSE of the quantized mesh according to the quantization error of each 4 subband

Proposed Method Input signal : II. Distortion criterion for multiresolution meshes Proposed Method Input signal : Quantization error model : « additive noise » S is one realization of a stationar and ergodic random process => deterministic quantity => MSE of the input signal

Proposed Method: Hypothesis II. Distortion criterion for multiresolution meshes Proposed Method: Hypothesis Uncorrelated error in each subband Subband errors mutually uncorrelated Synthesis filter energy Quantization error energy

Proposed Method: principle II. Distortion criterion for multiresolution meshes Proposed Method: principle Synthesis filter energy Polyphase components of the filters Cauchy theorem Quantization error energy Uncorrelated error in each subband

Proposed Method: solution II. Distortion criterion for multiresolution meshes Proposed Method: solution For 1 decomposition level MSE of the subband i Weights relative to the non-orthogonal filters with Polyphase component

polyphase representation II. Distortion criterion for multiresolution meshes polyphase representation Lifting scheme: => Polyphase components depend on only the prediction and update opérators New formulation : => can be applied easily to lifting scheme

Proposed Method : solution II. Distortion criterion for multiresolution meshes Proposed Method : solution For N decomposition levels avec et

Outline This formulation can be applied to lifting scheme II. Distortion criterion for multiresolution meshes Outline This formulation can be applied to lifting scheme Global formulation of the weights for any : Grid and related subsampling number of channels M Number of decomposition levels N

=> PSNR Gain : up to 3.5 dB II. Distortion criterion for multiresolution meshes Experimental Results => PSNR Gain : up to 3.5 dB

Visual impact Without the weights Original With the weights II. Distortion criterion for multiresolution meshes Visual impact Without the weights Original With the weights

Target bitrate or distortion II. Distortion criterion for multiresolution meshes Coding/Decoding Semi-regular Entropy coding Q 1010… Transform Remeshing Bit Allocation Target bitrate or distortion Preprocessing Inverse transform Entropy Decoding Q* Quantized semi-regular

=> Is the MSE suitable to control the quality? II. Distortion criterion for multiresolution meshes MSE and irregular mesh Quality of the reconstructed mesh : Reference : irregular mesh Used metric: geometrical distance between two surfaces: the «surface-to-surface distance (s2s) » => Is the MSE suitable to control the quality?

Quality of the reconstructed mesh II. Distortion criterion for multiresolution meshes Quality of the reconstructed mesh Forward distance: distance between one point and one surface: Quantized mesh (semi-regular) Input mesh (irregular)

Simplifying approximations II. Distortion criterion for multiresolution meshes Simplifying approximations Normal meshes: => infinitesimal remeshing error => uniform and regular geometry sampling Highly detailed meshes: => densely sampled geometry Relation with the quantization error?

Hypothesis: asymptotical case II. Distortion criterion for multiresolution meshes Hypothesis: asymptotical case => Preservation of the LF subbands => normal orientations slightly modified => errors lie in the normal direction (normal meshes) θ ε(v2 ) n’ n n’ n θ ε(v2 )

II. Distortion criterion for multiresolution meshes Proposed heuristic Approximating formulation: Asymptotical case + normal meshes => MSE : suitable criterion to control the quality of the reconstructed mesh

Summary Background Distortion criterion for multiresolution meshes Optimization of the Rate-Distorsion trade-off Experimental results Conclusions and perpectives

Optimization of the Rate-Distorsion trade-off III.Optimization of the Rate-Distorsion trade-off Optimization of the Rate-Distorsion trade-off Objective : find the quantization steps that maximize the quality of the reconstructed mesh Scalar quantization (less complex than VQ) 3D Coefficients => data structuring?

Local frames Normal at the surface: z-axis of the local frame III.Optimization of the Rate-Distorsion trade-off Local frames Normal at the surface: z-axis of the local frame => Coefficient : Tangential components (x and y-coordinates) Normal components (z-coordinates) z x Global frame x z Local frame

Histogram of the polar angle III.Optimization of the Rate-Distorsion trade-off Histogram of the polar angle Local frame: θ x y z 0° 90° 180° => Components treated separately (2 scalar subbands) => Most of coefficients have only normal components

MSE of one subband i MSE relative to the tangential components III.Optimization of the Rate-Distorsion trade-off MSE of one subband i MSE relative to the tangential components MSE relative to the normal components

How solving the problem? III.Optimization of the Rate-Distorsion trade-off How solving the problem? Find the quantization steps and lambda that minimize the following lagrangian criterion: Method: => first order conditions Distortion Constraint relative to the bitrate

Solution Need to solve (2N + 4) equations with (2N + 4) unknowns III.Optimization of the Rate-Distorsion trade-off Solution Need to solve (2N + 4) equations with (2N + 4) unknowns PDF of the component sets: Generalized Gaussian Distribution (GGD) => model-based algorithm (C. Parisot, 2003)

Model-based algorithm III.Optimization of the Rate-Distorsion trade-off Model-based algorithm compute the variance and α for each subband compute the bitrates for each subband λ Complexity : 12 operations / semi-regular Example : 0.4 second (PIII 512 Mb Ram) => Fast process. Target bitrate reached? Look-up tables new λ compute the quantization step of each subband

Summary Background Distortion criterion for multiresolution meshes Optimization of the Rate-Distorsion trade-off Experimental results Conclusions and perpectives

Connectivity coding* (coarse mesh connectivity) IV. Experimental results Compression scheme 1010… Connectivity coding* (coarse mesh connectivity) MPX Unlifted Butterfly SQ 3D-CbAC* Normal meshes Bit Allocation Target Bitrate Preprocessing * Context-based Bitplane Arithmetic Coder (EBCOT-like) * Touma-Gotsman coder

Input mesh (irregular) IV. Experimental results Visual results Input mesh (irregular) CR = 83 2.2 bits/iv CR = 226 0.82 bits/iv CR = 900 0.2 bits/iv Compression ratio:

Comparison Quality criterion : State-of-the-art methods: IV. Experimental results Comparison Quality criterion : State-of-the-art methods: NMC (Normal meshes + Butterfly NL + zerotree) EQMC (Normal meshes + Butterfly NL + EQ) PGC (MAPS + Loop) Bounding box diagonal s2s between the irregular input mesh and the quantized semi-regular one

PSNR-bitrate curve: Rabbit IV. Experimental results PSNR-bitrate curve: Rabbit

PSNR-bitrate curve: Feline IV. Experimental results PSNR-bitrate curve: Feline => PSNR Gain: up to 7.5 dB

PSNR-bitrate curve: Horse IV. Experimental results PSNR-bitrate curve: Horse

Geometrical comparison IV. Experimental results Geometrical comparison NMC (62.86 dB) Proposed algorithm (65.35 dB) Bitrate = 0.71 bits/iv

Summary Background Distortion criterion for multiresolution meshes Optimization of the Rate-Distorsion trade-off Experimental results Conclusions and perpectives

=> Better results than the state-of-the-art methods. V. Conclusions and perspectives Conclusions New shape compression method: Contributions : Weighted MSE : suitable distortion criterion Original formulation of the weights (suitable in case of lifting scheme) Bit alllocation of low computational complexity that optimizes the quality of a quantized mesh. An original Context-based Bitplane Arithmetic Coder => Better results than the state-of-the-art methods.

V. Conclusions and perspectives Take into account some visual properties the human eye appreciates (local curvature, volume, smoothness…) Reference : Z.Karni and C.Gotsman, 2000 Algorithm for huge meshes: « on the flow » compression Reference : A. Elkefi et al., 2004

End