1. Wavelet-based Compression of 3D Mesh Sequences
Frédéric Payan
Coauthor: Marc Antonini
I3S laboratory - CReATIVe Research Group
Universite de Nice Sophia Antipolis FRANCE
2nd ACIDCA-ICMI, Tozeur, Tunisia, november, 2005.

2. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Motivations
Design a compression algorithm for mesh sequences (with a fixed connectivity).
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

3. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Problem Statement
Raw representation: set of irregular meshes
Talking Face: meshes of 539 vertices
Each frame defined by the coordinates of its vertices
VRML: more than 137 Mb! Compression needed…
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

4. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Summary
Background
Related works
Contributions
Proposed temporal lifting scheme
Bit allocation
Simulation results
Conclusion & perspectives
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

5. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Summary
Background
Related works
Contributions
Proposed temporal lifting scheme
Bit allocation
Simulation results
Conclusion & perspectives
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

6. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
I. Background
Main Related Works
Principal component analysis (PCA): Alexa & Müller (2000)
exploit the temporal correlation of sequence geometry
but… very complex and long analysis step and efficient only when few shape deformations
PCA+linear prediction coding (LPC): Karni & Gotsman (2004)
more efficient than PCA
but… same problem than PCA!
Spatial multiresolution analysis: Guskov & Khodakovsky (2004)
efficient to capture local shape deformations
but… less efficient when global deformations
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

7. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
I. Background
Contributions
Observation: no temporal wavelet-based coder !
Proposed approach:
Temporal lifting scheme
to exploit the temporal coherence
fast and low complex synthesis but also analysis
Bit allocation process
optimal coding
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

8. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Summary
Background
Related works
Contributions
Proposed temporal lifting scheme
Bit allocation
Simulation results
Conclusion & perspectives
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

9. Lifting scheme: principle
II. Proposed temporal lifting scheme
Lifting scheme: principle
Split -P U +
HF details (wavelet coefficients)
Low frequency (LF) signal
Signal
3 steps:
split => two cosets
Prediction operator => set of details
Update operator => low frequency (LF) signal
defined by a pair [n,m]
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

10. Temporal Lifting scheme: principle
II. Proposed temporal lifting scheme
Temporal Lifting scheme: principle
Time
Original Sequence …
TLS
LF Sequence …
Sets of HF details …
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

11. Temporal Lifting Scheme for mesh sequences
II. Proposed temporal lifting scheme
Temporal Lifting Scheme for mesh sequences
Principle: for each vertex, apply a 1D lifting scheme on its successive positions along the time axis
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

12. Temporal Lifting Scheme for mesh sequences
II. Proposed temporal lifting scheme
Temporal Lifting Scheme for mesh sequences
Example: filter [2,0], with 2 levels of decomposition
Successive positions of the vertex V(i)
wavelet coefficients
LF data
wavelet coefficients
LF signal
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

13. Temporal Lifting Scheme for mesh sequences
II. Proposed temporal lifting scheme
Temporal Lifting Scheme for mesh sequences
=> Multiresolution decomposition
Input sequence
time …
First set of temporal details …
h(1)(1)
h(1)(2)
h(1)(3)
h(1)(4)
Second set of temporal details …
h(2)(1)
h(2)(2) …
LF sequence
Decomposition
level
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

14. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Summary
Background
Related works
Contributions
Proposed temporal lifting scheme
Bit allocation
Simulation results
Conclusion & perspectives
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

15. Bit allocation: principle
III. Bit allocation
Bit allocation: principle D
Compression: optimize the rate-distortion tradeoff
Multiresolution representation => how dispatching the bits across the different sets of details ? R
Solution: use a bit allocation to maximize the quality for a user-given bitrate
Decomposition
level
h(1)(1)
h(1)(2)
h(1)(3)
h(1)(4)
h(2)(1)
h(2)(2) …
Bit budget ?
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

16. Proposed bit allocation
III. Bit allocation
Proposed bit allocation
Objective: find the quantization steps (used to encode the different set of details) that minimize the MSE of the reconstructed mesh sequence for a user-given target bitrate.
This problem can be modeled by
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

17. How solving this problem?
III. Bit allocation
How solving this problem?
Find the quantization steps and lambda that minimize the following lagrangian criterion:
Method: first order conditions
Algorithm: iterative and model-based
Distortion
Constraint relative to the bitrate
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

18. connectivity encoding
III. Bit allocation
Overall coding scheme
connectivity encoding
Sequence connectivity
MUX
1011…
Temporal DWT SQ
Entropy Coding
Geometry encoding
{q}
Target Bitrate
Bit Allocation
SQ: scalar quantization
Connectivity encoding: valence-based coder of Touma & Gotsman
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

19. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Summary
Background
Related works
Contributions
Proposed temporal lifting scheme
Bit allocation
Simulation results
Conclusion & perspectives
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

20. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
III. Simulation results
Distortion measure
K&G in [2] => relative discrete L2-norm both in time and space
with:
matrix G: geometry of the sequence
matrix hat G: quantized geometry
Matrix E(G): mean coordinates of each frame
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

21. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
III. Simulation results
Different models!
Talking face
Few global motion
Surprised chicken
high global motion
high local deformations
Mad cow
few local deformations
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

22. Different lifting schemes…
III. Simulation results
Different lifting schemes…
Filter [4,2]: less computing resources in time and memory
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

23. Different numbers of decomposition…
III. Simulation results
Different numbers of decomposition…
high motion => few decomposition levels (4)
few motion => a lot of decomposition levels (7)
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

24. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
III. Simulation results
Comparison with…
TG (Touma & Gotsman)
Dynapack (Ibarria & Rossignac)
PCA (Alexis & Müller)
KG (PCA+LPC, Karni & Gotsman)
WCA (Guskov & Khodakovsky)
on…
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

25. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
III. Simulation results
…The talking face!
few global motion
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

26. …the surprised chicken!
III. Simulation results
…the surprised chicken!
high global motion and local deformations
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

27. …the mad cow! high global motion and few local deformations
III. Simulation results
…the mad cow!
high global motion and few local deformations
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

28. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
Summary
Background
Related works
Contributions
Proposed temporal lifting scheme
Bit allocation
Simulation results
Conclusion & perspectives
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

29. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
IV. Conclusions and perspectives
Conclusions
Temporal lifting scheme for mesh sequences
simpler and faster than other analysis tools (for analysis and synthesis) like PCA…
Optimal encoding of the different subbands
Model-based bit allocation
=> Efficient and fast compression method for mesh sequences
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

30. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
IV. Conclusions and perspectives
Perspectives
Additional spatial multiresolution analysis on LF sequences
Explicit motion estimation technique (like in video coding) ?
should reduce geometry information
transmit side information (motion vectors)
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.

31. 2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.
This is the end…
My fpayan.i3s.unice.fr
My webpage:
Acknowledgement
Microsoft Inc. For the surprised chicken
D. Terzopoulos For the talking face
Z. Karni for providing us with these data
I. Guskov for providing us with mad cow
Results extracted from
paper of Z. Karni & C. Gotsman
provided by I. Guskov
23/02/2019
2nd ACIDCA-ICMI, Tozeur, Tunisia, november 2005.