1. Compression of 3D images using disparity based coder
Department Of Communication System Engineering 2007
Compression of 3D images using disparity based coder
Presenting:
Shay Yannay & Arik Wasserman
Supervising:
Dr. Adi Stern & Dr. Ofer Hadar

2. Introduction & Motivation
3D imaging has great potential in the fields of home entertainment, industry, medicine etc.
Auto-stereoscopy: the ability to represent a 3D image which is visible without the need of special equipment or lighting source.
One way of implementation – Light Field and multiview array.
Our goal – to purpose an efficient method for encoding such data for storing and transmitting.

3. Multiview array & Light Field
Main Scheme
Encoder (recursive)
Multiview array & Light Field
Disparity Map
Map Estimation
Image Compensation
Export

4. LightField & Multiview Array
Light Field - A reliable representation of a 3D scene independent of it’s complexity.
Acquisition  process & Display: Integral Image system (IIS).
Viewing zone
Projection screen
multiview lenslet projector
Arranging a 2-dimentional array from the captured pictures : A Multiview Image.

5. Amount of Required Information – Example…
Proper viewing condition: 1 pixel distance between each adjacent pictures.
Image resolution X256 x
Array size – 32X32 x
Bit Per Pixel (RGB) – 24
Total: MB
So compression is crucial!
We will try to take advantage of the redundancy between the pictures in order to compress the formed data base.

6. What is a Disparity?
The offset of identical objects in two adjacent pictures is known as disparity.
The direction of the offset is known – according to the image position. We need only to get the magnitude of the shift.
For optimum (theory) – disparity values per each pixel.
In practice - the distance between blocks (8X8) is taken.
Hence, we can build for each picture a Disparity-Map.

7. Building the Disparity Map
MSE 5
Disparity Value

8. Disparity Map Array

9. Encoding principals: step 1– Map Estimation
Estimated Map
Missing Map

10. Encoding principals: step 2– Image Compensation
Missing Image
Estimated Map 5

11. The Coder – Decision tree
To meet a reconstruction quality of a missing image, we define a threshold parameter – Qmin (as PSNR).
The purpose: to achieve Qmin at the decoder while saving the minimum necessary.
Therefore, there are 3 possible methods for reconstructing a missing image:
The compensation is based upon the estimated map.
The compensation is based upon the original map (from initial array).
The compensation is done as in (2) and an error is also added.
The compensation stage goes along with a decision tree:

12. Compensate image using estimated disparity map
Save nothing
PSNR>Qmin
Yes No
Compensation With Original Disparity Map
Save Original
Disparity Maps
PSNR>Qmin
Yes No
Calculate residue
( original-estimated )
Save Original Map + Residue

13. Encoding- recursion order
Demo- 9X9 Light Field array : A C A C B C A C A

14. Encoding- recursion order
Demo- 9X9 Light Field array : A C A C A C B C B C A C A C A C B C B C A C A C A

15. Encoding- recursion order
Demo- 9X9 Light Field array : A C A C A C A C A C B C B C B C B C A C A C A C A C A C B C B C B C B C A C A C A C A C A C B C B C B C B C A C A C A C A C A C B C B C B C B C A C A C A C A C A

16. Disparity Coder Syntax
Encoder Data Base
Encoder Header
Decoder Guide
JPEG(4 Base ‘A’ Images)
Huffman(Encoder Disparity Map)
JPEG(Error)

17. Disparity Coder - Benchmark
We tested our encoder Based on several parameters:
Block size - 4x4 ; 8x8 ; 16x16
Qb & Qc - Qmin threshold is assigned for each ‘Compensation type’ in the decision tree :
Qb – for B type images ; Qc – for C type images. {Qb;Qc | 30->46 , 30->46 – incremented by 1 dB}.

18. Disparity Coder - Benchmark
Performance As A function of Qc & Qb
Psnr(Qc,Qb)
BitRate(Qc,Qb)
Display Performance
Time(Qc,Qb)
Encoding Decision
Ratio (%)

19. Images Array Quality Measurements

20. Images Array Bitrate – [bpp]

21. Images Array Encoding Elapsed Time

22. Encoding Decision Ratio (%)
Encoding with Estimated Map
Encoding with Original Map
Encoding with Original Map + resideu

23. Encoding Decision Ratio (%)
Encoding with Estimated Map
Encoding with Original Map
Encoding with Original Map + resideu

24. Encoding Decision Ratio (%)
Encoding with Estimated Map
Encoding with Original Map
Encoding with Original Map + resideu

25. Encoding Decision Ratio (%)
Encoding with Estimated Map
Encoding with Original Map
Encoding with Original Map + resideu

26. Encoding- Performance
Reconstructed Multi-view Array
Original Images
Reconstructed Images
Q(global) = 40, compression ratio  1:200

27. Disparity Coder – Compression Ratio

28. Further work
Testing various image types - synthetic and real, - higher content complexity, - larger array sizes (wider viewing angle)
Optimizing the algorithm (block size, minimizing residue saving)
Implementing concepts of video coding
Decoder and GUI

29. Questions?