1. Distributed Compression For Still Images
Kivanc Ozonat
Distributed Compression For Still Images

2. Distributed Compression For Still Images
Introduction
Description of the Problem
Related Concepts from Information Theory
Application of Bit-Plane Encoding
as a Possible Solution Strategy
Proposed Solution: Using Transform Coding
- Basic Scheme
- Relation to the Information Theory Concepts
Distributed Compression For Still Images

3. Distributed Compression For Still Images
Problem Description
Given two still images, a noisy version, X, at the decoder and the original, Y, at the encoder, how to transmit Y with the best coding efficiency?
No communication of X and Y at the encoder
Encoder
Decoder Y X
Distributed Compression For Still Images

4. Information Theory Background
Slepian-Wolf : Given the following scheme,
(X,Y)
(X,Y) R1
Encode X
Encode Y R2
Distributed Compression For Still Images

5. Information Theory Background
Can transmit X and Y, if:
- R1 > H(X|Y) , R2 > H(Y|X), and
- R1+ R2 > H(X,Y). R2
H(Y)
H(Y|X) R1
H(X|Y)
H(X)
Distributed Compression For Still Images

6. Information Theory Background
Our problem is a special case of this: R2
H(Y)
H(Y|X)
H(Y|X) R1
H(X|Y)
H(X)
H(X)
Distributed Compression For Still Images

7. General Solution Strategy
Form cosets with 3 requirements:
- Members of the same coset should be maximally
separated.
- Members of the same coset should have the same (or
very close) probabilities of occurrence.
- Coset construction should be practically implementable.
Distributed Compression For Still Images

8. Distributed Compression For Still Images
Underlying Approach
Use the Idea of Jointly Typical Sets:
- Encode a long sequence (length n) of i.i.d. sources
together, and form the typical set.
- As n gets large, the typical set contains almost all of the
probability of occurrence.
- Further, the typical set has its members uniformly
distributed.
Distributed Compression For Still Images

9. Distributed Compression For Still Images
Underlying Approach
The typical set contains most of the probability of occurrence,
but it has only power (of 2) nH elements.
Typical Set
Distributed Compression For Still Images

10. Distributed Compression For Still Images
Underlying Approach
Given i.i.d. X and i.i.d. Y, can form long sequences to get
the typical X and typical Y sequences.
Then, there are power (of 2) H(X,Y) jointly typical sequences.
Typical Y
Typical X
Distributed Compression For Still Images

11. Distributed Compression For Still Images
A Possible Scheme?
Use bit-plane encoding (followed by gray coding) to divide the image at the encoder into bit-planes.
Exploit the correlation between the adjacent pixels through
the upper bit planes.
The lower bit planes contain i.i.d. (or almost i.i.d) distribution of 0’s and 1’s.
Distributed Compression For Still Images

12. Distributed Compression For Still Images
A Possible Scheme?
Example: A lower bit-plane, with i.i.d 0’s and 1’s:
0.7
0.3
0.3 1 1
0.7
Distributed Compression For Still Images

13. Distributed Compression For Still Images
A Possible Scheme?
Given X , the noisy version, the jointly typical set is
(X,Y) such that X is as given, and Y is the set with
a Hamming distance of (0.3)n from X.
As n increases, Pr(X, Y=(X-0.3n)) will approach 1, with
each (X,Y) pair having the same distribution.
Hence, can perform an efficient coset construction
Distributed Compression For Still Images

14. Distributed Compression For Still Images
A Possible Scheme?
Problems:
The lower bit-planes, which are of interest, have error probabilities of close to 0.5, even with moderate noise
variances.
- Cannot compete with transform domain methods in terms of bit rates.
Distributed Compression For Still Images

15. Distributed Compression For Still Images
Proposed Solution
Using transform coding is better because:
- Energy compaction, resulting in lower bit rates
for PSNR’s of around dB.
- The addition / averaging process involved in transform coding reduces the effect of noise through the central limit theorem.
- The coded sequences of coefficients are de-correlated to
a significant extent.
Distributed Compression For Still Images

16. Distributed Compression For Still Images
Proposed Solution
Schematically,
Modified
Huffman
Coder Y
DCT
Zonal
Coder
Quantizer
Coset
Constructer
Bit-Plane
Encoder
Distributed Compression For Still Images

17. Distributed Compression For Still Images
Proposed Solution
The coefficients can be grouped in pairs with almost equal probabilities of occurrence. (because they are Laplacian)
One member from each pair is selected and Huffman-coded.
The other member of the pair is to have exactly the reverse
(0’s and 1’s switched) code.
The coded coefficients are placed in bit-planes.
Distributed Compression For Still Images

18. Distributed Compression For Still Images
Proposed Solution
1, 2, 3, 4, 5
01, 10, 11, 000, 111 (w.p , 0.125, 0.10 , 0.075, 0.075)
-1, -2, -3, -4, -5
10, 01, 00, 111, 000 (w.p , 0.125, 0.10 , 0.075, 0.075)
Assume: need to code 1,-4,-2, 2, -3,-1,5,2,3
Distributed Compression For Still Images

19. Distributed Compression For Still Images
Proposed Solution
Advantages:
- Equal likelihood of 0’s and 1’s. This makes the use of the
error coding simple.
- Essentially Huffman coding. Not a significant increase,
if the upper bit-planes are run-length coded.
- Maximal distribution of cosets possible, due to the uniformity and equality of probabilities.
Distributed Compression For Still Images

20. Distributed Compression For Still Images
Conclusions
Asymptotic Equipartition Property is essential in forming the typical sets.
Having equal probabilities of occurrences make the error-coding simpler.
Decorrelation is maintained through DCT.
The Laplacian Distribution of the DCT coefficients is important in getting equally probable pairs.
Distributed Compression For Still Images