1. 2018/9/16
Distributed Source Coding Using Syndromes (DISCUS): Design and Construction
S.Sandeep Pradhan, Kannan Ramchandran
IEEE Transactions on Information Theory,
vol. 49, no.3, pp , Mar 2003
2018/9/16

2. Outline Introduction Preliminaries Encoding with a Fidelity Criterion
2018/9/16
Outline
Introduction
Preliminaries
Encoding with a Fidelity Criterion
Problem Formulation
Design Algorithm
Constructions based on Trellis Codes
Simulation Results
Conclusion
2018/9/16

3. Introduction Slepian-Wolf theorem:
2018/9/16
Introduction
Slepian-Wolf theorem:
By knowing joint distribution of X and Y, without explicitly knowing Y, encoder of X can perform as well as encoder who knows Y.
Both encoder and decoder have access to side information Y
Only decoder has access to side information Y
2018/9/16

4. Introduction Wyner-Ziv Problem: Prior work on source quantizer design.
2018/9/16
Introduction
Wyner-Ziv Problem:
If decoder knows Y, then the information-theoretic rate-distortion performance for coding X is identical, no matter encoder knows Y or not.(X &Y are Gaussian.)
Prior work on source quantizer design.
Contributions:
Construction of a framework resting on algebraic channel coding principles
Performance analysis on Gaussian signals.
Source: discrete-alphabet  continuous-valued
Compression: lossless  lossy
2018/9/16

5. Outline Preliminaries Introduction Encoding with a Fidelity Criterion
2018/9/16
Outline
Introduction
Preliminaries
Encoding with a Fidelity Criterion
Problem Formulation
Design Algorithm
Constructions based on Trellis Codes
Simulation Results
Conclusion
2018/9/16

6. Preliminaries Example: Solution? X, Y: equiprobable 3-bit binary words
Hamming distance is no more than 1.
Y is available to decoder.
Solution?
Cosets: {000,111},{100,011},{010,101},{001, 110}
Only transmit coset index/syndrome.
2018/9/16

7. Preliminaries Quantization: Digitizes an analog signal.
Two parameters: a partition and a codebook.
Codebook: [-2, 0.4, 2.3, 6]
-0.5 1 +3
2018/9/16

8. Preliminaries Lloyd Max Quantization: yi yi-1 ai yi-2 ai-1
partition: ai are midpoints.
codebook: yi are centroids.
Optimal scalar quantization. yi
yi-1 ai
yi-2
ai-1
2018/9/16

9. Preliminaries Trellis Coded Quantization (TCQ):[24] Dual of TCM
Example:
Uniformly distributed source in [-A, A]
Implemented by Viterbi algorithm
[24] M.W. Marcellin and T. R. Fischer, “Trellis coded quantization of memoryless and Gauss-Markov sources,” IEEE Trans. Commun., vol. 38, pp.82–93, Jan
2018/9/16

10. Outline Encoding with a Fidelity Criterion Introduction Preliminaries
2018/9/16
Outline
Introduction
Preliminaries
Encoding with a Fidelity Criterion
Problem Formulation
Design Algorithm
Constructions based on Trellis Codes
Simulation Results
Conclusion
2018/9/16

11. Encoding with a Fidelity Criterion
Problem Formulation
X, Y: correlated, memoryless, i.i.d distributed sequences
Yi = Xi + Ni
Xi, Yi, Ni: continuous-valued
Ni: i.i.d distributed, independent from X
Xi, Ni: zero-mean Gaussian random variables with known variance
Decoder alone has access to Y.
Goal: Form best approximation to X given R bits per sample
Encoding in blocks of length L
Distortion measure:
Min R, s.t. reconstruction fidelity is less than given value D.
2018/9/16

12. Encoding with a Fidelity Criterion
System Model: encoder and decoder.
Interplay of source coding, channel coding and estimation
2018/9/16

13. Encoding with a Fidelity Criterion
Design Algorithm
Source Coding (M1, M2):
Partition source space:
Defining source codebook (S)
Characterizing active codeword by W (r.v.)
Estimation (M3):
Get best estimate of X (minimizing distortion) conditioned on outcome of Y and the element in .
Channel Coding (M4, M5):
Transmit over an error-free channel with rate R (less than Rs)
Doable: I(W;Y) > 0, so H(W|Y) = H(W) – I(W;Y)
Build channel code with rate Rc on channel P(Y|W)
R = Rs – Rc.
2018/9/16

14. Encoding with a Fidelity Criterion
Summary of Design Algorithm:
M1 and M3:
minimize Rs, s.t. reconstruction distortion within given criterion.
M2: maximize I(W;Y).
M4:
maximize Rc, s.t. error probability meets a desired tolerance level.
M5: minimize decoding computational complexity.
2018/9/16

15. Encoding with a Fidelity Criterion
Scalar Quantization and Memoryless Coset Construction (C1):
Lloyd-Max (memoryless) quantizer
Memoryless coset partition (M4)
Example:
L=1, (sample by sample)
Quantization codebook: {r0, r1, …, r7}, (Rs = 3)
Channel coding codebook: {r0, r2, r4, r6}, {r1, r3, r5, r7}. (Rc = 2)
R = Rs – Rc = 1 bit/sample.
2018/9/16

16. Encoding with a Fidelity Criterion
Scalar Quantization and Trellis-Based Coset Construction (C2):
Scalar quantizer for {Xi}i=1L
Coset partition (M4) by trellis code.
Codebook (size of 8L), Rs = 3 bits/sample, two cosets
2018/9/16

17. Encoding with a Fidelity Criterion
Example:
Computing syndrome (Rs = 3, Rc = 2)
outcome of quantization be 7, 3, 2, 1, 4.
L = 5,
Syndrome is given by for 5 samples.
2018/9/16

18. Encoding with a Fidelity Criterion
Trellis-Based Quantization and Memoryless Coset Construction (C3):
Trellis coded quantizer
Memoryless coset partition
Example:
Quantization codebook: Rs = 2
D0={r0, r4}, D1={r1, r5}, D2={r2, r6}, D3={r3, r7}.
Memoryless channel code: Rc = 1
1 coded bit with another 1 uncoded bit (from Y) to recover Di.
2018/9/16

19. Encoding with a Fidelity Criterion
Trellis-Based Quantization and Trellis-Based Coset Coset Construction (C4):
Trellis coded quantizer
Trellis coded coset partition
Comparison between C3 and C4.
2018/9/16

20. Encoding with a Fidelity Criterion
Distance Property
Given a uniform partition, four cases of coset constructions have same distance property.
Non-uniform quantizer, analyze performance by simulations.
2018/9/16

21. Outline Simulation Results Introduction Preliminaries
2018/9/16
Outline
Introduction
Preliminaries
Encoding with a Fidelity Criterion
Problem Formulation
Design Algorithm
Four Constructions
Simulation Results
Conclusion
2018/9/16

22. Quantization levels decrease distortion. (C1)
Simulation Results
Correlation
-SNR:
ratio of X’s
variance and
N’s variance.
Quantization levels decrease distortion. (C1)
2018/9/16

23. Quantization levels increase prob. Of error. (C1)
Simulation Results
Correlation
-SNR:
ratio of X’s
variance and
N’s variance.
Quantization levels increase prob. Of error. (C1)
2018/9/16

24. Error probability comparison of C1 and C2
Simulation Results
Correlation
-SNR:
ratio of X’s
variance and
N’s variance.
Error probability comparison of C1 and C2
(3-4dB gain)
2018/9/16

25. Error probability of C4 codes.
Simulation Results
Correlation
-SNR:
ratio of X’s
variance and
N’s variance.
Error probability of C4 codes.
2018/9/16

26. Conclusions
Constructive practical framework based on algebraic trellis codes.
Promising performance.
2018/9/16