2018/9/16 Distributed Source Coding Using Syndromes (DISCUS): Design and Construction S.Sandeep Pradhan, Kannan Ramchandran IEEE Transactions on Information Theory, vol. 49, no.3, pp.626-643, Mar 2003 2018/9/16

Outline Introduction Preliminaries Encoding with a Fidelity Criterion 2018/9/16 Outline Introduction Preliminaries Encoding with a Fidelity Criterion Problem Formulation Design Algorithm Constructions based on Trellis Codes Simulation Results Conclusion 2018/9/16

Introduction Slepian-Wolf theorem: 2018/9/16 Introduction Slepian-Wolf theorem: By knowing joint distribution of X and Y, without explicitly knowing Y, encoder of X can perform as well as encoder who knows Y. Both encoder and decoder have access to side information Y Only decoder has access to side information Y 2018/9/16

Introduction Wyner-Ziv Problem: Prior work on source quantizer design. 2018/9/16 Introduction Wyner-Ziv Problem: If decoder knows Y, then the information-theoretic rate-distortion performance for coding X is identical, no matter encoder knows Y or not.(X &Y are Gaussian.) Prior work on source quantizer design. Contributions: Construction of a framework resting on algebraic channel coding principles Performance analysis on Gaussian signals. Source: discrete-alphabet  continuous-valued Compression: lossless  lossy 2018/9/16

Outline Preliminaries Introduction Encoding with a Fidelity Criterion 2018/9/16 Outline Introduction Preliminaries Encoding with a Fidelity Criterion Problem Formulation Design Algorithm Constructions based on Trellis Codes Simulation Results Conclusion 2018/9/16

Preliminaries Example: Solution? X, Y: equiprobable 3-bit binary words Hamming distance is no more than 1. Y is available to decoder. Solution? Cosets: {000,111},{100,011},{010,101},{001, 110} Only transmit coset index/syndrome. 2018/9/16

Preliminaries Quantization: Digitizes an analog signal. Two parameters: a partition and a codebook. Codebook: [-2, 0.4, 2.3, 6] -0.5 1 +3 2018/9/16

Preliminaries Lloyd Max Quantization: yi yi-1 ai yi-2 ai-1 partition: ai are midpoints. codebook: yi are centroids. Optimal scalar quantization. yi yi-1 ai yi-2 ai-1 2018/9/16

Preliminaries Trellis Coded Quantization (TCQ):[24] Dual of TCM Example: Uniformly distributed source in [-A, A] Implemented by Viterbi algorithm [24] M.W. Marcellin and T. R. Fischer, “Trellis coded quantization of memoryless and Gauss-Markov sources,” IEEE Trans. Commun., vol. 38, pp.82–93, Jan. 1990. 2018/9/16

Outline Encoding with a Fidelity Criterion Introduction Preliminaries 2018/9/16 Outline Introduction Preliminaries Encoding with a Fidelity Criterion Problem Formulation Design Algorithm Constructions based on Trellis Codes Simulation Results Conclusion 2018/9/16

Encoding with a Fidelity Criterion Problem Formulation X, Y: correlated, memoryless, i.i.d distributed sequences Yi = Xi + Ni Xi, Yi, Ni: continuous-valued Ni: i.i.d distributed, independent from X Xi, Ni: zero-mean Gaussian random variables with known variance Decoder alone has access to Y. Goal: Form best approximation to X given R bits per sample Encoding in blocks of length L Distortion measure: Min R, s.t. reconstruction fidelity is less than given value D. 2018/9/16

Encoding with a Fidelity Criterion System Model: encoder and decoder. Interplay of source coding, channel coding and estimation 2018/9/16

Encoding with a Fidelity Criterion Design Algorithm Source Coding (M1, M2): Partition source space: Defining source codebook (S) Characterizing active codeword by W (r.v.) Estimation (M3): Get best estimate of X (minimizing distortion) conditioned on outcome of Y and the element in . Channel Coding (M4, M5): Transmit over an error-free channel with rate R (less than Rs) Doable: I(W;Y) > 0, so H(W|Y) = H(W) – I(W;Y) Build channel code with rate Rc on channel P(Y|W) R = Rs – Rc. 2018/9/16

Encoding with a Fidelity Criterion Summary of Design Algorithm: M1 and M3: minimize Rs, s.t. reconstruction distortion within given criterion. M2: maximize I(W;Y). M4: maximize Rc, s.t. error probability meets a desired tolerance level. M5: minimize decoding computational complexity. 2018/9/16

Encoding with a Fidelity Criterion Scalar Quantization and Memoryless Coset Construction (C1): Lloyd-Max (memoryless) quantizer Memoryless coset partition (M4) Example: L=1, (sample by sample) Quantization codebook: {r0, r1, …, r7}, (Rs = 3) Channel coding codebook: {r0, r2, r4, r6}, {r1, r3, r5, r7}. (Rc = 2) R = Rs – Rc = 1 bit/sample. 2018/9/16

Encoding with a Fidelity Criterion Scalar Quantization and Trellis-Based Coset Construction (C2): Scalar quantizer for {Xi}i=1L Coset partition (M4) by trellis code. Codebook (size of 8L), Rs = 3 bits/sample, two cosets 2018/9/16

Encoding with a Fidelity Criterion Example: Computing syndrome (Rs = 3, Rc = 2) outcome of quantization be 7, 3, 2, 1, 4. L = 5, Syndrome is given by 10110 for 5 samples. 2018/9/16

Encoding with a Fidelity Criterion Trellis-Based Quantization and Memoryless Coset Construction (C3): Trellis coded quantizer Memoryless coset partition Example: Quantization codebook: Rs = 2 D0={r0, r4}, D1={r1, r5}, D2={r2, r6}, D3={r3, r7}. Memoryless channel code: Rc = 1 1 coded bit with another 1 uncoded bit (from Y) to recover Di. 2018/9/16

Encoding with a Fidelity Criterion Trellis-Based Quantization and Trellis-Based Coset Coset Construction (C4): Trellis coded quantizer Trellis coded coset partition Comparison between C3 and C4. 2018/9/16

Encoding with a Fidelity Criterion Distance Property Given a uniform partition, four cases of coset constructions have same distance property. Non-uniform quantizer, analyze performance by simulations. 2018/9/16

Outline Simulation Results Introduction Preliminaries 2018/9/16 Outline Introduction Preliminaries Encoding with a Fidelity Criterion Problem Formulation Design Algorithm Four Constructions Simulation Results Conclusion 2018/9/16

Quantization levels decrease distortion. (C1) Simulation Results Correlation -SNR: ratio of X’s variance and N’s variance. Quantization levels decrease distortion. (C1) 2018/9/16

Quantization levels increase prob. Of error. (C1) Simulation Results Correlation -SNR: ratio of X’s variance and N’s variance. Quantization levels increase prob. Of error. (C1) 2018/9/16

Error probability comparison of C1 and C2 Simulation Results Correlation -SNR: ratio of X’s variance and N’s variance. Error probability comparison of C1 and C2 (3-4dB gain) 2018/9/16

Error probability of C4 codes. Simulation Results Correlation -SNR: ratio of X’s variance and N’s variance. Error probability of C4 codes. 2018/9/16

Conclusions Constructive practical framework based on algebraic trellis codes. Promising performance. 2018/9/16