Adaptive Slepian-Wolf Decoding using Particle Filtering based Belief Propagation Samuel Cheng, Shuang Wang and Lijuan Cui University of Oklahoma Tulsa, OK

Slepian-Wolf (SW) Problem X Encoder RX X and Y are discrete, correlated sources Y) , (X ˆ Separate Encoding: R=RY+RX=H(X,Y) < H(X)+H(Y) (if RX≥H(X|Y), RY≥H(Y|X)) Joint Decoder RX and RY are compression rates Y RY Encoder Joint Encoding: R=RY+RX=H(X,Y) < H(X)+H(Y) H(X)+H(Y) – separate enc and dec Separate encoding is as efficient as joint encoding!

SW Problem: The Rate Region Y separate encoding and decoding H(X,Y) H(Y) Focus of this work H(Y|X) H(X|Y) H(X) H(X,Y) R X Achievable rate region

Source Coding with Decoder Side Information (Asymmetric SW) ^ X X Source X Lossless Encoder Decoder Y R X H(Y|X) H(X) H(Y) H(X|Y) Y H(X,Y) A B Y – decoder side information (SI)

Prior Work of “Asymmetric” SW Coding Trellis code based Pradhan et al. ‘99 Turbo code based Garcia-Frias et al. ’01 Bajcsy & Mitran ’01 Aaron & Girod ’02 Li et al. ’04 LDPC code based Schonberg et al. ’02 Liveris et al. ’02, ’03 Garcia-Frias et al. ’03 None of the prior work is adaptive. The correlation statistics is assumed to be static and known a priori

H= Systematic (7,4) Hamming code C (can correct one bit error) ^ X X Syndrome former sT=HxT s Conventional channel decoder Source X Correlation Channel Y Systematic (7,4) Hamming code C (can correct one bit error) 1 0 0 1 0 1 1 0 1 0 1 1 0 1 0 0 1 0 1 1 1 H= Ry (3,7) Suppose that realizations are: xT = [ 0100000 ] sT = [ 010 ] 7 6 5 4 3 yT = [ 1100000 ] 3 4 5 6 7 Rx

LDPC based SW Coding Correlation model Encoding Decoding 1 1 ? 1 ? 1 1 ? 1 ? BSC p X Y ? 1 ? ? X and Y are binary ? ? p is static and known a priori X S Y X S

Belief Propagation Review Variable node update i a Factor node update Belief update i

Adaptive LDPC based SW Coding Correlation model Encoding Decoding 1 ? 1 ? 1 ? BSC p X Y ? 1 ? ? ? ? X S P Y X S Pj are continuous

BP cannot apply directly since p are continuous Approximate distribution of p using Np particles (at {p1, p2, …, pNp} and with weights {w1, w2, …, wNp}) The message passing steps do not change But locations and weights of particles of p should be updated appropriately  particle filtering

Particle Filter Particle Filter Steps: Particle locations obtained from previous iteration 2. Particle weights obtained from belief resulted generated by last BP iteration 3. Resampling 4. Random walk

Random Walk After the resampling step, particles congregate round the values with large weights. RW ensures the diversity of the particles. RW is implemented by adding a Gaussian random variable with zero mean and variance on the current value of each new particle generated in resample step.

Adaptive LDPC based SW Coding Correlation model Encoding Decoding 1 ? 1 ? 1 ? BSC p X Y ? 1 ? ? ? ? X S P Y X S Connection ratio = 1:1

Adaptive LDPC based SW Coding Correlation model Encoding Decoding 1 1 ? ? 1 ? BSC p X Y ? ? 1 ? ? ? ? ? ? X S P Y X S Connection ratio  1:2

Results 16 particles were assigned to each pj For the Random Walk, we assumed and λ=0.01 The following results were obtained by taking average of 30 different codewords. Code length = 20K Regular codes were used

Correlation Estimation

Decoding Performance Linearly changing correlations (1:16)

Correlation Estimation

Decoding Performance Sinusoidally changing correlations

Conclusions Adaptive decoding for asymmetric SW coding using BP + particle filtering is proposed. Can accurately estimate dynamic change of correlation (connection ratio should not be too small) The work has been extended to non-asymmetric case using code partitioning technique (submitted to ICASSP 2010); adaptive LDPC decoding was presented in CISS 2009. Note that the theoretical limit (SW limit) shown is really an outer bound. Because the original SW limit is derived assuming the model statistics are known. Future work: non parametric BP

Resampling Calculate Compare The cumulative sum of the particle weights and updated uniform number where is drawn from the uniform distribution . . Compare and to determine the number of replications for particle k in the range Systematic resampling process for an example with Np particles. Weight of particles is listed in the table.