1 Distributed LT Codes Srinath Puducheri, Jörg Kliewer, and Thomas E. Fuja. Department of Electrical Engineering, University of Notre Dame, Notre Dame, IN 46556, USA IEEE International Symposium on Information Theory (ISIT), 2006.

2 Outline Introduction Description of LT Code Decomposition of LT Code Simulation Results Conclusions References

3 Introduction Consider a scenario: T : k information symbols Distributed LT LT N S 1 : k/2 information symbols S 2 : k/2 information symbols

4 Description of LT Code Luby Transform (LT) Codes: –Encoding process For the i th encoded symbol, select degree d i by Soliton distribution Choose d i source symbols Perform XOR on chosen symbols –Decoding process Decode degree-one encoded symbols Remove degree-one edges iteratively … x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 y1y1 y2y2 y3y3 y4y4 y5y5 x1x3x1x3 x2x2 x2x5x2x5 x3x5x6x3x5x6 Degree 123…k probability μ1μ1 μ2μ2 μ3μ3 …μkμk x4x4

5 Description of LT Code LT process: A B D y1y1 y2y2 y4y4 A  B A B  D covered : in the ripple S processed unprocessed: L C The most likely degree of encoded symbols that get released: k/L y3y3 A  C  D

6 Description of LT Code The intuition for the degree distribution: –More small degree than large degree. Easy to be decoded. –One single degree-one encoded symbol can trigger the LT process. Ideal Soliton Distribution: k : number of total original symbols i : degree of encoded symbols

7 Description of LT Code Ideal Soliton Distribution: probability degree

8 Description of LT Code In practice: –Only one single degree-one encoded symbol is too fragile. Number of degree-one encoded symbol has to be more. A larger Ripple. –Large degree encoded symbols are necessary evils. Because of a larger Ripple. To give a better coverage of original symbols. –A modified Soliton Distribution, concerning the size of Ripple during the LT process. Robust Solition Distribution.

9 Description of LT Code LT process: C F D y5y5 C  D  E  F covered : in the ripple S processed unprocessed: L E … When L = S, we expect all the original symbols to be covered. We need the encoding symbols that get released then must cover S original symbols by degree: k/S

10 Description of LT Code Robust Soliton Distribution (RSD): –Robust Soliton Distribution μ( · ) is given by: k : number of total original symbols i : degree of encoded symbols

11 Description of LT Code ρ( · ) is a distribution over 1 ≦ i ≦ k, and for constants c > 0, 0 <δ< 1, the τ( · ) is:, where S define the size of Ripple:

12 Description of LT Code Robust Soliton Distribution K : number of total original symbols d : degree of encoded symbols τ( k/S ) probability k = 10000

13 Decomposition of LT Code Recall: S 1 : k/2 information symbols S 2 : k/2 information symbols T : k information symbols Distributed LT LT Deconvolved Soliton Distribution Robust Soliton Distribution N

14 Decomposition of LT Code Proposed scenario: –Each data set contains k/2 information symbols. –Node N only performs one of two operations: Transmits the bitwise XOR of the symbols receives from S 1 and S 2. Transmits just one of them. The resulting encoded symbol received by T : –The degree of the resulting symbol has two cases: The sum of the degrees (up to i = k ). The individual degree of one of the received symbols.

15 Decomposition of LT Code In determining p (·), we employ the “deconvolution” of the RSD μ (·). –If node N XOR s the incoming symbols, the resulting degree is the sum of the in coming symbols. –If the degrees are independently chosen according to p (·), their sum will follow the distribution p * p. ‘ * ’ denotes the discrete-time convolution operation:

16 Decomposition of LT Code However, a direct attempt at deconvolving μ(·) does not yield satisfactory results. –μ(0) = 0, it is necessary that p(0) = 0. We cannot permit a degree of zero. –It implies that the smallest degree in p * p is two. However, this does not match the RSD as μ(1) > 0. degree = 1 would have problem. ( )

17 Decomposition of LT Code Even if we neglect μ(1), and attempt to deconvolve the remaining part of μ(·) by solving for p(·) from: –owing to the fact that μ(i) has a spike at i = k/S, we obtain negative values of p(·) for some arguments. degree = k/S would have problem.

18 Decomposition of LT Code An alternative approach (1/3): : ρ : τ Prob. Deg. Deg = 1 Deg = k/S Deg = 1 μ(i) : ρ : τ Prob. Deg. μ’(i)μ’(i) : ρ : τ Prob. μ ’’ (i)

19 Decomposition of LT Code An alternative approach (2/3): –Split RSD μ(·) into two distributions – μ ’ (·) and μ’’ (·). with the normalization factor β’ given by: Deconvolving μ’(·) yields a valid probability distribution function:

20 Decomposition of LT Code An alternative approach (3/3): –Split RSD μ(·) into two distributions – μ ’ (·) and μ’’ (·). with the normalization factor β’’ given by: and:

21 Decomposition of LT Code ∵ β ’ + β ’’ = β, the RSD can be rewritten as: Hence, the RSD μ (·) can be viewed as a mixture of the distributions μ’ (·) and μ’’ (·) with mixing parameter β’ / β’’.

22 Decomposition of LT Code Then we deconvolve μ’ (·) to yield a valid probability distribution f (·). –By solving: –Using the same approach as earlier mentioned ( recall ), with p(·) replaced by f (·) and μ (·) by μ’ (·), we obtain the following set of recursive equations: recall

23 Decomposition of LT Code The Deconvolved Soliton Distribution (DSD) p(·) is given by: with the parameter, given by: p(·) can be viewed as the mixture of f (·) and μ ’’ (·) with mixing parameter λ.

24 Decomposition of LT Code Notice that: μ':μ': 12 … k/S k/2 k f :f : 12 … k/S k/2k 12 … k/S k/2 f * f : k/2 + 1 tail Deg = k/S : ρ : τ Deg. μ(i) Prob.

25 Decomposition of LT Code Encoding procedure: D 1 : k/2 information symbols DSD : p(·) encoded symbols: Y Robust Soliton Distribution N D 2 : k/2 information symbols S1S1 S2S2 DSD p(·) encoded symbols: X 1 encoded symbols: X 2 Node N knows the degrees and neighbors of X 1 and X 2 Y = X 1  X 2 or Y = X 1 or Y = X 2 T DLT-2 Code

26 Decomposition of LT Code Node N generates a symbol Y as follows: otherwise (1/2): Generates independent random variables: U 1 and U 2 uniformly on the interval (0, 1). Based on the values of U 1 and U 2, one of the following operations is performed.

27 Decomposition of LT Code otherwise (2/2):

28 Decomposition of LT Code The code obtained by combining the two DLT-2 Codes generated by S 1 and S 2 is not, strictly speaking, an LT Code.

29 Simulation Results An application of LT Code decomposition: T : k information symbols Distributed LT LT N S 1 : m information symbols S J : m information symbols … … … J ≧ 2 source nodes - k = Jm Lossy channel

30 Simulation Results Two schemes (1/2): –Let every source employ an LT code for its data, and these LT codes are multiplexed in time at N and sent to T. Works well except when source data blocks are small ( information symbols). Overhead is fairly large (about 15-20%).

31 Simulation Results Two schemes (2/2): –It would be beneficial if we could form an LT code over the entire set of k = Jm information symbols at N and send this to T. J -fold deconvolution of the RSD. –Due to the constraint that N has limited processing and storage capabilities. Code symbols from different sources are XOR ed together at N. It yields symbols that obey the RSD over the range of degrees 1 through k = Jm.

32 Simulation Results Consider the case J = 4 (1/2): –The DLT-2 code can be decomposed further into two more subcodes called DLT-4 codes. –By performing a deconvolution of the DSD p(·) similar to how the DSD was derived from the RSD: Write DSD p(·) as a mixture of two distributions p ’(·) and p ’’(·), where p ’’(·) contributes to the degrees 1 and k/S. The distribution p ’(·) is then deconvolved to get g (·) and the final degree distribution q (·) is obtained by taking the appropriate mixture of g (·) and p ’’(·).

33 Simulation Results Consider the case J = 4 (2/2): –By performing a deconvolution of the DSD p(·) similar to how the DSD was derived from the RSD: The degree distribution q (·) is used by source nodes to generate encode symbols from their data, and these code symbols constitute the DLT-4 codeword. The node N performs a selective XOR of the four DLT-4 code symbols received from four sources, and forwards the resulting code symbol to T. Node T needs to collect slightly greater than k = Jm encode symbols from N to recover all source data.

34 Simulation Results Compare two schemes (1/2): –In terms of the number of code symbols that must be received by the sink T to recover all the sources’ data, for J = 4, m = 500, k = Jm = 2000 : Scheme A : –Each source generates LT code symbols from its 500- symbol data set. –The code symbols from different sources are time-multiplexed at N and transmitted to the sink T. –The sink separately decodes each LT code to recover the data. –Let N A denote the total number of code symbols that are needed by T to recover all the data.

35 Simulation Results Compare two schemes (2/2): Scheme B : –Each source generates DLT-4 codewords from its 500- symbol data set. –At every instant, the four encode symbols generated by the sources are selectively XOR -ed at N and forwarded to the sink T. –The sink thus receives a big LT code over all the sources’ data, which is then decoded to recover all the data simultaneously. – Let N B denote the total number of code symbols that are needed by T to recover all the data.

36 Simulation Results The average values of N A and N B obtained via simulations: P.S. Probable Max Degree of RSD in Scheme A : up to 500 Probable Max Degree of RSD in Scheme B : up to 2000

37 Simulation Results

38 Conclusions Presents a novel technique that allows distributed users to encode their data by selectively XOR ed, and results in an efficient rateless code, similar to an LT Code, at a common destination. Exploits the improved efficiency for large blocklengths: by forming a single codeword at the destination representing all the distributed data sets. –A reduction in overhead compared with a system in which each data set is encoded with its own LT code having a smaller blocklength.

39 References M. Luby, “LT Codes,” Proc. of the 43rd Annual IEEE Symp. on Foundations of Comp. Sc., pp , Vancouver, Canada, November D.J.C. MacKay, “Fountain Codes”, IEE Proceedings – Commun. Vol. 152, No 6, December Ashish Khisti, “Tornado Codes and Luby Transform Codes, ” Course Lecture of “6.454: Graduate Seminar in Area I” at MIT, Fall 2003.