Improving turbocode performance by cross-entropy Yannick Saouter 24/02/2019

Turbocodes Turbo code encoding : data frame X=(x1,…,xn) is permuted by  to give X’=(x(1),…,x(n)). The two frames are encoded by a RSC and gives redundancies Y1 and Y2. Invented by Claude Berrou and Alain Glavieux [BerrouGlavieux93]

Soft decoding Turbodecoding code : computation of a posteriori estimates given as a priori to the second decoder SISO decoders : generally MAP or SUBMAP for convolutive turbocodes

Definition : Let p, q be probability density functions over  then: Cross-entropy Definition : Let p, q be probability density functions over  then: Also known as Kullback distance. Property : CE(p,q)0 with CE(p,q)=0 iff. p=q almost everywhere.

Cross-entropy decoding CE decoding : code C, a priori distribution q(x) , outputs distribution p(x) such that [Battail87] : Generally computationally infeasible Turbocode decoding is a suboptimal cross entropy decoding [MoherGulliver98] With SISO decoders, cross entropy can give a numeric valuation of quality of decoding If CE(p,q) is low (resp. large), decoding is reliable (resp. unreliable). p(x) satifies parity constraints of C CE(p,q) is minimal

Pratical computation of cross-entropy X=(x1,…,xn), R=(r1,…,rn) convolutive frame with input extrinsic values Lein=(Lein1,…,Leinn). SISO decoder produces output extrinsic values Leout=(Leout1,…,Leoutn). A priori distribution Lin=Lc.X+Lein and A posteriori distribution Lout=Lc.X+Leout. Gaussian hypothesis : We have [Hagenauer96] : Too computational.

Hypothesis 2 : IfLiini is low, then Liouti also. Some hypothesis Goal : Minimizing computational load, avoid use of exp and log functions Hypothesis 1 : If Liini is large, then Liouti also and has the same sign. Hypothesis 2 : IfLiini is low, then Liouti also. Correspond generally to correct decoded samples Unreliable inputs cannot produce reliable ouputs

The « large » case Cross entropy : Assume Lini large and positive. Same for Louti, thus : Same result if Lini large and negative.

Hypothesis : Lini and Louti are near 0. The « low » case Hypothesis : Lini and Louti are near 0. Use Taylor developments. We obtain:

Practical criterion Criterion : Definition consistent with the « large » case where T is a positive threshold value, Lin=Lc.X+Lein and Lout=Lc.X+Leout

Use with turbocodes Idea : use the cross entropy criterion to advantage good sets of extrinsic values and penalize bad sets. We set: (Lein)step+1= (Lein)step with

Simulation results Details : Frame length 2000 bits, ARP permutation model, SUBMAP decoding Comparison cross entropy scaling vs. Fixed scaling (8 iterations) Comparison cross entropy scaling (6 iterations) vs. Fixed scaling (6-8 iterations)

Further axes of research Summary : Implementation issues Adaptation to duo-binary turbocodes Other types of codes : LDPC, product codes Using gaussian mixtures for likelihood representation More complex correction : Chebyshev interpolation - Use of cross entropy principle in turbocode systems - Derivation of simplified cross entropy formula - Improvements in terms of speed of convergence and performance in the waterfall area

Thank you for your attention !