Towards ideal codes: looking for new turbo code schemes Ph.D student: D. Kbaier Ben Ismail Supervisor: C. Douillard Co-supervisor: S. Kerouédan

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 1/44 What is a good code? Ph.D defense Monday 26 th September 2011 Extract from «Codes and Turbo Codes» Under the direction of Claude Berrou Dilemma: good convergence versus high Minimum Hamming Distance Good convergence High asymptotic gain Ideal system Limits to the correction capability of any code Established by Shannon ( ) Asymptotic gain

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 2/44 Turbo codes: a breakthrough in digital communications How to combat the floor while keeping a good convergence? Turbo codes (TCs): various communication standards (-) High floors of errors Lower error rates are required for real-time & demanding applications 3D TCs [1] Irregular TCs [2] Asymmetric turbo codes with different RSC encoders Devising more sophisticated internal permutations Component encoders with a large number of states Different types of concatenation: serial, hybrid, multiple… [1] C. Berrou, A. Graell i Amat, Y. Ould-Cheikh-Mouhamedou, C. Douillard, and Y. Saouter, “Adding a rate-1 third dimension to turbo codes,” in Proc. IEEE Inform. Theory Workshop, Lake Tahoe, CA, Sep. 2007, pp. 156–161. [2] B. Frey and D. MacKay. Irregular turbocodes. In Proc. 37th Allerton Conference on Communication, Control and Computing, Illinois, page 121, September 1999.

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 3/44 Outline Introduction 3-Dimensional turbo codes (3D TCs) 3D coding scheme Parameters: post-encoder, Π’ and λ Improving the asymptotic performance Improving the convergence threshold Irregular turbo codes Conclusion

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 4/44 Ph.D defense Monday 26 th September 2011 The added part is placed just behind the pre-existing turbo encoder λ =1/4  {1000} 3D coding scheme: encoding structure Π dataRSC 1 X Y1Y1 Y2Y2 λ Y 1 λ Y 2 P/S Π’ Post Encoder (1-λ) Y 1 (1-λ) Y 2 W PUNCTURINGPUNCTURING RSC 2 Classical turbo encoder Parameters: Permeability rate λ Post-encoder Permutation Π’ C. Berrou, A. Graell i Amat, Y. Ould-Cheikh-Mouhamedou, C. Douillard, and Y. Saouter, “Adding a rate-1 third dimension to turbo codes,” in Proc. IEEE Inform. Theory Workshop, Lake Tahoe, CA, Sep. 2007, pp. 156–161

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 5/44 Ph.D defense Monday 26 th September 2011 Choice of the post-encoder Influences performance in the waterfall and error floor region Must be simple  low memory RSC codes The code is made tail biting  accumulator Must not exhibit too much error amplification Our contribution: EXIT analysis

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 6/44 Post encoders EXIT analysis k = 570 bits λ = 1/4 R = 1/3 Max-Log-MAP 10 iterations AWGN channel

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 7/44 Ph.D defense Monday 26 th September 2011 Permutation Π’ Role? A "composite" input weight 4 square error pattern Weight of the codeword: d=28 Puncturing to R=1/2  d=16 Role of the 3D part: A few 1s of the redundancy part of the error pattern will be moved away to each other Produce a significant of additional 1s Increasing the total codeword weight Importance of the spread Regular permutation i=Π’(j)=(P 0 j+i 0 ) mod P P 0 =sqrt(2P) i 0 ~P 0 /2

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 8/44 Ph.D defense Monday 26 th September 2011 Choice of the permeability rate λ Convergence loss / required d min trade-off A large value of λ :(+) a higher d min (-) convergence FER / BER E b /N 0 (dB) R 1, λ 1 R 1, λ 2 > λ 1

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 9/44 Performance of 3GPP2 based 3D TCs All simulations use the MAP algorithm with 10 decoding iterations k = 570 R = 4/5 d min = 4 k = 3066 R= 1/3 d min = 23 d min = 38 d min <= 43

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 10/44 Improving the asymptotic performance of 3D TCs: optimization method All-zero iterative decoding algorithm [3]  determine low weight codewords & estimate multiplicity First terms : low multiplicity … …… … xxxxxxxxxxxxxxxx Y1 Y2 Regular pattern λ = 1/4 Systematic part Parity y Parity w Low weight codeword [3] R. Garello and A. Casado, “The All-Zero Iterative Decoding Algorithm for Turbo Code Minimum Distance Computaion," IEEE International Conference on Communications, pp. 361–364, June 2004.

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 11/44 Improving the asymptotic performance of 3D TCs: optimization method All-zero iterative decoding algorithm [3]  determine low weight codewords & estimate multiplicity First terms : low multiplicity Pattern of post-encoding: not regular any more … …… … Systematic part Parity y Parity w Low weight codeword Non regular pattern

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 12/44 Ph.D defense Monday 26 th September 2011 Optimization results for k = 1146 data bits k = 1146 R = 2/3 λ = 1/4 Distance Multiplicity13≥1≥2 Address 1 Address 5 Address 9 Address 13 xxxxxxxxxxxxxxxx Y1 Y2 Ones concentrated in the systematic part The new minimum distance of the optimized 3D TC is 33 (compared to 7)

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 13/44 Ph.D defense Monday 26 th September 2011 Assessment: optimization method Yes! Optimization method applicable for any family of TCs Provided that the distance spectrum has low multiplicities at the beginning For the 3GPP2: Tail bits  singular points in the trellis Tail bits  cause the codewords to be truncated But the method “cannot” be applied with the WiMAX permutation (ARP) Periodic distribution of the bits High codewords multiplicity Tail biting termination  better distances Can we generalize? A slight irregular post-encoding pattern  improvement in the distance properties Optimistic results  implement the optimization method especially for high coding rates

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 14/44 Outline Introduction 3-Dimensional turbo codes (3D TCs) 3D coding scheme Parameters: post-encoder, Π’ and λ Improving the asymptotic performance Improving the convergence threshold Irregular turbo codes Conclusion

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 15/44 Improving the convergence threshold of the 3D TC Loss in the convergence threshold (dB) for 3GPP2 3D TCs over AWGN channel: R = 1/3R = 1/2R = 2/3R = 4/5 λ =1/ λ =1/ Reducing the convergence loss of 3D TCs: Costello [4]  Time Varying (TV) post-encoder Specific Gray mapping for 3D TCs associated with high order constellations 0.19 Rayleigh channel λ R [4] D. Costello Jr. Free distance bounds for convolutional codes. IEEE Transactions on Information Theory, 20(3): , May 1974.

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 16/44 Input 4 7 Recursivity : polynomial 5 4-state post-encoder with time-varying parity construction (5, 4:7) Convergence/distance trade-off Reducing the convergence loss of 3D TCs: time varying post encoder 4-state post-encoder with time-varying parity construction (5, 4:7) Replace periodically some redundancies W 1 =4 by W 2 =7 BER out = 2* (BER in +ξ) (5,4:7)  Distance = 2 (5,4)  Distance =3 and (5,7)  Distance =5 time W 1 (4) W 2 (7) W 1 (4) W 2 (7) W 1 (4) Replacement period L W 1 (4) W 2 (7) Time varying trellis

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 17/44 General results for the time varying technique Loss of convergence reduced by 10% to 50% of the value expressed in dB The asymptotic performance is not degraded For a fixed code memory, the choice of the post-encoder does not influence d min of the 3D TC Higher local minimum distance of the post-encoder = Better level of the extrinsic information which the predecoder supplies to the two SISO decoders The TV technique acts as a convergence accelerator of the 3D TC

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 18/44 Error rate performance example of time varying 3D TCs k = 1146 bits Loss of convergence reduced by 35% from 0.23 dB to 0.15 dB Max-Log-MAP 10 iterations

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 19/44 3D TCs for high spectral efficiency transmissions BICM approach Among the bits forming a symbol in M-QAM or M-PSK modulations, the average probability of error is not the same for all the bits Three constellation mappings: Configuration 1: mapping uniformly distributed Configuration 2: systematic bits mapped to better protected places as a priority Configuration 3: systematic bits (then if possible) post- encoded parity bits protected as a priority

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 20/44 Example: 3D TCs associated with a 16- QAM modulator Systematic bits & post-encoded parity bits mapped to better protected places QAM symbols 4 bits of a 16-QAM symbol 2298 x 2298 y y w k = 2298 bits R = 1/3 λ = 1/8 Gaussian channel Gain: 0.22

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 21/44 Design rules Configuration 1  loss of convergence still observed Configuration 2 or 3  gain in the waterfall region Configuration 3 must be used as far as possible Otherwise, implement at least the configuration 2 Significant gain: Even for transmissions over Rayleigh fading channels Increases with the coding rate R for the same λ

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 22/44 Properties of 3-Dimensional turbo codes Increase in d min But: Loss in the convergence threshold Increase in complexity -Why? -The answer is in the decoding process

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 23/44 What about the 3D decoding complexity? 8-state SISO DEC1 8-state SISO DEC2 4-state SISO PRE-DEC Π’ -1 S/P Π P/S Π’ w y2y2 Extrinsic information about the post-encoded parity bits Π Π -1 y1y1 Classical Turbo Decoder

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 24/44 Complexity figures High throughputs # Proc increases  additional complexity decreases k = 1530 bits λ = 1/8 R = 1/2

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 25/44 Summary: 3D TCs (1/2) BER/FER E b /N 0 (dB) Classical TC 3D TC Time varying 3D TCs + high order modulations + specific Gray mapping Optimization method Time varying post- encoder (5, 4:7) with a little irregularity Irregularity in the Gray mapping for 3D TCs associated with high order modulations Non regular post- encoding pattern to improve the asymptotic performance

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 26/44 Summary: 3D TCs (2/2) Irregularity The next step of the study concerns the investigation of irregular TCs Why? Obtain an irregular TC which performs well in both the waterfall and the error floor regions  Work on irregular LDPC codes  significant gain  Frey & MacKay introduced irregularity to TCs  Sawaya & Boutros  lower the floor of irregular TCs Time varying post- encoder (5, 4:7) with a little irregularity Irregularity in the Gray mapping for 3D TCs associated with high order modulations Non regular post- encoding pattern to improve the asymptotic performance

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 27/44 Outline Introduction 3-Dimensional turbo codes (3D TCs) Irregular turbo codes Basics of irregular TCs Selecting the degree profile EXIT diagrams Design of suitable permutations for irregular TCs Principle & simulation results Irregular TCs with post-encoding Conclusion

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 28/44 Self-concatenated turbo encoder Equivalent encoding structure for a regular turbo encoder: Merge two trellis encoders  double size interleaver + 2-fold repetition Interest: introduce an irregular structure

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 29/44 Irregular turbo encoder Repetition (d j ) Interleaver RSC Information bits Repetition (d 2 ) Repetition (d max ) k info bits k f2k f2 k f max k fjk fj Degree profile  (2, 3,…, d max ) or (f 2, f 3,…, f max ) Parity bits  Two non-zero fractions: d =2 and d >2 : f 2 + f max =1 2 f 2 + d max f max = d Average  Only three parameters  Performance of an irregular TC strongly depends on the degree profile  Number of degrees and fractions: 2(d max -1)  Only two equations to optimize all these parameters!

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 30/44 What is a good irregular turbo code? Our approach = we separate the problems Π Degree profile RSC code It depends on: 1.Search for a good degree profile using a random interleaver 2.Optimize the interleaver Fixed Our contribution: analyzing the degree profile using hierarchical EXIT charts

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 31/44 Analyzing the degree profile using hierarchical EXIT charts

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 32/44 Performance example of irregular TCs Interleaver length: 3438 d av = 3 R = ¼ MAP 8 iterations k = 1146 bits

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 33/44 Outline Introduction 3-Dimensional turbo codes (3D TCs) Irregular turbo codes Basics of irregular TCs Selecting the degree profile EXIT diagrams Design of suitable permutations for irregular TCs Principle & simulation results Irregular TCs with post-encoding Conclusion

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 34/44 Proposed algorithm for the permutation design (1/2) Reduce the correlation effect between the pilot groups while improving the distance properties of irregular TCs Information sequence: … Appropriate repetition weight Original Address =565 Copy 2 Address =273 weight 0 Copy 3 / Address =120 Interleaver size: 576 The Dijkstra’ s algorithm [5]: [5] E. Dijkstra. A note on two problems in connexion with graphs. Numerische mathematik, 1(1): , 1959.

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 35/44 Proposed algorithm for the permutation design (2/2) Reduce the correlation effect between the pilot groups while improving the distance properties of irregular TCs Information sequence: … Appropriate repetition weight Original Address =565 Copy 2 Address =273 weight 0 Copy 3 / Address =120 Copy 5 Address =356 Copy 4 Address =440 Copy 2 Address =273 Copy 6 Address =189 Copy 6 Address =500 Copy 7 Address =47 Original Address =565 weight = 0 weight = 1 Address =1 In the example: d = 8

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 36/44 Error rate performance of irregular TCs with an optimized interleaver All simulations use the MAP algorithm with 10 decoding iterations R = 1/4 Interleaver size: 144 Gain: 2.5 decades Interleaver size: 576 Gain: 3.5 decades

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 37/44 Error rate performance of irregular TCs with an optimized interleaver Proposed algorithm: very fast for short block sizes For medium sizes and large blocks: Unacceptable computational time Uncertainty about detecting all the possible cases Drawback: Necessity to store all the interleaved addresses Devising good interleavers for irregular TCs proves to be a difficult task All simulations use the MAP algorithm with 8 decoding iterations R = 1/4 Interleaver size: 3438 Gain: > 2 decades CPU: Two quad core processors (Xéon) RAM: 8Go

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 38/44 Outline Introduction 3-Dimensional turbo codes (3D TCs) Irregular turbo codes Basics of irregular TCs Selecting the degree profile EXIT diagrams Design of suitable permutations for irregular TCs Principle & simulation results Irregular TCs with post-encoding Conclusion

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 39/44 Adding a post-encoder to irregular TCs We propose an irregular TC inspired by our work about 3D TCs Ensure large asymptotic gain at very low error rates Even with non optimized internal permutation Improve the distance properties of irregular TCs Non- uniform repetition ΠRSC Information bits Parity bits λ Π’ Post- encoder 1-λ

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 40/44 Performance example of irregular TCs with post-encoding All simulations use the MAP algorithm with 10 decoding iterations Degree profile (f 2,f 8 ), d av = 3, R = 1/4, λ = 1/8 and k = 4096 bits 3GPP2 interleaver, interleaver size: Gain: 2.5 decades d min = 33 d min = 44 d min = 50

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 41/44 Summary: irregular TCs BER/FER E b /N 0 (dB) Classical TC Irregular TC Suitable permutations Irregular TC + Post-encoder

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 42/44 Conclusion Towards ideal codes? 3D TCs Asymptotic performance: The 3D TC significantly improves performance in the error floor region Convergence: We can implement methods which reduce significantly the loss of convergence Irregular TCs Performance: Closer to capacity but very poor asymptotic performance Improve the distance properties: Graph-based permutations (Dijkstra's algorithm + estimation of the minimum distance) Irregular TCs + post-encoder

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 43/44 Perspectives Towards ideal codes? 3D TCs: New structures Diversity techniques: MIMO, rotated constellations… Double binary Hardware implementation complexity of 3D turbo decoder Irregular TCs: Post-encoding pattern The design of suitable permutations for irregular TCs is an important future research work -Eliminate the interleavers producing low minimum distances early in the search process  Reduce the space of search  Promising algorithm even for large blocks

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26 th September 2011 page 44/44 Thank you for your attention Contributions to the literature Conference papers: 1. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Improving 3-dimensional turbo codes using 3GPP2 interleavers", ComNet'09: 1st International Conference on Communications and Networking, November 2009, Hammamet, Tunisia, KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Reducing the convergence loss of 3- dimensional turbo codes", 6th International Symposium on Turbo Codes & Iterative Information Processing, September 2010, France, pp Journal papers: 3. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Analysis of 3-dimensional turbo codes", Annals of Telecommunications, available online at 4. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Design of suitable permutations for irregular turbo codes", Electronics Letters, June 2011, vol. 47, n° 13, pp KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Improving irregular turbo codes", Electronics Letters, to appear. Submitted journal paper: 6. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Improving 3GPP2 3-dimensional turbo codes and aspects of irregular turbo codes", submitted to EURASIP Journal on Wireless Communications and Networking.