1. Extremal Problems of Information Combining Alexei Ashikhmin  Information Combining: formulation of the problem  Mutual Information Function for the Single Parity Check Codes  More Extremal Problems of Information Combining  Solutions (with the help of Tchebysheff Systems) for the Single Parity Check Codes Joint work with Yibo Jiang, Ralf Koetter, Andrew Singer

2. Encoder Channel APP Decoder Information Transmission Density function of the channel is not known We only know

3. Optimization Problem We assume that and that the channel is symmetric Problem 1 Among all probability distributions such that determine the probability distribution that maximizes (minimizes) the mutual information at the output of the optimal decoder

4. Variable nodes processingCheck nodes processing Interleaver Input from channel From variable nodes To variable nodes Decoder of single parity check code

5. Problem is Solved Already 1. I.Land, P. Hoeher, S.Huettinger, J. Huber, 2003 2. I.Sutskover, S. Shamai, J. Ziv, 2003

6. erasure Repetition code: The Binary Erasure Channel (BEC) is the best The Binary Symmetric Channel (BSC) is the worst Single Parity Check Code: is Dual of Repetition Code The Binary Erasure Channel (BEC) is the worst The Binary Symmetric Channel (BSC) is the best

7. Our Goals  We would like to solve the optimization problem for the Single Parity Check Codes directly (without using duality)  Get some improvements

8. Soft Bits We call soft bit, it has support on Channel

9. erasure

10. Binary symmetric channel, Gaussian Channel:

11. Channel Decoder Single Parity Check Code Encoder Single Parity Check Code E.Sharon, A. Ashikhmin, S. Litsyn Results:

12. Properties of the moments Lemma 1. is nonnegative and nonincreasing 2. The ratio sequence is nonincreasing Lemma In the Binary Erasure Channel all moments are the same

13. Problem 2 Among all T-consistent probability distributions on [0,1] such that determine the probability distribution that maximizes (minimizes) the second moment

14. Solution of Problem 2 Theorem Among all binary-input symmetric-output channel distributions with a fixed mutual information Binary Symmetric Channel maximizes and Binary Erasure Channel minimizes the second moment Proof: We use the theory of Tchebysheff Systems

15. Lemma Binary Symmetric, Binary Erasure and an arbitrary channel with the same mutual information have the following layout of

16. Lemma Let and 1) 2) if for and for then

17. This is exactly our case satisfy conditions of the previous lemma

18. Problem 1 on extremum of mutual information and Problem 2 on extremum of the second moment are equivalent

19. Extrema of MMSE It is known that the channel soft bit is the MMSE estimator fo the channel input Theorem Among all binary-input symmetric-output channels with fixed the Binary Symmetric Channel has the minimum MMSE: and the Binary Erasure Channel has the maximum MMSE: Channel

20. How good the bounds are

21. Problem 3 1) 2) Among all T-consistent channels find that maximizes (minimizes) Channel Decoder Single Parity Check Code Encoder Single Parity Check Code

22. Problem 4 Among all T-consistent probability distributions on [0,1] such that 1) 2) determine the probability distribution that maximizes (minimizes) the fourth moment

23. Theorem The distribution with mass at, mass at and mass at 0 maximizes The distribution with mass at, mass at and mass at 1 minimizes

24. Extremum densities Maximizing Minimizing:

25. Lemma Channel with minimum and maximum and an arbitrary channel with the same mutual information have the followin layout of

26. Problem 3 on extremum of mutual information and Problem 4 on extremum of the fourth moment are equivalent

27. Assume that and is the same as in AWGN channel with this

28. Tchebysheff Systems Definition A set of real continues functions is called Tchebysheff system (T-system) if for any real the linear combination has at most distinct roots at Definition A distribution is a nondecreasing, right-continues function The moment space, defined by ( is the set of valid distributions), is a closed convex cone. For define

29. Problem For a given find that maximizes (minimizes)

30. Theorem If and are T-systems, and then the extrema are attained uniquely with distrtibutions and with finitely many mass points Lower principal representation Upper principal representation

31. Soft Bits We call soft bit, it has support on Lemma (Sharon, Ashikhmin, Litsyn) If then Channel Random variables with this property are called T-consistent

32. Find extrema of Under constrains

33. Theorem Systems and are T-systems on [0,1]. --------------------------------------------------------------------------------- the distribution that maximizes has only one mass point at : has probability mass at and at This is exactly the Binary Symmetric Channel