Information theory Multi-user information theory A.J. Han Vinck Essen, 2004

content Some examples of channels Additive coding for the broadcasting Superposition coding for multi-access Coding for the two-way channel Coding for the switching channel Some more

Goal of the lectures: Introduction of some classical models two-way; two access; broadcast; Problems connected: calculation and formulation of capacity Development of coding strategies

Time Sharing (TDMA) User 1 User 2 User 3 Time sharing: easy to organize inefficient if not many users active efficiency depends on channel message idle Common channel

Two-way X1X2 Y1Y2 X1 and X2 communicate by observing Y1 and Y2 R1= I(X1;Y2|X2) R2= I(X2;Y1|X1) Maximize (R1,R2) over any input distribution P(X1,X2) I(X1;Y2|X2) := H(X1|X2)-H(X1|X2,Y2)

Note: I(X1;Y2|X2) := H(X1|X2)-H(X1|X2,Y2) H(X1|X2) = minimum average # bits needed to specify X1 given X2 H(X1|X2,Y2) = minimum average # bits needed to specify X1 given X2 and the observation Y2 Difference = what we learned from the transmission over the channel = the reduction in average specification length of X1|X2

Example: AND channel X1 X2 Y X1 01 X Y When X1 = 0, he does not know X2 X1 = 1, he knows X2 Same for X2

A coding example X X if y = 0 Transmit inverse (red) If y = 0 Inputs are known Rate: 1/( 2*3/4 + 1*¼) = 4/7 = 0.57 > 1 !! X1 X2 Y

Another coding example X X Rate: log 2 3/( 2*3/9 + 3 * 6/9 ) =.59 > 1 !! X1 X2 Y

dependent inputs X1 and X2 P(X1=0, X2=0) = 0 P(X1=0, X2=1) = P(X1=1, X2=0) = p P(X1=1,X2=1) = 1-2p Then, P(X1=1) = P(X1=1, X2=0) + P(X1=1, X2=1) = 1-p. R2 = R1 = I(X2;Y|X1) = I(X1;Y|X2) = H(Y|X1) = (1-p)h(p/(1-p)) The maximum = p 1 p 1-2p

Note: P(Y=0|X1=1) = P(Y=0, X1=1)/P(X1=1) = p/(1-p) P(Y=1|X1=1) = P(Y=1, X1=1)/P(X1=1) = (1-2p)/(1-p)

A lower bound Let X1 and X2 transmit independently P(X1 = 1) = 1 – P(X1=0) = a P(X2 = 1) = 1 – P(X2=0) = a Then: R1 = I(X1;Y|X2) = H(Y|X2) – H(Y|X1,X2) = ah(a) = R2 The maximum = > 4/7 X1 X2 Y

The upper (outer) bound inner The inner bound ( for independent transmission) is outer < the Shannon outer bound ( X1 and X2 dependent). For R1 = R2inner rate = outer rate  The exact capacity is unknown! X1 X2 Y

bounds R1 1 1 R2 Outer bound inner bound 0 X1 X2 Y

Broadcast X Z Y Z transmits information to X same information to Y R1  I( X; Z ) R2  I( Y; Z ) R1 + R2  I ( Z; (X,Y)) = I( Z; X)

broadcast X Z Y Z transmits information to X different information to Y R1  I( X; Z ) R2  I( Y; Z ) R1 + R2  I( Z; (X,Y) ) = I( Z; X) + I(Z;Y|X)

example: Blackwell BC ZXY ZXY R1  I( X; Z ) = H(X)-H(X|Z) R2  I( Y; Z ) =H(Y) –H(Y|Z) R1 + R2  I( Z; (X,Y))  log 2 3 X Z Y

example Y 00/11 01/ X Z Z X Y I(Y;Z) = 1 I(X;Z) = log 2 3 R sum = (1+ log 2 3)/2 = 1.29 bit/tr. X Z Y

2-access channel X1 X2 Y X1 and X2 want to communicate with Y at the same time! Obvious bound on the sum rate: R1+R2  H(Y) – H(Y|X1,X2)  H(Y)

Two-access models Switchingtwo-Adder x1 x2y x1 x2 y 00   Y Y X1 X2 X1 X2 y y  

Two-adder (1) Capacity region X X2 1 X1 X2 Y R1  1 from X1 to Y R2  1 from X2 to Y R1+R2  H(Y)  R2 R timesharing

Two-adder (2) Coding strategy:  -error User 1: transmit at rate R = 1 bit, i.e. P(X1 = 0) = ½ User 2: sees erasure channel. 0 User Max H(X)-H(X|Y) = ½ Hence: rate pair (R1,R2) = ( 1, ½ ) X1 X2 Y

Two-adder (3) A simple a-symmetric strategy: 0-error X1 X Efficiency = 1/3 log 2 14 = 1.27 X1 X2 Y

Two-adder with feedback (1) Question: can we enlarge the capacity region? Yes (Wolf)! Is this a surprise? R R2 Capacity region characterization: modified by Cover-Leung Willems X1 X2 Y

Two-adder with feedback (2) X1 X2 N independent transmissions uncertainty Solve uncertainty insteps Total efficiency: = 1.52! Joint output selection X1 X2 Y

Switching channel (1) X101 X20  0 1  1 Y X2={0,1} X1={0,1} { ,0,1} tri-state logic P(  \ pass info ) = ( 1-a, a ) R sum (max) = a + h(a)  log 2 3

simple coding example (2) For n = 2: code 1: code     1 Sum rate: R sum = (log )/2 = 1.3 X2={0,1} X1={0,1} { ,0,1}

Switching channel with general coding (3) Strategy: code 1 ( ) < d min zeros Linear code 2 ( ) k  n – d min + 1 receive: (  1 ...  0  ) correct d min - 1 erasures = C!! PERFORMANCE: = C!! n

Extensions (2) T-user ADDER + {0,1} {0,1,, T}

T-user ADDER + (1) Input output User 1{ 00 11} { } User 2 { 10 01} User 3 { 00 10} Efficiency: 3 * ½ = 1.5

T-user ADDER + (2) Input User 1 { } User 2{ } User 3{ } Outputs: { etc. } Efficiency = ( 2 + log 2 6 )/3 = 1.53 bits/tr. record: by van Tilborg (1991)