Agenda On Quiz 2 Network Information Theory Typical Networked Communication Scenarios Gaussian Networked Communication Channels Joint Typical Sequences

Network Communication Theory Point-to-point Information Theory Lossless data compression Channel capacity Lossy data communication, rate-distortion Information Theory for Communication Network Multiple point communication Data flow in the network

Typical Communication Topology Multiple Access Communication System Broadcast Communication System Relay Communication System Transmitter Transmitter 1 Transmitter 2 Relay Transmitter Receiver Receiver Receiver 1 Receiver 2

Typical Communication Topology Communication Network Max-Flow Min-Cut Theorem C = min{C1+C2, C1-C3+C5, C2+C3+C4, C4+C5} C1 C4 C3 C2 C5

Gaussian Multi-user Channels Gaussian Single User Channel Y = X + Z, Z ~ Gaussian(0, N) C = (1/2)log(1 + P/N) Gaussian Multi-access Channel with M Users Y = Σ1≤m≤MXm + Z, Z ~ Gaussian(0, N) Capacity Region: Σm \in SRm ≤ (1/2)log(1 + |S|P/N) Two users? Three users?

Gaussian Multi-user Channels Gaussian Broadcast Channels Y1 = X + Z1, Z1 ~ Gaussian(0, N1) Y2 = X + Z2, Z2 ~ Gaussian(0, N2) User Rate Region: (R1, R2), for 0 ≤ α ≤ 1 R1 ≤ C(αP/N1), R2 ≤ C((1-α)P/(αP+N2))

Gaussian Interference Channel Y1 = X1 + aX2 + Z1, Z1 ~ Gaussian(0, N) Y2 = X2 + aX1 + Z2, Z2 ~ Gaussian(0, N) Strong Interference = No Interference; What is a >> 1? X1 Y1 a a X2 Y2

Joint Typical Sequence Communication Network Signal Vector (X1, X2, …, XN) Let S Denote any Order Sets of (X1, X2, …, XN) S = (X1, X2), S = (X1, XN), S = (X1, X3, XN-1), S1, S2, …, SN be realizations of S -(1/n)log p(S1, S2, …, SN)  H(S) Typical: |-(1/n)log p(S1, S2, …, SN) - H(S)| < ε All 2N-1 choices of S

Joint Typical Sequence A(n)ε(S): the ε-typical sequence w.r.t. S For sufficiently large n, we have Pr(A(n)ε(S)) > 1 – ε For s in A(n)ε(S), we have p(s) ≈ 2-nH(s) |A(n)ε(S)| ≈ 2nH(s) For s1, s2 in A(n)ε(S), we have p(s1|s2) ≈ 2-nH(s1|s2)

Joint Typical Sequence A(n)ε(S1|s2): set of S1 sequence jointly typical with s2 sequence For sufficiently large n, we have |A(n)ε(S1|s2)| ≤ 2n(H(S1|S2) + 2ε) (1-ε)2n(H(S1|S2)-2ε) ≤ Σs2p(s2)|A(n)ε(S1|s2)|

Homework 15.2, 15.4, 15.6, 15.23, 15.25, 15.33