1. Agenda On Quiz 2 Network Information Theory
Typical Networked Communication Scenarios
Gaussian Networked Communication Channels
Joint Typical Sequences

2. Network Communication Theory
Point-to-point Information Theory
Lossless data compression
Channel capacity
Lossy data communication, rate-distortion
Information Theory for Communication Network
Multiple point communication
Data flow in the network

3. Typical Communication Topology
Multiple Access Communication System
Broadcast Communication System
Relay Communication System
Transmitter
Transmitter 1
Transmitter 2
Relay
Transmitter
Receiver
Receiver
Receiver 1
Receiver 2

4. Typical Communication Topology
Communication Network
Max-Flow Min-Cut Theorem
C = min{C1+C2, C1-C3+C5, C2+C3+C4, C4+C5} C1 C4 C3 C2 C5

5. Gaussian Multi-user Channels
Gaussian Single User Channel
Y = X + Z, Z ~ Gaussian(0, N)
C = (1/2)log(1 + P/N)
Gaussian Multi-access Channel with M Users
Y = Σ1≤m≤MXm + Z, Z ~ Gaussian(0, N)
Capacity Region: Σm \in SRm ≤ (1/2)log(1 + |S|P/N)
Two users?
Three users?

6. Gaussian Multi-user Channels
Gaussian Broadcast Channels
Y1 = X + Z1, Z1 ~ Gaussian(0, N1)
Y2 = X + Z2, Z2 ~ Gaussian(0, N2)
User Rate Region: (R1, R2), for 0 ≤ α ≤ 1
R1 ≤ C(αP/N1), R2 ≤ C((1-α)P/(αP+N2))

7. Gaussian Interference Channel
Y1 = X1 + aX2 + Z1, Z1 ~ Gaussian(0, N)
Y2 = X2 + aX1 + Z2, Z2 ~ Gaussian(0, N)
Strong Interference = No Interference;
What is a >> 1? X1 Y1 a a X2 Y2

8. Joint Typical Sequence
Communication Network Signal Vector
(X1, X2, …, XN)
Let S Denote any Order Sets of (X1, X2, …, XN)
S = (X1, X2), S = (X1, XN), S = (X1, X3, XN-1),
S1, S2, …, SN be realizations of S
-(1/n)log p(S1, S2, …, SN)  H(S)
Typical: |-(1/n)log p(S1, S2, …, SN) - H(S)| < ε
All 2N-1 choices of S

9. Joint Typical Sequence
A(n)ε(S): the ε-typical sequence w.r.t. S
For sufficiently large n, we have
Pr(A(n)ε(S)) > 1 – ε
For s in A(n)ε(S), we have p(s) ≈ 2-nH(s)
|A(n)ε(S)| ≈ 2nH(s)
For s1, s2 in A(n)ε(S), we have p(s1|s2) ≈ 2-nH(s1|s2)

10. Joint Typical Sequence
A(n)ε(S1|s2): set of S1 sequence jointly typical with s2 sequence
For sufficiently large n, we have
|A(n)ε(S1|s2)| ≤ 2n(H(S1|S2) + 2ε)
(1-ε)2n(H(S1|S2)-2ε) ≤ Σs2p(s2)|A(n)ε(S1|s2)|

11. Homework
15.2, 15.4, 15.6, 15.23, 15.25, 15.33