Universal Linked Multiple Access Source Codes Sidharth Jaggi Prof. Michelle Effros

zIntroduction Source Codes (SCs) Models for Source Coding

zIntroduction Source Codes (SCs) Universal SCs, memoryless distributions on, Models for Source Coding R

zIntroduction Source Codes (SCs) Multiple Access SCs (Slepian-Wolf) Models for Source Coding zmemoryless distributions on Y X Z Xavier Yvonne Zorba R X (Q) R Y (Q)

Source Codes (SCs) Multiple Access SCs Models for Source Coding Y X Z R X (Q) R Y (Q)

Source Codes (SCs) Universal SCsMultiple Access SCs Models for Source Coding R Y X Z R X (Q) R Y (Q)

Slepian-Wolf Rate Region R X (Q) R Y (Q)

Source Codes (SCs) Multiple Access SCs Universal MASCs? Models for Source Coding memoryless distributions on

Source Codes (SCs) Universal SCsMultiple Access SCs Universal MASCs? Models for Source Coding memoryless distributions on

Universal MASCs? Let

Universal MASCs?

Source Codes (SCs) Universal SCsMultiple Access SCs Missing Link Linked MASCs Models for Source Coding

Linked MASC (LMASC) Model Y X Z Xavier Yvonne Zorba

(0,0)-LMASCs Y X Z Xavier Yvonne Zorba

RXRX RYRY (0,0)-LMASC Rate Region z(0,0)-LMASC Rate Region = Slepian-Wolf Rate Region

Source Codes (SCs) Universal SCsMultiple Access SCs Linked MASCs Universal LMASCs?

Universal (0,0)-LMASCs Code Y X Z Xavier Yvonne Zorba

Universal (0,0)-LMASCs Code Y X Z Xavier Yvonne Zorba

Results for (0,0)-LMASCs If Example: then zTradeoffs

Y X Z Xavier Yvonne Zorba z LMASCs

Y X Z zAchievable Region zUniversal Coding Possible z LMASCs

Y X Z zAchievable Region zUniversal Coding possible z LMASCs

Y X Z Yvonne Zorba Xavier Algernon A z -encoder LMASCs= -encoder MASC z - encoder LMASCs zUniversal Coding possible

Y X Z Xavier Yvonne Zorba z(0,0)-FMASCs =(0,0)-LMASCs z(,)-FMASCs =(0,0)-LMASCs zUniversal Coding possible zFeedback MASCs

Proof Sketch - Universal LMASCs Y X Z Xavier Yvonne Zorba

Let be the type of Tell Zorba value of in bits. Y X Z Xavier Yvonne Zorba Proof Sketch - Universal LMASCs

Y X Z Xavier Yvonne Zorba Proof Sketch - Universal LMASCs Let be the type of

Proof Sketch - Universal LMASCs

What could possibly go wrong? Estimate “far off” Probability of ErrorRate Redundancy

What could possibly go wrong? Probability of ErrorRedundancy Atypicality Code fails Source mismatch

zProbability of ErrorzRedundancy What could possibly go wrong?

Conclusions X Z zMASC z(0,0)-LMASC Universality Y X Z Y z - LMASC Universality X Z Y

Conclusions X Z z(0,0)-FMASC Y zl -encoder - LMASC Universality X Z Y z - FMASC Complicated diagrams

The bottom line is… It WORKS!

Universal SCs Let, class of memoryless distributions on zPre-designed codes (“Guess”): Code C such that Eg: Csiszár and Körner, zAdaptive Codes (“Estimate”): Code C such that Eg: Lempel-Ziv

Multiple Sources Individual Encoding z Xavier and Yvonne encode using individually optimal strategies Y X Z Xavier Yvonne R2R2 Individual Encoding Rate region R2R2 H(X) H(Y)

Multiple Sources Joint Encoding z Xavier and Yvonne encode together Y X Z Xavier Yvonne R2R2 H(X,Y) Joint Encoding Rate region R2R2 H(X,Y)

Universal (0,0)-LMASCs “Guess-timate”… Let be the type of Tell Zorba value of in bits. Y X Z Xavier Yvonne Zorba

Universal (0,0)-LMASCs Choosing the following rates works Parameters of code Choose a pre-designed Slepian-Wolf code matched to pmf

Sketch of Proof By Sanov’s Theorem, probability of being “far-off” from is “small”

Sketch of Proof Assume Then

Sketch of Proof If Probability of Error

Sketch of Proof … or if Probability of Error

Sketch of Proof … or if code fails for Probability of Error

Sketch of Proof … or if code fails for Probability of Error

Sketch of Proof Expected Rate Overhead Rate Overhead in Code Design for pmf …

Sketch of Proof Expected Rate Overhead … and Source Mismatch. If …

Sketch of Proof Expected Rate Overhead … and Source Mismatch. If …

Sketch of Proof Expected Rate Overhead … and Source Mismatch. … and if …

Results Inter-Encoder Communication Probability of Error Expected Rate Overhead

Other System Models z LMASC Rate Region “Transfer of rate” LMASC Rate Region R X (Q) R Y (Q)

Main Result For any m(n)  satisfying (1), and i.I.D. Sources X and Y, there exists a sequence of encoders and decoder (f n,g n,h n ) such that. zE((r(X n, Y n )) differs from the boundary of the Slepian- wolf region by at most –3|x||y|  (n)log  (n)+n -1 m(n). (For any  (n) > m -½ (n)). zE(Prob(error)) = 2 -o(m(n) ). Further, the rate region for UMASC under the above constraints is identical to that of Slepian-wolf encoding for the same source.  2 (n)

Sketch of Proof Estimate of p(x,y) = p’(x,y) = m -1 (n)  i,j 1(x i =x,y j =y) Define max (x,y) |p(x,y)-p’(x,y)| =  0 By Sanov’s theorem, Pr(  0 >  ) = 2 -o(m(n)D(P ||P)) where D(P * ||P) = min P’  S D(P’||P), S={p’(x,y):max (x,y) |p(x,y)-p’(x,y)| >  } * S P P* D(P * ||P) 

Lemma 1 D(P*||P) = d(p(x,y)+  || p(x,y)) for some particular (x,y) (Lagrange optimization) =  (  2 ) for sufficiently small   D(P * ||P) c 2  2

Lemma 2 1. Max (x,y) |p(x,y)-p’(x,y)| <  (n)  |H P (X,Y)-H P’ (X,Y)|< -|x||y|  (n)log(  (n)) 2. |H p (x,y)-h p’ (x,y)|>   Max (x,y) |p(x,y)-p’(x,y)| >  (|X||Y|) -1 {p(x,y)} H P (X,Y)  

Choice of R(X n,Y m(n) ), R(X m(n),Y n ) 1.Estimate p’(x,y) 2.Choose m(n) and  (n) satisfying Theorem statement 3.Find p’’(x,y) such that 1.max (x,y) |p’(x,y)-p’’(x,y)| =  (n) 2.p’’(x,y) = argmax H P’’ (X,Y) subject to above 4.Encode using a Slepian-Wolf-like codebook for p’’(x,y) {p(x,y)} H P’’ (X,Y)    Probability Entropy 0 0 H P’ (X,Y)H P (X,Y) {p’’(x,y)}{p’(x,y)} A (n) A’ (n)  

Excess Rate over Slepian-Wolf Encoding 1(a) With high probability, max (x,y) |p’(x,y)-p’’(x,y)| < 2  (n) Contribution to excess rate at most –2|X||Y|  (n)log(  (n)) 1(b) If 1(a) not satisfied, contribution to expected excess rate at most 2 -o(m(n) ) log((|X||Y|). Absorb into 1(a) 2. Rate communicated to Zorba to inform him of choice of codebook = n -1 m(n)  2 (n)

Probability of Error 1. Probability of catastrophically incorrect p’(x,y) at most exp(-O(m(n)  2 (n))) 2. Probability of atypical (x n,y n ) at most exp(-O(n  2 (n))) 3. Probability of distinct typical elements decoding to the same codewords at most exp(-O(-n  (n)log  (n))) 1. Dominates over 2. and 3.

THE END