1. General Strong Polarization
Madhu Sudan
Harvard University
Joint work with Jaroslaw Blasiok (Harvard), Venkatesan Gurswami (CMU), Preetum Nakkiran (Harvard) and Atri Rudra (Buffalo)
August 11, 2017
IITB: General Strong Polarization

2. Shannon and Channel Capacity
Shannon [‘48]: For every (noisy) channel of communication, ∃ capacity 𝐶 s.t. communication at rate 𝑅<𝐶 is possible and 𝑅>𝐶 is not.
Needs lot of formalization – omitted.
Example:
Acts independently on bits
Capacity =1 – 𝐻(𝑝) ; 𝐻(𝑝) = binary entropy!
𝐻 𝑝 =𝑝⋅ log 1 𝑝 + 1−𝑝 ⋅ log 1 1−𝑝
𝑋∈ 𝔽 2
𝑋 w.p. 1−𝑝
1−𝑋 w.p. 𝑝
𝐵𝑆𝐶(𝑝)
August 11, 2017
IITB: General Strong Polarization

3. “Achieving” Channel Capacity
Commonly used phrase. Many mathematical interpretations.
Some possibilities:
How small can 𝑛 be?
Get 𝑅>𝐶−𝜖 with polytime algorithms?
[Luby et al.’95] Make running time poly 1 𝜖 ?
Open till 2008
Arikan’08: Invented “Polar Codes” …
Final resolution: Guruswami+Xia’13, Hassani+Alishahi+Urbanke’13 – Strong analysis
Shannon ‘48 : 𝑛=𝑂 𝐶−𝑅 −2
Forney ‘66 :time =𝑝𝑜𝑙𝑦 𝑛, 𝜖 2
August 11, 2017
IITB: General Strong Polarization

4. Context for today’s talk
[Arikan’08]: General class of codes.
[GX’13, HAU’13]: One specific code within.
Why?
Bottleneck: Strong Polarization
Arikan et al.: General class polarizes
[GX,HAU]: Specific code polarizes strongly!
This work/talk:
Introduce Local Polarization
Local ⇒ Strong Polarization
Thm: All known codes that polarize also polarize strongly!
August 11, 2017
IITB: General Strong Polarization

5. IITB: General Strong Polarization
This talk:
What are Polar Codes
“Local vs. Global” (Strong) Polarization
Analysis of Polarization
August 11, 2017
IITB: General Strong Polarization

6. Lesson 0: Compression ⇒ Coding
Linear Compression Scheme:
𝐻,𝐷 for compression scheme for bern 𝑝 𝑛 if
Linear map 𝐻: 𝔽 2 𝑛 → 𝔽 2 𝑚
Pr 𝑍∼bern 𝑝 𝑛 𝐷 𝐻⋅𝑍 ≠𝑍 ≤𝜖
Hopefully 𝑚 𝑛 →𝐻(𝑝)
Compression ⇒ Coding
Let 𝐻 ⊥ be such that 𝐻⋅𝐻 ⊥ =0;
Encoder: 𝑋↦ 𝐻 ⊥ ⋅𝑋
Error-Corrector: 𝑌= 𝐻 ⊥ ⋅𝑋+𝜂↦𝑌−𝐷 𝐻⋅𝑌
= 𝑤.𝑝. 1−𝜖 𝐻 ⊥ ⋅𝑋
August 11, 2017
IITB: General Strong Polarization

7. Question: How to compress?
Arikan’s key idea:
Start with 2×2 “Polarization Transform”:
𝑈,𝑉 → 𝑈+𝑉,𝑉
Invertible – so does nothing?
If 𝑈,𝑉 independent,
then 𝑈+𝑉 “more random” than either
𝑉 | 𝑈+𝑉 “less random” than either
Iterate (ignoring conditioning)
End with bits that are completely random, or completed determined (by others).
Output “totally random part” to get compression!
August 11, 2017
IITB: General Strong Polarization

8. Information Theory Basics
Entropy:
Defn: for random variable 𝑋∈Ω,
𝐻 𝑋 = 𝑥∈Ω 𝑝 𝑥 log 1 𝑝 𝑥 where 𝑝 𝑥 = Pr 𝑋=𝑥
Bounds: 0≤𝐻 𝑋 ≤ log Ω
Conditional Entropy
Defn: for jointly distributed 𝑋,𝑌
𝐻 𝑋 𝑌 = 𝑦 𝑝 𝑦 𝐻(𝑋|𝑌=𝑦)
Chain Rule: 𝐻 𝑋,𝑌 =𝐻 𝑋 +𝐻(𝑌|𝑋)
Conditioning does not increase entropy: 𝐻 𝑋 𝑌 ≤𝐻(𝑋)
Equality ⇔ Independence: 𝐻 𝑋 𝑌 =𝐻 𝑋 ⇔ 𝑋⊥𝑌
August 11, 2017
IITB: General Strong Polarization

9. Iteration by Example: 𝐴,𝐵,𝐶,𝐷 → 𝐴+𝐵+𝐶+𝐷,𝐵+𝐷,𝐶+𝐷, 𝐷
𝐸,𝐹,𝐺,𝐻 → 𝐸+𝐹+𝐺+𝐻,𝐹+𝐻,𝐺+𝐻,𝐻
𝐵+𝐷
𝐹+𝐻
𝐵+𝐷+𝐹+𝐻
𝐻 𝐵+𝐷 𝐴+𝐵+𝐶+𝐷)
𝐻 𝐵+𝐷+𝐹+𝐻| 𝐴+𝐵+𝐶+𝐷, 𝐸+𝐹+𝐺+𝐻
𝐻 𝐹+𝐻| 𝐴+𝐵+𝐶+𝐷, 𝐸+𝐹+𝐺+𝐻. 𝐵+𝐷+𝐹+𝐻
𝐻 𝐹+𝐻 𝐸+𝐹+𝐺+𝐻)
August 11, 2017
IITB: General Strong Polarization

10. IITB: General Strong Polarization
Algorithmics
Summary: Iterating 𝑡 times yields 𝑛×𝑛-polarizing transform for 𝑛= 2 𝑡 :
(𝑍 1 ,…, 𝑍 𝑛 )→( 𝑊 1 ,…, 𝑊 𝑛 ) → 𝑊 𝑆 ( 𝑆 ≈𝐻 𝑝 ⋅𝑛)
Encoding/Compression: Clearly poly time.
Decoding/Decompression:
“Successive cancellation decoder”: implementable in poly time.
Can compute Pr 𝑍∼bern(𝑝) 𝑊 𝑖 =1 | 𝑊 <𝑖 recursively
Can reduce both run times to 𝑂(𝑛 log 𝑛)
Only barrier: Determining 𝑆
Resolved by [Tal-Vardy’13]
August 11, 2017
IITB: General Strong Polarization

11. Analysis: The Polarization martingale
Martingale: Sequence of r.v., 𝑋 0 , 𝑋 1 ,…
∀𝑖, 𝑥 0 ,…, 𝑥 𝑖−1 , 𝔼 𝑋 𝑖 | 𝑥 0 ,…, 𝑥 𝑖−1 = 𝑥 𝑖−1
Polarization martingale:
𝑋 𝑖 = Conditional entropy of random output at 𝑖th stage.
Why martingale?
At 𝑖th stage two inputs take (𝑈,𝐴); (𝑉,𝐵)
Output = (𝑈+𝑉, A∪𝐵) and 𝑉, 𝐴∪𝐵∪𝑈+𝑉
Input entropies = 𝑥 𝑖−1 (𝐻 𝑈 𝐴 =𝐻 𝑉 𝐵 = 𝑥 𝑖−1 )
⇒ Output entropies average to 𝑥 𝑖−1
𝐻 𝑈+𝑉 𝐴,𝐵 +𝐻 𝑉 𝐴,𝐵,𝑈+𝑉 =2 𝑥 𝑖−1
August 11, 2017
IITB: General Strong Polarization

12. IITB: General Strong Polarization
Quantitative issues
How “quickly” does this martingale converge? (𝜆)
How “well” does it converge? 𝜖
Formally, want:
Pr 𝑋 𝑡 ∈ 𝜆,1−𝜆 ≤𝜖
𝜖: Gives gap to capacity
𝜆: Governs decoding failure ≈𝑛⋅ 𝜆
Reminder: 𝑛= 2 𝑡
For useable code, need: 𝜆≪ 4 −𝑡
For poly 1 𝜖 block length, need: 𝜖≤ 𝛼 𝑡 , 𝛼<1
August 11, 2017
IITB: General Strong Polarization

13. Regular and Strong Polarization
(Regular) Polarization:
∀𝛾>0, Lim 𝑡→∞ Pr 𝑋 𝑡 ∈ 𝛾 𝑡 ,1− 𝛾 𝑡 →0.
Strong Polarization:
∀𝛾>0 ∃𝛼>0 ∀𝑡, Pr 𝑋 𝑡 ∈ 𝛾 𝑡 , 1− 𝛾 𝑡 ≤ 𝛼 𝑡
Both “global” properties (large 𝑡 behavior)
Construction yields: Local Property
How does 𝑋 𝑖 relate to 𝑋 𝑖−1 ?
Local vs. Global Relationship?
Regular polarization well understood.
Strong understood only 𝑈,𝑉 ↦(𝑈+𝑉,𝑉)
What about 𝑈,𝑉,𝑊 ↦ 𝑈+𝑉,𝑉,𝑊 ?, more generally?
August 11, 2017
IITB: General Strong Polarization

14. IITB: General Strong Polarization
General Polarization
Basic underlying ingredient 𝑀∈ 𝔽 2 𝑘×𝑘
Local polarization step: Pick 𝑈 𝑗 , 𝐴 𝑗 𝑗∈[𝑘] i.i.d.
One step: generate 𝐿 1 ,…, 𝐿 𝑘 =𝑀⋅ 𝑈 1 ,…, 𝑈 𝑘
𝑋 𝑖−1 =𝐻 𝑈 𝑗 | ∪ ℓ 𝐴 ℓ , 𝑈 <𝑗 =𝐻 𝑈 𝑗 | 𝐴 𝑗
𝑋 𝑖 =𝐻 𝐿 𝑗 | ∪ ℓ 𝐴 ℓ , 𝐿 <𝑗
Examples
𝐻 2 = ; 𝐻 3 ′ =
Which matrices polarize? Strongly?
August 11, 2017
IITB: General Strong Polarization

15. IITB: General Strong Polarization
Mixing matrices
Observation 1: Identity matrix does not polarize.
Observation 1’: Upper triangular matrix does not polarize.
Observation 2: (Row) permutation of non-polarizing matrix does not polarize!
Corresponds to permuting 𝑈 1 ,…, 𝑈 𝑘
Definition: 𝑀 mixing if no row permutation is upper-triangular.
[KSU]: 𝑀 mixing ⇒ martingale polarizes.
[Our main theorem]: 𝑀 mixing ⇒ martingale polarizes strongly.
August 11, 2017
IITB: General Strong Polarization

16. Key Concept: Local (Strong) Polarization
Martingale 𝑋 0 , 𝑋 1 ,… locally polarizes if it exhibits
Variance in the middle:
∀𝜏>0 ∃𝜎>0, Var 𝑋 𝑖 𝑋 𝑖−1 ∈ 𝜏,1−𝜏 >𝜎
Suction at the end:
∃𝜃>0 ∀𝑐<∞, ∃𝜏>0 Pr 𝑋 𝑖 < 𝑋 𝑖−1 𝑐 | 𝑋 𝑖−1 ≤𝜏 ≥𝜃
(similarly with 1 − 𝑋 𝑖 )
Definition local: compares 𝑋 𝑖 with 𝑋 𝑖−1
Implications global:
Thm 1: Local Polarization ⇒ Strong Polarization.
Thm 2: Mixing matrices ⇒ Local Polarization.
August 11, 2017
IITB: General Strong Polarization

17. Mixing Matrices ⇒ Local Polarization
2×2 case:
Ignoring conditioning, we have
𝐻 𝑝 → 𝐻 2𝑝 1−𝑝 w.p 𝐻 𝑝 −𝐻 2𝑝 1−𝑝 w.p
Variance, Suction ⇐ Taylor series for log . …
𝑘×𝑘 case:
Reduction to 2×2 case: probs. →∼ 1 𝑘
August 11, 2017
IITB: General Strong Polarization

18. Local ⇒ (Global) Strong Polarization
Step 1: Medium Polarization:
∃𝛼,𝛽<1 Pr 𝑋 𝑡 ∈ 𝛽 𝑡 ,1− 𝛽 𝑡 ≤ 𝛼 𝑡
Proof: Potential Φ 𝑡 =𝔼 min 𝑋 𝑡 , 1− 𝑋 𝑡
Variance+Suction ⇒ ∃𝜂<1 s.t. Φ 𝑡 ≤𝜂⋅ 𝜙 𝑡−1
(Aside: Why 𝑋 𝑡 ? Clearly 𝑋 𝑡 won’t work. And 𝑋 𝑡 2 increases!)
⇒ Φ 𝑡 ≤ 𝜂 𝑡 ⇒ Pr 𝑋 𝑡 ∈ 𝛽 𝑡 ,1− 𝛽 𝑡 ≤ 𝜂 𝛽 𝑡
August 11, 2017
IITB: General Strong Polarization

19. Local ⇒ (Global) Strong Polarization
Step 1: Medium Polarization:
∃𝛼,𝛽<1 Pr 𝑋 𝑡 ∈ 𝛽 𝑡 ,1− 𝛽 𝑡 ≤ 𝛼 𝑡
Step 2: Medium Polarization + 𝑡-more steps ⇒ Strong Polarization:
Proof: Assume 𝑋 0 ≤ 𝛼 𝑡 ; Want 𝑋 𝑡 ≤ 𝛾 𝑡 w.p. 1 −𝑂 𝛼 1 𝑡
Pick 𝑐 large & 𝜏 s.t. Pr 𝑋 𝑖 < 𝑋 𝑖−1 𝑐 | 𝑋 𝑖−1 ≤𝜏 ≥𝜃
2.1: Pr ∃𝑖 𝑠.𝑡. 𝑋 𝑖 ≥𝜏 ≤ 𝜏 −1 ⋅ 𝛼 𝑡 (Doob’s Inequality)
2.2: Let 𝑌 𝑖 = log 𝑋 𝑖 ; Assuming 𝑋 𝑖 ≤𝜏 for all 𝑖
𝑌 𝑖 − 𝑌 𝑖−1 ≤− log 𝑐 w.p. ≥𝜃; & ≥𝑘 w.p. ≤ 2 −𝑘 .
𝔼xp 𝑌 𝑡 ≤−𝑡 log 𝑐 𝜃 +1 ≤2𝑡⋅ log 𝛾
Concentration! Pr 𝑌 𝑡 ≥𝑡⋅ log 𝛾 ≤ 𝛼 2 𝑡 QED!
August 11, 2017
IITB: General Strong Polarization

20. IITB: General Strong Polarization
Conclusion
Simple General Analysis of Strong Polarization
But main reason to study general polarization:
Maybe some matrices polarize even faster!
This analysis does not get us there.
But hopefully following the (simpler) steps in specific cases can lead to improvements.
August 11, 2017
IITB: General Strong Polarization

21. IITB: General Strong Polarization
Thank You!
August 11, 2017
IITB: General Strong Polarization