1. General Strong Polarization
Madhu Sudan
Harvard University
Based on joint works with Jaroslaw Blasiok (Harvard), Venkatesan Guruswami (CMU), Preetum Nakkiran (Harvard) and Atri Rudra (Buffalo)
March 18-22, 2019
Oberwolfach: General Strong Polarization

2. Addendum: (After Alex’s talk yesterday)
Another talk on Polar Codes.
Emphasis
BSC – “errors”
Focus on asymptotics and theorems!
… and proofs
… hopefully some teachable material
March 18-22, 2019
Oberwolfach: General Strong Polarization

3. Shannon and Channel Capacity
BSC(𝑝):
Acts independently on bits
Capacity =1 –ℎ(𝑝) ; ℎ(𝑝) = binary entropy!
ℎ 𝑝 =𝑝⋅ log 1 𝑝 + 1−𝑝 ⋅ log 1 1−𝑝
This talk: Price of communication at rate 𝑅=𝐶−𝜖
Smallest 𝑛, smallest running times.
𝑋∈ 𝔽 2
𝑋 w.p. 1−𝑝
1−𝑋 w.p. 𝑝
𝐵𝑆𝐶(𝑝)
March 18-22, 2019
Oberwolfach: General Strong Polarization

4. “Achieving” Shannon Capacity
How small can 𝑛 be?
Get 𝑅>𝐶−𝜖 with polytime algorithms?
Problem articulated by [Luby et al.’95]
running time poly 𝑛 𝜖 ?
(equiv. want block length 𝑛=poly 1 𝜖 ?)
Open till 2008
Arikan’08: Invented “Polar Codes” …
Resolution of open question: Guruswami+Xia’13, Hassani+Alishahi+Urbanke’13 – Strong analysis
Shannon ‘48: 𝑛=Θ 1 𝜖 2 ; 𝜖≝𝐶−𝑅
Forney ‘66 :time =𝑝𝑜𝑙𝑦 𝑛, 𝜖 2
March 18-22, 2019
Oberwolfach: General Strong Polarization

5. Polar Codes and Martingales
Arikan: Defined Polar Codes, one for every integer 𝑡
Associated “martingale” 𝑋 0 ,…, 𝑋 𝑡 ,… 𝑋 𝑡 ∈[0,1]
𝑡th code: Length 𝑛= 2 𝑡
If Pr 𝑋 𝑡 ∈ 𝜏 𝑡 ,1− 𝛿 𝑡 ≤ 𝜖 𝑡 then
𝑡th code is 𝜖 𝑡 + 𝛿 𝑡 -close to capacity, and
Pr Decode BSC Encode 𝑚 ≠𝑚 ≤𝑛⋅ 𝜏 𝑡
Need 𝜏 𝑡 =𝑜 1 𝑛 or 𝜏 𝑡 = 1 𝑛 𝜔(1)
Need 𝜖 𝑡 , 𝛿 𝑡 =1/ 𝑛 Ω(1)
Arikan et al. 𝜏=𝑛𝑒𝑔 𝑛 ;𝜖=𝑜 1 ; [GX13,HAU13] ↑
𝑋 0 , 𝑋 1 , …, 𝑋 𝑡 , … form a martingale if
∀𝑡, 𝔼 𝑋 𝑡 𝑋 0 ,…, 𝑋 𝑡−1 ]= 𝑋 𝑡−1
“Strong”
Polarization
March 18-22, 2019
Oberwolfach: General Strong Polarization

6. Part II: Polar Codes Encoding, Decoding, Martingale, Polarization
March 18-22, 2019
Oberwolfach: General Strong Polarization

7. Lesson 0: Compression ⇒ Coding
Defn: Linear Compression Scheme:
𝑀,𝐷 form compression scheme for 𝐵ern 𝑝 𝑛 if
Linear map 𝐻: 𝔽 2 𝑛 → 𝔽 2 𝑚
Pr 𝑍∼𝐵ern 𝑝 𝑛 𝐷 𝐻⋅𝑍 ≠𝑍 =𝑜 1
Want: 𝑚 𝑛 ≤ℎ 𝑝 +𝜖, 𝐷 efficient
Compression ⇒ Coding
Let 𝐺 be such that 𝐻⋅𝐺=0;
Encoder: 𝑋↦𝐺⋅𝑋
Error-Corrector: 𝑌=𝐺⋅𝑋+𝑍↦𝑌−𝐷 𝐻⋅𝑌
=𝑌−𝐷 𝐻⋅𝐺⋅𝑋+𝐻⋅𝑍 = 𝑤.𝑝. 1−𝑜(1) 𝐺⋅𝑋 𝐻 𝐺
March 18-22, 2019
Oberwolfach: General Strong Polarization

8. Question: How to compress?
Arikan’s key idea:
Start with 2×2 “Polarization Transform”:
𝑈,𝑉 → 𝑈+𝑉,𝑉
Invertible – so does nothing?
If 𝑈,𝑉 independent,
then 𝑈+𝑉 “more random” than either
𝑉 | 𝑈+𝑉 “less random” than either
Iterate (ignoring conditioning)
End with bits that are almost random, or almost determined (by others).
Output “random part” to get compression!
March 18-22, 2019
Oberwolfach: General Strong Polarization

9. The Polarization Butterfly
𝑃 𝑛 𝑈,𝑉 = 𝑃 𝑛 2 𝑈+𝑉 , 𝑃 𝑛 2 𝑉
Martingale: 𝑋 𝑡 = 𝐻 𝑍 𝑖,𝑡 𝑍 <𝑖,𝑡
for random 𝑖
𝑍 1
𝑍 2
𝑍 𝑛 2
𝑍 𝑛 2 +1
𝑍 𝑛 2 +2
𝑍 𝑛
𝑍 1 + 𝑍 𝑛 2 +1
𝑍 2 + 𝑍 𝑛 2 +2
𝑍 𝑛 2 + 𝑍 𝑛
𝑍 𝑛 2 +1
𝑍 𝑛 2 +2
𝑍 𝑛
𝑊 1
𝑊 2
𝑊 𝑛 2
𝑊 𝑛 2 +1
𝑊 𝑛 2 +2
𝑊 𝑛
Polarization ⇒∃𝑆⊆ 𝑛
𝐻 𝑊 𝑛 −𝑆 𝑊 𝑆 →0
𝑆 ≤ 𝐻 𝑝 +𝜖 ⋅𝑛 𝑈
𝑍 𝑖,0
𝑍 𝑖,1 … Z 𝑖,𝑡 … 𝑊 𝑖 =𝑍 𝑖, log 𝑛
Compression 𝐸 𝑍 = 𝑃 𝑛 𝑍 𝑆
Encoding time =𝑂 𝑛 log 𝑛
(given 𝑆) 𝑉
To be shown:
1. Decoding = 𝑂 𝑛 log 𝑛
2. Martingale Polarization
March 18-22, 2019
Oberwolfach: General Strong Polarization

10. The Polarization Butterfly: Decoding
Decoding idea:
Given 𝑆, 𝑊 𝑆 = 𝑃 𝑛 𝑈,𝑉
Compute 𝑈+𝑉∼Bern 2𝑝−2 𝑝 2
Determine 𝑉∼ ?
𝑍 1
𝑍 2
𝑍 𝑛 2
𝑍 𝑛 2 +1
𝑍 𝑛 2 +2
𝑍 𝑛
𝑍 1 + 𝑍 𝑛 2 +1
𝑊 1
𝑊 2
𝑊 𝑛 2
𝑊 𝑛 2 +1
𝑊 𝑛 2 +2
𝑊 𝑛
𝑍 2 + 𝑍 𝑛 2 +2
𝑉|𝑈+𝑉∼ 𝑖 Bern( 𝑟 𝑖 )
Key idea: Stronger induction!
- non-identical product distribution
𝑍 𝑛 2 + 𝑍 𝑛
𝑍 𝑛 2 +1
Decoding Algorithm:
Given 𝑆, 𝑊 𝑆 = 𝑃 𝑛 𝑈,𝑉 , 𝑝 1 ,…, 𝑝 𝑛
Compute 𝑞 1 ,.. 𝑞 𝑛 2 ← 𝑝 1 … 𝑝 𝑛
Determine 𝑈+𝑉=𝐷 𝑊 𝑆 + , 𝑞 1 … 𝑞 𝑛 2
Compute 𝑟 1 … 𝑟 𝑛 2 ← 𝑝 1 … 𝑝 𝑛 ;𝑈+𝑉
Determine 𝑉=𝐷 𝑊 𝑆 − , 𝑟 1 … 𝑟 𝑛 2
𝑍 𝑛 2 +2
𝑍 𝑛
March 18-22, 2019
Oberwolfach: General Strong Polarization

11. Part III: Martingales, Polarization, Strong & Local
March 18-22, 2019
Oberwolfach: General Strong Polarization

12. Martingales: Toy examples
𝑋 𝑡+1 = 𝑋 𝑡 + 2 − 𝑡 2 w.p 𝑋 𝑡 − 2 − 𝑡 2 w.p
𝑋 𝑡+1 = 𝑋 𝑡 + 2 −𝑡 w.p 𝑋 𝑡 − 2 −𝑡 w.p
𝑋 𝑡+1 = 𝑋 𝑡 w.p if 𝑋 𝑡 ≤ 𝑋 𝑡 w.p if 𝑋 𝑡 ≤ 1 2
𝑋 𝑡+1 = 𝑋 𝑡 2 w.p if 𝑋 𝑡 ≤ 𝑋 𝑡 − 𝑋 𝑡 2 w.p if 𝑋 𝑡 ≤ 1 2
Converges!
Uniform on [0,1]
Polarizes (weakly)!
Polarizes
(strongly)!
March 18-22, 2019
Oberwolfach: General Strong Polarization

13. Oberwolfach: General Strong Polarization
Arikan Martingale
Issues:
Local behavior – well understood
Challenge: Limiting behavior
March 18-22, 2019
Oberwolfach: General Strong Polarization

14. Main Result: Definition and Theorem
Strong Polarization: (informally)
Pr 𝑋 𝑡 ∈(𝜏,1−𝜏) ≤𝜖 if 𝜏= 2 −𝜔(𝑡) and 𝜖= 2 −𝑂(𝑡)
formally ∀𝛾>0 ∃𝛽<1,𝑐 𝑠.𝑡. ∀𝑡 Pr 𝑋 𝑡 ∈( 𝛾 𝑡 ,1− 𝛾 𝑡 ) ≤𝑐⋅ 𝛽 𝑡
Local Polarization:
Variance in the middle: 𝑋 𝑡 ∈(𝜏,1−𝜏)
∀𝜏>0 ∃𝜎>0 𝑠.𝑡. ∀𝑡, 𝑋 𝑡 ∈ 𝜏,1−𝜏 ⇒𝑉𝑎𝑟 𝑋 𝑡+1 𝑋 𝑡 ≥𝜎
Suction at the ends: 𝑋 𝑡 ∉ 𝜏,1−𝜏
∃𝜃>0, ∀𝑐<∞, ∃𝜏>0 𝑠.𝑡. 𝑋 𝑡 <𝜏⇒ Pr 𝑋 𝑡+1 < 𝑋 𝑡 𝑐 ≥𝜃
Theorem: Local Polarization ⇒ Strong Polarization.
Both definitions qualitative!
“low end” condition. Similar
condition for high end
March 18-22, 2019
Oberwolfach: General Strong Polarization

15. Oberwolfach: General Strong Polarization
Proof (Idea):
Step 1: The potential Φ 𝑡 ≝ min 𝑋 𝑡 , 1− 𝑋 𝑡 decreases by constant factor in expectation in each step.
⇒𝔼 Φ 𝑇 = exp −𝑇 ⇒ Pr 𝑋 𝑇 ≥ exp −𝑇/2 ≤ exp −𝑇/2
Step 2: Next 𝑇 time steps, 𝑋 𝑡 plummets whp
2.1: Say, If 𝑋 𝑡 ≤𝜏 then Pr 𝑋 𝑡+1 ≤ 𝑋 𝑡 ≥1/2.
2.2: Pr ∃𝑡∈ 𝑇,2𝑇 𝑠.𝑡. 𝑋 𝑡 >𝜏 ≤ 𝑋 𝑇 /𝜏 [Doob]
2.3: If above doesn’t happen 𝑋 2𝑇 < 5 −𝑇 whp
QED
March 18-22, 2019
Oberwolfach: General Strong Polarization

16. Local Polarization of Arikan Martingale
Variance in the Middle:
Roughly: 𝐻 𝑝 ,𝐻 𝑝 →(𝐻 2𝑝−2 𝑝 2 , 2𝐻 𝑝 −𝐻 2𝑝−2 𝑝 2 )
𝑝∈ 𝜏,1−𝜏 ⇒2𝑝−2 𝑝 2 far from 𝑝
+ continuity of 𝐻 ⋅ ⇒ 𝐻 2𝑝−2 𝑝 2 far from 𝐻(𝑝)
Suction:
High end:𝐻 1 2 −𝛾 →𝐻 1 2 − 𝛾 2
𝐻 1 2 −𝛾 =1 −Θ 𝛾 2 ⇒1− 𝛾 2 →1 −Θ 𝛾 4
Low end: 𝐻 𝑝 ≈𝑝 log 1 𝑝 ;2𝐻 𝑝 ≈2𝑝 log 1 𝑝 ;
𝐻 2𝑝−2 𝑝 2 ≈𝐻 2𝑝 ≈2𝑝 log 1 2𝑝 ≈2𝐻 𝑝 −2𝑝≈ 2− 1 log 1 𝑝 𝐻 𝑝
Dealing with conditioning – more work (lots of Markov)
March 18-22, 2019
Oberwolfach: General Strong Polarization

17. Oberwolfach: General Strong Polarization
“New contributions”
So far: Reproduced old work (simpler proofs?)
Main new technical contributions:
Strong polarization of general transforms
E.g. 𝑈,𝑉,𝑊 → 𝑈+𝑉,𝑉,𝑊 !
Exponentially strong polarization [BGS’18]
Suction at low end is very strong!
Random 𝑘×𝑘 matrix yields 𝑋 𝑡+1 ≈ 𝑋 𝑡 𝑘 whp
Strong polarization of Markovian sources [GNS’19?]
Separation of compression of known sources from unknown ones.
March 18-22, 2019
Oberwolfach: General Strong Polarization

18. Oberwolfach: General Strong Polarization
Conclusions
Importance of Strong Polarization!
Generality of Strong Polarization!
Some technical questions:
Best poly 1 𝜖 ?
Now turn to worst case errors? (with list-decoding)
March 18-22, 2019
Oberwolfach: General Strong Polarization

19. Oberwolfach: General Strong Polarization
Thank You!
March 18-22, 2019
Oberwolfach: General Strong Polarization