Imperfectly Shared Randomness in Communication Madhu Sudan Harvard Joint work with Clément Canonne (Columbia), Venkatesan Guruswami (CMU) and Raghu Meka (UCLA). 11/16/2016 UofT: ISR in Communication

Classical theory of communication Shannon (1948) Clean architecture for reliable communication. Remarkable mathematical discoveries: Prob. Method, Entropy, (Mutual) Information, Compression schemes, Error-correcting codes. But does Bob really need all of 𝑥? 𝑥 Alice Bob Encoder Decoder 11/16/2016 UofT: ISR in Communication

[Yao’80] Communication Complexity The model (with shared randomness) 𝑥 𝑦 𝑓: 𝑥,𝑦 ↦Σ 𝑅=$$$ Alice Bob 𝐶𝐶 𝑓 =# bits exchanged by best protocol 𝑓(𝑥,𝑦) w.p. 2/3 11/16/2016 UofT: ISR in Communication

Communication Complexity - Motivation Standard: Lower bounds Circuit complexity, Streaming, Data Structures, extended formulations … This talk: Communication complexity as model for communication Food for thought: Shannon vs. Yao? Which is the right model. Typical (human-human/computer-computer) communication involves large context and brief communication. Contexts imperfectly shared. 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication This talk Example problems with low communication complexity. New form of uncertainty and overcoming it. 11/16/2016 UofT: ISR in Communication

Aside: Easy CC Problems Equality testing: 𝐸𝑄 𝑥,𝑦 =1⇔𝑥=𝑦; 𝐶𝐶 𝐸𝑄 =𝑂 1 Hamming distance: 𝐻 𝑘 𝑥,𝑦 =1⇔Δ 𝑥,𝑦 ≤𝑘; 𝐶𝐶 𝐻 𝑘 =𝑂(𝑘 log 𝑘 ) [Huang etal.] Small set intersection: ∩ 𝑘 𝑥,𝑦 =1⇔ wt 𝑥 , wt 𝑦 ≤𝑘 & ∃𝑖 𝑠.𝑡. 𝑥 𝑖 = 𝑦 𝑖 =1. 𝐶𝐶 ∩ 𝑘 =𝑂(𝑘) [Håstad Wigderson] Gap (Real) Inner Product: 𝑥,𝑦∈ ℝ 𝑛 ; 𝑥 2 , 𝑦 2 =1; 𝐺𝐼 𝑃 𝑐,𝑠 𝑥,𝑦 =1 if 𝑥,𝑦 ≥𝜖; =0 if 𝑥,𝑦 ≤0; 𝐶𝐶 𝐺𝐼 𝑃 𝑐,𝑠 =𝑂 1 𝜖 2 ; [Alon, Matias, Szegedy] Protocol: Fix ECC 𝐸: 0,1 𝑛 → 0,1 𝑁 ; Shared randomness: 𝑖← 𝑁 ; Exchange 𝐸 𝑥 𝑖 , 𝐸 𝑦 𝑖 ; Accept iff 𝐸 𝑥 𝑖 =𝐸 𝑦 𝑖 . 𝑝𝑜𝑙𝑦 𝑘 Protocol: Use common randomness to hash 𝑛 →[ 𝑘 2 ] 𝑥=( 𝑥 1 , …, 𝑥 𝑛 ) 𝑦= 𝑦 1 , …, 𝑦 𝑛 〈𝑥,𝑦〉≜ 𝑖 𝑥 𝑖 𝑦 𝑖 Main Insight: If 𝐺←𝑁 0,1 𝑛 , then 𝔼 𝐺,𝑥 ⋅ 𝐺,𝑦 =〈𝑥,𝑦〉 Summary from [Ghazi, Kamath, S.’16] 11/16/2016 UofT: ISR in Communication

Uncertainty in Communication Overarching question: Are there communication mechanisms that can overcome uncertainty? What is uncertainty? This talk: Alice, Bob don’t share randomness perfectly; only approximately. A B f R 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Rest of this talk Model: Imperfectly Shared Randomness Positive results: Coping with imperfectly shared randomness. Negative results: Analyzing weakness of imperfectly shared randomness. 11/16/2016 UofT: ISR in Communication

Model: Imperfectly Shared Randomness Alice ←𝑟 ; and Bob ←𝑠 where 𝑟,𝑠 = i.i.d. sequence of correlated pairs 𝑟 𝑖 , 𝑠 𝑖 𝑖 ; 𝑟 𝑖 , 𝑠 𝑖 ∈{−1,+1};𝔼 𝑟 𝑖 =𝔼 𝑠 𝑖 =0;𝔼 𝑟 𝑖 𝑠 𝑖 =𝜌≥0 . Notation: 𝑖𝑠 𝑟 𝜌 (𝑓) = cc of 𝑓 with 𝜌-correlated bits. 𝑐𝑐(𝑓): Perfectly Shared Randomness cc. 𝑝𝑟𝑖𝑣 𝑓 : cc with PRIVate randomness Starting point: for Boolean functions 𝑓 𝑐𝑐 𝑓 ≤𝑖𝑠 𝑟 𝜌 𝑓 ≤𝑝𝑟𝑖𝑣 𝑓 ≤𝑐𝑐 𝑓 + log 𝑛 What if 𝑐𝑐 𝑓 ≪log 𝑛? E.g. 𝑐𝑐 𝑓 =𝑂(1) =𝑖𝑠 𝑟 1 𝑓 =𝑖𝑠 𝑟 0 𝑓 𝜌≤𝜏⇒ 𝑖𝑠 𝑟 𝜌 𝑓 ≥𝑖𝑠 𝑟 𝜏 𝑓 11/16/2016 UofT: ISR in Communication

Distill Perfect Randomness from ISR? Agreement Distillation: Alice ←𝑟; Bob ←𝑠; 𝑟,𝑠 𝜌-corr. unbiased bits Outputs: Alice →𝑢; Bob →𝑣; 𝐻 ∞ 𝑢 , 𝐻 ∞ 𝑣 ≥𝑘 Communication = 𝑐 bits; What is max. prob. 𝜏 of agreement (𝑢=𝑣)? Well-studied in the literature! [Ahlswede-Csiszar ‘70s]: 𝜏→1⇒𝑐=Ω 𝑘 (one-way) [Bogdanov-Mossel ‘2000s]: 𝑐=0⇒𝜏≤exp(−𝑘) [Our work]: 𝜏= exp(−𝑘+𝑂 𝑐 ) (two-way) Summary: Can’t distill randomness! 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Results Model first studied by [Bavarian,Gavinsky,Ito’14] (“Independently and earlier”). Their focus: Simultaneous Communication; general models of correlation. They show 𝑖𝑠𝑟 Equality =𝑂 1 (among other things) Our Results: Generally: 𝑐𝑐 𝑓 ≤𝑘⇒𝑖𝑠𝑟 𝑓 ≤ 2 𝑘 Converse: ∃𝑓 with 𝑐𝑐 𝑓 ≤𝑘 & 𝑖𝑠𝑟 𝑓 ≥ 2 𝑘 11/16/2016 UofT: ISR in Communication

Equality Testing (our proof) Key idea: Think inner products. Encode 𝑥↦𝑋=𝐸(𝑥);𝑦↦𝑌=𝐸 𝑦 ;𝑋,𝑌∈ −1,+1 𝑁 𝑥=𝑦⇒ 〈𝑋,𝑌〉=𝑁 𝑥≠𝑦⇒ 〈𝑋,𝑌〉≤𝑁/2 Estimating inner products: Building on sketching protocols … Alice: Picks Gaussians 𝐺 1 ,… 𝐺 𝑡 ∈ ℝ 𝑁 , Sends 𝑖∈ 𝑡 maximizing 𝐺 𝑖 ,𝑋 to Bob. Bob: Accepts iff 𝐺′ 𝑖 ,𝑌 ≥0 Analysis: 𝑂 𝜌 (1) bits suffice if 𝐺 ≈ 𝜌 𝐺′ Gaussian Protocol 11/16/2016 UofT: ISR in Communication

General One-Way Communication Idea: All communication ≤ Inner Products (For now: Assume one-way-cc 𝑓 ≤ 𝑘) For each random string 𝑅 Alice’s message = 𝑖 𝑅 ∈ 2 𝑘 Bob’s output = 𝑓 𝑅 ( 𝑖 𝑅 ) where 𝑓 𝑅 : 2 𝑘 → 0,1 W.p. ≥ 2 3 over 𝑅, 𝑓 𝑅 𝑖 𝑅 is the right answer. 11/16/2016 UofT: ISR in Communication

General One-Way Communication For each random string 𝑅 Alice’s message = 𝑖 𝑅 ∈ 2 𝑘 Bob’s output = 𝑓 𝑅 ( 𝑖 𝑅 ) where 𝑓 𝑅 : 2 𝑘 → 0,1 W.p. ≥ 2 3 , 𝑓 𝑅 𝑖 𝑅 is the right answer. Vector representation: 𝑖 𝑅 ↦ 𝑥 𝑅 ∈ 0,1 2 𝑘 (unit coordinate vector) 𝑓 𝑅 ↦ 𝑦 𝑅 ∈ 0,1 2 𝑘 (truth table of 𝑓 𝑅 ). 𝑓 𝑅 𝑖 𝑅 =〈 𝑥 𝑅 , 𝑦 𝑅 〉; Acc. Prob. ∝ 𝑋,𝑌 ;𝑋= 𝑥 𝑅 𝑅 ;𝑌= 𝑦 𝑅 𝑅 Gaussian protocol estimates inner products of unit vectors to within ±𝜖 with 𝑂 𝜌 1 𝜖 2 communication. 11/16/2016 UofT: ISR in Communication

Two-way communication Still decided by inner products. Simple lemma: ∃ 𝐾 𝐴 𝑘 , 𝐾 𝐵 𝑘 ⊆ ℝ 2 𝑘 convex, that describe private coin k-bit comm. strategies for Alice, Bob s.t. accept prob. of 𝜋 𝐴 ∈ 𝐾 𝐴 𝑘 , 𝜋 𝐵 ∈ 𝐾 𝐵 𝑘 equals 〈 𝜋 𝐴 , 𝜋 𝐵 〉 Putting things together: Theorem: 𝑐𝑐 𝑓 ≤𝑘⇒𝑖𝑠𝑟 𝑓 ≤ 𝑂 𝜌 ( 2 𝑘 ) 11/16/2016 UofT: ISR in Communication

Main Technical Result: Matching lower bound Theorem: There exists a (promise) problem 𝑓 s.t. 𝑐𝑐 𝑓 ≤𝑘 𝑖𝑠 𝑟 𝜌 𝑓 ≥exp⁡(𝑘) The Problem: Gap Sparse Inner Product (G-Sparse-IP). Alice gets sparse 𝑥∈ 0,1 𝑛 ; wt 𝑥 ≈ 2 −𝑘 ⋅𝑛 Bob gets 𝑦∈ 0,1 𝑛 Promise: 〈𝑥,𝑦〉 ≥ .9 2 −𝑘 ⋅𝑛 or 𝑥,𝑦 ≤ .6 2 −𝑘 ⋅𝑛. Decide which. G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

𝒑𝒔𝒓 Protocol for G-Sparse-IP Note: Gaussian protocol takes 𝑂 2 𝑘 bits. Need to get exponentially better. Idea: 𝑥 𝑖 ≠0 ⇒ 𝑦 𝑖 correlated with answer. Use (perfectly) shared randomness to find random index 𝑖 s.t. 𝑥 𝑖 ≠0 . Shared randomness: 𝑖 1 , 𝑖 2 , 𝑖 3 , … uniform over [𝑛] Alice → Bob: smallest index 𝑗 s.t. 𝑥 𝑖 𝑗 ≠0. Bob: Accept if 𝑦 𝑖 𝑗 =1 Expect 𝑗≈ 2 𝑘 ; 𝑐𝑐≤𝑘. G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Lower Bound Idea Catch: ∀ Dist., ∃ Det. Protocol w. comm. ≤𝑘. Need to fix strategy first, construct dist. later. Main Idea: Protocol can look like G-Sparse-Inner-Product But implies players can agree on common index to focus on … Agreement is hard Protocol can ignore sparsity Requires 2 𝑘 bits Protocol can look like anything! Invariance Principle [MOO, Mossel … ] 𝑥 11/16/2016 UofT: ISR in Communication

Invariance Principle [MOO’08] Informally: Analysis of prob. processes with bits often hard. Analysis of related processes with Gaussian variables easy. “Invariance Principle” – under sufficiently general conditions … prob. of events “invariant” when switching from bits to Gaussian Theorem: For every convex 𝐾 1 , 𝐾 2 ⊆ −1,1 ℓ ∃ transformations 𝑇 1 , 𝑇 2 s.t. if 𝑓: 0,1 𝑛 → 𝐾 1 and 𝑔: 0,1 𝑛 → 𝐾 2 have no common influential variable, then 𝐹= 𝑇 1 𝑓: ℝ 𝑛 → 𝐾 1 and 𝐺= 𝑇 2 𝑔: ℝ 𝑛 → 𝐾 2 satisfy Exp 𝑥,𝑦 〈𝑓 𝑥 ,𝑔 𝑦 〉 ≈ Exp 𝑋,𝑌 〈𝐹 𝑋 ,𝐺 𝑌 〉 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Summarizing 𝑘 bits of comm. with perfect sharing → 2 𝑘 bits with imperfect sharing. This is tight Invariance principle for communication Agreement distillation Low-influence strategies G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Conclusions Imperfect agreement of context important. Dealing with new layer of uncertainty. Notion of scale (context LARGE) Many open directions+questions: Imperfectly shared randomness: One-sided error? Does interaction ever help? How much randomness? More general forms of correlation? 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Thank You! 11/16/2016 UofT: ISR in Communication

Towards a lower bound: Ruling out a natural approach Alice and Bob use (many) correlated bits to agree perfectly on few random bits? For G-Sparse-IP need 𝑂( 2 𝑘 log 𝑛) random bits. Agreement Distillation Problem: Alice & Bob exchange 𝑡 bits; generate 𝑘 random bits, with agreement probability 𝛾. Lower bound [Bogdanov, Mossel]: 𝑡≥𝑘 −𝑂 log 1 𝛾 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Towards Lower Bound Explaining two natural protocols: Gaussian Inner Product Protocol: Ignore sparsity and just estimate inner product. Uses ~ 2 2𝑘 bits. Need to prove it can’t be improved! G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

Optimality of Gaussian Protocol Problem: 𝑥,𝑦 ← 𝜇 𝑛 : 𝜇= 𝜇 𝑌𝐸𝑆 𝑜𝑟 𝜇 𝑁𝑂 supported on ℝ×ℝ 𝜇 𝑌𝐸𝑆 :𝜖-correlated Gaussians 𝜇 𝑁𝑂 : uncorrelated Gaussians Lemma: Separating 𝜇 𝑌𝐸𝑆 𝑛 𝑣𝑠. 𝜇 𝑁𝑂 𝑛 requires Ω 𝜖 −1 bits of communication. Proof: Reduction from Disjointness Conclusion: Can’t ignore sparsity! G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Towards Lower Bound Explaining two natural protocols: Gaussian Inner Product Protocol: Ignore sparsity and just estimate inner product. Uses ~ 2 2𝑘 bits. Need to prove it can’t be improved! Protocol with perfectly shared randomness: Alice & Bob agree on coordinates to focus on: ( 𝑖 1 , 𝑖 2 , …, 𝑖 2 𝑘 , …); Either 𝑖 1 has high entropy (over choice of 𝑟,𝑠) Violates agreement distillation bound Or has low-entropy: Fix distributions of 𝑥,𝑦 s.t. 𝑥 𝑖 1 ⊥ 𝑦 𝑖 1 G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

Aside: Distributional lower bounds Challenge: Usual CC lower bounds are distributional. 𝑐𝑐(G-Sparse-IP) ≤𝑘, ∀ inputs. ⇒𝑐𝑐(G-Sparse-IP) ≤𝑘 ∀ distributions. ⇒ det-cc (G-Sparse-IP) ≤𝑘 ∀ distributions. So usual approach can’t work … Need to fix strategy first and then “identify” a hard distribution for the strategy … G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

UofT: ISR in Communication Towards lower bound Summary so far: Symmetric strategy ⇒ 2 𝑘 bits of comm. Strategy asymmetric; 𝑥 1 , 𝑦 1 … 𝑥 𝑘 , 𝑦 𝑘 have high influence ⇒ fix the distribution so these coordinates do not influence answer. Strategy asymmetric; with random coordinate having high influence ⇒ violates agreement lower bound. Are these exhaustive? How to prove this? Invariance Principle!! [Mossel, O’Donnell, Oleskiewisz], [Mossel] … 11/16/2016 UofT: ISR in Communication

ISR lower bound for GSIP. One-way setting (for now) Strategies: Alice 𝑓 𝑟 𝑥 ∈ 𝐾 ; Bob 𝑔 𝑠 𝑦 ∈ 0,1 𝐾 ; Distributions: If 𝑥 𝑖 , 𝑦 𝑖 have high influence on (𝑓 𝑟 , 𝑔 𝑠 ) w.h.p. over 𝑟,𝑠 then set 𝑥 𝑖 = 𝑦 𝑖 =0. [𝑖 is BAD] Else 𝑦 𝑖 correlated with 𝑥 𝑖 in YES case, and independent in NO case. Analysis: 𝑖∈𝐵𝐴𝐷 influential in both 𝑓 𝑟 , 𝑔 𝑠 ⇒ No help. 𝑖∉𝐵𝐴𝐷 influential … ⇒ violates agreement lower bound. No common influential variable ⇒ 𝑥,𝑦 can be replaced by Gaussians ⇒ 2 𝑘 bits needed. G-Sparse-IP: 𝒙, 𝒚 ∈ 𝟎,𝟏 𝒏 ;𝒘𝒕 𝒙 ≈ 𝟐 −𝒌 ⋅𝒏 Decide 𝒙,𝒚 ≥(.𝟗) 𝟐 −𝒌 ⋅𝒏 or 𝒙,𝒚 ≤(.𝟔) 𝟐 −𝒌 ⋅𝒏? 11/16/2016 UofT: ISR in Communication

Invariance Principle + Challenges Informal Invariance Principle: 𝑓,𝑔 low-degree polynomials with no common influential variable ⇒ Exp 𝑥,𝑦 𝑓 𝑥 𝑔 𝑦 ≈ Exp 𝑋,𝑌 [𝑓(𝑋)𝑔(𝑌)] (caveat 𝑓≈𝑓;𝑔≈𝑔) where 𝑥,𝑦 Boolean 𝑛-wise product dist. and 𝑋,𝑌 Gaussian 𝑛-wise product dist Challenges [+ Solutions]: Our functions not low-degree [Smoothening] Our functions not real-valued 𝑔: 0,1 𝑛 → 0,1 ℓ : [Truncate range to 0,1 ℓ ] 𝑓: 0,1 𝑛 → ℓ : [???, [work with Δ ℓ ]] 11/16/2016 UofT: ISR in Communication

Invariance Principle + Challenges Informal Invariance Principle: 𝑓,𝑔 low-degree polynomials with no common influential variable ⇒ Exp 𝑥,𝑦 𝑓 𝑥 𝑔 𝑦 ≈ Exp 𝑋,𝑌 [𝑓(𝑋)𝑔(𝑌)] (caveat 𝑓≈𝑓;𝑔≈𝑔) Challenges Our functions not low-degree [Smoothening] Our functions not real-valued [Truncate] Quantity of interest is not 𝑓 𝑥 ⋅𝑔 𝑦 … [Can express quantity of interest as inner product. ] … (lots of grunge work …) Get a relevant invariance principle (next) 11/16/2016 UofT: ISR in Communication

Invariance Principle for CC Theorem: For every convex 𝐾 1 , 𝐾 2 ⊆ −1,1 ℓ ∃ transformations 𝑇 1 , 𝑇 2 s.t. if 𝑓: 0,1 𝑛 → 𝐾 1 and 𝑔: 0,1 𝑛 → 𝐾 2 have no common influential variable, then 𝐹= 𝑇 1 𝑓: ℝ 𝑛 → 𝐾 1 and 𝐺= 𝑇 2 𝑔: ℝ 𝑛 → 𝐾 2 satisfy Exp 𝑥,𝑦 〈𝑓 𝑥 ,𝑔 𝑦 〉 ≈ Exp 𝑋,𝑌 〈𝐹 𝑋 ,𝐺 𝑌 〉 Main differences: 𝑓,𝑔 vector-valued. Functions are transformed: 𝑓↦𝐹;𝑔↦𝐺 Range preserved exactly ( 𝐾 1 =Δ ℓ ; 𝐾 2 = 0,1 ℓ )! So 𝐹,𝐺 are still communication strategies! 11/16/2016 UofT: ISR in Communication