Communication Amid Uncertainty Madhu Sudan Harvard Based on Juba, S. (STOC 2008, ITCS 2011) Goldreich, Juba, S. (JACM 2011) Juba, Kalai, Khanna, S. (ITCS 2011) Haramaty, S. (ITCS 2014) Canonne, Guruswami, Meka, S. (ITCS 2015) Ghazi, Kamath, S. (SODA 2016) Ghazi, Komargodski, Kothari, S. (SODA 2016) Leshno, S. (manuscript) 02/24/2017 UCR: Uncertain Communication

Communication vs. Computation Interdependent technologies: Neither can exist without other Technologies/Products/Commerce developed (mostly) independently. Early products based on clean abstractions of the other. Later versions added other capability as afterthought. Today products … deeply integrated. Deep theories: Well separated … and have stayed that way Turing ‘36 Shannon ‘48 02/24/2017 UCR: Uncertain Communication

Consequences of the wall Computing theory: Fundamental principle = Universality You can program your computer to do whatever you want. ⇒ Heterogeneity of devices Communication theory: Centralized design (Encoder, Decoder, Compression, IPv4, TCP/IP). You can NOT program your device! ⇐ Homogeneity of devices Contradiction! But does it matter? Yes! 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Role of theory? Ideally: Foundations of practice! Application Theory layer 02/24/2017 UCR: Uncertain Communication

Communication vs. Computing Option 1 Computing Communication 02/24/2017 UCR: Uncertain Communication

Communication vs. Computing Option 2 Computing Communication 02/24/2017 UCR: Uncertain Communication

Communication vs. Computing Option 3 Computing Communication 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Sample problems: Universal printing: You are visiting a friend. You can use their Wifi network, but not their printer. Why? Projecting from your laptop: Machines that learn to communicate, and learn to understand each other. Digital libraries: Data that lives forever (communication across time), while devices change. 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication A new theory? Computing Communication 02/24/2017 UCR: Uncertain Communication

Essence of mismatch: Uncertainty Shannon: “The significant aspect is that the actual message is one selected from a set of possible messages” Essence of unreliability today: Context: Determines set of possible messages. dictionary, grammar, general knowledge coding scheme, prior distribution, communication protocols … Context is HUGE; and not shared perfectly; 02/24/2017 UCR: Uncertain Communication

Modelling uncertainty Uncertain Communication Model Classical Shannon Model A1 B1 B2 A2 Channel A B B3 A3 New Class of Problems New challenges Needs more attention! Ak Bj 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Hope Better understanding of existing mechanisms In natural communication In “ad-hoc” (but “creative”) designs What problems are they solving? Better solutions? Or at least understand how to measure the quality of a solution. 02/24/2017 UCR: Uncertain Communication

II: Uncertain Compression 02/24/2017 UCR: Uncertain Communication

Human-Human Communication 𝑀 1 = 𝑤 11 , 𝑤 12 , … 𝑀 2 = 𝑤 21 , 𝑤 22 , … 𝑀 3 = 𝑤 31 , 𝑤 32 , … 𝑀 4 = 𝑤 41 , 𝑤 42 , … … Human-Human Communication Role of dictionary = ? [Juba, Kalai, Khanna, S. 11] Dictionary: list of words representing message words appear against multiple messages multiple words per message. How to decide which word to use? Context! Encoding: Given message, use shortest unambiguous word in current context. Decoding: Given word, use most likely message in current context, (among plausible messages) Context = ???. 𝑃 𝑖 = Prob [message = 𝑀 𝑖 ] Prob. distribution on messages 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication 𝑀 1 = 𝑤 11 , 𝑤 12 , … 𝑀 2 = 𝑤 21 , 𝑤 22 , … 𝑀 3 = 𝑤 31 , 𝑤 32 , … 𝑀 4 = 𝑤 41 , 𝑤 42 , … … Human Communication - 2 Good (Ideal?) dictionary Should compress messages to entropy of context: 𝐻(𝑃 =〈 𝑃 1 ,…, 𝑃 𝑁 〉). Even better dictionary? Should not assume context of sender/receiver identical! Compression should work even if sender uncertain about receiver (or receivers’ context). Receiver context Sender Theorem [JKKS]: If dictionary is “random” then compression achieves message length 𝐻(𝑃) +Δ, if sender and receiver distributions are “Δ-close”. 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Implications Reflects tension between ambiguity resolution and compression. Larger the gap in context (Δ), larger the encoding length. Coding scheme reflects human communication? “Shared randomness’’ debatable assumption: Dictionaries do have more structure. Deterministic communication? [Haramaty+S,14] Randomness imperfectly shared? Next … 02/24/2017 UCR: Uncertain Communication

III: Imperfectly Shared Randomness 02/24/2017 UCR: Uncertain Communication

Communication (Complexity) Compression (Shannon, Noiseless Channel) What will Bob do with 𝑥? Often knowledge of 𝑥 is overkill. Hopefully 𝑥 𝑥 ∼𝑃=( 𝑃 1 ,…, 𝑃 𝑛 ) Alice Bob Compress Decompress 02/24/2017 UCR: Uncertain Communication

(Recall) Communication Complexity The model (with shared randomness) 𝑥 𝑦 𝑓: 𝑥,𝑦 ↦Σ 𝑅=$$$ Usually studied for lower bounds. This talk: CC as +ve model. Alice Bob 𝐶𝐶 𝑓 =# bits exchanged by best protocol 𝑓(𝑥,𝑦) w.p. 2/3 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Brief history ∃ problems where Alice can get away with much fewer bits of communication. Example: ⊕ 𝑥,𝑦 ≜ ⊕ 𝑖 ( 𝑥 𝑖 ⊕ 𝑦 𝑖 ) But very few such deterministically. Enter Randomness: Alice & Bob share random string 𝑟 (ind. of 𝑥,𝑦) Many more problems; Example: Eq 𝑥,𝑦 =1 if 𝑥=𝑦 and 0 otherwise Deterministically:Θ 𝑛 Randomized: 𝑂(1) Uncertainty-motivated question: Does randomness have to be perfectly shared? Bob 𝑠 Alice 𝑟 Unstated philosophical contribution of CC a la Yao: Communication with a focus (“only need to determine 𝑓 𝑥,𝑦 ”) can be more effective (shorter than 𝑥 ,𝐻 𝑥 ,𝐻 𝑦 ,𝐼(𝑥;𝑦)… ) [Ghazi, Kamath, S., SODA 2016]:Taxonomy of simple problems; Many interesting problems and protocols! 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Results [Newman ‘90s]: 𝐶𝐶 without sharing≤𝐶𝐶 with sharing+ log 𝑛 But additive cost of log 𝑛 may be too much. Compression! Equality!! Model recently studied by [Bavarian et al.’14] Equality: 𝑂 1 bit protocol w. imperfect sharing Our Results: [Canonne, Guruswami, Meka, S.’15] Compression: 𝑂 𝐻 𝑃 +Δ Generally: 𝑘 bits with shared randomness ⇒ 𝑂(2 𝑘 ) bits with imperfect sharing. 𝑘→ 2 𝑘 loss is necessary. 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Some General Lessons Compression Protocol: Adds “error-correction” to [JKKS] protocol. Send shortest word that is far from words of other high probability messages. Another natural protocol. General Protocol: Much more “statistical” Classical protocol for Equality: Alice sends random coordinate of ECC(x) New Protocol ~ Alice send # 1’s in random subset of coordinates. 02/24/2017 UCR: Uncertain Communication

IV: Focussed Communication 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Model Bob wishes to compute 𝑓(𝑥,𝑦); Alice knows 𝑔≈𝑓; Alice, Bob given 𝑔,𝑓 explicitly. (New input size ~ 2 𝑛 ) Modelling Questions: What is ≈? Is it reasonable to expect to compute 𝑓 𝑥,𝑦 ? E.g., 𝑓 𝑥,𝑦 = 𝑓 ′ 𝑥 ? Can’t compute 𝑓 𝑥,𝑦 without communicating 𝑥 Our Choices: Assume 𝑥,𝑦 come from a distribution 𝜇 𝑓≈𝑔 if 𝑓(𝑥,𝑦) usually equals 𝑔(𝑥,𝑦) Suffices to compute ℎ 𝑥,𝑦 for ℎ≈𝑓 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Results Thm [Ghazi,Komargodski,Kothari,S]: ∃𝑓≈𝑔, with 𝐶𝐶 𝑓 ,𝐶𝐶 𝑔 =1, but uncertain complexity ≈ 𝑛 Thm [GKKS]: But not if 𝑥 independent of 𝑦 (in 1-way setting). 2-way setting, even 2-round, open! Main Idea: Canonical 1-way protocol for 𝑓: Alice + Bob share random 𝑦 1 , … 𝑦 𝑚 ∈ 0,1 𝑛 . Alice sends 𝑔 𝑥, 𝑦 1 , …, 𝑔 𝑥, 𝑦 𝑚 to Bob. Protocol used previously … but not as “canonical”. Canonical protocol robust when 𝑓≈𝑔. 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Conclusions Positive view of communication complexity: Communication with a focus can be effective! Context Important: (Elephant in the room: Huge, unmentionable) New layer of uncertainty. New notion of scale (context LARGE) Importance of 𝑜( log 𝑛 ) additive factors. Many “uncertain” problems can be solved without resolving the uncertainty Many open directions+questions (which is a good thing) 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Thank You! 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication IV: Coordination 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Communicate meaning? Ultimate goal: Message ⇒ Instructions. What is this dictionary? Can it be learned by communication? At first glance: Ambiguity can never be resolved by communication (even a theorem [JS’08]). Second look: Needs more careful definitions. Meaning = mix of communication + actions + incentives. 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Receiver dictionary Sender (Mis) Understanding? Uncertainty problem: Sender/receiver disagree on meaning of bits Definition of Understanding? Sender sends instructions; Receiver follows? Errors undetectable (by receiver) Not the right definition anyway: Does receiver want to follow instructions What does receiver gain by following instructions? Must have its own “Goal”/”Incentives”. [Goldreich,Juba,S. 2012]: Goal-oriented communication: 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Receiver dictionary Sender (Mis) Understanding? Uncertainty problem: Sender/receiver disagree on meaning of bits Definition of Understanding? Receiver has goals/incentives. [Goldreich,Juba,S. 2012]: Goal-oriented communication: Define general communication problems (and goals) Show that if Sender can help receiver achieve goal (from any state) Receiver can sense progress towards goal then Receiver can achieve goal. Functional definition of understanding. 02/24/2017 UCR: Uncertain Communication

Illustration: (Repeated) Coordination [Leshno,S.] Basic Coordination Game: Alice and Bob simultaneously choose actions ∈ 0,1 If both pick same action, both win. If they pick opposite actions, both lose. Main challenge: Don’t know what the other person will choose when making our choice. Repeated version: Play a sequence of games, using outcome of previous games to learn what the other player may do next. Goal: Eventual perpetual coordination. 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Our setting Repeated coordination game with uncertainty: Bob’s perspective: Knows his payoffs – 1 for coord.; 0 for not. Does not know Alice’s payoffs (uncertainty): May vary with round But for every action of Alice, payoff does not decrease if Bob coordinates (compatibility). Knows a set 𝑆 𝐴 of strategies she may employ (“reasonable behaviors”). Can he learn to coordinate eventually? 02/24/2017 UCR: Uncertain Communication

Coordination with Uncertainty Mixes essential ingredients: Communication: Actions can be use to communicate (future actions). Control: Communication (may) influence future actions. Incentives: Bob has incentive to coordinate. Alice not averse. What do the general results say? ∃ Universal strategy U s.t. ∀ Alice s.t. ∃ Bob who coordinates with Alice from any state. U coordinates with Alice. 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Lessons Coordination is possible: Even in extreme settings where Alice has almost no idea of Bob Bob has almost no idea of Alice Alice is trying to learn Bob Bob is trying to learn Alice Learning is slow … Need to incorporate beliefs to measure efficiency. [Juba, S. 2011] Does process become more efficient when languages have structure? [Open] 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication Role of theory? Ideally: Foundations of practice! Application Theory layer 02/24/2017 UCR: Uncertain Communication

Communication vs. Computing Option 1 Computing Communication 02/24/2017 UCR: Uncertain Communication

Communication vs. Computing Option 2 Computing Communication 02/24/2017 UCR: Uncertain Communication

Communication vs. Computing Option 3 Computing Communication 02/24/2017 UCR: Uncertain Communication

UCR: Uncertain Communication A new theory? Computing Communication 02/24/2017 UCR: Uncertain Communication