Presentation is loading. Please wait.

Presentation is loading. Please wait.

Communication Amid Uncertainty

Published byAdelia Hawkins Modified over 6 years ago

Similar presentations

Presentation on theme: "Communication Amid Uncertainty"— Presentation transcript:

1 Communication Amid Uncertainty
Madhu Sudan Harvard Based on Juba, S. (STOC 2008, ITCS 2011) Goldreich, Juba, S. (JACM 2011) Juba, Kalai, Khanna, S. (ITCS 2011) Haramaty, S. (ITCS 2014) Canonne, Guruswami, Meka, S. (ITCS 2015) Ghazi, Kamath, S. (SODA 2016) Ghazi, Komargodski, Kothari, S. (SODA 2016) Leshno, S. (manuscript) 02/24/2017 UCR: Uncertain Communication

2 Communication vs. Computation
Interdependent technologies: Neither can exist without other Technologies/Products/Commerce developed (mostly) independently. Early products based on clean abstractions of the other. Later versions added other capability as afterthought. Today products … deeply integrated. Deep theories: Well separated … and have stayed that way Turing ‘36 Shannon ‘48 02/24/2017 UCR: Uncertain Communication

3 Consequences of the wall
Computing theory: Fundamental principle = Universality You can program your computer to do whatever you want. ⇒ Heterogeneity of devices Communication theory: Centralized design (Encoder, Decoder, Compression, IPv4, TCP/IP). You can NOT program your device! ⇐ Homogeneity of devices Contradiction! But does it matter? Yes! 02/24/2017 UCR: Uncertain Communication

4 UCR: Uncertain Communication
Role of theory? Ideally: Foundations of practice! Application Theory layer 02/24/2017 UCR: Uncertain Communication

5 Communication vs. Computing
Option 1 Computing Communication 02/24/2017 UCR: Uncertain Communication

6 Communication vs. Computing
Option 2 Computing Communication 02/24/2017 UCR: Uncertain Communication

7 Communication vs. Computing
Option 3 Computing Communication 02/24/2017 UCR: Uncertain Communication

8 UCR: Uncertain Communication
Sample problems: Universal printing: You are visiting a friend. You can use their Wifi network, but not their printer. Why? Projecting from your laptop: Machines that learn to communicate, and learn to understand each other. Digital libraries: Data that lives forever (communication across time), while devices change. 02/24/2017 UCR: Uncertain Communication

9 UCR: Uncertain Communication
A new theory? Computing Communication 02/24/2017 UCR: Uncertain Communication

10 Essence of mismatch: Uncertainty
Shannon: “The significant aspect is that the actual message is one selected from a set of possible messages” Essence of unreliability today: Context: Determines set of possible messages. dictionary, grammar, general knowledge coding scheme, prior distribution, communication protocols … Context is HUGE; and not shared perfectly; 02/24/2017 UCR: Uncertain Communication

11 Modelling uncertainty
Uncertain Communication Model Classical Shannon Model A1 B1 B2 A2 Channel A B B3 A3 New Class of Problems New challenges Needs more attention! Ak Bj 02/24/2017 UCR: Uncertain Communication

12 UCR: Uncertain Communication
Hope Better understanding of existing mechanisms In natural communication In “ad-hoc” (but “creative”) designs What problems are they solving? Better solutions? Or at least understand how to measure the quality of a solution. 02/24/2017 UCR: Uncertain Communication

13 II: Uncertain Compression
02/24/2017 UCR: Uncertain Communication

14 Human-Human Communication
𝑀 1 = 𝑤 11 , 𝑤 12 , … 𝑀 2 = 𝑤 21 , 𝑤 22 , … 𝑀 3 = 𝑤 31 , 𝑤 32 , … 𝑀 4 = 𝑤 41 , 𝑤 42 , … … Human-Human Communication Role of dictionary = ? [Juba, Kalai, Khanna, S. 11] Dictionary: list of words representing message words appear against multiple messages multiple words per message. How to decide which word to use? Context! Encoding: Given message, use shortest unambiguous word in current context. Decoding: Given word, use most likely message in current context, (among plausible messages) Context = ???. 𝑃 𝑖 = Prob [message = 𝑀 𝑖 ] Prob. distribution on messages 02/24/2017 UCR: Uncertain Communication

15 UCR: Uncertain Communication
𝑀 1 = 𝑤 11 , 𝑤 12 , … 𝑀 2 = 𝑤 21 , 𝑤 22 , … 𝑀 3 = 𝑤 31 , 𝑤 32 , … 𝑀 4 = 𝑤 41 , 𝑤 42 , … … Human Communication - 2 Good (Ideal?) dictionary Should compress messages to entropy of context: 𝐻(𝑃 =〈 𝑃 1 ,…, 𝑃 𝑁 〉). Even better dictionary? Should not assume context of sender/receiver identical! Compression should work even if sender uncertain about receiver (or receivers’ context). Receiver context Sender Theorem [JKKS]: If dictionary is “random” then compression achieves message length 𝐻(𝑃) +Δ, if sender and receiver distributions are “Δ-close”. 02/24/2017 UCR: Uncertain Communication

16 UCR: Uncertain Communication
Implications Reflects tension between ambiguity resolution and compression. Larger the gap in context (Δ), larger the encoding length. Coding scheme reflects human communication? “Shared randomness’’ debatable assumption: Dictionaries do have more structure. Deterministic communication? [Haramaty+S,14] Randomness imperfectly shared? Next … 02/24/2017 UCR: Uncertain Communication

17 III: Imperfectly Shared Randomness
02/24/2017 UCR: Uncertain Communication

18 Communication (Complexity)
Compression (Shannon, Noiseless Channel) What will Bob do with 𝑥? Often knowledge of 𝑥 is overkill. Hopefully 𝑥 𝑥 ∼𝑃=( 𝑃 1 ,…, 𝑃 𝑛 ) Alice Bob Compress Decompress 02/24/2017 UCR: Uncertain Communication

19 (Recall) Communication Complexity
The model (with shared randomness) 𝑥 𝑦 𝑓: 𝑥,𝑦 ↦Σ 𝑅=$$$ Usually studied for lower bounds. This talk: CC as +ve model. Alice Bob 𝐶𝐶 𝑓 =# bits exchanged by best protocol 𝑓(𝑥,𝑦) w.p. 2/3 02/24/2017 UCR: Uncertain Communication

20 UCR: Uncertain Communication
Brief history ∃ problems where Alice can get away with much fewer bits of communication. Example: ⊕ 𝑥,𝑦 ≜ ⊕ 𝑖 ( 𝑥 𝑖 ⊕ 𝑦 𝑖 ) But very few such deterministically. Enter Randomness: Alice & Bob share random string 𝑟 (ind. of 𝑥,𝑦) Many more problems; Example: Eq 𝑥,𝑦 =1 if 𝑥=𝑦 and 0 otherwise Deterministically:Θ 𝑛 Randomized: 𝑂(1) Uncertainty-motivated question: Does randomness have to be perfectly shared? Bob 𝑠 Alice 𝑟 Unstated philosophical contribution of CC a la Yao: Communication with a focus (“only need to determine 𝑓 𝑥,𝑦 ”) can be more effective (shorter than 𝑥 ,𝐻 𝑥 ,𝐻 𝑦 ,𝐼(𝑥;𝑦)… ) [Ghazi, Kamath, S., SODA 2016]:Taxonomy of simple problems; Many interesting problems and protocols! 02/24/2017 UCR: Uncertain Communication

21 UCR: Uncertain Communication
Results [Newman ‘90s]: 𝐶𝐶 without sharing≤𝐶𝐶 with sharing+ log 𝑛 But additive cost of log 𝑛 may be too much. Compression! Equality!! Model recently studied by [Bavarian et al.’14] Equality: 𝑂 1 bit protocol w. imperfect sharing Our Results: [Canonne, Guruswami, Meka, S.’15] Compression: 𝑂 𝐻 𝑃 +Δ Generally: 𝑘 bits with shared randomness ⇒ 𝑂(2 𝑘 ) bits with imperfect sharing. 𝑘→ 2 𝑘 loss is necessary. 02/24/2017 UCR: Uncertain Communication

22 UCR: Uncertain Communication
Some General Lessons Compression Protocol: Adds “error-correction” to [JKKS] protocol. Send shortest word that is far from words of other high probability messages. Another natural protocol. General Protocol: Much more “statistical” Classical protocol for Equality: Alice sends random coordinate of ECC(x) New Protocol ~ Alice send # 1’s in random subset of coordinates. 02/24/2017 UCR: Uncertain Communication

23 IV: Focussed Communication
02/24/2017 UCR: Uncertain Communication

24 UCR: Uncertain Communication
Model Bob wishes to compute 𝑓(𝑥,𝑦); Alice knows 𝑔≈𝑓; Alice, Bob given 𝑔,𝑓 explicitly. (New input size ~ 2 𝑛 ) Modelling Questions: What is ≈? Is it reasonable to expect to compute 𝑓 𝑥,𝑦 ? E.g., 𝑓 𝑥,𝑦 = 𝑓 ′ 𝑥 ? Can’t compute 𝑓 𝑥,𝑦 without communicating 𝑥 Our Choices: Assume 𝑥,𝑦 come from a distribution 𝜇 𝑓≈𝑔 if 𝑓(𝑥,𝑦) usually equals 𝑔(𝑥,𝑦) Suffices to compute ℎ 𝑥,𝑦 for ℎ≈𝑓 02/24/2017 UCR: Uncertain Communication

25 UCR: Uncertain Communication
Results Thm [Ghazi,Komargodski,Kothari,S]: ∃𝑓≈𝑔, with 𝐶𝐶 𝑓 ,𝐶𝐶 𝑔 =1, but uncertain complexity ≈ 𝑛 Thm [GKKS]: But not if 𝑥 independent of 𝑦 (in 1-way setting). 2-way setting, even 2-round, open! Main Idea: Canonical 1-way protocol for 𝑓: Alice + Bob share random 𝑦 1 , … 𝑦 𝑚 ∈ 0,1 𝑛 . Alice sends 𝑔 𝑥, 𝑦 1 , …, 𝑔 𝑥, 𝑦 𝑚 to Bob. Protocol used previously … but not as “canonical”. Canonical protocol robust when 𝑓≈𝑔. 02/24/2017 UCR: Uncertain Communication

26 UCR: Uncertain Communication
Conclusions Positive view of communication complexity: Communication with a focus can be effective! Context Important: (Elephant in the room: Huge, unmentionable) New layer of uncertainty. New notion of scale (context LARGE) Importance of 𝑜( log 𝑛 ) additive factors. Many “uncertain” problems can be solved without resolving the uncertainty Many open directions+questions (which is a good thing) 02/24/2017 UCR: Uncertain Communication

27 UCR: Uncertain Communication
Thank You! 02/24/2017 UCR: Uncertain Communication

28 UCR: Uncertain Communication
IV: Coordination 02/24/2017 UCR: Uncertain Communication

29 UCR: Uncertain Communication
Communicate meaning? Ultimate goal: Message ⇒ Instructions. What is this dictionary? Can it be learned by communication? At first glance: Ambiguity can never be resolved by communication (even a theorem [JS’08]). Second look: Needs more careful definitions. Meaning = mix of communication + actions + incentives. 02/24/2017 UCR: Uncertain Communication

30 UCR: Uncertain Communication
Receiver dictionary Sender (Mis) Understanding? Uncertainty problem: Sender/receiver disagree on meaning of bits Definition of Understanding? Sender sends instructions; Receiver follows? Errors undetectable (by receiver) Not the right definition anyway: Does receiver want to follow instructions What does receiver gain by following instructions? Must have its own “Goal”/”Incentives”. [Goldreich,Juba,S. 2012]: Goal-oriented communication: 02/24/2017 UCR: Uncertain Communication

31 UCR: Uncertain Communication
Receiver dictionary Sender (Mis) Understanding? Uncertainty problem: Sender/receiver disagree on meaning of bits Definition of Understanding? Receiver has goals/incentives. [Goldreich,Juba,S. 2012]: Goal-oriented communication: Define general communication problems (and goals) Show that if Sender can help receiver achieve goal (from any state) Receiver can sense progress towards goal then Receiver can achieve goal. Functional definition of understanding. 02/24/2017 UCR: Uncertain Communication

32 Illustration: (Repeated) Coordination
[Leshno,S.] Basic Coordination Game: Alice and Bob simultaneously choose actions ∈ 0,1 If both pick same action, both win. If they pick opposite actions, both lose. Main challenge: Don’t know what the other person will choose when making our choice. Repeated version: Play a sequence of games, using outcome of previous games to learn what the other player may do next. Goal: Eventual perpetual coordination. 02/24/2017 UCR: Uncertain Communication

33 UCR: Uncertain Communication
Our setting Repeated coordination game with uncertainty: Bob’s perspective: Knows his payoffs – 1 for coord.; 0 for not. Does not know Alice’s payoffs (uncertainty): May vary with round But for every action of Alice, payoff does not decrease if Bob coordinates (compatibility). Knows a set 𝑆 𝐴 of strategies she may employ (“reasonable behaviors”). Can he learn to coordinate eventually? 02/24/2017 UCR: Uncertain Communication

34 Coordination with Uncertainty
Mixes essential ingredients: Communication: Actions can be use to communicate (future actions). Control: Communication (may) influence future actions. Incentives: Bob has incentive to coordinate. Alice not averse. What do the general results say? ∃ Universal strategy U s.t. ∀ Alice s.t. ∃ Bob who coordinates with Alice from any state. U coordinates with Alice. 02/24/2017 UCR: Uncertain Communication

35 UCR: Uncertain Communication
Lessons Coordination is possible: Even in extreme settings where Alice has almost no idea of Bob Bob has almost no idea of Alice Alice is trying to learn Bob Bob is trying to learn Alice Learning is slow … Need to incorporate beliefs to measure efficiency. [Juba, S. 2011] Does process become more efficient when languages have structure? [Open] 02/24/2017 UCR: Uncertain Communication

36 UCR: Uncertain Communication
Role of theory? Ideally: Foundations of practice! Application Theory layer 02/24/2017 UCR: Uncertain Communication

37 Communication vs. Computing
Option 1 Computing Communication 02/24/2017 UCR: Uncertain Communication

38 Communication vs. Computing
Option 2 Computing Communication 02/24/2017 UCR: Uncertain Communication

39 Communication vs. Computing
Option 3 Computing Communication 02/24/2017 UCR: Uncertain Communication

40 UCR: Uncertain Communication
A new theory? Computing Communication 02/24/2017 UCR: Uncertain Communication

Download ppt "Communication Amid Uncertainty"

Similar presentations

1+eps-Approximate Sparse Recovery Eric Price MIT David Woodruff IBM Almaden.

1+eps-Approximate Sparse Recovery Eric Price MIT David Woodruff IBM Almaden.
Of 23 09/24/2013HLF: Reliable Meaningful Communication1 Reliable Meaningful Communication Madhu Sudan Microsoft, Cambridge, USA.

Of 23 09/24/2013HLF: Reliable Meaningful Communication1 Reliable Meaningful Communication Madhu Sudan Microsoft, Cambridge, USA.
Of 35 05/30/2012CSOI-Summer: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.

Of 35 05/30/2012CSOI-Summer: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.
Of 12 12/04/2013CSOI: Communication as Coordination1 Communication as Coordination Madhu Sudan Microsoft, Cambridge, USA -

Of 12 12/04/2013CSOI: Communication as Coordination1 Communication as Coordination Madhu Sudan Microsoft, Cambridge, USA -
Of 10 09/27/2011Communication, Computing & Technology: Communication, Computing, & Technology Madhu Sudan MSR New England.

Of 10 09/27/2011Communication, Computing & Technology: Communication, Computing, & Technology Madhu Sudan MSR New England.
Of 13 10/08/2013MSRNE 5 th Anniversary: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.

Of 13 10/08/2013MSRNE 5 th Anniversary: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.
Of 29 12/02/2013Purdue: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.

Of 29 12/02/2013Purdue: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.
Of 24 11/20/2012TIFR: Deterministic Communication Amid Uncertainty1 ( Deterministic ) Communication amid Uncertainty Madhu Sudan Microsoft, New England.

Of 24 11/20/2012TIFR: Deterministic Communication Amid Uncertainty1 ( Deterministic ) Communication amid Uncertainty Madhu Sudan Microsoft, New England.
Of 30 10/31/2013Cornell: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.

Of 30 10/31/2013Cornell: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.
Of 30 09/16/2013PACM: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.

Of 30 09/16/2013PACM: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.
Gillat Kol (IAS) joint work with Ran Raz (Weizmann + IAS) Interactive Channel Capacity.

Gillat Kol (IAS) joint work with Ran Raz (Weizmann + IAS) Interactive Channel Capacity.
Rational Oblivious Transfer KARTIK NAYAK, XIONG FAN.

Rational Oblivious Transfer KARTIK NAYAK, XIONG FAN.
Of 7 10/01/2013LIDS Lunch: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.

Of 7 10/01/2013LIDS Lunch: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.
Universal Communication Brendan Juba (MIT) With: Madhu Sudan (MIT)

Universal Communication Brendan Juba (MIT) With: Madhu Sudan (MIT)
Of 30 09/04/2012ITW 2012: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.

Of 30 09/04/2012ITW 2012: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.
Of 27 01/06/2015CMI: Uncertain Communication1 Communication Amid Uncertainty Madhu Sudan Microsoft Research Based on Juba, S. (STOC 2008, ITCS 2011) Juba,

Of 27 01/06/2015CMI: Uncertain Communication1 Communication Amid Uncertainty Madhu Sudan Microsoft Research Based on Juba, S. (STOC 2008, ITCS 2011) Juba,
Of 13 October 6-7, 2010Emerging Frontiers of Information: Kickoff 1 Madhu Sudan Microsoft Research + MIT TexPoint fonts used in EMF. TexPoint fonts used.

Of 13 October 6-7, 2010Emerging Frontiers of Information: Kickoff 1 Madhu Sudan Microsoft Research + MIT TexPoint fonts used in EMF. TexPoint fonts used.
Of 10 Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.

Of 10 Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.
Of 12 03/22/2012CISS: Compression w. Uncertain Priors1 Compression under uncertain priors Madhu Sudan Microsoft, New England Based on joint works with:

Of 12 03/22/2012CISS: Compression w. Uncertain Priors1 Compression under uncertain priors Madhu Sudan Microsoft, New England Based on joint works with:
1 School of Computing Science Simon Fraser University, Canada PCP: A Probabilistic Coverage Protocol for Wireless Sensor Networks Mohamed Hefeeda and Hossein.

1 School of Computing Science Simon Fraser University, Canada PCP: A Probabilistic Coverage Protocol for Wireless Sensor Networks Mohamed Hefeeda and Hossein.

Similar presentations

Ads by Google