of 27 01/06/2015CMI: Uncertain Communication1 Communication Amid Uncertainty Madhu Sudan Microsoft Research Based on Juba, S. (STOC 2008, ITCS 2011) Juba, S. (STOC 2008, ITCS 2011) Goldreich, Juba, S. (JACM 2011) Goldreich, Juba, S. (JACM 2011) Juba, Kalai, Khanna, S. (ITCS 2011) Juba, Kalai, Khanna, S. (ITCS 2011) Haramaty, S. (ITCS 2014) Haramaty, S. (ITCS 2014) Canonne, Guruswami, Meka, S. (ITCS 2015) Canonne, Guruswami, Meka, S. (ITCS 2015) Leshno, S. (manuscript) Leshno, S. (manuscript)

of 27 01/06/2015CMI: Uncertain Communication2 Congratulations, CMI! Bravo!!!

of 27 Communication vs. Computation Communication vs. Computation Interdependent technologies: Neither can exist without other Interdependent technologies: Neither can exist without other Technologies/Products/Commerce developed (mostly) independently. Technologies/Products/Commerce developed (mostly) independently. Early products based on clean abstractions of the other. Early products based on clean abstractions of the other. Later versions added other capability as afterthought. Later versions added other capability as afterthought. Today products … deeply integrated. Today products … deeply integrated. Deep theories: Deep theories: 01/06/2015CMI: Uncertain Communication3 Well separated … and have stayed that way Turing ‘36 Shannon ‘48

of 27 Consequences of the wall 01/06/2015CMI: Uncertain Communication4

of 27 Sample problems: Universal printing: Universal printing: You are visiting a friend. You can use their Wifi network, but not their printer. Why? You are visiting a friend. You can use their Wifi network, but not their printer. Why? Projecting from your laptop: Projecting from your laptop: Machines that learn to communicate, and learn to understand each other. Machines that learn to communicate, and learn to understand each other. Digital libraries: Digital libraries: Data that lives forever (communication across time), while devices change. Data that lives forever (communication across time), while devices change. 01/06/2015CMI: Uncertain Communication5

of 27 Essence of “semantics”: Uncertainty Shannon: Shannon: “The significant aspect is that the actual message is one selected from a set of possible messages” “The significant aspect is that the actual message is one selected from a set of possible messages” Essence of unreliability today: Essence of unreliability today: Context: Determines set of possible messages. Context: Determines set of possible messages. dictionary, grammar, general knowledge dictionary, grammar, general knowledge coding scheme, prior distribution, communication protocols … coding scheme, prior distribution, communication protocols … Context is HUGE; and not shared perfectly; Context is HUGE; and not shared perfectly; 01/06/2015CMI: Uncertain Communication6

of 27 Modelling uncertainty Classical Shannon Model 01/06/2015CMI: Uncertain Communication7 A B Channel B2B2B2B2 AkAkAkAk A3A3A3A3 A2A2A2A2 A1A1A1A1 B1B1B1B1 B3B3B3B3 BjBjBjBj Uncertain Communication Model New Class of Problems New challenges Needs more attention!

of 27 Hope Better understanding of existing mechanisms Better understanding of existing mechanisms In natural communication In natural communication In “ad-hoc” designs In “ad-hoc” designs What problems are they solving? What problems are they solving? Better solutions? Better solutions? Or at least understand how to measure the quality of a solution. Or at least understand how to measure the quality of a solution. 01/06/2015CMI: Uncertain Communication8

of 27 01/06/2015CMI: Uncertain Communication9 II: Uncertain Compression

of 27 Human-Human Communication 01/06/2015CMI: Uncertain Communication10 Prob. distribution on messages

of 27 Human Communication /06/2015CMI: Uncertain Communication11ReceivercontextSendercontext

of 27 Implications 01/06/2015CMI: Uncertain Communication12

of 27 01/06/2015CMI: Uncertain Communication13 III: Imperfectly Shared Randomness

of 27 Communication (Complexity) 01/06/2015CMI: Uncertain Communication14 AliceAlice BobBob CompressDecompress In general, model allows interaction. For this talk, only one way comm.

of 27 Brief history 01/06/2015CMI: Uncertain Communication15

of 27 Results 01/06/2015CMI: Uncertain Communication16

of 27 Some General Lessons Compression Protocol: Compression Protocol: Adds “error-correction” to [JKKS] protocol. Adds “error-correction” to [JKKS] protocol. Send shortest word that is far from words of other high probability messages. Send shortest word that is far from words of other high probability messages. Another natural protocol. Another natural protocol. General Protocol: General Protocol: Much more “statistical” Much more “statistical” Classical protocol for Equality: Classical protocol for Equality: Alice sends random coordinate of ECC(x) Alice sends random coordinate of ECC(x) New Protocol New Protocol ~ Alice send # 1’s in random subset of coordinates. ~ Alice send # 1’s in random subset of coordinates. 01/06/2015CMI: Uncertain Communication17

of 27 01/06/2015CMI: Uncertain Communication18 IV: Coordination

of 27 Communicate meaning? 01/06/2015CMI: Uncertain Communication19

of 27 (Mis) Understanding? Uncertainty problem: Uncertainty problem: Sender/receiver disagree on meaning of bits Sender/receiver disagree on meaning of bits Definition of Understanding? Definition of Understanding? Sender sends instructions; Receiver follows? Sender sends instructions; Receiver follows? Errors undetectable (by receiver) Errors undetectable (by receiver) Not the right definition anyway: Not the right definition anyway: Does receiver want to follow instructions Does receiver want to follow instructions What does receiver gain by following instructions? Must have its own “Goal”/”Incentives”. What does receiver gain by following instructions? Must have its own “Goal”/”Incentives”. [ Goldreich,Juba,S ]: Goal-oriented communication: [ Goldreich,Juba,S ]: Goal-oriented communication: 01/06/2015CMI: Uncertain Communication20ReceiverdictionarySenderdictionary

of 27 (Mis) Understanding? Uncertainty problem: Uncertainty problem: Sender/receiver disagree on meaning of bits Sender/receiver disagree on meaning of bits Definition of Understanding? Definition of Understanding? Receiver has goals/incentives. Receiver has goals/incentives. [ Goldreich,Juba,S ]: Goal-oriented communication: [ Goldreich,Juba,S ]: Goal-oriented communication: Define general communication problems (and goals) Define general communication problems (and goals) Show that if Show that if Sender can help receiver achieve goal (from any state) Sender can help receiver achieve goal (from any state) Receiver can sense progress towards goal Receiver can sense progress towards goal then then Receiver can achieve goal. Receiver can achieve goal. Functional definition of understanding. Functional definition of understanding. 01/06/2015CMI: Uncertain Communication21ReceiverdictionarySenderdictionary

of 27 Illustration: (Repeated) Coordination 01/06/2015CMI: Uncertain Communication22

of 27 Our setting 01/06/2015CMI: Uncertain Communication23

of 27 Coordination with Uncertainty 01/06/2015CMI: Uncertain Communication24

of 27 Lessons Coordination is possible: Coordination is possible: Even in extreme settings where Even in extreme settings where Alice has almost no idea of Bob Alice has almost no idea of Bob Bob has almost no idea of Alice Bob has almost no idea of Alice Alice is trying to learn Bob Alice is trying to learn Bob Bob is trying to learn Alice Bob is trying to learn Alice Learning is slow … Learning is slow … Need to incorporate beliefs to measure efficiency. [Juba, S. 2011] Need to incorporate beliefs to measure efficiency. [Juba, S. 2011] Does process become more efficient when languages have structure? [Open] Does process become more efficient when languages have structure? [Open] 01/06/2015CMI: Uncertain Communication25

of 27 Conclusions 01/06/2015CMI: Uncertain Communication26

of 27 Thank You! 01/06/2015CMI: Uncertain Communication27