of 13 October 6-7, 2010Emerging Frontiers of Information: Kickoff 1 Madhu Sudan Microsoft Research + MIT TexPoint fonts used in EMF. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A Read the TexPoint manual before you delete this box.: A A A A Communication & Computation A need for a new unifying theory

of 13 04/28/2009UT-Austin: Communication & Computation2 Theory of Communication Shannon’s architecture for Shannon’s architecture for communication over noisy channel communication over noisy channel Yields reliable communication Yields reliable communication (and storage (= communication across time)). (and storage (= communication across time)). Noisy Channel Encoder Decoder Y Ŷ m = E(m) D(Ŷ) = m?

of 13 Encoders & Decoders? Mathematically: (arbitrary) functions. Mathematically: (arbitrary) functions. Physically: Pieces of hardware – to compute certain functions. Physically: Pieces of hardware – to compute certain functions. (Mathematical/Physical question): (Mathematical/Physical question): Why should the set of functions required have nice hardware to compute them? Why should the set of functions required have nice hardware to compute them? October 6-7, 2010Emerging Frontiers of Information: Kickoff3

of 13 04/28/2009UT-Austin: Communication & Computation4 Theory of Computing Turing architecture Turing architecture FiniteState Control Control R/W UniversalMachine Encodings of other machines One machine to rule them all! → von Neumann architecture CPU RAM

of 13 Consequence Any ((polynomial time) computable) function does have nice hardware. Any ((polynomial time) computable) function does have nice hardware. However … However … Requires reliable CPU $ RAM Requires reliable CPU $ RAM Requires reliable RAM Requires reliable RAM Conclusion: Conclusion: Computing needs Communication Computing needs Communication Communication needs Computing Communication needs Computing (we are extremely lucky to get out of the loop) (we are extremely lucky to get out of the loop) October 6-7, 2010Emerging Frontiers of Information: Kickoff5

of 13 ~ 70 years since Computing and Communication thriving Computing and Communication thriving Separate theoretically Separate theoretically Unified physically (in gadgets in my pocket). Unified physically (in gadgets in my pocket). My Opinion: Need to unify the theories My Opinion: Need to unify the theories Why? Why? Need to know the limits of what can be done Need to know the limits of what can be done Need to know what is being done? Need to know what is being done? What are ? What are ? Modern practice has rich mix of computing/communication apps? What is the problem space? What is the solution space? Modern practice has rich mix of computing/communication apps? What is the problem space? What is the solution space? October 6-7, 2010Emerging Frontiers of Information: Kickoff6

of 13 Communication: Goals, Semantics To make for reliable, useful, communication: To make for reliable, useful, communication: Need to formalize: Goals of communication Need to formalize: Goals of communication In each usage In each usage Why is each player communicating? What is the end effect that it desires? Why is each player communicating? What is the end effect that it desires? What is each player assuming about others? What is each player assuming about others? How will it know if something goes wrong? How will it know if something goes wrong? Communication should preserve meaning: Communication should preserve meaning: Players allow for, and recover from, misunderstandings at endpoints. Players allow for, and recover from, misunderstandings at endpoints. October 6-7, 2010Emerging Frontiers of Information: Kickoff7

of 13 October 6-7, 2010Emerging Frontiers of Information: Kickoff8 The Meaning of Bits Why did miscommunication occur? Why did miscommunication occur? Alice assumed Bob knows colloquial English; Alice assumed Bob knows colloquial English; But she’s actually talking to some Bob’ who doesn’t. But she’s actually talking to some Bob’ who doesn’t. Channel Alice Bob Bob Bob Freeze!

of 13 Modelling Miscommunication Classical Shannon Model October 6-7, 2010Emerging Frontiers of Information: Kickoff9 A B Channel B2B2B2B2 AkAkAkAk A3A3A3A3 A2A2A2A2 A1A1A1A1 B1B1B1B1 B3B3B3B3 BjBjBjBj Semantic Communication Model

of 13 Our claims ( based on joint works with Juba, Goldreich, Kalai, Khanna) Semantic Communication can be achieved with almost full diversity Semantic Communication can be achieved with almost full diversity If goals of communication are formally captured and “sensible”; and players are “helpful” [GJS] If goals of communication are formally captured and “sensible”; and players are “helpful” [GJS] Communication does offer advantages even in purely informational settings [JS1] Communication does offer advantages even in purely informational settings [JS1] Can be attained almost w.l.o.efficiency [JS2] Can be attained almost w.l.o.efficiency [JS2] Explains difference between engineered and natural communication (e.g., ambiguity in natural language) [JKKS] Explains difference between engineered and natural communication (e.g., ambiguity in natural language) [JKKS] October 6-7, 2010Emerging Frontiers of Information: Kickoff10

of 13 Communicating Knowledge [JS1] Knowledge (X) = Subset of Information (X) Knowledge (X) = Subset of Information (X) Useful/Useable Useful/Useable Not known without X. [Goldwasser,Micali,Rackoff ~86] Not known without X. [Goldwasser,Micali,Rackoff ~86] Knowledge · Information (usually <<) Knowledge · Information (usually <<) In all practical settings: Suffices to communication Knowledge. In all practical settings: Suffices to communication Knowledge. Can be communicated with misunderstanding. Can be communicated with misunderstanding. Can be used to overcome misunderstanding. Can be used to overcome misunderstanding. Because it is useful! Because it is useful! October 6-7, 2010Emerging Frontiers of Information: Kickoff11

of 13 Going Forward Need to build robust theory of “why (anyone/everyone) communicate(s)” Need to build robust theory of “why (anyone/everyone) communicate(s)” Combine with Semantics Combine with Semantics Needed to improve Needed to improve Improve Reliability Improve Reliability Reduce Vulnerability Reduce Vulnerability Increase efficiency Increase efficiency Safely transfer all information to digital form. Safely transfer all information to digital form. October 6-7, 2010Emerging Frontiers of Information: Kickoff12 Channel B2B2B2B2 AkAkAkAk A3A3A3A3 A2A2A2A2 A1A1A1A1 B1B1B1B1 B3B3B3B3 BjBjBjBj

of 13 References [JS1]: Juba and Sudan, STOC [JS1]: Juba and Sudan, STOC [GJS]: Goldreich, Juba, and Sudan, ECCC 2009 [GJS]: Goldreich, Juba, and Sudan, ECCC 2009 [JS2]: Juba and Sudan, ICS 2011 (to appear) [JS2]: Juba and Sudan, ICS 2011 (to appear) [JKKS]: Juba, Kalai, Khanna, Sudan, ICS 2011 (to appear) [JKKS]: Juba, Kalai, Khanna, Sudan, ICS 2011 (to appear) October 6-7, 2010Emerging Frontiers of Information: Kickoff13