Presentation is loading. Please wait.

Presentation is loading. Please wait.

Of 12 03/22/2012CISS: Compression w. Uncertain Priors1 Compression under uncertain priors Madhu Sudan Microsoft, New England Based on joint works with:

Published byEustacia Clarke Modified over 9 years ago

Similar presentations

Presentation on theme: "Of 12 03/22/2012CISS: Compression w. Uncertain Priors1 Compression under uncertain priors Madhu Sudan Microsoft, New England Based on joint works with:"— Presentation transcript:

1 of 12 03/22/2012CISS: Compression w. Uncertain Priors1 Compression under uncertain priors Madhu Sudan Microsoft, New England Based on joint works with: Adam Kalai (MSR), Sanjeev Khanna (U.Penn) and Brendan Juba (Harvard)

2 of 12 Motivation: Human Communication Human communication (dictated by languages, grammars) very different. Human communication (dictated by languages, grammars) very different. Grammar: Rules, often violated. Grammar: Rules, often violated. Dictionary: Often multiple meanings to a word. Dictionary: Often multiple meanings to a word. Redundant: But not as in any predefined way (not an error-correcting code). Redundant: But not as in any predefined way (not an error-correcting code). Theory? Theory? Information theory? Information theory? Linguistics? (Universal grammars etc.)? Linguistics? (Universal grammars etc.)? 03/22/2012CISS: Compression w. Uncertain Priors2

3 of 12 Behavioral aspects of natural communication (Vast) Implicit context. (Vast) Implicit context. Sender sends increasingly long messages to receiver till receiver “gets” (the meaning of) the message. Sender sends increasingly long messages to receiver till receiver “gets” (the meaning of) the message. Sender may use feedback from receiver if available; or estimates receiver’s knowledge if not. Sender may use feedback from receiver if available; or estimates receiver’s knowledge if not. Language provides sequence of (increasingly) long ways to represent a message. Language provides sequence of (increasingly) long ways to represent a message. Question: What is the benefit of choosing short/long messages? Question: What is the benefit of choosing short/long messages? 03/22/2012CISS: Compression w. Uncertain Priors3

4 of 12 Model: Reason to choose short messages: Compression. Reason to choose short messages: Compression. Channel is still a scarce resource; still want to use optimally. Channel is still a scarce resource; still want to use optimally. Reason to choose long messages (when short ones are available): Reducing ambiguity. Reason to choose long messages (when short ones are available): Reducing ambiguity. Sender unsure of receiver’s prior (context). Sender unsure of receiver’s prior (context). Sender wishes to ensure receiver gets the message, no matter what its prior (within reason). Sender wishes to ensure receiver gets the message, no matter what its prior (within reason). 03/22/2012CISS: Compression w. Uncertain Priors4

5 of 12 Model 03/22/2012CISS: Compression w. Uncertain Priors5

6 of 12 Contrast with some previous models Universal compression? Universal compression? Doesn’t apply: P,Q are not finitely specified. Doesn’t apply: P,Q are not finitely specified. Don’t have a sequence of samples from P; just one! Don’t have a sequence of samples from P; just one! K-L divergence? K-L divergence? Measures inefficiency of compressing for Q if real distribution is P. Measures inefficiency of compressing for Q if real distribution is P. But assumes encoding/decoding according to same distribution Q. But assumes encoding/decoding according to same distribution Q. Semantic Communication: Semantic Communication: Uncertainty of sender/receiver; but no special goal. Uncertainty of sender/receiver; but no special goal. 03/22/2012CISS: Compression w. Uncertain Priors6

7 of 12 Closeness of distributions: 03/22/2012CISS: Compression w. Uncertain Priors7

8 of 12 Dictionary = Shared Randomness? 03/22/2012CISS: Compression w. Uncertain Priors8

9 of 12 Solution (variant of Arith. Coding) 03/22/2012CISS: Compression w. Uncertain Priors9

10 of 12 Performance 03/22/2012CISS: Compression w. Uncertain Priors10

11 of 12 Implications 03/22/2012CISS: Compression w. Uncertain Priors11

12 of 12 Future work? Upcoming: Upcoming: Some partial derandomization Some partial derandomization [w. E. Haramaty and G.Ranade] [w. E. Haramaty and G.Ranade] Neat connections to fractional graph chromaticity and the Knesser conjecture/Lovasz theorem. Neat connections to fractional graph chromaticity and the Knesser conjecture/Lovasz theorem. Needed: Needed: Better understanding of forces on language. Better understanding of forces on language. Information-theoretic Information-theoretic Computational Computational Evolutionary Evolutionary 03/22/2012CISS: Compression w. Uncertain Priors12

13 of 12 Thank You 03/22/2012CISS: Compression w. Uncertain Priors13

Download ppt "Of 12 03/22/2012CISS: Compression w. Uncertain Priors1 Compression under uncertain priors Madhu Sudan Microsoft, New England Based on joint works with:"

Similar presentations

Pretty-Good Tomography Scott Aaronson MIT. Theres a problem… To do tomography on an entangled state of n qubits, we need exp(n) measurements Does this.

Pretty-Good Tomography Scott Aaronson MIT. Theres a problem… To do tomography on an entangled state of n qubits, we need exp(n) measurements Does this.
Of 23 09/24/2013HLF: Reliable Meaningful Communication1 Reliable Meaningful Communication Madhu Sudan Microsoft, Cambridge, USA.

Of 23 09/24/2013HLF: Reliable Meaningful Communication1 Reliable Meaningful Communication Madhu Sudan Microsoft, Cambridge, USA.
Of 35 05/30/2012CSOI-Summer: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.

Of 35 05/30/2012CSOI-Summer: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.
Of 12 12/04/2013CSOI: Communication as Coordination1 Communication as Coordination Madhu Sudan Microsoft, Cambridge, USA -

Of 12 12/04/2013CSOI: Communication as Coordination1 Communication as Coordination Madhu Sudan Microsoft, Cambridge, USA -
Of 19 03/21/2012CISS: Beliefs in Communication1 Efficient Semantic Communication & Compatible Beliefs Madhu Sudan Microsoft, New England Based on joint.

Of 19 03/21/2012CISS: Beliefs in Communication1 Efficient Semantic Communication & Compatible Beliefs Madhu Sudan Microsoft, New England Based on joint.
Of 13 10/08/2013MSRNE 5 th Anniversary: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.

Of 13 10/08/2013MSRNE 5 th Anniversary: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.
Of 29 12/02/2013Purdue: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.

Of 29 12/02/2013Purdue: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.
Of 24 11/20/2012TIFR: Deterministic Communication Amid Uncertainty1 ( Deterministic ) Communication amid Uncertainty Madhu Sudan Microsoft, New England.

Of 24 11/20/2012TIFR: Deterministic Communication Amid Uncertainty1 ( Deterministic ) Communication amid Uncertainty Madhu Sudan Microsoft, New England.
Of 30 10/31/2013Cornell: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.

Of 30 10/31/2013Cornell: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.
Of 30 09/16/2013PACM: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.

Of 30 09/16/2013PACM: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: -Universal Semantic.
Language Models Naama Kraus (Modified by Amit Gross) Slides are based on Introduction to Information Retrieval Book by Manning, Raghavan and Schütze.

Language Models Naama Kraus (Modified by Amit Gross) Slides are based on Introduction to Information Retrieval Book by Manning, Raghavan and Schütze.
Of 7 10/01/2013LIDS Lunch: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.

Of 7 10/01/2013LIDS Lunch: Communication Amid Uncertainty1 Communication Amid Uncertainty Madhu Sudan Microsoft Research.
Semantic Communication Madhu Sudan (based on joint work with Brendan Juba (MIT); and upcoming work with Oded Goldreich (Weizmann) & J. )

Semantic Communication Madhu Sudan (based on joint work with Brendan Juba (MIT); and upcoming work with Oded Goldreich (Weizmann) & J. )
The Roles of Uncertainty and Randomness in Online Advertising Ragavendran Gopalakrishnan Eric Bax Raga Gopalakrishnan 2 nd Year Graduate Student (Computer.

The Roles of Uncertainty and Randomness in Online Advertising Ragavendran Gopalakrishnan Eric Bax Raga Gopalakrishnan 2 nd Year Graduate Student (Computer.
Of 30 09/04/2012ITW 2012: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.

Of 30 09/04/2012ITW 2012: Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.
1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,

1 Information complexity and exact communication bounds April 26, 2013 Mark Braverman Princeton University Based on joint work with Ankit Garg, Denis Pankratov,
Of 27 01/06/2015CMI: Uncertain Communication1 Communication Amid Uncertainty Madhu Sudan Microsoft Research Based on Juba, S. (STOC 2008, ITCS 2011) Juba,

Of 27 01/06/2015CMI: Uncertain Communication1 Communication Amid Uncertainty Madhu Sudan Microsoft Research Based on Juba, S. (STOC 2008, ITCS 2011) Juba,
Of 13 October 6-7, 2010Emerging Frontiers of Information: Kickoff 1 Madhu Sudan Microsoft Research + MIT TexPoint fonts used in EMF. TexPoint fonts used.

Of 13 October 6-7, 2010Emerging Frontiers of Information: Kickoff 1 Madhu Sudan Microsoft Research + MIT TexPoint fonts used in EMF. TexPoint fonts used.
Of 10 Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.

Of 10 Uncertainty in Communication1 Communication amid Uncertainty Madhu Sudan Microsoft, Cambridge, USA Based on: Universal Semantic.
Amparo Urbano (with P. Hernandez and J. Vila) University of Valencia. ERI-CES Pragmatic Languages with Universal Grammars: An Equilibrium Approach.

Amparo Urbano (with P. Hernandez and J. Vila) University of Valencia. ERI-CES Pragmatic Languages with Universal Grammars: An Equilibrium Approach.

Similar presentations

Ads by Google