1. of 27 12/03/2015 Boole-Shannon: Laws of Communication of Thought 1 Laws of Communication of Thought? Madhu Sudan Harvard

2. of 27 George Boole (1815-1864) 12/03/2015 Boole-Shannon: Laws of Communication of Thought2 (page 34)

3. of 27 Boole’s Mathematics: Boole’s Mathematics: Focus on tiny part of mathematical universe. Focus on tiny part of mathematical universe. 12/03/2015Boole-Shannon: Laws of Communication of Thought3 {0,1} Progress In Math

4. of 27 Boole’s “modest” ambition “The design of the following treatise is to investigate the fundamental laws of those operations of the mind by which reasoning is performed; to give expression to them in the symbolical language of a Calculus, and upon this foundation to establish the science of Logic and construct its method; to make that method itself the basis of a general method for the application of the mathematical doctrine of Probabilities; and, finally, to collect from the various elements of truth brought to view in the course of these inquiries some probable intimations concerning the nature and constitution of the human mind.” [G.Boole, “On the laws of thought …” p.1] “The design of the following treatise is to investigate the fundamental laws of those operations of the mind by which reasoning is performed; to give expression to them in the symbolical language of a Calculus, and upon this foundation to establish the science of Logic and construct its method; to make that method itself the basis of a general method for the application of the mathematical doctrine of Probabilities; and, finally, to collect from the various elements of truth brought to view in the course of these inquiries some probable intimations concerning the nature and constitution of the human mind.” [G.Boole, “On the laws of thought …” p.1] 12/03/2015Boole-Shannon: Laws of Communication of Thought4 {0,1}

5. of 27 Shannon (1916-2001) “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.... The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design.” [Mathematical Theory of Communication. 1948 ] “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.... The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design.” [Mathematical Theory of Communication. 1948 ] 12/03/2015Boole-Shannon: Laws of Communication of Thought5 Engineering is the motivation! Engineering is the motivation! Mathematics emerges from the theory Mathematics emerges from the theory Information (Bit), Entropy, Capacity, Rate … Information (Bit), Entropy, Capacity, Rate … The Probabilistic Method! The Probabilistic Method! … And captures natural phenomena … And captures natural phenomena Yet …

6. of 27 E.g. “Series of approx. to English” 12/03/2015Boole-Shannon: Laws of Communication of Thought6

7. of 27 12/03/2015Boole-Shannon: Laws of Communication of Thought7 ? X+ X X !

8. of 27 The broader picture 12/03/2015Boole-Shannon: Laws of Communication of Thought8

9. of 27 Some CS Adventures 12/03/2015Boole-Shannon: Laws of Communication of Thought9

10. of 27 1950-today Old “axioms” Old “axioms” Discrete communication Discrete communication Discrete reasoning Discrete reasoning New “axiom” New “axiom” Resource bounded computing. (P,NP …) Resource bounded computing. (P,NP …) Many phenomena captured: Many phenomena captured: Cryptography, (Pseudo-)Randomness, Knowledge, Privacy Cryptography, (Pseudo-)Randomness, Knowledge, Privacy 12/03/2015Boole-Shannon: Laws of Communication of Thought10

11. of 27 Pseudorandomness? 12/03/2015Boole-Shannon: Laws of Communication of Thought11

12. of 27 (computational in-) Distinguishability 12/03/2015Boole-Shannon: Laws of Communication of Thought12

13. of 27 Knowledge? Some sequences have lots of information but little knowledge. Some sequences have lots of information but little knowledge. E.g., my email … every day! E.g., my email … every day! Why? What is the difference? Why? What is the difference? Can sequences have more knowledge than information? Can sequences have more knowledge than information? E.g., your banking password: E.g., your banking password: Your internet provider already has sufficient “information”, but hopefully not “knowledge” Your internet provider already has sufficient “information”, but hopefully not “knowledge” 12/03/2015Boole-Shannon: Laws of Communication of Thought13

14. of 27 Knowledge [ GoldwasserMicaliRackoff’86 ] 12/03/2015Boole-Shannon: Laws of Communication of Thought14 Would you pay for it? Shouldn’t pay for random bits!

15. of 27 Communication of Thought 12/03/2015Boole-Shannon: Laws of Communication of Thought15

16. of 27 Motivation Communication as a “network phenomenon” Communication as a “network phenomenon” Intellectual Pursuit: Intellectual Pursuit: Start with existing knowledge. Start with existing knowledge. Add to the body of knowledge. Add to the body of knowledge. Find new facts/concepts/designs that rely on knowledge from past. Find new facts/concepts/designs that rely on knowledge from past. Repeat. Repeat. Two questions: Two questions: What is the error-correction mechanism? What is the error-correction mechanism? How do we communicate “knowledge” (language, dictionary, journals …) How do we communicate “knowledge” (language, dictionary, journals …) 12/03/2015Boole-Shannon: Laws of Communication of Thought16

17. of 27 Error-correction via proofs Communication of Mathematical knowledge: Communication of Mathematical knowledge: Doesn’t matter how you arrive at the new piece of knowledge. Doesn’t matter how you arrive at the new piece of knowledge. But must provide “self-contained proof” of the new piece. But must provide “self-contained proof” of the new piece. Sound approach: Sound approach: Is it slowing us down? Is it slowing us down? Verification time-consuming Verification time-consuming Proofs long … Proofs long … 12/03/2015Boole-Shannon: Laws of Communication of Thought17

18. of 27 Variations on Proofs Central part of “CS Adventures” Central part of “CS Adventures” Relates to computation [ Turing, Church, Gödel ] Relates to computation [ Turing, Church, Gödel ] Proofs need to be “easy” to verify. Proofs need to be “easy” to verify. Relates to cryptography [ GMR ] Relates to cryptography [ GMR ] How can user convince system it has the right to access some resource. How can user convince system it has the right to access some resource. Relates to optimization [ Cook, Levin, Karp ] Relates to optimization [ Cook, Levin, Karp ] Better mechanisms for proofs lead to stronger barriers to optimization. Better mechanisms for proofs lead to stronger barriers to optimization. 12/03/2015Boole-Shannon: Laws of Communication of Thought18

19. of 27 Variations on Proofs - 2 12/03/2015Boole-Shannon: Laws of Communication of Thought19

20. of 27 Variations on Proofs - 3 12/03/2015Boole-Shannon: Laws of Communication of Thought20

21. of 27 Variations on Proofs - 4 C omputationally S ound (CS) Proofs [ Kilian’90,Micali’94 ] C omputationally S ound (CS) Proofs [ Kilian’90,Micali’94 ] “proofs” much shorter than classical proofs. “proofs” much shorter than classical proofs. Proofs of incorrect statements can exist, but are very hard to find! Proofs of incorrect statements can exist, but are very hard to find! Incrementally Verifiable Proofs [P.Valiant ‘08] Incrementally Verifiable Proofs [P.Valiant ‘08] Can utilize pre-existing knowledge+proofs to generate (short, CS) proofs of new facts Can utilize pre-existing knowledge+proofs to generate (short, CS) proofs of new facts Proofs for Delegated computing, Quantum Proofs … Proofs for Delegated computing, Quantum Proofs … 12/03/2015Boole-Shannon: Laws of Communication of Thought21

22. of 27 Do proofs solve the problem? No. (Even in Mathematics) Proofs are never self- contained. No. (Even in Mathematics) Proofs are never self- contained. Always assume (even to state theorem) … Always assume (even to state theorem) … Language (Unboundedly Large). Language (Unboundedly Large). Background/Context (Bounded, but still enormous). Background/Context (Bounded, but still enormous). Most effective communication relies on the fact that context is shared (even if imperfectly). Most effective communication relies on the fact that context is shared (even if imperfectly). 12/03/2015Boole-Shannon: Laws of Communication of Thought22 “ We give a elementary self-contained proof of *** theorem”. But two sentences earlier: “We assume reader is familiar with [***]” (60 pages)

23. of 27 Unbounded Language Problem 12/03/2015Boole-Shannon: Laws of Communication of Thought23

24. of 27 Communicating with large context 12/03/2015Boole-Shannon: Laws of Communication of Thought24

25. of 27 More about Context Many forms of communication rely on shared context. Many forms of communication rely on shared context. Natural Communication: Natural Communication: Language, Grammar, Common Knowledge … (typically each unbounded). Language, Grammar, Common Knowledge … (typically each unbounded). Designed Communication: Designed Communication: Protocols, Encoding/Decoding functions (used to be bounded, now no longer so). Protocols, Encoding/Decoding functions (used to be bounded, now no longer so). Are they resilient to imperfect sharing? Are they resilient to imperfect sharing? Natural … maybe! What are the axioms? Natural … maybe! What are the axioms? Designed … no! What can we do? Designed … no! What can we do? 12/03/2015Boole-Shannon: Laws of Communication of Thought25 No Certainty (ever) No Certainty (ever) Trust but check Trust but check Simplicity first Simplicity first Survival of fittest Survival of fittest

26. of 27 Broader investigation of Communication Social/Network Process: Social/Network Process: No one player makes all rules No one player makes all rules In fact may have few central players In fact may have few central players Cooperative game Cooperative game What are good practices: What are good practices: Can we use Boolean calculus and information theoretic principles to compare solutions. Can we use Boolean calculus and information theoretic principles to compare solutions. Can we recommend good solutions: Can we recommend good solutions: Ultimate archival format for documents in digital libraries? Ultimate archival format for documents in digital libraries? Let machines (learn to) talk to each other without human intervention. Let machines (learn to) talk to each other without human intervention. Can we measure societal (intellectual) progress? Will it be monotone? Can we measure societal (intellectual) progress? Will it be monotone? 12/03/2015Boole-Shannon: Laws of Communication of Thought26

27. of 27 Thank You! 12/03/2015Boole-Shannon: Laws of Communication of Thought27