1. Mobility, Security, and Proof-Carrying Code Peter Lee Carnegie Mellon University Lecture 1 Course Overview July 10, 2001 Lipari School on Foundations of Wide Area Network Programming

2. Opportunities and Challenges

3. Carnegie Mellon Arianne 5 On June 4, 1996, the Arianne 5 took off on its maiden flight. 40 seconds into its flight it veered off course and exploded. It was later found to be an error in reuse of a software component. For the next two years, virtually every research presentation used this picture.

4. Carnegie Mellon “Better, Faster, Cheaper” In Sept.’99, NASA lost both the Mars Polar Lander and the Climate Orbiter. Later investigations determined software errors were to blame. Orbiter: Component reuse error. Lander: Precondition violation.

5. Carnegie Mellon USS Yorktown “After a crew member mistakenly entered a zero into the data field of an application, the computer system proceeded to divide another quantity by that zero. The operation caused a buffer overflow, in which data leaked from a temporary storage space in memory, and the error eventually brought down the ship's propulsion system. The result: the Yorktown was dead in the water for more than two hours.”

6. Carnegie Mellon Observations Failures often due to simple problems “in the details.” Reuse is critical but perilous. Performance still matters a lot.

7. Carnegie Mellon Safety Engineering Small theorems about large programs would be useful. Need clearly specified interfaces and checking of interface compliance. Must not sacrifice performance.

8. But in the Real World?

9. Carnegie Mellon Security Attacks According to CERT, the majority of security attacks exploit input validation failure buffer overflow VBS http://www.cert.org/summaries/CS-2000-04.html

10. Carnegie Mellon BSOD embarrassments

11. Carnegie Mellon Warrantees? LIMITED WARRANTY. Microsoft warrants that (a) the SOFTWARE PRODUCT will perform substantially in accordance with the accompanying written materials for a period of ninety (90) days from the date of receipt, … LIMITATION OF LIABILITY. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL MICROSOFT OR ITS SUPPLIERS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT, OR CONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING, …) ARISING OUT OF THE USE OF … THE SOFTWARE PRODUCT… MICROSOFT’S ENTIRE LIABILITY … SHALL BE LIMITED TO THE GREATER OF THE AMOUNT ACTUALLY PAID BY YOU FOR THE SOFTWARE PRODUCT OR U.S. $5.00; PROVIDED...

12. Carnegie Mellon Automotive Analogy “If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and...

13. Carnegie Mellon Automotive Analogy “If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year killing everyone inside." - Robert Cringely

14. Carnegie Mellon Mobile/Wireless Devices In ‘97, 101M mobile phones vs 82M PCs. (40% vs 14%.) 95% phones will be WAP enabled by ‘04. 64Mbits of RAM in 2002. Battery life a primary factor. Efficiency and bandwidth will still be precious.

15. Carnegie Mellon Bluetooth 670M Bluetooth-enabled devices by ‘03. 70% of mobile phones Bluetooth-enabled by ‘04. Priceline.com’s grocery- store scenario. Commercial world creates demand for “push” technologies.

16. Carnegie Mellon Networked Appliances By far the largest- growing segment. Enormous diversity of platforms. Reliability and longevity are expected. Major challenges for OS and language standards.

17. Carnegie Mellon Commercial Demands Performance. Mobility/extensibility. Reliability/quality. Well-defined languages. Scalable security.

18. Carnegie Mellon Opportunities High assurance depends fundamentally on our ability to reason about programs. The opportunities for computational logic, type theory, and formal semantics are great.

19. Carnegie Mellon Challenges The impact and cost of software failures will increase, as will the demand for extensibility. The distinction between “safety- critical” and “consumer electronics” software will fade away. Somebody will provide technology for “safe” systems. Will it be us?

20. Is the World Ready?

21. Carnegie Mellon Is the World Ready? What we start with: What we want: What we get along the way:

22. Carnegie Mellon Is the World Ready? What we start with: What we want: What we get along the way:

23. Carnegie Mellon Cheese and the Sum Total of Human Knowledge

24. The Code Safety Problem

25. Carnegie Mellon The Code Safety Problem Please install and execute this.

26. Carnegie Mellon Code Safety CPU Code Trusted Host Is this safe to execute?

27. Carnegie Mellon Approach 1 Trust the Code Producer CPU Code Trusted Host sig Trusted 3rd Party PK1 PK2 Trust is based on personal authority, not program properties Scaling problems?

28. Carnegie Mellon Approach 2 Baby-sit the Program CPU Code Trusted Host Execution monitor Expensive Limited in expressive power (Why?) E.g., Software Fault Isolation [Wahbe & Lucco], Inline Reference Monitors [Schneider]

29. Carnegie Mellon Approach 3 Java CPU Code Trusted Host Interp/ JIT Expensive and/or big Limited in expressive power Verifier

30. Carnegie Mellon Theorem Prover Approach 4 Formal Verification CPU Code Flexible and powerful. Trusted Host But really really really hard and must be correct.

31. Carnegie Mellon A Key Idea: Explicit Proofs Certifying Prover CPU Proof Checker Code Proof Trusted Host

32. Carnegie Mellon A Key Idea: Explicit Proofs Certifying Prover CPU Code Proof No longer need to trust this component. Proof Checker

33. Carnegie Mellon Proof-Carrying Code [Necula & Lee, OSDI’96] A B Formal proof or “explanation” of safety Typically native or VM code rlrrllrrllrlrlrllrlrrllrrll…

34. Carnegie Mellon Proof-Carrying Code Certifying Prover CPU Code Proof Simple, small (<52KB), and fast. No longer need to trust this component. Proof Checker Reasonable in size (0-10%).

35. Carnegie Mellon But......How to generate the proofs? Proving theorems about real programs is hard. Most useful safety properties of low-level programs are undecidable. Theorem-proving systems are unfamiliar to programmers and hard to use even for experts.

36. Carnegie Mellon The Role of Programming Languages Civilized programming languages can provide “safety for free”. Well-formed/well-typed  safe. Idea: Arrange for the compiler to “explain” why the target code it generates preserves the safety properties of the source program.

37. Carnegie Mellon Certifying Compilers [Necula & Lee, PLDI’98] Intuition: Compiler “knows” why each translation step is semantics-preserving. So, have it generate a proof that safety is preserved. “Small theorems about big programs.” Don’t try to verify the whole compiler, but only each output it generates.

38. Carnegie Mellon Automation via Certifying Compilation Certifying Compiler CPU Looks and smells like a compiler. % spjc foo.java bar.class baz.c -ljdk1.2.2 Source code Proof Object code Certifying Prover Proof Checker

39. Overview of the Necula/Lee Approach to PCC

40. Carnegie Mellon Note Our current approach seems to work for many problems. But it is the only one we have tried — there are many others. PCC is a general concept and we have just barely scratched the surface.

41. Carnegie Mellon Overview of Our Approach Please install and execute this. OK, but let me quickly look over the instructions first. Code producerHost

42. Carnegie Mellon Overview of Our Approach Code producerHost

43. Carnegie Mellon Overview of Our Approach This store instruction is dangerous! Code producerHost

44. Carnegie Mellon Overview of Our Approach Can you prove that it is always safe? Code producerHost

45. Carnegie Mellon Overview of Our Approach Can you prove that it is always safe? Yes! Here’s the proof I got from my certifying Java compiler! Code producerHost

46. Carnegie Mellon Overview of Our Approach Your proof checks out. I believe you because I believe in logic. Code producerHost

47. Course Overview

48. Carnegie Mellon This Short Course This short course will focus on the concept of proof-carrying code. PCC addresses code safety issues. Reducing the trusted computing base. Introducing a concept of “proof engineering”. Exploiting modern ideas in compiling, theorem-proving, and logic programming.

49. Carnegie Mellon Proof Engineering This course will spend much of its time on engineering matters. In particular, the problems of “scaling up” ideas to handle realistic problems. A formal or systematic understanding of many of the concepts has not yet been attained.

50. Carnegie Mellon Outline In four parts: 0) Introduction and informal overview. 1) Safety infrastructure: proof representation and checking. 2) Verification and programming tools. 3) System engineering and related work.

51. Summary

52. Carnegie Mellon Summary The code safety problem presents great opportunities and challenges for applied logic and programming language design. Proof-carrying code may be an example of how current knowledge can be applied to practical problems.