1. Mobility, Security, and Proof-Carrying Code Peter Lee Carnegie Mellon University Lecture 1 Course Overview July 10, 2001 Lipari School on Foundations of Wide Area Network Programming

2. Opportunities and Challenges

3. Arianne 5 On June 4, 1996, the Arianne 5 took off on its maiden flight. 40 seconds into its flight it veered off course and exploded. It was later found to be an error in reuse of a software component. For the next two years, virtually every research presentation used this picture.

4. “Better, Faster, Cheaper” In 1999, NASA lost both the Mars Polar Lander and the Climate Orbiter. Later investigations determined software errors were to blame. Orbiter: Component reuse error. Lander: Precondition violation.

5. USS Yorktown “After a crew member mistakenly entered a zero into the data field of an application, the computer system proceeded to divide another quantity by that zero. The operation caused a buffer overflow, in which data leaked from a temporary storage space in memory, and the error eventually brought down the ship's propulsion system. The result: the Yorktown was dead in the water for more than two hours.”

6. Programmable mobile devices By 2003, one in five people will own a mobile communications device. Nokia expects to sell 500M Java-enabled phones in 2003. Most of these devices will be power and memory limited.

7. Observations Failures often due to simple problems “in the details.” Reuse is critical but perilous. Performance still matters a lot.

8. Safety Engineering Small theorems about large programs would be useful. Need clearly specified interfaces and checking of interface compliance. Must not sacrifice performance.

9. But in the Real World?

10. Security Attacks According to CERT, the majority of security attacks exploit input validation failure buffer overflow VBS http://www.cert.org/summaries/CS-2000-04.html

11. BSOD embarrassments

12. Warrantees? LIMITED WARRANTY. Microsoft warrants that (a) the SOFTWARE PRODUCT will perform substantially in accordance with the accompanying written materials for a period of ninety (90) days from the date of receipt, … LIMITATION OF LIABILITY. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL MICROSOFT OR ITS SUPPLIERS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT, OR CONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING, …) ARISING OUT OF THE USE OF … THE SOFTWARE PRODUCT… MICROSOFT’S ENTIRE LIABILITY … SHALL BE LIMITED TO THE GREATER OF THE AMOUNT ACTUALLY PAID BY YOU FOR THE SOFTWARE PRODUCT OR U.S. $5.00; PROVIDED...

13. Automotive Analogy “If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and...

14. Automotive Analogy “If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year killing everyone inside." - Robert Cringely

15. Mobile/Wireless Devices In ‘97, 101M mobile phones vs 82M PCs. (40% vs 14%.) 95% phones will be WAP enabled by ‘04. 64Mbits of RAM in 2002. Battery life a primary factor. Efficiency and bandwidth will still be precious.

16. Bluetooth 670M Bluetooth-enabled devices by ‘03. 70% of mobile phones Bluetooth-enabled by ‘04. Priceline.com’s grocery- store scenario. Commercial world creates demand for “push” technologies.

17. Networked Appliances By far the largest- growing segment. Enormous diversity of platforms. Reliability and longevity are expected. Major challenges for OS and language standards.

18. Commercial Demands Performance. Mobility/extensibility. Reliability/quality. Well-defined languages. Scalable security.

19. Opportunities High assurance depends fundamentally on our ability to reason about programs. The opportunities for computational logic, type theory, and formal semantics are great.

20. Challenges The impact and cost of software failures will increase, as will the demand for extensibility. The distinction between “safety- critical” and “consumer electronics” software will fade away. Somebody will provide technology for “safe” systems. Will it be us?

21. Is the World Ready?

22. What we start with: What we want: What we get along the way:

23. Is the World Ready? What we start with: What we want: What we get along the way:

24. Cheese and the Sum Total of Human Knowledge

25. The Code Safety Problem

26. Please install and execute this.

27. Code Safety CPU Code Trusted Host Is this safe to execute?

28. Approach 1 Trust the Code Producer CPU Code Trusted Host sig Trusted 3rd Party PK1 PK2 Trust is based on personal authority, not program properties Scaling problems?

29. Approach 2 Baby-sit the Program CPU Code Trusted Host Execution monitor Expensive Limited in expressive power (Why?) E.g., Software Fault Isolation [Wahbe & Lucco], Inline Reference Monitors [Schneider]

30. Approach 3 Java CPU Code Trusted Host Interp/ JIT Expensive and/or big Limited in expressive power Verifier

31. Theorem Prover Approach 4 Formal Verification CPU Code Flexible and powerful. Trusted Host But really really really hard and must be correct.

32. A Key Idea: Explicit Proofs Certifying Prover CPU Proof Checker Code Proof Trusted Host

33. A Key Idea: Explicit Proofs Certifying Prover CPU Code Proof No longer need to trust this component. Proof Checker

34. Proof-Carrying Code [Necula & Lee, OSDI’96] A B Formal proof or “explanation” of safety Typically native or VM code rlrrllrrllrlrlrllrlrrllrrll…

35. Proof-Carrying Code Certifying Prover CPU Code Proof Simple, small (<52KB), and fast. No longer need to trust this component. Proof Checker Reasonable in size (0-10%).

36. But......How to generate the proofs? Proving theorems about real programs is hard. Most useful safety properties of low-level programs are undecidable. Theorem-proving systems are unfamiliar to programmers and hard to use even for experts.

37. The Role of Programming Languages Civilized programming languages can provide “safety for free”. Well-formed/well-typed  safe. Idea: Arrange for the compiler to “explain” why the target code it generates preserves the safety properties of the source program.

38. The Role of Java in this Short Course Java will be the main focus of the PCC examples in this course. Java is just barely a civilized programming language. We can and should do better.

39. Java Java is a worthwhile subject of research. However, it contains many outrageous and mostly inexcusable design errors. As researchers, we should not forget that we have already done much better, and must continue to do better in the future.

40. Certifying Compilers [Necula & Lee, PLDI’98] Intuition: Compiler “knows” why each translation step is semantics-preserving. So, have it generate a proof that safety is preserved. “Small theorems about big programs.” Don’t try to verify the whole compiler, but only each output it generates.

41. Automation via Certifying Compilation Certifying Compiler CPU Looks and smells like a compiler. % spjc foo.java bar.class baz.c -ljdk1.2.2 Source code Proof Object code Certifying Prover Proof Checker

42. Overview of the Necula/Lee Approach to PCC

43. Note Our current approach seems to work for many problems. But it is the only one we have tried — there are many others. PCC is a general concept and we have just barely scratched the surface.

44. Overview of Our Approach Please install and execute this. OK, but let me quickly look over the instructions first. Code producerHost

45. Overview of Our Approach Code producerHost

46. Overview of Our Approach This store instruction is dangerous! Code producerHost

47. Overview of Our Approach Can you prove that it is always safe? Code producerHost

48. Overview of Our Approach Can you prove that it is always safe? Yes! Here’s the proof I got from my certifying Java compiler! Code producerHost

49. Overview of Our Approach Your proof checks out. I believe you because I believe in logic. Code producerHost

50. Course Overview

51. This Short Course This short course will focus on the concept of proof-carrying code. PCC addresses code safety issues. Reducing the trusted computing base. Introducing a concept of “proof engineering”. Exploiting modern ideas in compiling, theorem-proving, and logic programming.

52. Proof Engineering This course will spend much of its time on engineering matters. In particular, the problems of “scaling up” ideas to handle realistic problems. A completely formal or systematic understanding of many of the concepts has not yet been attained.

53. Outline In four parts: 0) Introduction and informal overview. 1) Safety infrastructure: proof representation and checking. 2) Verification and programming tools. 3) System engineering and related work.

54. Summary

55. The code safety problem presents great opportunities and challenges for applied logic and programming language design. Proof-carrying code may be an example of how current knowledge can be applied to practical problems.

56. Homework Exercise 1 Certifying Compiler CPU Source code Proof Object code Certifying Prover Proof Checker The architecture shown in this lecture has the compiler and prover as separate communicating components. An alternative would be to have a single component that compiles and proves simultaneously. What are some advantages and disadvantages of the separate- component approach?