1. CS703 – Program verification and synthesis Loris D’Antoni
Course introduction
CS703 – Program verification and synthesis
Loris D’Antoni

2. Introduce yourself Name What year Who do you work with
What do you work on
Any hobby
Example:
Hi everyone, I’m Loris D’Antoni and I’m a professor in the programming languages group
I work on program verification and program synthesis
Among other things I like playing badminton, rock-climbing, and cooking

3. Logistics: Course structure
Two parts:
Program verification (more established topic)
Program synthesis (new topic and recent papers)
For every lecture
I will assign material to read before the lecture
I will present the material in class and we will discuss it together

4. Logistics: What do you have to do?
35% : HW assignments – there will be 3-4 problem sets
35% : Research project
10% : Paper reviews (for second part of course)
10% : Class/Presentations participation
10% : Project presentation

5. Logistics: Homework Assignments
On paper (please type them so I can read them) or programming assignments
Have to be done individually
Submission details
Send me an with the solution
Due the midnight on the posted due date
Late penalty 10% of grade each day up to 4 days max

6. Logistics: Project Most important part of the course
Teams of 1 or 2 people
Expectations commensurate with size of team
Deliverables
Sep 28: me a list of 3 project ideas
Oct 16: Project proposal
Nov 1: Description of progress 1
Nov 22: Description of progress 2
Early Dec: Presentation to the class
Dec 15: Final write-up

7. Logistics: What project to choose?
Something that excites you!
A theoretical problem (related to formal methods)
Extend one of the models we study
Synthesize some programming task
Extend existing tools with new ideas
Apply the concepts learned in class to a new domain
Something you can show to other people later and brag about!
If the project is good and new we can turn it into a research paper!

8. Logistics
We’ll end lectures a few weeks before end of course to focus on project
Office hours:
Tue 4-5 pm
Course website
pages.cs.wisc.edu/~loris/cs703f17/cs703.html
I will setup something for paper reviews

9. What is this course about?

10. Bugs cost Time, Money, Lives, …
Ariane disaster, 1996
$500 million software failure
Bugs cost Time, Money, Lives, …
FDIV error, 1994
$500 million
Estimated worst-case worm cost:
> $50 billion
For many years researchers have been trying to
improve programmers productivity by answering this question.
How do we build reliable software?
Given some piece of code we would like some techniques that
Tell us whether our code does more or less what we intend it to do
The techniques developed towards this goal can be divided into 2 classes:
Those that target general programs and try to answer questions such as…
And techniques that target restricted domains such as…
In this domains one can try to build simpler but complete abstractions to perform powerful analysis
Credit: S. A. Seshia

11. Does my program do what it is supposed to do?
Solutions
Program verification
Does my program do what it is supposed to do?
Program
synthesis
Can you generate a program that does what I have in mind and does not contain bugs?

12. Program verification
In 4 slides

13. What is program verification?
Input
Output
Program
YES Proof
UNDECIDABLE
Verifier
Property
No null pointer exception is ever triggered
NO Counterexample

14. How bad is it?
Even the “easiest” verification problem is NP-Complete (SAT)

15. Never give up

16. Reasons behind success
Wonderful theory
Domain-specific knowledge
Engineering efforts
Programming device drivers
Malware fingerprinting
XML querying
Router filtering
Security protocols
String encoding
We’ll see some in this class
SAT solvers SMT solvers…

17. Program synthesis

18. What is program synthesis?

19. What is software synthesis
When someone says "I want a programming language in which I need only say what I wish done," give him a lollipop.
-Allan Perlis 1982
The hubris is best captured by this quote from Allan Perlis from the early eighties.
Since then, and in particular in the last 8 years, the technology for synthesis has advanced a lot, but one could argue that Perlis quote is still current. Beyond a certain level of complexity, it is unreasonable to expect a synthesizer to derive from scratch algorithms that have taken years to develop by the best minds in our field.
In addition to the harness question, there was a second critique to the Dreams=>Programs agenda: dreams don’t make very good specs.

20. What is program synthesis?
EVEN MORE
UNDECIDABLE
Input
Output
User intent
Program
1 -> 0
340 -> 300
568 -> 500
Function f(x){
If(length(x)<3)
return 0
Else
return x[0]+`00’ }
Synthesizer
Domain knowledge
Program can only use:
Length(x), if(x)then y else z, x[i],…

21. Never give up Automatic program repair Parsers from examples
Network updates from specification
Biological models from mutations
Automatic feedback for programming assignments [Singh+14]
Reactive controllers
FlashFill video [Gulwani11]

23. Synthesis of AMBA AHB from Formal Spec
AMBA: Advanced Microcontroller Bus Architecture
AMBA AHB: a high-performance system backbone bus
Formal Spec written in LTL
Circuit automatically synthesized!
AHB Slave synthesized in 13 second,
(has 276 gates with area 545 square units)

24. FlashFill: a feature of Excel 2013 (Sumit Gulwani et al.)
One recent exciting example of domain specific synthesis was developed by Gulwani for automatically producing Excel macros from examples. In this slide we see that column A contains a line of text with personal data. Columns B, C, D, E, and F contain extracted elements of the data in column A, as shown in the first row. Using the first row, Gulwani’s system will synthesize a program that, given data in column A, will produce the data on columns B to E. As shown in the next slide, this program can then be applied to the remaining rows to produce the desired result.

25. FlashFill: a feature of Excel 2013 (Sumit Gulwani et al.)
Result of the program generated from the first row applied to the remaining rows.

26. Real world application of synthesis

27. A bit about my research

28. Program Synthesis with Quantities
What should the program do? (e.g., I/O examples)
Program
Give me the smallest program
Give me the fastest program
Loris’ work revolves around the foundations and practical applications of program synthesis. Program synthesis is the art of generating programs directly from an high level specification, such as a test of input-output example.
<Click Next>
One of the limitations of current synthesis engines is that they do not take into account quantitative objectives. For example there is no way to specify that one wants a small or fast program that satisfies the functional specification.
Adding this kind of objectives to program synthesis has been the focus of both Loris’ theoretical and practical research.
Let me show you some examples of practical applications that this work has resulted in.

29. Emulative program repair
Learn abstract transformation
Automatically fix programs
with similar bugs
Learn directly
from GitHub code
In emulative program repair, Loris uses synthesis to learn how to fix programs by observing how actual people fix their programs.
Whenever someone fixes a bug, Loris’ tool synthesizes the action they did and use it to help future people that encounter similar errors.
<Click Next> Loris is now applying this paradigm to learn arbitrary code fixes by mining GitHub repositories.

30. Direct code manipulation
<Click to start demo>
In this next application, Loris asked: can we give programmers new way to express the intended program behavior?
In this project Loris allows the programmer to directly manipulate intermediate variable values in a program run and tries to automatically synthesize a program that matches the modified user-given values. In this case, the quantitative objective is, what is the “best” modified program?

31. Proving that programs do not discriminate
Q: Based on the statistical distribution of students in USA, does the classifier favor male students?
Prove or disprove using program analysis:
Probabilistic programs
Hoare logic
Classifier
Admit
Do not admit
In this last application, Loris and Aws use program synthesis to fix bias in machine learned programs.
Programs are increasingly taking care of sensitive decisions that can have widespread impact. For example, programs decide who gets interviewed, who gets hired, who gets a loan, how long a person's prison sentence is…and the list is only growing longer! How can we prove that a program does not discriminate? This project looks at the theory and practice of automatically proving that programs do not discriminate, and automatically fixing discriminatory programs!

32. Synthesis of network configurations
Q: Based on the statistical distribution of students in USA, does the classifier favor male students?
Prove or disprove using program analysis:
Probabilistic programs
Hoare logic
Classifier
Admit
Do not admit
In this last application, Loris and Aws use program synthesis to fix bias in machine learned programs.
Programs are increasingly taking care of sensitive decisions that can have widespread impact. For example, programs decide who gets interviewed, who gets hired, who gets a loan, how long a person's prison sentence is…and the list is only growing longer! How can we prove that a program does not discriminate? This project looks at the theory and practice of automatically proving that programs do not discriminate, and automatically fixing discriminatory programs!

33. Synthesis of network configurations
Switch forwarding tables
Genesis
SMT
High-level policies on paths and switches (reachability, waypoint, isolation, fault tolerance)
SMT + LP
Zeppelin
OSPF and BGP configurations
QuARC
Efficient optimal repair
Probabilities
Temporal specificaitons
Thus, in conclusion, we presented a generalized synthesis framework for enforcing policies in multi-tenant clouds. We discussed various domain-specific optimizations to improve the synthesis performance and various seamless extensions to handle network failure scenarios.
Thank you for your attention and I would like to address any questions.
<Last Slide>