1. CS265: Program Analysis, Testing, and Debugging
Lecture 1
Koushik Sen
EECS, UC Berkeley

2. My Background Assistant Professor in CS since Fall 2006
Office: Parlab (581 Soda Hall)
Ph.D. and M.S. in Computer Science
University of Illinois at Urbana Champaign ( )
Spent 1 year in software industry as a software developer
B-Tech
IIT Kanpur ( )
Research Interests:
Software Engineering,
Programming Languages, and
Formal Methods
Verification, Testing, Model Checking, Program Analysis

3. Please introduce yourselves

4. Logistics

5. Course Goals
To help students start research in the area of program analysis, model checking, testing, and debugging of sequential, concurrent, and distributed systems.
To help students to apply the techniques learned in this course in their ongoing research in other areas such as operating systems, computer networks, security, and database systems.
Should meet your Th+Sys prelims requirement

6. Course Communication All class materials will be on the website
See website for announcements
Class meets on Monday and Wednesday, 1:00 PM – 2:30 PM at 320 Soda Hall
Office hour by appointment

7. Course Structure We will study papers
You will read a paper and write a review of the paper before each class
There will be some guest lectures
You need to do 2-3 programming assignments
Test generation
Dynamic analysis of concurrent programs
Points-to analysis
Project in teams of 1-2.

8. Course Grading Reviews and class participation: 30%
2-3 homework assignments: 30%
Project: 40%
A project must involve new research
Some sample projects will be posted online
Choose a project topic by 25th September, 2009
1-2 page project proposal due by 9th October, 2009
A 5-7 minute mid-semester project presentation
Final project demo or presentation
6 page project report in ACM SIGPLAN conference format

9. About the course This course is about software reliability
Why is software reliability important?
As society becomes more dependent on software, the consequences of software failures are non-trivial.
Money lost.
Lives lost.
Market share lost.
Clients lost.
Jobs lost.

10. Software Bug => Space Disaster
Ariane 5 Space mission
$7, 000, 000, 000
10 Years in the making
40 seconds after take off the rocket exploded

11. Software Bug => Space Disaster
Attempt to cram a 64-floating point number to a 16-bit integer failed

12. AT&T long distance service failed for 9 hours
On January 15, 1990, one of AT&T's #4ESS toll switching systems in New York City experienced an intermittent failure that caused a major service outage.
Wrong BREAK statement in C-Code
Complete coverage could have revealed this bug during testing
/* ``C'' Fragment to Illustrate AT&T Defect */
do { switch expression { case (value): if (logical) { … break; } else { … }
case (value2):
… } …
} while (expression)
Summary
On January 15, 1990, one of AT&T's #4ESS toll switching systems in New York City experienced an intermittent failure that caused a major service outage on the AT&T U.S. National Telephone Network [Reference 1]. The error was introduced due to a software flaw that had escaped detection even by AT&T's extremely sophisticated software test methods. The error, however (as described in [Reference 2]), would have been revealed with attainment of complete C1 coverage???
Error Description
What was reported in ACM's Software Engineering Notes [Reference 2] is that the software defect was traced to an elementary programming error, which is described as follows:
In the offending "C" program text there was a construct of the form:
/* ``C'' Fragment to Illustrate AT&T Defect */ do { switch expression { case (value): if (logical) { sequence of statements break } else { another sequence of statements } statements after if...else statement } statements after case statement } while (expression) statements after do...while statement
Programming Mistake Described
The mistake is that the programmer thought that the break statement applied to the if statement in the above passage, was clearly never exercised. If it had been, then the testers would have noticed the abnormal behavior and would have been able to corr
The only caveat to this statement is the following: it is possible that tests applied to the code contain information which would reveal the error; however, if the testers do not examine the output and notice the error, then the deficiency is not with th
In the case of a misplaced break statement, it is very likely that the error would have been detected.
References
1. "Can We Trust Our Software?", Newsweek, 29 January 1990.
2. ACM SIGSOFT, Software Engineering Notes, Vol. 15, No. 2, Page 11ff, April 1990.

13. Software Bugs: Cause of Deaths
Several deaths of cancer patients were due to overdoses of radiation resulting from a race condition between concurrent tasks in the Therac-25 software.

14. 180 Degree Bug
Torpedoes, that deviate more than 90 degree, explode to avoid self destruction of the ship.
Once upon a time a ship fired a torpedo but the torpedo was jammed in the tube. Then the captain gave the command: Let's turn around and return to the harbour! What happened next is no mystery.

15. Cost of Failure
Software failures were estimated to cost the US economy about $60 billion annually [NIST 2002]
Improvements in software testing infrastructure may save one-third of this cost
Testing accounts for an estimated 50%-80% of the cost of software development [Beizer 1990]

16. Methods for Building Reliable Software
Safe Programming Languages and Type systems
Testing
Model Checking and Theorem Proving
Static Program Analysis
Dynamic Program Analysis
Model Based Software Development and Analysis

17. Methods for Building Reliable Software
Safe Programming Languages and Type systems
Testing
Model Checking and Theorem Proving
Static Program Analysis
Dynamic Program Analysis
Model Based Software Development and Analysis

18. Course Contents Automated test generation
Software model checking and various theoretical results that form the foundation of software model checking
Concurrent program analysis
Abstract interpretation and points-to analysis

19. Course Contents Automated test generation
Software model checking and various theoretical results that form the foundation of software model checking
Concurrent program analysis
Abstract interpretation and points-to analysis

20. Automated Test Generation
Korat:
Reading assignment: Korat: Automated Testing Based on Java Predicates. Chandrasekhar Boyapati, Sarfraz Khurshid, Darko Marinov (ISSTA 2002)
Must submit review to the course wiki by 9/8 11:59 PM
Concolic Testing: Homework 1

21. Automated Test Generation
Generate test inputs
To reveal bugs: assertion violation, crashes, wrong output
Improve software reliability
Often cannot prove program correct
Need to check program for all possible inputs
Input domain is often infinite
Pick inputs to satisfy certain coverage criteria
Generate all legal inputs of bounded size
Generate all legal inputs for full branch coverage

22. Course Contents Automated test generation
Software model checking and various theoretical results that form the foundation of software model checking
Concurrent program analysis
Abstract interpretation and points-to analysis

23. Software Model Checking
Attempt to prove programs correct
Abstract domain
Create an abstraction of the program
Show that the abstraction does not contain a “bad state”
Predicate abstraction and boolean programs
Successfully used for model checking device drivers

24. Software Model Checking
BLAST: predicate abstraction
Decidability results for various boolean program models
Forms theoretical foundation of various software model checking algorithms
Pushdown systems
Pushdown systems with multiset
Petri-Nets
Multi-pushdown systems with bounded context switch

25. Course Contents Automated test generation
Software model checking and various theoretical results that form the foundation of software model checking
Concurrent program analysis
Abstract interpretation and points-to analysis

26. Concurrent Program Analysis
Bugs due to concurrency are notorious
Intermittent and hard to reproduce
Common causes: data race, atomicity violations, deadlocks, and other synchronization issues
Much more difficult to analyze than sequential programs
Need to check program
all schedules
all inputs
Pushdown system with stacks => undecidable

27. Concurrent Program Analysis
Classic dynamic race detection algorithms
Atomicity checking and deadlock analysis
Explicit State Model Checking
Partial Order Reduction
Iterative preemption bounded model checking
Homework 2

28. Course Contents Automated test generation
Software model checking and various theoretical results that form the foundation of software model checking
Concurrent program analysis
Abstract interpretation and points-to analysis

29. Abstract Interpretation and Points-to Analysis
Note software model checking is an instantiation of abstract interpretation
forms basis of most static program analyses
Mayur Naik on Points-to analysis
The most important analysis
Anderson’s points-to analysis
BDDs (Binary Decision Diagrams) for scalable points-to analysis
And a homework

30. Course Contents Automated test generation
Software model checking and various theoretical results that form the foundation of software model checking
Concurrent program analysis
Abstract interpretation and points-to analysis

31. Summary
3 homework assignments will give you hands-on experience of program analysis
In my opinion, this is quite useful to get a deep understanding of the subject, and
to jump start research in program analysis

32. Symbolic Execution Use symbolic values for input variables
Execute the program symbolically on symbolic input values
Collect symbolic path constraints
Use constraint solver to generate test inputs for each execution path

33. Computation Tree: Execution Paths of a Program
Can be seen as a binary tree with possibly infinite depth
Computation tree
Each node represents the execution of a “if then else” statement
Each edge represents the execution of a sequence of non-conditional statements
Each path in the tree represents an equivalence class of inputs 1

34. Example of Computation Tree
void test_me(int x, int y) {
if(2*x==y){
if(x != y+10){
printf(“I am fine here”);
} else {
printf(“I should not reach here”);
ERROR; }
2*x==y N Y
x!=y+10 N Y
ERROR

35. Computation Tree vs. Program Dependence Graph
void testme1(int x) {
for (int j=0; j < 2; j++) {
if (x==j) {
printf(“Good\n”); }
void testme2(int x, unsigned int N) {
for (int j=0; j < N; j++) {
if (x==j) {
printf(“Good\n”); }

36. Concrete Execution int x = read(); int y = read(); int t; t = x;
x = x + y;
if (x > y) {
y = 2*t;
if (x+1 == y) {
assert(false); }

37. Concrete Execution int x = read(); int y = read(); int t; t = x;
x = x + y;
if (x > y) {
y = 2*t;
if (x+1 == y) {
assert(false); }
x = 4
y = 9
t = 4
x = 13
(13 > 9)
y = 8
(14 == 8)
END

38. Symbolic Execution int x = read(); int y = read(); int t; t = x;
(true | x = x0 )
int x = read();
int y = read();
int t;
t = x;
x = x + y;
if (x > y) {
y = 2 * t;
if (x+1 == y) {
assert(false); }
(true | x = x0,y=y0)
(true | x = x0,y=y0,t=x0)
(true | x = x0+y0,y=y0,t=x0)
x0+y0 > y0
(x0 <= 0 | x = x0+y0,y=y0,t=x0)
(x0>0 | x = x0+y0,y=2x0,t=x0)
x0+y0 + 1 == 2x0
(x0 > 0 && y0 + 1  x0 | x = x0+y0,y=2x0,t=x0)
(x0 > 0 && y0 + 1 == x0 | x = x0+y0,y=2x0,t=x0)

39. Test Generation int x = read(); int y = read(); int t; t = x;
(true | x = x0 )
int x = read();
int y = read();
int t;
t = x;
x = x + y;
if (x > y) {
y = 2 * t;
if (x+1 == y) {
assert(false); }
(true | x = x0,y=y0)
Solve: x0 > 0 && y0+1 = x0
Solution: x = 2, y=1
(true | x = x0,y=y0,t=x0)
Solve: x0<= 0
Solution: x = 0, y=0
(true | x = x0+y0,y=y0,t=x0)
x0+y0 > y0
(x0 <= 0 | x = x0+y0,y=y0,t=x0)
(x0 > 0 | x = x0+y0,y=2x0,t=x0)
x0+y0 + 1 == 2x0
Solve: x0 > 0 && y0+1  x0
Solution: x = 1, y=9
(x0 > 0 && y0 + 1  x0 | x = x0+y0,y=2x0,t=x0)
(x0 > 0 && y0 + 1 == x0 | x = x0+y0,y=2x0,t=x0)

40. Abstract Interpretation: Test to Proof
Example ( ) { int new = read(); lock = 0; 1: do{ lock = 1; old = new; b = read(); 2: if (b){ 3: lock=0; new ++; } 4:} while(new != old); 5: assert (lock==1);

41. Abstract Interpretation: Test to Proof
Example ( ) { int new = read(); lock = 0; 1: do{ lock = 1; old = new; b = read(); 2: if (b){ 3: lock=0; new ++; } 4:} while(new != old); 5: assert (lock==1);
Predicates: LOCK, new==old

42. Abstract Interpretation: Test to Proof
: LOCK 1 2 3 4
LOCK , new=old 5
Reachability Tree
LOCK , new==old
: LOCK , : new = old
: LOCK,
: new == old
Example ( ) { int new = read(); lock = 0; 1: do{ lock = 1; old = new; b = read(); 2: if (b){ 3: lock=0; new ++; } 4:} while(new != old); 5: assert (lock==1);
Predicates: LOCK, new==old