1. Introduction to Software Testing (Paul deGrandis) [Reading assignment: Chapter 15, pp. 231-252 and notes by Paul deGrandis]

2. Overview Terms and Definitions Tool Overview Static Checkers Unit Testing Coverage and Profiling Memory Analysis Bug Reporting and Tracking Continuous Integration

3. NOTE All the material and examples from here on are generated using a single makefile. Log into tux, then: wget http://www.pauldee.org/vandvmakefile mv vandvmakefile Makefilehttp://www.pauldee.org/vandvmakefile Export your classpath: export CLASSPATH=`pwd`/junit/junit-4.4.jar:`pwd`/emma/emma.jar:. To make and download all tools and examples: make

4. Tool Overview Verification Static Checking Unit Tests for code correctness Memory and performance specifications Validation Unit Tests for behavior and requirements

5. Static Checkers Does not execute the code Finds suspicious code or security vulnerabilities Tools C - SPLINT, GCC Warnings Java - FindBugs, PMD Cyclone - a safe C GNU Compiler Collection FindBugs - Find Bugs in Java

6. Static Checkers - C Screenshots

7. Static Checkers - C Demo Installation of SPLINT Compiling using GCC Warning Running Splint

8. Static Checkers - C Demo Installation of SPLINT Download wget http://www.splint.org/downloads/splint-3.1.2.src.tgz wget http://www.splint.org/downloads/splint-3.1.2.src.tgz Untar tar -xzvf splint-3.1.2.src.tgz Make (during make, splint checks itself) cd splint-3.1.2/./configure make

9. Static Checkers - C Demo Compiling using GCC Warnings see the screenshot three slides back

10. Static Checkers - C Demo Running SPLINT Simple and Basic splint -preproc +weak +show-summary +stats../src/*.c Dependable Software splint -preproc +checks +show-summary +stats../src/*.c

11. Static Checkers - Java Screenshots

12. Static Checkers - Java Demo Installation of FindBugs Running FindBugs

13. Static Checkers - Java Demo Installation of FindBugs Download wget http://prdownloads.sourceforge.net/findbugs/findbugs-1.2.1.tar.gzhttp://prdownloads.sourceforge.net/findbugs/findbugs-1.2.1.tar.gz Untar tar -xvzf findbugs-1.2.1.tar.gz cd findbugs-1.2.1

14. Static Checkers - Java Demo Running FindBugs Run It./findbugs

15. Unit Tests Validate a unit of code; smallest testable part Executes the code in a sandboxed environment Testing mostly for functional requirements can also test some non-functional requirements Many approaches and schools of thought History Based, Risk Based, Data Path, DOE

16. Unit Tests Frameworks-C Screenshots Check Unit Test Framework http://check.sourceforge.net

17. Demo Unit Tests Frameworks- C Installation of Check Writing a Check unit test Compiling your test(s) Running your test(s) http://check.sourceforge.net/doc/check.html/index.html

18. Demo Unit Tests Frameworks- C Installation of Check Download wget http://downloads.sourceforge.net/check/check-0.9.5.tar.gzhttp://downloads.sourceforge.net/check/check-0.9.5.tar.gz Untar tar -xvzf check-0.9.5.tar.gz Make (notice the GCC warnings passed) cd check-0.9.5./configure make

19. Demo Unit Tests Frameworks- C Writing a Check unit test Write your test fixtures Write your test(s) Write your test suite Write your main method (execute your suite)

20. Demo Unit Tests Frameworks- C Writing a Check unit test Write your test fixtures

21. Demo Unit Tests Frameworks- C Writing a Check unit test Write your test(s)

22. Demo Unit Tests Frameworks- C Writing a Check unit test Write your test suite

23. Demo Unit Tests Frameworks- C Writing a Check unit test Write your main method (execute your suite)

24. Demo Unit Tests Frameworks- C Compiling your test(s) See README in check/check- example/square

25. Demo Unit Tests Frameworks- C Running your test(s) See README in check/check- example/square

26. Unit Tests Frameworks-Java Screenshots

27. Demo Unit Tests Frameworks- Java Installation of Junit Writing a Junit unit test Compiling your test(s) Just use javac Running your test(s) just use java http://junit.sourceforge.nethttp://junit.sourceforge.net/ (look at the cookbook)

28. Demo Unit Tests Frameworks- Java Installation of Junit Download wget http://downloads.sourceforge.net/junit/junit-4.4.jarhttp://downloads.sourceforge.net/junit/junit-4.4.jar Add the jar to your classpath export CLASSPATH=$CLASSPATH:`pwd`/junit-4.4.jar:.

29. Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test fixtures Write your test(s) Write your test suite Write your main method (execute your suite)

30. Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test fixtures

31. Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test(s)

32. Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your test suite(s)

33. Demo Unit Tests Frameworks- Java Writing a Junit unit test Write your main method

34. Code Coverage Degree to how much code was tested (E.g., How confident are we in our tests and code) Criteria: Statement - Has each line been executed Condition - Every evaluation (if/else, try/catch, switch, loop) Path - Every route of code

35. Code Coverage - C Screenshots GCC - GCOV

36. Demo Code Coverage - C See the Screenshot one slide back

37. Code Coverage - Java Screenshots EMMA http://emma.sourceforge.net/intro.html

38. Demo Code Coverage - Java Installation of Emma Download wget http://downloads.sourceforge.net/emma/emma-2.0.5312-lib.zip Unzip unzip emma-2.0.5312-lib.zip Instrument Your Code java -cp emma.jar:./sample:../junit/junit-4.4.jar emma instr -outdir./sampleout -cp./sample/ SimpleTest Run Your Code and Generate a report java SimpleTest java -cp emma.jar emma report -r html -in sampleout/coverage.em,sampleout/coverage.ec

39. Memory Analysis Sandboxes an application in a VM Uses: Memory Leak - allocate memory, but don’t free Buffer Overflow - access memory outside of a fixed buffer Profiling - performance analysis

40. Memory Analysis - C Screenshots Valgrind

41. Demo Memory Analysis - C Installation of Valgrind Download wget http://downloads.sourceforge.net/check/check-0.9.5.tar.gzhttp://downloads.sourceforge.net/check/check-0.9.5.tar.gz Untar tar -xvjf valgrind-3.2.3.tar.bz2 Make (notice the GCC warnings passed) cd valgrind-3.2.3./configure make

42. Demo Memory Analysis - C Running of Valgrind Compile with debugging enabled gcc -g myprog.c Run Valgrind valgrind --leak-check=yes myprog arg1 arg2 Full tutorial and infoformation http://valgrind.org/docs/manual/quick-start.html#quick-start.intro

43. Memory Analysis - Java Screenshots JConsole

44. Issue and Bug Tracking Organizes and tracks issues, enhancements, features, and bugs in a software system Usually contains a database backend and a web frontend Generate report of project and team metrics and productivity

45. Issue and Bug Tracking Trac ( http://trac.edgewall.org/ ) http://trac.edgewall.org Roundup ( http://roundup.sourceforge.net/ ) Tracker ( http://tracker.rubyforge.org/ )

46. Continuous Integration Committing changes early and often Building and testing at least nightly If code breaks, revert Integration issues discovered early as developers work with the latest code

47. You now know … … static checkers … unit testing tools … coverage and profiling tools … memory analysis tools … Bug reporting and tracking tools