1. Michael Hildebrandt H UMBOLDT U NIVERSITY B ERLIN D EPARTMENT O F C OMPUTER S CIENCE C HAIR O F S OFTWARE E NGINEERING

2. motivation testing is expensive purposes of SOTA static program analysis structure oriented program test approach source code parsing instrumenting workflow source code preparation test execution evaluation 2 DAAD Workshop Neum, 2009

3. given: function overlaps parameter: (time) interval A (start A, end A ) (time) interval B (start B, end B ) result: true if A and B overlap, otherwise false How many test cases exist? 3 Software Engineering II, Prof. Dr. Holger Schlingloff DAAD Workshop Neum, 2009

4. 1. test cases for overlapping intervals 2. test cases for non-overlapping intervals 3. test cases for A is before B and vice versa 4. test cases for A B and B A 5. test cases for the same starting/ending point of A and B 6. test case for A = B 4 DAAD Workshop Neum, 2009

5. 7. test cases for intervals consisting of one point 8. test cases for identical intervals consisting of point 9. test cases for intervals containing integer / real numbers 10. test cases for invalid intervals 11. test cases for intervals with values at the borders of number range (maxint, maxreal) 5 DAAD Workshop Neum, 2009

6. about 20 test cases for this simple example imagine a project with 1.5 MLOC conclusion: testing every possible input is expensive or even impossible problem: Which subset of the possible test cases has the highest probability to detect the most possible failures? possible solution: structure oriented tests (using SOTA) 6 DAAD Workshop Neum, 2009

7. S tructure O riented T est and A nalysis static program analysis metrics control flow graph (CFG) structure oriented tests evaluation of test cases by code coverage development of additional test cases 7 author: Ronny Treyße (diploma thesis) DAAD Workshop Neum, 2009

8. calculated by parsing the source code provides several metrics lines of code number of statements, branches, atoms, conditions, … cyclomatic complexity essential complexity 8 DAAD Workshop Neum, 2009

9. 9 public static int fac(int n) { int result = 1; if (n == 0 | n == 1) { return result; } else { for (int i = n; i >= 2; i--) { result *= i; } return result; } public static int fac(int n) { int result = 1; if (n == 0 | n == 1) { return result; } else { for (int i = n; i >= 2; i--) { result *= i; } return result; } DAAD Workshop Neum, 2009

10. calculated by instrumenting and test execution 10 DAAD Workshop Neum, 2009 FEEC C 0 C 1 function entries and exits coverage statement coverage branch coverage C 2 MMCC MCDC C 3 simple condition coverage minimal multiple condition coverage modified condition decision coverage multiple condition coverage MBI BI modified boundary interior coverage boundary interior coverage

11. 11 C0C0 C0C0 C1C1 C1C1 C2C2 C2C2 C3C3 C3C3 MCDC MMCC MBI BI A A B B A is necessary for B practical relevant FEEC DAAD Workshop Neum, 2009

12. instrumentation enrichment of the original source code by adding logging statements breaking up conditions, transforming loops, … source code grows bigger (up to ten times) logging information during test execution values of condition, atoms used function entries / exits and branches of control structures (reconstruction of the control flow) 12 DAAD Workshop Neum, 2009

13. 13 DAAD Workshop Neum, 2009

14. level 0 no instrumentation to mark structures for no instrumentation level 1 all entries and exists of a method, branching control structures logged all coverage measures based on the control flow computable level 2 additional to level 1 the assignment of every atom is logged all coverage measures computable level 3 complete instrumentation: additional to level 2 the execution of every statement is logged allows control flow analysis of abnormal terminating programs 14 DAAD Workshop Neum, 2009

15. 15 if (n == 0 | n == 1) { return result; } if (n == 0 | n == 1) { return result; } DAAD Workshop Neum, 2009

16. 16 for (int i = n; i >= 2; i--) { result *= i; } for (int i = n; i >= 2; i--) { result *= i; } DAAD Workshop Neum, 2009

17. 3 modes of usage manual program testing pre procession: SOTA test execution: by hand post procession: SOTA testing with an external test system (e.g. ATOSj) pre procession: SOTA test execution: external test system post procession: SOTA testing with an automatic test system usage of non-gui functionality by SOTA library accessible via ASTManager object or command line 17 DAAD Workshop Neum, 2009

18. phase 1: preparation read in and parsing the source code configure the instrumentation instrumentation of source code phase 2: test execution compiling of the instrumented source code test execution including the logging of information phase 3: test evaluation restoration of the original code read In the log files calculation of the coverage measure visualization of the results creation of report 18 DAAD Workshop Neum, 2009

19. Questions ? Comments? 19 DAAD Workshop Neum, 2009