1. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 1 Tutorial / lab 2: Code instrumentation Goals of this session: 1.Create some tests for a Java class. 2.Measure the code coverage.

2. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 2 Code Instrumentation Tools Tools that modify production code for various purposes during code execution: –Debuggers –Add information to allow for break points or step-by-step execution. –Coverage measurement –Report on the number of times various code elements have been executed. –Profilers –Measure amount of time spent in various parts of the code.

3. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 3 Coverage tools Program is typically compiled with special options, to add extra source or object code. –Additional data structures, such as a flow graph, may also be created. Program is run, possibly via test cases –During execution, information is accumulated and written to an output file. Post-processing phase: –User report is generated from output file.

4. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 4 How to measure coverage? Instrument the source code before compilation Instrument the virtual machine or object code. –The Apache Byte Code Engineering Library (BCEL) is used by several Java coverage tools –http://jakarta.apache.org/bcel Use a customized virtual machine.

5. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 5 Source code instrumentation Create a copy of the source code, instrument the copy, and compile the instrumented copy to.class files. –This is the approach used by the Clover code coverage tool. Typical method: –Set up an array of counters. –Insert source code at points of interest that increment specific counters. –Coverage tool calls main() method –After main method returns, dump values of counters.

6. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 6 Source code before instrumentation public final void add( final MatrixRow newValues ) { if ( getNumColumns( ) == 0 ) { contents = newValues.copy( ).getContents( ); } else { contents.addAll( newValues.copy( ).getContents( ) ); } A counter is inserted for each method, statement, and branch. Statement counters are before each statement, in case the statement throws an exception. Branch counters need to identify true and false branches taken.

7. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 7 Same code after instrumentation public final void add( final MatrixRow newValues ) { CLOVER.M[348]++; CLOVER.S[1814]++; if ( ( getNumColumns( )==0 && ++CLOVER.CT[338]!=0 | true ) || ( ++CLOVER.CF[338] == 0 & false ) ) { CLOVER.S[1815]++; contents = newValues.copy( ).getContents( ); } else { CLOVER.S[1816]++; contents.addAll( newValues.copy( ).getContents( ) ); } method counter statement counters branch counters

8. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 8 Counting branches Count number of times that a branch condition evaluates to true and false. if ( ( aBoolean && ++CT[338]!=0 | true ) || ( ++CF[338] == 0 & false ) ) If aBoolean is true, then the && is also evaluated, which will increment the CT counter. We want the expression’s overall evaluation to be true. If aBoolean is false, then execution switches to the right side of the || clause. Increment the CF counter, but then ensure that the entire expression still evaluates to false to take the else branch. Short-circuit evaluation means that as soon as the expression result can be determined, evaluation stops. This avoids: –Incrementing the true counter if aBoolean is false. –Incrementing the false counter if aBoolean is true.

9. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 9 Analysis Advantages: –Results are directly mapped to source code. –If a statement throws an exception, it will be counted as having been executed. Disadvantages: –Requires keeping a second version of source code, and ensuring that original source is not over-written. –Instrumented code must not affect original code execution. –Management of.class files from compiling alternate source code. –Memory usage: for statement coverage alone, every statement requires one integer as a counter.

10. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 10 Byte code instrumentation Can be done in two ways: –Instrument object files in advance to produce a alternate version. –Example: create instrumented.class files for Java. –Instrument object files as they are used. –Example: create customized.class file loader that inserts the instrumentation at run time. This is the approach used by the Emma code coverage tool –This is an open-source tool; see http://emma.sourceforge.net –Eclipse plug-in version: http://www.eclemma.org

11. © Dr. A. Williams, Fall 2008 - Present Software Quality Assurance – Clover Lab 11 Analysis Pre-instrumentation: –Advantages: –source code is not modified. –can be done as a batch process before execution –Disadvantages: –management of two.class file versions. –mapping of source code statements to/from object code –counting source code statements can be difficult –object code may need extra information such as special compiler options