Minimal Instrumentation for Software Feature Location Sharon Simmons Dennis Edwards Norman Wilde Eric Daniels

Software Reconnaissance Fall 2009 ShowcaseUWF - Simmons2 Where in this program is feature X implemented? 100’s KLOC, 10’s MLOC Modification required Interactions forgotten 2. Run tests with the feature Feature is active Feature is NOT active ~Set difference Solution: track execution 3. Compare what was executed 1. Run tests without the feature Components executed in the first group of tests, but not in the second group Marker Components Begin exploration here!

(demo of version 3) TraceGraph 4 Fall 2009 ShowcaseUWF - Simmons3 Row Instrumentation point Column test case Col 1 without feature Col 2+ with feature Identifies “markers” Gives big picture Subroutine (En… 0 be_aas_bspq3.c /research/ope… 147

Software Recon Status Fall 2009 ShowcaseUWF - Simmons4 Developed for several years at UWF –Stable, usable, and free –Published works enumerate benefits and limitations Brief Benefits –Identifies a small number of marker components –Identifies good places to start efficient code exploration –Can decrease software maintenance time

Instrumentation Alternatives Atom (Digital Equipment Corp) – Instrumentation at load time Pin (Intel Corp) – Instrumentation at dynamic link time CodeTEST (Freescale Semiconductor) – Instrumentation of preprocessed source code Recon (UWF) – Instrumentation of source code Gcov (GNU) – Instrumentation by (GNU) compiler Fall 2009 ShowcaseUWF - Simmons5 None completely successful

Problems Encountered Fall 2009 ShowcaseUWF - Simmons6 Newer systems time-sensitive Deadlines must be met Tracing can not be intrusive Target must execute normally Initialization was hours instead of minutes Deadlines were not met Tracing intrusive and prevented normal execution Hard / soft real-time Race conditions Timing faults

Instrument preprocessed source Efficient code coverage tracing Build IDB prior to execution Provide facilities to partition execution into intervals Simplify installation and use Fall 2009 ShowcaseUWF - Simmons7 Our Approach

Information Flow Fall 2009 ShowcaseUWF - Simmons8 Driver Collection Engine Target IDB Bitstring Traces Trace Records Data Joiner Source Instrumenter Interval Delimiter Test Cases During Compilation During Execution After Execution

MinInst Overview Fall 2009 ShowcaseUWF - Simmons9 Instrumentation point represented as bit – 1MB RAM ≈ 8.4M instrumentation points Ordered set of bits (bit string) represents code – Bit index identifies IDB record – Bit string cleared at interval start – Bit set when instrumentation point executed 0  not executed; 1  executed – Execution order not retained – Execution frequency not retained – Bit string saved at interval end

MinInst Details Fall 2009 ShowcaseUWF - Simmons10 Two bitstrings allocated in shared memory –Active (A): currently accessed by target –Passive (P): prepared for next interval Collection Engine –Started as first instruction in main() 1.Wait for signal (no overhead here) 2.Exchange A & P bitstrings (pointer swap) 3.Save P bitstring to trace file (I/O overhead) 4.Clear P bitstring for next interval 5.Go back to step 1

Version A Instruments C/C++ Linux platform CodeTEST – Identifies instrumentation points – Builds IDB MinInst – Inserts instrumentation code – Builds bit index  IDB map Driver – Delimits trace intervals – Performs test cases Collection Engine – Records bitstring trace at runtime Joiner – Combines IDB, MAP and bitstring trace – Creates usable trace file Fall 2009 ShowcaseUWF - Simmons11

Information Flow Fall 2009 ShowcaseUWF - Simmons12 Driver Collection Engine Target IDB Bitstring Traces Trace Records Data Joiner Source Instrumenter Interval Delimiter Test Cases During Compilation During Execution After Execution Map CodeTEST MinInst

Test Cases Generated Code – 1 file, simple functions (5 stmts, 5 inst pts) – 15KLOC, 30KLOC, 45KLOC, 60KLOC – CodeTEST: +200%MinInst: +11% Apache web server (httpd) – 550 files, 300KLOC, multithreaded, multiprocess – CodeTEST: +717%MinInst: +1% Openarena & OpenGL – 1K files, 700KLOC, time sensitive, multiplayer, networked – CodeTEST: +9371%MinInst: +6% Fall 2009 ShowcaseUWF - Simmons13

Shortcomings Complexity of use Reliance on CodeTEST CodeTEST failed some test cases Execution differences between platforms Licensed software Unsupported for a while Fall 2009 ShowcaseUWF - Simmons14

Version B Prototype written in C for C/C++ source – Removed CodeTEST dependency – Tested in Linux – Windows port is pending Simplified installation and use – Installation/removal scripts – Integrated into GCC/Make in Linux – Integrated into Visual Studio IDE Interval options – Timed or user controlled Fall 2009 ShowcaseUWF - Simmons15

Information Flow Fall 2009 ShowcaseUWF - Simmons16 Driver Collection Engine Target IDB Bitstring Traces Trace Records Data Joiner Source Instrumenter Interval Delimiter Test Cases During Compilation During Execution After Execution MinInst

Summary Version B status – Execution time unaltered from Version A – Linux and Windows XP – GCC and Visual Studio – Test cases in progress Next steps – Conclude test cases – Northrop Grumman large system testing – Complete distribution package Future Work – Modify TraceGraph for bit string operation Fall 2009 ShowcaseUWF - Simmons17

Information Flow Fall 2009 ShowcaseUWF - Simmons18 Driver Collection Engine Target IDB Bitstring Traces Source MinInst Interval Delimiter Test Cases During Compilation During Execution After Execution

Contact Information Sharon Dennis Norman Fall 2009 ShowcaseUWF - Simmons19