1. University of Maryland parseThat: A Robust Arbitrary-Binary Tester for Dyninst Ray Chen

2. University of Maryland Motivation Extensive test suite exists –118 individual sub-tests in 15 major categories –Attach mode flag –Relocate mode flag –Multiple mutatee compilers (GCC, Native) –Multiple mutatee languages (C/C++/Fortran) Total of over 700 possible individual tests per platform! –Currently 8 platforms tested each night

3. University of Maryland Motivation Tests written to exercise functionality –Mutatees tailored for mutators –Involve simple/artificial code Test1.35 involves hand modified assembly output of gcc to trigger function relocation Poor stress tests of Dyninst performance and capabilities Mutatees relatively simple compared to real- world applications

4. University of Maryland Motivation Goals –Allow regular, automated testing on production- scale application binaries Firefox, GCC, MySQL –Allow a useful way for API users to easily report faults back to developers

5. University of Maryland Overview Three-tiered process hierarchy Testing methodology –Snippets in mutatee kept simple –Mutator wraps API calls and announces results –Monitor records mutator state and watches for abnormal conditions MonitorMutatorMutatee

6. University of Maryland Basic Usage Operational Modes –parseThat –parseThat –c Parsing Levels –Module –Function –CFG Instrumentation Levels –No instrumentation –Function Entry –Function Exit –Basic Block –Memory Access

7. University of Maryland Inserted Snippets Keep instrumentation simple –Goal is to test range of mutatee support Pre-insertion –Force mutatee to allocate integer from heap One integer for each level of instrumentation granularity specified per function Snippet –Increment integer associated with point of insertion, like function entry

8. University of Maryland Mutator Responsibilties Wrap Dyninst API calls –Before each Dyninst API call Announce API routine name –After return from API call Announce result (Success, Warning, Failure) Report mutatee status –Announce values of inserted counters –Announce abnormal mutatee exit

9. University of Maryland Mutator Output Modeled after traditional Internet protocols (HTTP, FTP, etc) –Driven by numeric identifiers –Designed to be readable by humans But also easily parsed by other programs –Transfers data and state information Message Structure – ::=

10. University of Maryland Mutator Output Example message: 01a73 “” Test Message 3 Digit Hexadecimal Numeric Identifier 1 Digit Severity Identifier 1 Digit Status Identifier Data String (Possibly quoted) Description String

11. University of Maryland Mutator Output Sample output 01031 sigalrm Attempting to instrument function. 01141 "" Attempting to instrument function entry. 01c61 "" Instructing mutatee to call malloc(). 01c64 "" Success. 02151 "" Inserting counter-increment snippet. 02154 "" Success. 01144 "" Success. 01034 "" Success.

12. University of Maryland Monitor Responsibilities Interprets mutator output –Provides internal stack for presenting mutator messages to user –Based on verbosity value User decides how much information they want to see History records –Provides ability to track changes across runs Trace pipe –Provides ability to track progress of generated code

13. University of Maryland Monitor Interpreter Monitor output [ Processing funccall ] =================================== Creating new BPatch object.... done. Requesting notification of mutatee exit.... done. Requestion notification of mutator fork.... done. Forking mutatee process.... Mutatee launched as process 151690. Retrieving BPatch_image object.... done. Parsing image for module information.... Found 20 module(s). Parsing function data from funccall.c module... Found 6 function(s). Parsing CFG data from functions in funccall.c module... 6 warning(s).

14. University of Maryland History Records Did it work? –Cannot determine success from mutatee alone Mutatee output? Mutatee return value? Track history in logs –Record all output from monitor and mutator –Collect cumulative data for comparison –Report changes detected –Allow user to ultimately determine success

15. University of Maryland History Records Track mutatees by signature –History records must be locatable by mutatee –Hash of mutatee should provide unique ID –Don’t forget the shared libraries! An upgraded shared library can significantly alter the output of a mutatee Mutatee signature –SHA-1 of binary –List of program arguments –SHA-1 of dependant libraries

16. University of Maryland History Records Each successive run appends output to history record –Differences from latest run displayed to user Allows us to achieve automated testing on production-scale binaries –Logging diff output over time is a valid regression test

17. University of Maryland Errors In Generated Code Only API function errors checked so far –Entire class of faults possible after mutatee continues execution How to report faults of this class? –Almost always results in abnormal mutatee exit –All information is lost when fault detected –Need a method to isolate fault location

18. University of Maryland Errors In Generated Code Fault Isolation –New userEventCallback feature Mutatee sends data through inefficient debug channels –Force mutatee to call printf at each instPoint Additional stream or file to manage Take best of both worlds: Named Pipe –Written to by mutatee, read by mutator –Mutatee reports at function entry and exit –Mutator forwards data to monitor –Last N trace points stored in monitor

19. University of Maryland Advanced Usage Advanced Dyninst features available –Attach to existing process –Save-the-World –Enable merge tramps –Enable transactions Additional parseThat features –Timeout value –Include and exclude functions/modules by regular expression

20. University of Maryland Avalibility To be released with DyninstAPI 5.0 Demo tomorrow