CS527 Topics in Software Engineering (Software Testing and Analysis) Darko Marinov September 22, 2011

Schedule First few lectures to help you select projects –Shared memory: CHESS, IMUnit, CAPP –Message passing: Setac –Comment analysis: iComment –Regression testing: survey paper –Model-based testing: Spec Explorer –Test assertions: Orstra –Today: mock-based testing: Moles Project proposals due on Sep 27 (week-!) –Your project can be something we didn’t cover

Next Week Project proposals (due 9/27) –If we didn’t talk ( /Skype/in person), schedule a meeting with me –Create a Wiki page (low overhead but helps in keeping track) Paper reports (due 9/29) –Choose one paper, ideally related to your project You’re welcome to discuss your choice with me –Write a four-bullet report on that paper

Paper Today Moles: Tool-Assisted Environment Isolation with Closures Jonathan de Halleux and Nikolai Tillmann (TOOLS 2010)Moles: Tool-Assisted Environment Isolation with Closures Follows: one-slide overview, one empty slide, discussion (project ideas and questions)

Paper Overview Problem –“Stub out” code that your unit under test depends on: can use mocking for OO tests –Mocking is hard(er) without interfaces Solution –Moles –Use instrumentation to add lambdas/delegates Evaluation –Testing SharePoint Foundation –Achieved full code coverage

Basic Understanding? Do we all agree WHAT and WHY this does? Is there any other way to write the code in figure 3 and still get the same results.

Discussion (1) Are there any mock-based testing tools for Java? Using these tools to assist us to write unit test cases for some programs. [YL] –Is it possible to implement this for Java? [JT] Integrated automated assertion generation with mock-based test cases. [YL] Does this tool promote bad coding habits? Shouldn't code that is non deterministic be heavily factored out? [SB] –When to use Moles (non-determinism)? [SO] –Isolating a unit is always a good idea? [FS]

Discussion (2) use run-time data (from a previous run) to determine what the values should be? [SB] a simpler approach that just generates arbitrary data still work with PEX? [DeG] any potential disadvantages to using mocking to isolate tests by reaching inside the behavior of the method being tested? Could this miss bugs or overspecify tests? [DwG] In multi-threaded applications under test to isolate the scheduler, Moles + Chess. [FS] –extend Moles for multithreaded code [CM]

Discussion (3) automatically detecting [what to mock] [SO] –simplify writing models by a skilled developer[AD] how to guarantee a correct [model]? [FS] using Moles and SpecExplorer together[DeG] Moles to have behavior more similar to other mocking frameworks like EasyMock [DwG] (something missing or different?) could this be used in development (not just testing) to further customize third-party software. E.g., add more possible return values to a sealed class's method. [JT]

Discussion (4) completely eliminate the code-injection and simply override all function calls, by simply changing the call/jmp addresses? [AY] integrate Moles with some other more standard mock testing framework [KN] Support for other unit testing frameworks [AK] determine if [detouring] could've been avoided if a better design was used [AK] Are there methods or classes that can not be mocked/delegated using Moles? [AK] Support for VB.NET or.NET 4.0? [AD]

Discussion (5) Extend this to other languages like Java, C/C++ [AY] To what extent are tools like Moles used outside of academia? [CM]