Tools for Automated Testing Presented by: Žygimantas Mockus

What is automated software testing? A process which covers: the use of software to control the execution of tests the comparison of actual outcomes to predicted outcomes other test control and test reporting functions.

Why should we rely on automated software testing?

Time savings translates directly into cost savings It improves testing productivity It improves accuracy It increases test coverage It allows to run regression tests on a code that is continuously changing

What are the disadvantages of automated software testing Bigger initial investments You cannot automate everything, some tests still have to be done manually

Main approaches of test automation Code-driven testing. The public (usually) interfaces of classes, modules or libraries are tested with a variety of input arguments to validate that the results that are returned are correct. Graphical user interface testing. A testing framework generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.

Tools for code-driven testing Unit-testing frameworks (xUnit). Test runners and code-coverage tools (NCrunch, dotCover)

Unit Testing framework example (MSTest)

Unit Testing framework example (NUnit)

Usual TDD (Test-Driven development) cycle without tools Write the test Stop and run the test Write the code under test Stop and run tests Refactor the code under test Stop and run tests

Usual TDD (Test-Driven development) cycle with NCrunch Write the test Write the code under test Refactor the code under test

Test runners and code-coverage tools (NCrunch)

NCrunch and code coverage NCrunch collects test coverage for your code while it runs your tests. This is shown next to your code in coloured markers showing which lines the tests touched, with marker colours indicating pass or fail status. You can also navigate to any covering tests from any line of code, making it easy to see which tests you might impact with a change. Full code coverage metrics are also available for your entire solution, allowing you to see where your code coverage is heavy and where it's light.

Test runners and code-coverage tools (NCrunch)

NCrunch and performance metrics NCrunch profiles your tests during their execution to pick up the execution time of every line of code under test. Metrics are shown inline conveniently with a tooltip, and 'hot spots' are shown with special colouring on the code coverage markers.

Test runners and code-coverage tools (NCrunch)

NCrunch and exception handling The stack traces of exceptions thrown from your tests are processed by NCrunch and projected over the code coverage markers. This makes it really easy to spot where your tests went wrong, without the information getting in your way.

Test runners and code-coverage tools (NCrunch)

NCrunch and smart test execution NCrunch tracks all sorts of interesting statistics about your tests, and it uses this information in the most intelligent way possible. Tests that you have recently impacted with your code changes are highly prioritised for execution. NCrunch uses a powerful weighting system designed to give you the most important feedback as fast as possible.

Test runners and code-coverage tools (NCrunch)

NCrunch – more info More info available at:

dotCover -.NET Code Coverage Tool Reporting statement-level coverage in.NET Framework and Silverlight applications. Integration with Visual Studio 2005, 2008, 2010, 2012 and Highlighting for covered and uncovered code in Visual Studio. Detecting which tests cover a particular location in code. Managing and running unit tests with ReSharper's unit test runner or dotCover's own. Support for multiple unit testing frameworks: MSTest, NUnit, xUnit, and MSpec. Console utility to use with a Continuous Integration server. In fact, a server-side version of dotCover is built into TeamCity and available free of charge. Fetching coverage analysis data from TeamCity to Visual Studio. Generating code coverage reports as XML, HTML, or JSON.

dotCover - Running and Managing Unit Tests

dotCover – Navigation to covering tests

dotCover – Coverage highlighting in Visual Studio

AutoHotkey – Free keyboard macro program Automate almost anything by sending keystrokes and mouse clicks. You can write a mouse or keyboard macro by hand or use the macro recorder. Create hotkeys for keyboard, joystick, and mouse. Virtually any key, button, or combination can become a hotkey.

AutoHotkey – Automated GUI testing software

TestComplete – Automated GUI testing software A choice of scripting languages (VBScript, JScript, C++Script, C#Script and DelphiScript) Automated test scripts can call Windows API, DLL and WMI functions,.NET and Java classes, ADO and BDE database access functions and use ActiveX and COM objects. This way, testers can verify the application state and behavior far beyond the user interface.

TestComplete – Automated test recording

TestComplete – Automated GUI testing software

TestComplete – View screenshots of each test operation

TestComplete – Visual modifications of tests in special editor

Questions?