Visualizing Regression Test Results Ray Chen Tugrul Ince

Motivation Regression tests are essential to Software Development Lifecycle A minor fix in some part can break the software Regression Tests should be executed frequently Ideal: After each update Many test cases Hard to identify tests that fail Visualize them! Shorter time to identify failures Running tests is not a burden

Design Goals Overview of data should be immediately available No scrolling allowed Test failures should catch the eye Test results of different dates should be comparable Relationships between files and tests should be emphasized The system should be portable

Previous System Data in tabular format Letters and Colors represent the test results Two-day comparison Requires many clicks/scrolls to access information Portable (Web based) P P+ F F-

Previous System

Test Result Visualization Tool All test results fit on the screen Use Fisheye visualization Failures are identified immediately Everything is visible Display the test results of more than two days Aggregate results of a time frame Portable (Java)

Test Result Visualization Tool Details of each major test by clicking on the cell On-demand Test-Source Code Relations Spot files that might be related to a failure Similarly, find out which tests should be first executed after an update No need to have the source code Fun!

Conclusions Regression tests are essential Visualizing test results increases our understanding Visualizations let users immediately identify the tests that failed Test-Source Code correlation helps us pinpoint possible reasons for failures