1. Visualizing Regression Test Results
Ray Chen
Tugrul Ince

2. Motivation
Regression tests are essential to Software Development Lifecycle
A minor fix in some part can break the software
Regression Tests should be executed frequently
Ideal: After each update
Many test cases
Hard to identify tests that fail
Visualize them!
Shorter time to identify failures
Running tests is not a burden

3. Design Goals Overview of data should be immediately available
No scrolling allowed
Test failures should catch the eye
Test results of different dates should be comparable
Relationships between files and tests should be emphasized
The system should be portable

4. Previous System Data in tabular format
Letters and Colors represent the test results
Two-day comparison
Requires many clicks/scrolls to access information
Portable (Web based) P P+ F F-

5. Previous System

6. Test Result Visualization Tool
All test results fit on the screen
Use Fisheye visualization
Failures are identified immediately
Everything is visible
Display the test results of more than two days
Aggregate results of a time frame
Portable (Java)

7. Test Result Visualization Tool
Details of each major test by clicking on the cell
On-demand
Test-Source Code Relations
Spot files that might be related to a failure
Similarly, find out which tests should be first executed after an update
No need to have the source code
Fun!

8. Conclusions Regression tests are essential
Visualizing test results increases our understanding
Visualizations let users immediately identify the tests that failed
Test-Source Code correlation helps us pinpoint possible reasons for failures