1. IMPACTED TESTS BASED ON
FUNCTION COVERAGE
Shagufta, Software Engineer
Shubhi Gupta, Software Engineer
Rakesh Jha, Quality Engineering Manager
Adobe Systems India Pvt Ltd.

2. ABSTRACT
Rapid Release model of Software Development demands an expeditious shipment of product to market. The continuous integration process in turn requires frequent certification of the product build which necessitates the execution of a large number of tests. An optimization to this can be attained by an intelligent selection of a test suite for execution. Code changes can serve as reliable indicator for identifying the tests which need to be run for a particular build of the product. Assessment of the probability of failure can be done by running only the tests which invokes THE FUNCTIONS THAT HAVE BEEN CHANGED in the code (based on Code Coverage). This leads to a shorter test execution window and saving of computing resources. So, the extent of testing / the set of test cases that are to be chosen for a particular requirement are the ones which have their test requirements impacted by the function changes. In products like Adobe Acrobat that have 8k+ test cases, running the full test suite is a time consuming and redundant process. Executing only a limited set of tests thus reduces the test count significantly which can be run in no time, like in Acrobat test count reduces to approximately 300, and any break caused by code change is immediately caught. Thus, change based automation not only optimizes the test execution but also significantly reduces the turnaround time for the developer to be reported any regression caused by the change. The implementation of “Impacted Tests based on function coverage" in Adobe Acrobat team has proven to be very useful in obtaining accurate results in a short span of time.

3. CURRENT PROCESS Automation Framework New Build`
Scope of Improvement
CL 3: Version bump
CL 5: Dummy CL
CL 1: Test fix
CL 2: Bug Fix
CL 4: DLL Change
Build System
Automation
Framework
Executes 8K+ Test cases
Time: ~24 hours
‘N’ Machines
Test results available in
24 hours
Entire QE team triages the results
New Build
Fixes and Changes
Turn around time in reporting regressions ~ 30hours. Can we improve this???
Based on triage and testing, new bugs logged.

4. Component Based Testing
(Executing Feature Specific Tests)
Create PDF
PDF View
Security
Acrobat Components
Acrobat Test Suite
Create PDF Tests (500)
PDF View Tests (700)
Security Tests (300)
CONS
Tests dependent on other Components are not executed

5. Function Change Based Testing
(Executing Tests invoking the functions changed)
Function level
coverage
Product Build
Identify functions being changed
F1, F2,…….,F(n)
Query DB to get test list {T} that covers the changed functions {F1…F(n)}
Execute Tests {T} affected by the changed functions
PRO’s
Ensures testing of all components impacted by code change.
Only required tests executed, so time is reduced.

6. Architecture of “IMPACTED” Test Tool
Creates the Functional Profile of a test which is actually a list of functions covered by a given test.
TEST
PROFILER
PROVIDER
IMPACTED
TEST TOOL
Provides list of tests which covers the functions changed in a given submission.

7. (TESTS TO FUNCTION MAPPING)
TEST PROFILER
(TESTS TO FUNCTION MAPPING)
Tests
C++ Function Name
1. Commenting_Main
void comments()
2. Highlight_Text
void highligtText()
3.Commenting_TODO
int toDoCommenting()
4. CreatePDF_Maker
bool makeNewPDF()
5. …. …
8000+ tests
Executed on
Code Coverage files
Code coverage product build
Parse code coverage file
Database
COLLECT DATA
Fortnightly Task
Determining Functions invoked by a test can be done easily by Code Coverage Tools

8. TEST PROVIDER
New Product Build
Parse the files of the new build using Code Difference Tool
Identify the lines which have been changed
Query DB to get the tests corresponding to functions changed
CTAG library determines the Functions Changed .
List of Tests to be executed due to function changes

9. EXAMPLE – TEST RESULTS Before
After Implementing Function Based Automation
Before

10. Return of Investment
Saves Computing Resources:
Runs only the tests impacted by the function change without wastage of computing resources.
Stable build available all the time. Enables frequent Beta releases.
Reduction in regression turn around time:
Shorter Test Execution Window provides quick results to the developer enabling him to promptly fix the issue when the code change context is fresh in his mind.
Saves QE hours in isolating the change which caused test failure (Only the failures due to a particular code change are reported).

11. REFERENCES & APPENDIX https://stackify.com/code-coverage-tools/

12. AUTHOR BIOGRAPHY
The author is Shagufta, Software Engineer in Acrobat Team. Involved in Security Testing, Automation Tools, and Feature Testing.
The co-author is Shubhi Gupta, Software Engineer in Acrobat Team. Involved into developing Automation Tools and writing Automation Tests.
The co-author is Rakesh Jha, Quality Engineering Manager in Acrobat Team. Involved majorly in Acrobat security, commenting and Automation projects.
We are collectively responsible for developing Impacted Tests Based On Function Coverage.

13. THANK YOU!!!