1. SPI Software Process & Infrastructure
Quality Assurance (QA)
QA reports generation - 22 June 2005
Johanne BENARD

2. SPI - Software Process & Infrastructure
QA reports generation
Test coverage reports
Basics
Some examples
Other utilities
Savannah Quality Assurance (QA) reports
The web interface
Capabilities
J.Benard
SPI - Software Process & Infrastructure

3. Test coverage reports - Basics
Aim at:
Showing what percentage of an application has been executed by the test process,
Measuring effectiveness of a system test.
Two different principles in GCOV:
Statement coverage analysis:
Code break down into basic blocs,
Block solicitations counting
Branch coverage analysis:
Conditional possible paths
Path usage frequency
J.Benard
SPI - Software Process & Infrastructure

4. Test coverage reports - Basics
Code instrumentalisation:
-fprofile-arcs
-ftest-coverage
LTP GCOV
Suite (*):
lcov, genhtml, etc.
Compilation time
Runtime
Coverage analysis
File.bb
File.cxx, .c , .h, etc.
File.da
File.info
File.bbg
(*) LTP: Linux Test Project
LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed while running a particular test case.
J.Benard
SPI - Software Process & Infrastructure

5. SPI - Software Process & Infrastructure
Some examples - POOL
J.Benard
SPI - Software Process & Infrastructure

6. SPI - Software Process & Infrastructure
Some examples - POOL
J.Benard
SPI - Software Process & Infrastructure

7. SPI - Software Process & Infrastructure
Some examples - CLHEP
J.Benard
SPI - Software Process & Infrastructure

8. SPI - Software Process & Infrastructure
Some examples - PI
J.Benard
SPI - Software Process & Infrastructure

9. SPI - Software Process & Infrastructure
Some examples - SEAL
J.Benard
SPI - Software Process & Infrastructure

10. SPI - Software Process & Infrastructure
Some examples - ROOT
J.Benard
SPI - Software Process & Infrastructure

11. Test Coverage - Other utilities Tools
Genhtml:
Creates an HTML view from coverage data found in INFOFILE
Geninfo:
Converts .da files into .info files
Genpgn:
Creates an overview image for a given source code file of either plain text or .gcov file format
Gendesc:
Converts a test case description file into a format as understood by genhtml INPUTFILE
J.Benard
SPI - Software Process & Infrastructure

12. SPI - Software Process & Infrastructure
QA Savannah Reports
J.Benard
SPI - Software Process & Infrastructure

13. QA Savannah report – Web Form
Bugs, Tasks,
Support and
Patches
Default:
Project beginning
Today
.ini file: header,
Style sheet
and options
Category,
Assigned To
Severity, etc.
J.Benard
SPI - Software Process & Infrastructure

14. QA Savannah report – Web Form
Potential users: projects or experiments that actively use Savannah trackers (bugs, tasks, support, patch)
Current users: JRA1 coordination, JRA1 Middleware, LCG 2 Sites
Ready for LCG projects:
POOL,
PI,
SEAL
Tried on several other projects:
Athena,
Gaudi,
Cobra,
Iguana
J.Benard
SPI - Software Process & Infrastructure

15. QA Savannah report - Capabilities
Options
used
Global
processing
Breakdown
Into
Savannah
fields
J.Benard
SPI - Software Process & Infrastructure

16. QA Savannah report - Capabilities
Statistics on a
given period
Translated into
graph
J.Benard
SPI - Software Process & Infrastructure

17. Some examples – LCG 2 Sites
J.Benard
SPI - Software Process & Infrastructure

18. Some examples – JRA1 Middleware
J.Benard
SPI - Software Process & Infrastructure

19. SPI - Software Process & Infrastructure
Some examples - POOL
J.Benard
SPI - Software Process & Infrastructure

20. SPI - Software Process & Infrastructure
Some examples - SEAL
J.Benard
SPI - Software Process & Infrastructure

21. SPI - Software Process & Infrastructure
Some example - PI
J.Benard
SPI - Software Process & Infrastructure

22. SPI - Software Process & Infrastructure
Quality Assurance
J.Benard
SPI - Software Process & Infrastructure

23. Web site: http://spi.cern.ch/qa
J.Benard
SPI - Software Process & Infrastructure

24. Thank you for your attention!
Conclusion
Proposed services:
To generate customized test coverage reports
To set parameters for Savannah QA reports
To repeat this talk for projects / experiments
Thank you for your attention!
Question?
J.Benard
SPI - Software Process & Infrastructure