1. Toward a Quality Certification Model for Grid Research Projects: an ETICS feasibility Study
Andrea Manieri
Adriano Rippa
- Engineering Ingegneria Informatica s.p.a.
On behalf of ETICS Project
Chi siamo, cosa facciamo

2. The starting point Current quality assurance standards are useful but:
are process oriented
provide theoretical guidelines on the organization
The problem:
Guidelines can be applied but it’s hard to systematically verify goodness of results
Our solution:
A quality assurance model
fully automatable in measuring and verifying activities
not subjective
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

3. ECHOGRID/EUChinagrid
The timeline
Now
OGF 21
15-19 Oct
ALL HANDS
MEETING
23-25 May
OGF 20/EGEE UF
7-11 May
FIRST BETA
VERSION
FINAL BETA
VERSION
FIRST OFFICIAL
VERSION
FEBRUARY
MAY
OCTOBER
DECEMBER
ECHOGRID/EUChinagrid
Conf. Beijing April
QUALIPSO Conference
Dec 2007
Belief Conference
RIO – BRASIL June
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

4. Terminology
Measure: a value (number or category) assigned to an attribute of an entity (e.g. software component)
Measurement: The act or process of assigning a number or category to an entity to describe an attribute of that entity.
Metric: A function whose inputs are software measures and whose output is a single numerical value that can be interpreted as the degree to which software possesses a given attribute that affects its quality.
Perspective: A set of homogenous software attributes that lead to some quality assumptions
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

5. Gird Quality Assurance Certification Model (GQACM ): a 3D QA Model
Let’s suppose A, B, C to be different perspectives
Most of Common QA Standards (e.g. CMMI) are Waterfall-like
GQACM uses 3 independent perspectives to provide client-friendly results
Users can be interested in some attributes and less in others..
So-made perspectives can have sublevels… A B C A C B
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

6. GQACM: Preliminary remarks
GQACM is independent from the ETICS tools
However, any GQACM implementation must use automation
– the most useful characteristic of the model is that it can be automatically applied!
GQACM has three perspectives (I.e. independent points of view) to derive the quality degree of a software :
Code and test
Platforms
Standards
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

7. Perspectives composition
Code Test Perspective: to qualify the goodness of the code and of the test
Coding Conventions
Code Commenting
Other QA metrics related to the corresponding programming language
Unit and functional tests
Platform perspective: to qualify B&T requirements on different platforms
The set of GQACM available platforms
Standard perspective: to qualify the compliance with standards (e.g. IPV6, BES, JSDL, SRM, WS-I)
The set of GQACM available standards
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

8. Test execution metrics
Cyclomatic complexity
CCT (method level):
if C is the cyclomatic complexity (of the method) and T is the number of executed tests then CCT = C/T
RFCT (class level):
if CK RFC is the the number of methods which can be invoked from an instance of the class target to answer a message ) and T is the number of executed tests on it then RFCT = RFC/T
ET (class level):
if E is the number of events which interact with the class and T is the number of executed tests then ET = E/T
NOCT (class level):
if CK NOC is the Number Of Children of a class and T is the number of executed tests on the class then NOCT = NOC/T
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

9. Code comment metrics FC: AD:
if F is the number of functions and C is the number of comment paragraphs then FC = F/C
AD:
Automatical documentation programs use: e.g. doxygen
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

10. Coding convention metrics 1/2
Only one class definition per header file
Filename = Classname
Filename in lowercase
No cycles in “include” operations
No C instruction (in C++ programming)
Coherent Programming Convention
Curly Brackets positioning
First character in variables names
Lower case in macros names
Coherent use of new lines
Semicolons only at the end of a line
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

11. Coding convention metrics 2/2
No Never used functions
No Not reachable code paths
No Not defined prototypes
No Never used constants
No Never used macros
No Never used types definitions
No Never used variables
Too frequent “breaks” use
No absence of variables de-alloaction
No Null pointers
Management of one instruction long “if” or one instruction long “cycle”
Shared resources management
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

12. other metrics Other software related metrics
(e.g. Chidamber and Kemerer (CK) Metrics for OO software)
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

13. Final Score1/2 For every perspective must be genrated:
a score between 0 and 100
a list of passed and not passed tests
The way to calculate the score is different for every perspective.
For code perspective
The score is simply calculated using this mathematical formula:
(X/N)*100
where X is the number of successes, N the number of used metrics
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

14. Final Score 2/2 For platform perspective For standard perspective If
DP is the number of declared compliant platforms,
CP the number of really compliant platforms,
TP the number of GQACM available platforms ,
CTP the total number of compliant platforms ,
then the score can be calculated with this mathematical formula:
(CP/DP)*50+(CTP/TP)*50
For standard perspective
DS is the number of declared compliant standards,
CS is the number of really compliant standards,
TS is the number of available standards,
CTS is the total number of compliant standards (declared + not declared),
then score can be calculated with this mathematical formula:
(CS/DS)*50+(CTS/TS)*50
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

15. Score visualization
The graphical view of the score is related to the used test tool
In general, all tools have to use at least the “classical version”:
Final score: <total>/<max possible score> ([<perspective><score>])
Where
Perspective can be: Code (C), Platforms (P), Standards (S)
Score: (where 0 is 0% compliant, 100 is 100% compliant)
Total score: sum of the scores
E.g. component X: GQACM score = 151/300 (C90-P51-S10)
ETICS could complement the standard basic visualization using a 3d graph to give the user a better feeling interface
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

16. GQACM integration in an ISO 9126 certified organization
Facts:
ISO 9126 define quality attributes .
GQACM support the measure and evaluation of quality metrics.
Hypothesis:
ISO9126 Portability: adaptability, installability
ISO9126 asks OS adaptability and installability for software of certified organizations.
GQACM may test deployment of SW on different platforms (HW+OS)
ISO9126 Maintainability: all.
ISO9126 asks to develop a software that is easy to analyse, easy to modify, Stable
GQACM metrics (such as the CK for OO or coding convetions metrics) helps to verify in concrete these requirements
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

17. GQACM integration in an ISO 9126 certified organization
ISO9126 Functionality: accuracy.
ISO9126 asks that a software have to give results according with requirements.
GQACM certification can run functional test (or plug-in external test suits) of declared compliances.
ISO9126 Functionality: adequacy.
ISO9126 asks for correct functions with specific goals.
GQACM may check not dead functions.
Our thesis:
Many ISO9126 quality metrics can be measured and evaluated based on GQACM
Any tool implementing it and can be integrated as supporting ISO9126 adoption
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

18. GQACM integration in a CMMi certified organization
Facts
CMM is process oriented (it gives guidelines).
GQACM is product oriented.
Hypotesis:
CMM2
asks to “control and value”
GQACM propose the measures and metrics to be collected, hence controlling and evaluating specific sw related items
CMM3
asks to “use internal standards and have a reference model”
GQACM proposes code and implementation conventions (how to organise and structure code, how to define tests, etc.)
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

19. GQACM integration in a CMM certified organization
asks to “have measured and controlled processes using quantitative and statistical techniques”
GQACM propose the continuous collection of quality data to perform trends analysis.
CMM5
asks to have an “improvement process and quality based on measurement”
GQACM provide numerical reference to set the improvement processes!!
Our Thesis
GQACM is not in contrast with CMMI and can be integrated in organization as a tool to support CMMI adoption
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

20. ETICS B&T tool and GQACM
Could perform many measurements and calculate metrics (e.g. JPmdPlugin and JFindbugsPlugin)  Code and test perspective
Can build and test software on several distinct platforms  Platform perspective
Could test the software compliance with standards  Standard perspective
Can automate all the processes (build, deployment, test)
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

21. Conclusions GQACM is a certification model
Automatable 3D
Not in contrast with classical standards
Free
GQACM doesn’t require big efforts to be used
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

22. Questions? Feedbacks? Thanks http://www.eu-etics.org
ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007