1. Defining measures for WMS and VOMS services evaluation
Sergio Andreozzi
INFN-CNAF
JRA1 IT/CZ meeting, 10 november 2005

2. WMS-related measurers review VOMS-related measurers review
OUTLINE
Facts
Why measuring
What is a measure
Basics of measurement theory
Document structure
WMS-related measurers review
VOMS-related measurers review
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

3. Facts See SA1 Metric WG wiki are meaningful are correct
During the last EGEE Operations workshop (Sep05), SA1 created a Metric WG for the definition of a set of metrics useful to evaluate, functionality and performance of the EGEE Grid
It is a mixed approach where both passive and active monitoring are in place
There is a fast push to have something in place “now”
It is vital to enter the definition process in order to be sure that the defined metrics
are meaningful
are correct
can be measured
See
SA1 Metric WG wiki
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

4. Why Measuring measurement Numerical objects, Real-world objects,
relationships and operationts
Real-world objects,
relationships and operationts
Intelligence
barrier
mathematics,
statistics
Results relevant
to the world
Numerical results
interpretation
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

5. What is a Measure
Measurement: the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to describe them according to clearly defined rules.
Measure: Let A be a set of physical or empirical objects. Let B be a set of formal objects, such as numbers. A measure μ is defined to be a one-to-one mapping μ : A B.
Measurement Method: logical sequence of operations, described generically, used in quantifying an attribute with respect to a specified scale
Measurement Unit: a particular quantity, defined and adopted by convention, with which other quantities of the same kind are compared in order to express their magnitude relative to that quantity
Measurement Scale: (informally) a mathematical structure assuring that predicates about entities in the real world are equivalent to predicates in the numerical systems
For more information:
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

6. Technical Document Structure
We plan to write a reference document to be published in EDMS and representing the official definitions of measures
Starting from WMS and VOMS, one evolving document for all IT/CZ services?
STRUCTURE:
Introduction
Background on Measurement Theory
Measures Definition by Systems
WMS
VOMS …
TITLE: Definition of Measures for the …
WMS and VOMS
gLite IT/CZ cluster services
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

7. Template for Measure Definition
ID: e.g., an identifier at least unique within the document
System: e.g., WMS
Component: e.g., Job Controller
Attribute: e.g., Waiting Requests
Measurement Unit: e.g., request
Measurement Scale: i.e., nominal, ordinal, interval, ratio, absolute
Description:
Measurement Method
e.g., grep " g$“ $GLITE_WMS_TMP/workload_manager/input.fl|wc -l
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

8. The Measurement Process
Once a measure has been defined, the developers responsible for the affected piece of software must provide an efficient recipe to extract the information by using e.g., a shell command
The ideal would be to have:
short term: the recipe to gather each attribute measure by using a shell script (e.g., grep of a log file)
medium term: a shell command returning the last measure for all the define attributes in (at least) two format: xml + raw
long term: a web service port getStats?
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

9. A Measurement Construct
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005

10. Measures for WMS and VOMS Review
EGEE JRA1 IT/CZ Meeting - Bologna, 10 November 2005