1 On the Pertinence of the Notion of Etalon (Standard) for Software Measure Valérie Paulus, Miguel Lopez, Gregory Seront, Simon Alexandre

Workshop on Software Metrics Validation - 5th of July Questions What is the concept of étalon for software measure? What is its use? In which case is it pertinent? –For internal, external measure? –For which scale?

Workshop on Software Metrics Validation - 5th of July Etalon in “Classical” Metrology “(measurement) standard étalon material measure, measuring instrument, reference material or measuring system intended to define, realize, conserve or reproduce a unit or one or more values of a quantity to serve as a reference “ ISO definition of vocabulary …

Workshop on Software Metrics Validation - 5th of July Etalon in “Classical” Metrology Grounded in the physical world –… although not anymore (meter) Serves a definition purpose –Etalons often precedes definition –… it is the other way round for software measure Dissemination purpose (secondary Etalons) Referenced during calibration Measurement results are expressed as a multiple (or fraction) of the Etalon

Workshop on Software Metrics Validation - 5th of July Why Etalon for Software Measure? Help definition process? Use for calibration Gives an idea of the magnitude of the values (what is small, large)? Useless in some context?

Workshop on Software Metrics Validation - 5th of July Etalon for which Scales? Measurement results are expressed as a multiple (or fraction) of the Etalon –Implies a ratio scale –Need for addition operator –Difficult to define “meaningful” operator –Mathematically meaningful operator might not be preserve the semantic Quid for other scales? –Set of Representative Entities for which the Empirical Order is known –Etalon for ordinal scales?

Workshop on Software Metrics Validation - 5th of July The Outside World The analogy seems more obvious for external measures, but –Performance = nb of records processed/ second –Defect density = nb of known defect/nb LOC –Usability?? –Reference to physical world Etalons? –What would the Etalons be?

Workshop on Software Metrics Validation - 5th of July The Inner World At first Etalon seem useless for Internal Measures Internal measure as a mapping to a mathematical structure –Mathematical definition self-sufficient Internal Measurement seen as deterministic operation –No errors on measure –No need for calibration

Workshop on Software Metrics Validation - 5th of July The Empirical World But, … [Lopez et al.’04] Errors due to –Measurement instrument misconception –Usage condition (OS overload, memory corruption)

Workshop on Software Metrics Validation - 5th of July The Empirical World The software draw its semantic from its environment (cf. next presentation) The compiler, the processor, the OS and in some case the developer are part of the environment Complexity is defined in reference to the developer mental processes The definition of Etalon might helps the understanding of the attribute Source Code CompilationExecution User Interaction

Workshop on Software Metrics Validation - 5th of July Open Questions Would Etalons help measure definition/attribute comprehension? What is the Etalon for Usability? Do Etalons make sense for internal measure? Do you believe in calibration? Do internal attributes exist in the outside world?