1. Emilia Mendes Professora Visitante CAPES/ Associate Professor Univ. Auckland, NZ. Introdução a Métricas, Qualidade e Medição de Software

2. Motivation Metric's Definition Goal-based Framework Empirical Investigations Threats to Validity Outline

3. Emilia Mendes3 Economic measurements to determine price and pay increase. Measurement in radar systems to detect aircrafts when direct vision obscured. Measurement in medical systems enabling physicians to diagnose illnesses. Height and size measurement to ensure our clothing will fit properly. Many more… Measurement Occurs Everywhere…

4. Emilia Mendes4 “Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to describe them according to clearly defined rules” Entity: an object or an event in the real world. Attribute: feature or property of an entity. Numbers and symbols are abstractions used to reflect perceptions of the real world. Measurement: Definition

5. Emilia Mendes5 People’s heights S M L  T-shirts’ sizes Examples of Entities & Attributes

6. Emilia Mendes6 Software measurement is the process of measuring attributes (measures) of software entities Software measures are used to quantify software, resources, and development processes Measures can be directly measured (lines of code, number of html pages), or indirectly measured (software quality, software usability). Software & Measurement

7. Emilia Mendes7 Software measurement is important for three basic activities: –Understand what is happening during development and maintenance. –Control what is happening on our projects. –Improve our resources, processes and products… Improve what we do, how it is done, and who does it! Software Measurement: Importance

8. Emilia Mendes8 Every measurement action must be motivated by a particular goal or need that is clearly defined and easily understandable. Measurement objectives must be specific, tied to what the managers, developers and users need to know. Motivation & Objectives

9. Emilia Mendes9 Our new technique guarantees 100% reliability! Our tool improves productivit y by 200%!!! Build your code with half the staff in a quarter of the time!!! Cuts test time by 2/3!! Measurement for Promotion

10. Emilia Mendes10 Cost and Effort Estimation Productivity Measures and Models Data Collection Quality Models and measures Reliability Models Performance Evaluation and Models Structural and Complexity Metrics Capability Maturity Assessment Management by Metrics Evaluation of Methods and Tools Scope of Software/Web Measurement

11. We Introduce here a Goal-based Framework to help organisations decide and focus on what they need, and from there choose what to measure and how it will be measured. The Framework described was proposed by Norman Fenton and Shari Lawrence Pfleeger on their book Software Measurement and Metrics. How to initiate a measurement programme? Emilia Mendes11

12. Emilia Mendes12 The primary question in a goal-based measurement is: –“What do we want to know or learn?” instead of “what measures shall we use?” The answers will depend on the organisation’s goals, therefore no fixed set of measures is universally adequate. Goal-based Framework

13. Emilia Mendes13 The framework is based on three principles: goals –Determining relevant measurement goals entities –Classifying the entities to be examined level of maturity –Identifying the level of maturity an organisation has reached Goal-based Framework

14. Emilia Mendes14 The Goal-Question-Metric (GQM) approach provides a framework involving three steps: –Goal –Goal: List the major goals of the development or maintenance project –Question –Question: Derive from each goal the questions that must be answered to determine if the goals are being met –Metric –Metric: Decide what must be measured in order to be able to answer the questions adequately GQM Approach

15. Emilia Mendes15 Goal: Evaluate effectiveness of coding standard using Fortran Questions: Who is using the standard? What is coder productivity? What is code quality? Measures: Proportion of coders - using standard - using Fortran Experience of coders -with standard -with Fortran -with environment -etc Code size (LOC, function Points etc) Effort Errors… GQM Example

16. Emilia Mendes16 PurposePurpose: To (characterize, evaluate, predict, motivate etc) the (process, product, model, measure etc) in order to (understand, assess, manage, engineer, learn, improve etc) it. –Example: To evaluate the maintenance process in order to improve it. Template for Goal Definition (1)

17. Emilia Mendes17 Perspective: Examine the (cost, effectiveness, correctness, defects, changes, product measures etc) from the viewpoint of the (developer, manager, customer etc). Example: Examine the cost from the viewpoint of the manager. Template for Goal Definition (2)

18. Emilia Mendes18 Environment: The environment consists of the following: process factors, people factors, problem factors, methods, tools, constraints etc. Example: The maintenance staff are poorly motivated programmers who have limited access to tools. Template for Goal Definition (3)

19. Emilia Mendes19 Purpose: –Determine a way to improve the productivity of our development team by evaluating their current productivity Perspective: –Examine the team personality factors, expertise of the development organisation, team’s analysis and design techniques, knowledge of programming languages used by our development team, from the point of view of the project manager Environment and constraints: –Payroll applications programming in C++ –100 software developers with 5 or more years experience in C++ –Customers are businesses –Do not maintain a reusable module database –Examine new projects completed from 1/1/2008 to 31/12/2009. Example

20. Emilia Mendes20 Three classes of entities: –Processes – collections of software related activities usually associated with some timescale. Different SW processes: development, maintenance, testing, reuse, configuration and management process etc. –Products – any artefacts, deliverables or documents that result from a process activity. –Resources – entities required by a process activity. What Entities to Measure?

21. Emilia Mendes21 Within each entity class, there can be two types of attributes: –Internal attributes of a product, process or resource are those that can be measured by examining the product, process or resource on its own, separate from its behaviour. –External attributes of a product, process or resource relates to its environment. The behaviour of the product, process or resource is important, rather than the entity itself. Attribute Types

22. Emilia Mendes22 Sample Process Measures (1)

23. Emilia Mendes23 Sample Process Measures (2)

24. Emilia Mendes24 Sample Product Measures (1)

25. Emilia Mendes25 Sample Product Measures (2)

26. Emilia Mendes26 Sample Resource Measures

27. Emilia Mendes27 We can relate the GQM template to the Entity & Attribute framework. A Goal or question can be associated with at least one pair of Entities and Attributes. Once a measure is obtained, it can be related back to questions and goals. GQM, Entities and Attributes

28. Emilia Mendes28 Predictable Process Standard, Consistent Process Disciplined Process Continuously Improving Process Maturity of an Organisation: CMM Levels

29. Emilia Mendes29 Since process maturity suggests that one can only measure what is visible, the incorporation of process maturity with GQM helps identify what measures will be most useful to the organisation at that particular moment in time. Combining GQM with Process Maturity

30. Emilia Mendes30 Five major types –nominal –ordinal –interval –ratio –absolute knowing the characteristics of each type helps us to interpret the measures Scale Types (1)

31. Emilia Mendes31 The most primitive form of measurement Defines classes or categories, and places entities in a particular class or category, based on the value of the attribute. Characteristics –Entities are only organised in different classes. –No notion of ordering among the classes –Any numbering or symbolic representation of the classes is acceptable, but there is no notion of magnitude associated with the numbers of symbols. Nominal Scale

32. Emilia Mendes32 Examples using Nominal Scale

33. Emilia Mendes33 Often useful to augment the nominal scale with information about an ordering of classes or categories. Characteristics Entities belong to classes that are ordered with respect to the attribute. The numbers represent ranking only, so addition, subtraction and other arithmetic operations have no meaning. Ordinal Scale

34. Emilia Mendes34 Examples using Ordinal Scale

35. Emilia Mendes35 Useful to augment the ordinal scale with information about the size of the intervals that separate the classes. Characteristics An interval scale preserves order, as with an ordinal scale An interval scale preserves differences but not ratios. That is, we know the difference between any two of the ordered classes in the range of the mapping, but computing the ratio of two classes in the range does not make sense. Addition and subtraction are acceptable operations, but not multiplication and division Interval Scale

36. Emilia Mendes36 Examples of Interval Scale

37. Emilia Mendes37 This is the most useful scale of measurement, common in the physical sciences. Characteristics It is a measurement that preserves ordering, the size of intervals between entities, and rations between entities. There is a zero element, representing total lack of the attribute. All arithmetic can be meaningfully applied to the classes in the range of the mapping. Ratio Scale

38. Emilia Mendes38 Examples using Ratio Scale

39. Emilia Mendes39 This is the most restrictive scale of measurement. Characteristics The measurement for an absolute scale is made simply by counting the number of elements in the entity set. The attribute always takes the form “number of occurrences of x in the entity”. There is only one possible measurement, namely the actual count. All arithmetic analysis of the resulting count is meaningful. Absolute Scale

40. Emilia Mendes40 Examples using Ratio Scale

41. Emilia Mendes41 Summary of Scale Types

42. Exercise