1. Empirical Validation of UML Statechart Diagram Metrics: A Family of Three Experiments José A. Cruz-Lemus Department of Computer Science University of Castilla – La Mancha (Spain)

2. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20042 Outline Introduction Family of Experiments Definition Planning Operation Analysis and Interpretation Validity Conclusions

3. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20043 Introduction Guarantee quality characteristics in OO systems since the initial lifecycle stages. MDD & MDA: conceptual models are backbone of OO systems development. Scarce references to behavioural diagrams.

4. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20044 Introduction SizeNEntryANExitANANSSNCSNGNE Structural Complexity NTCC (Genero et al., 2002) Theoretical validation Briand et al. Poels and Dedene Empirical validation Metrics for UML Statechart Diagrams (SD)

5. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20045 Outline Introduction Family of Experiments Family of Experiments Definition Definition Planning Planning Operation Operation Analysis and Interpretation Analysis and Interpretation Validity Validity Conclusions

6. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20046 Family of Experiments Chronology

7. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20047 Family of Experiments Definition Analyze Structural complexity and size metrics for UML SD For the purpose of purpose ofEvaluating With respect to respect to The capability of being used as indicators of the understandability of UML SD From the point of view of of view ofResearchers In the context of context of Undergraduate students of Computer Science and SE teachers of the Computer Science Department at the University of Castilla - La Mancha

8. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20048 Family of Experiments Planning E1R1R2 Context 8 teachers 10 fifth-year students 24 third-year students 49 third-year students Subjects high / averagelow experience Variables structural complexitysize Indep.: UML SD structural complexity and size understandability Dep.: UML SD understandability Instrumentation 20 UML SD Independent: metrics metrics Dependent: understandability time understandability time correctness (correct vs. answered) correctness (correct vs. answered) completeness (correct vs. asked) completeness (correct vs. asked)

9. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 20049 Family of Experiments Planning E1R1R2 Hypotheses H 0,1 No correlation UML SD structural complexity and size - understandability time H 0,2 No correlation UML SD structural complexity and size - correctness H 0,3 No correlation UML SD structural complexity and size - completeness Design Within-subjects. Diagrams in different orders.

10. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200410 Family of Experiments Operation

11. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200411 Family of Experiments Operation

12. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200412 Family of Experiments Analysis and Interpretation

13. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200413 Family of Experiments Analysis and Interpretation

14. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200414 Family of Experiments Analysis and Interpretation NEntryANExitANANSSNCSNENGNTCC UTime0.198-0.2130.4630.4920.3420.4000.5350.5210.406 Correct-0.0770.038-0.170-0.376-0.596-0.109-0.434-0.207-0.017 Complt0.0600.065-0.148-0.266-0.525-0.051-0.438-0.1250.029 UTime-0.021-0.3840.5290.5860.4280.3530.5720.5410.404 Correct-0.239-0.056-0.1690.009-0.201-0.132-0.118-0.154-0.238 Complt-0.1640.050-0.3780.092-0.1270.006-0.203-0.016-0.024 Utime-0.091-0.3910.6030.5280.3440.3580.5190.5470.436 Correct-0.220-0.035-0.1160.005-0.3850.048-0.5060.0200.058 Complt-0.2060.019-0.157-0.041-0.3690.027-0.519-0.0130.034 α = 0.05E1R1R2

15. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200415 Family of Experiments Validity Conclusion No pupulation sample (small and voluntary participation) ‘Pencil & Paper’ Construct Theoretical validation Internal Differences within subjects, knowledge of the universe of discourse, precision in time checking, learning and fatigue effects, motivation, plagiarism... External Materials and task used, not real projects data. Subjects. Expericenced enough.

16. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200416 Outline Introduction Family of Experiments Definition Planning Operation Analysis and Interpretation Validity Conclusions

17. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200417 Conclusions Metrics for structural complexity and size of UML Statechart Diagrams. Family of experiments. Descriptive statistics and Spearman’s correlation coefficients.  Experience  Results NA, NSS, NG, NT highly correlated to the understandability time.

18. Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th 200418 Conclusions Future work Further validation. Practitioners and profesionals. Real projects data. Prediction model based on the metrics....

19. Empirical Validation of UML Statechart Diagram Metrics: A Family of Three Experiments José A. Cruz-Lemus {JoseAntonio.Cruz@uclm.es} Department of Computer Science University of Castilla – La Mancha (Spain)