1. 1/24 Impact of design decisions on software quality Wiebe Hordijk 12-10-2004

2. 2/24 Goal of this talk Show you my research framework Show you the status of my PhD study Discuss research design ideas

3. 3/24 Agenda Introduction Overall goal Theoretical framework Related work Research designs

4. 4/24 Introduction Job daysPerWeek = 3 title = “ICT Consultant” Project client = “Ministry of Foreign Affairs” interesting = true CurrentAssignment PhD study daysPerWeek = 2 startDate = 1-1-2004 plannedEndDate = 1-1-2008 mutualBenefit

5. 5/24 Overall goal To investigate correlations between design decisions and software quality To build better software To build software more predictably Why? Theoretical Framework Empirical Research Case study Research How? Form:

6. 6/24 Example hypotheses Layered systems are more reusable than non-layered systems. Systems with a relational database are more reliable than systems with file storage. In the majority of systems above 500 function points, systems with a domain model have better changeability than systems with a transaction script.

7. 7/24 Theoretical framework Why? Reusable Rationale Blocks Design Space outline Quality Attributes & Indicators Examples

8. 8/24 Theoretical framework: Why Research Framework Hyp. 1Hyp. 2Hyp. 3 Enables Refines Web Site Projects HelpFeedback

10. 10/24 Quality Attributes

11. 11/24 Quality Indicator example

12. 12/24 Design Space outline example

13. 13/24 Effects Table example: Data Storage

14. 14/24 Example: Sorting Algorithm

15. 15/24 Framework Class Diagram

16. 16/24 Related work Patterns SEI Reasoning Frameworks Other design space theories –QOC –DRL –IBIS

17. 17/24 Research designs Criteria Difficulties Possible solutions Examples

18. 18/24 Criteria for research designs Construct validity, internal validity, external validity, reliability Conclusion must corroborate, reject or refine hypothesis Research must be feasible Hypothesis must be: –Non-trivial –Interesting to practitioners

19. 19/24 Difficulties in this research Long chains of influence Many rival hypotheses Many variables Little data Finding data costs much effort

20. 20/24 1: Reasoning Frameworks Described by SEI Deduct system properties from component properties Example: Failure probability with n-version majority voting:

21. 21/24 Drawbacks of Reasoning Frameworks Only available for some attributes Do not start from design choices, but from component properties

22. 22/24 2: Intermediate indicators « designProblem » Domain Logic Structure «option» Domain Model «option» Table Module «option» Transaction Script « qualityIndicator » Change Effort Hypothesis « intermediateIndicator » # Lines of Code changed « intermediateIndicator » Analysis Effort

23. 23/24 Intermediate Indicators for Reliability number of test cases/source lines of code (R1) ; number of test cases/number of requirements (R2); test lines of code/source lines of code (R3); number of asserts/source lines of code (R4). code coverage (C) “Reliability = 0.669 + 1.586*R1 + 0.0513*R2 – 0.0290*R3 + 0.192*R4 – 0.0774*C” N. Nagappan, 2004

24. 24/24 3: Rationale Harvesting Lots of consultants and system designers document rationale This rationale can be generalized into rationale blocks The support for this rationale must be documented This will only yield weak support

25. 25/24 Impact of design decisions on software quality Wiebe Hordijk 12-10-2004 Thank you!