1. DAIMIHenrik Bærbak Christensen1 What is Software Quality?

2. DAIMIHenrik Bærbak Christensen2 Reflection What is software quality to you ? Name a program/application/system of high quality? Low quality? What aspects of it make it ‘high quality’ or ‘low quality’?

3. DAIMIHenrik Bærbak Christensen3 Definition of quality Criteria for a good definition of quality: allow us to measure quality in a meaningful way Why? –avoid dogmas –measure if techniques really improve quality –what investment gives most ‘return on investment’?

4. DAIMIHenrik Bærbak Christensen4 Quality views Kitchenham & Pleeger Transcendental view –quality can be recognized but not defined User view –quality is fitness for purpose Manufacturing view –quality is conformance to specification Product view –quality is inherent characteristics of the product Value-based view –quality is the amount costumer will pay for product

5. DAIMIHenrik Bærbak Christensen5 View clashing Stakeholders have different view on same product Morale: Trade-offs between cost and quality The product Researchers and Teachers Product view Customers User view Developers Manufacturing view Management Value-based view

6. DAIMIHenrik Bærbak Christensen6 Reflection What views did we use in our discussion of good/bad systems before? dPaSS / dSoftArk course preaches: –Low coupling / high cohesion –Favor object composition over class inheritance What is the view here ???

7. DAIMIHenrik Bærbak Christensen7 Quality frameworks Quality has many faces...

8. DAIMIHenrik Bærbak Christensen8 Quality Attributes... measure quality in a meaningful way. This requires –that we define the aspects/qualities we measure –that we agree on some kind of scale: a metric

9. DAIMIHenrik Bærbak Christensen9 Measuring quality Performance Maintainability Usability Quality Attribute Metric Measurement Choose alternatives Quality Framework

10. DAIMIHenrik Bærbak Christensen10 Quality Frameworks Of course – a lot of different quality frameworks have been proposed. –Bass, Clements and Kazman: Architecture Quality Attributes –Sommerville: Software Engineering books –IEEE Standard

11. DAIMIHenrik Bærbak Christensen11 Quality framework (Bass et al.) System quality attributes –Availability –Modifiability –Performance –Security –Testability –Usability Business qualities –Time to market –Cost –Projected lifetime –Targeted market –Roll-out schedule –Integration with legacy sys. Architectural qualities –Conceptual integrity –Correctness and completeness –Buildability

12. DAIMIHenrik Bærbak Christensen12 Bass’ System Qualities Availability –probability that the system will be operational when needed Modifiability –ease with which the system supports change Performance –response time Security –ability to withstand attacks/threats Usability –how easy it is for the user to accomplish a desired task Testability –ease with which the software can be made to demonstrate its faults

13. DAIMIHenrik Bærbak Christensen13 Discussion Which of these qualities can be judged by a testing process? Which can be measured? –quantitatively (hard numbers: “4.3”) –qualitatively (judgment: “low/high”)

14. DAIMIHenrik Bærbak Christensen14 Dependability Sommerville Name systems where one is very important? What is the relation to ‘usefulness’? equates to its trustworthiness.

15. DAIMIHenrik Bærbak Christensen15 IEEE (Burnstein) IEEE Standard Glossary of SE terminology: 1. Quality relates to the degree to which a system, system component, or process meets specified requirements. 2. Quality relates to the degree to which a system, system component, or process meets customer or user needs, or expectations. Exercise: Relate to Kitchenham’s quality views...

16. DAIMIHenrik Bærbak Christensen16 War Story Major Danish company, highly active in services on the WWW. Requirement specification: Make something that is smarter than the competitors’ product...

17. DAIMIHenrik Bærbak Christensen17 Reliability and failure Reliability: Probability that a software system will not cause the failure of the system for a specified time under specified conditions. Failure is then defined by either –the specification (IEEE definition 1) –the user/customer (IEEE definition 2)

18. DAIMIHenrik Bærbak Christensen18 Reliability is in the eyes of the beholder Note that conditions are part of the definition. But conditions may change ! –Fault causing crash in Word’s fax service ? –Developer may be able to use his own prototype tool, but others may make it crash every time… –Our car’s engine would cut out during heavy rain 

19. DAIMIHenrik Bærbak Christensen19 Reliability and Failures Thus fixing any defect and removing a failure will not give same increase in reliability... Example: two defects –Word cannot save documents due to defect –Word cannot reconfigure button panel due to defect Which defect removal will increase reliability most?

20. DAIMIHenrik Bærbak Christensen20 Quality View Cem Kaner: The measure of a product’s or service’s quality is the satisfaction of their customers, not the match to a specification. A program’s quality is: –the features that make the customer want to use the program –the flaws that make the customer wish he’d bought something else

21. DAIMIHenrik Bærbak Christensen21 Yet another view Beizer There can never be an absolute definition for bugs, nor an absolute determination of the existence. The extent to which a program has bugs is measured by the extent to which it fails to be useful. This is a fundamentally human measure. Interesting… It makes many programs into defects instead

22. DAIMIHenrik Bærbak Christensen22 Ensuring reliability Reliability is compromised by failures. Failures happen when the system is in an error state, which may be caused by a defect. So – reliability can be enhanced by several measures. –Fault avoidance: simply avoid introducing defects! –Fault detection and removal: Find and remove the defects before they cause failures. –Fault tolerance: Ensure that faults does not lead to failures.

23. DAIMIHenrik Bærbak Christensen23 Fault avoidance Use methods, tools, languages, techniques that are known to minimize the probability of introducing defects. –Corollary: Avoid using m,t,l,t that are notoriously known for introducing defects That is: Never, ever, use C or C++ Use safe languages (Java, C#) Use proven design techniques –encapsulation, no goto’s, type-checking, etc.

24. DAIMIHenrik Bærbak Christensen24 Fault detection and removal The primary focus of this course: Hunt down the defects by what-ever means (execution, review) and remove them.

25. DAIMIHenrik Bærbak Christensen25 Fault tolerance If we know that cannot remove all defects then at least ensure that they do not lead to failures –Exception handling: handle IO errors gracefully null pointer exceptions? –N-version programming let n teams develop the same software unit vote on the n results Space shuttle maintains five parallel systems –one running base flight software –four running mission and flight software

26. DAIMIHenrik Bærbak Christensen26 Run-time cycle Faults cause failures when faulty code is executed with inputs that expose the fault. –I_e: input that will lead the system into error state Program execution state I_e error states Input space

27. DAIMIHenrik Bærbak Christensen27 Economic consequences Not all input have the same probability –Which function to you use most in Word? save document reconfigure button panel –If a defect is present which unit would you rather have it in? save document algorithm reconfigure button panel algorithm Thus with a given testing budget not all test cases are born equal!

28. DAIMIHenrik Bærbak Christensen28 Summary Quality is many things... There are several frameworks defining quality All, however, must deal with reliability –Reliability: Probability that a software system will not cause the failure of the system for a specified time under specified conditions Reliability techniques –fault avoidance; fault removal; fault tolerance Not all faults are equally important for reliability Testing budget should be used wisely