20. The Software-Quality Landscape SCMP-368 Software Development James Skon

Did you know….. Testing isn’t the best way to find errors! Taking more time to write code can actually increase error rate! No single method is sufficient for finding all errors!

Topics Characteristics of Software Quality Techniques for Improving Software Quality Relative Effectiveness of Quality Techniques When to Do Quality Assurance The General Principle of Software Quality

Characteristics of Software Quality External Correctness Usability Efficiency Reliability Integrity Robustness Internal Maintainability Flexibility Portability Reusability Readability Testability Understandability

Characteristics of Software Quality - Correctness The degree to which a system is free from faults in its specification, design, and implementation.

Characteristics of Software Quality - Usability The ease with which users can learn and use a system.

Characteristics of Software Quality - Efficiency Minimal use of system resources, including memory and execution time.

Characteristics of Software Quality - Reliability The ability of a system to perform its required functions under stated conditions whenever required—having a long mean time between failures.

Characteristics of Software Quality - Integrity Integrity The degree to which a system prevents unauthorized or improper access to its programs and its data. The idea of integrity includes restricting unauthorized user accesses as well as ensuring that data is accessed properly— that is, that tables with parallel data are modified in parallel, that date fields contain only valid dates, and so on.

Characteristics of Software Quality - Adaptability The extent to which a system can be used, without modification, in applications or environments other than those for which it was specifically designed.

Characteristics of Software Quality - Accuracy The degree to which a system, as built, is free from error, especially with respect to quantitative outputs. Accuracy differs from correctness; it is a determination of how well a system does the job it's built for rather than whether it was built correctly. .

Characteristics of Software Quality - Robustness The degree to which a system continues to function in the presence of invalid inputs or stressful environmental conditions.

Characteristics of Software Quality - Maintainability The ease with which you can modify a software system to change or add capabilities, improve performance, or correct defects.

Characteristics of Software Quality - Flexibility The extent to which you can modify a system for uses or environments other than those for which it was specifically designed.

Characteristics of Software Quality - Portability The ease with which you can modify a system to operate in an environment different from that for which it was specifically designed.

Characteristics of Software Quality - Reusability The extent to which and the ease with which you can use parts of a system in other systems.

Characteristics of Software Quality - Readability The ease with which you can read and understand the source code of a system, especially at the detailed-statement level.

Characteristics of Software Quality - Testability The degree to which you can unit-test and system-test a system; the degree to which you can verify that the system meets its requirements.

Characteristics of Software Quality - Understandability The ease with which you can comprehend a system at both the system-organizational and detailed-statement levels. Understandability has to do with the coherence of the system at a more general level than readability does.

Focus on one external Quality impacts other qualities

Techniques for Improving Software Quality Software quality assurance is a planned and systematic program of activities designed to ensure that a system has the desired characteristics. You focus on the process rather then the product

Software-quality objectives set explicit quality objectives from among the external and internal characteristics Without explicit goals, programmers might work to maximize characteristics different from the ones you expect them to maximize

Testing strategy Execution testing can provide a detailed assessment of a product's reliability It is important to have a carefully planned, and executed test plan that specifically addresses each requirement and objective of the system. However, testing alone is NOT sufficient to assure the objectives are met.

Software-engineering guidelines Guidelines should control the technical character of the software as it's developed. Such guidelines apply to all software development activities, including problem definition, requirements development, architecture, construction, and system testing.

Informal technical reviews Many software developers review their work before turning it over for formal review. Informal reviews include desk-checking the design or the code or walking through the code with a few peers.

Formal technical reviews Great for finding errors earlier rather than later Planned periodic tests or reviews can help determine whether the quality of the product at one stage is sufficient to support moving on to the next Quality gates - can be an inspection, a peer review, a customer review, or an audit.

External audits An external audit is a specific kind of technical review used to determine the status of a project or the quality of a product being developed. An audit team is brought in from outside the organization and reports its findings to whoever commissioned the audit, usually management.

DEVELOPMENT PROCESS Change-control procedures Measurement of results Avoid uncontrolled changes Measurement of results Provide evidence of success Prototyping Provides early feedback on how realistic the goals are

SETTING OBJECTIVES How can you meet a quality goal with well defined objectives? Programmers given a specific, ranked objective have been shown to meet that objective

Relative Effectiveness of Quality Techniques If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. — Gerald Weinberg Some practices are better at detecting defects than others, and different methods find different kinds of defects.

Defect-Detection Rates Removal Step Lowest Rate Model Rate Highest Rate Informal design reviews 25% 35% 40% Formal design inspections 45% 55% 65% Informal code reviews 20% Formal code inspections 60% 70% Modeling or prototyping 80% Personal desk-checking of code Unit test 15% 30% 50% New function (component) test Integration test Regression test System test Low-volume beta test (<10 sites) High-volume beta test (>1,000 sites) 75% 85%

Defect-Detection Rates modal rates don't rise above 75 percent for any single technique and that the techniques average about 40 percent. for the most common kinds of defect detection—unit testing and integration testing—the modal rates are only 30–35 percent.

Defect-Detection Rates The typical organization uses a test-heavy defect-removal approach and achieves only about 85 percent defect removal efficiency. Leading organizations use a wider variety of techniques and achieve defect-removal efficiencies of 95 percent or higher. Best to use a combination of techniques!

Extreme Programming's Estimated Defect-Detection Rate Removal Step Lowest Rate Modal Rate Highest Rate Informal design reviews (pair programming) 25% 35% 40% Informal code reviews (pair programming) 20% Personal desk-checking of code 60% Unit test 15% 30% 50% Integration test Regression test Expected cumulative defect-removal efficiency ∼74% ∼90% ∼97%

COST OF FINDING DEFECTS Some defect-detection practices cost more than others. The most economical practices result in the least cost per defect found Most studies have found that inspections are cheaper than testing! Ackerman, Buchwald, and Lewski found that it cost six times as much to detect design defects by using testing as by using inspections

COST OF FIXING DEFECTS The longer a defect remains in the system, the more expensive it becomes to remove. Microsoft's applications division has found that it takes 3 hours to find and fix a defect by using code inspection, a one-step technique, and 12 hours to find and fix a defect by using testing, a two-step technique

Quality Techniques Effective software-quality program must include a combination of techniques Formal inspections of all requirements, all architecture, and designs for critical parts of a system Modeling or prototyping Code reading or inspections Execution testing

When to Do Quality Assurance Defects creep into software at all stages. Consequently, you should emphasize quality-assurance work in the early stages and throughout the rest of the project. It should be planned into the project as work begins; it should be part of the technical fiber of the project as work continues; and it should punctuate the end of the project, verifying the quality of the product as work ends.

The General Principle of Software Quality

The General Principle of Software Quality Software projects with the lowest levels of defects had the shortest development schedules and the highest development productivity…. software defect removal is actually the most expensive and time-consuming form of work for software

How much time does it take to create quality code? In a 1985 study, 166 professional programmers wrote programs from the same specification. The resulting programs averaged 220 lines of code and a little under five hours to write. The fascinating result was that programmers who took the median time to complete their programs produced programs with the greatest number of errors.

How much time does it take to create quality code?

A QUALITY-ASSURANCE PLAN Have you identified specific quality characteristics that are important to your project? Have you made others aware of the project's quality objectives? Have you differentiated between external and internal quality characteristics? Have you thought about the ways in which some characteristics might compete with or complement others? Does your project call for the use of several different error-detection techniques suited to finding several different kinds of errors? Does your project include a plan to take steps to assure software quality during each stage of software development? Is the quality measured in some way so that you can tell whether it's improving or degrading? Does management understand that quality assurance incurs additional costs up front in order to save costs later?