Software Quality & Testing Unit 1

Lessons from the past

McDonalds Free MP3 Player Japan Promotional offer by Coca Cola 10,000 infected MP3 players 10 free songs QQPass Trojan

Mars Orbiter Priority inversion bug http://research.microsoft.com/en-us/um/people/mbj/mars_pathfinder/mars_pathfinder.html

NASA Mars Climate Orbiter speeding through space and speaking to NASA in metric. But the engineers on the ground were replying in non-metric English.

Denver Airport’s Baggage Handling System BAE Automated Systems

A 380 – The Largest Passenger Airliner

A380 Delays A380 is being developed in four countries Spain Germany France UK The whole program delayed by 2 years because of the configuration problems of the wiring software. According to the reports on October 2006, cost of the consequent two-year delay to Airbus is estimated to be 4.8 billion euros.

Is testing important?

Bug vs fault vs error vs failure

Cost of Software Bugs WASHINGTON (COMPUTERWORLD) - Software bugs are costing the U.S. economy an estimated $59.5 billion each year, with more than half of the cost borne by end users and the remainder by developers and vendors, according to a new federal study. Improvements in testing could reduce this cost by about a third, or $22.5 billion, but it won't eliminate all software errors, the study said. Of the total $59.5 billion cost, users incurred 64% of the cost and developers 36%.

What is Quality?

Software Quality Assurance : Definitions “The function of software quality that assures that the standards, processes, and procedures are appropriate for the project and are correctly implemented.” www.hq.nasa.gov/office/codeq/software/umbrella_defs.htm

Quality is conformance to specifications (British Defense Industries Quality Assurance Panel)

Quality is conformance to requirements (Philip Crosby)

Quality is fitness for purpose or use (Juran)

Quality is a predictable degree of uniformity and dependability, at low cost and suited to the market (Edward Deming)

Quality is synonymous with customer needs and expectations (R J Mortiboys)

Quality is meeting the (stated) requirements of the customer- now and in the future (Mike Robinson)

Quality is the total composite product and service characteristics of marketing, engineering, manufacturing and maintenance through which the product and service in use will meet the expectations by the customer (Armand Feigenbaum)

Totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs (ISO 8402 : 1994)

… implied needs must be turned into requirements … PMBOK

What is Software Quality? many aspects Popular View Intangible “I know it when I see it” Subjective NOT objective / discreet E.g

What is Software Quality? Professional View it is a moral and legal “best practices” product quality must be measurable in some way Product quality conformance to requirements fitness for use freedom from errors and failures customer satisfaction

Quality Parameters External Quality Parameters Correctness Usability Efficiency Reliability Integrity Adaptability Accuracy Robustness Internal Quality Parameters Maintainability Flexibility Portability Reusability Readability Testability Understandability

Effect of Quality Attributes

Software Quality Principle Principle 1: Know What You Are Doing We normally address this by following a software process with regular milestones, planning, scheduling, reporting and tracking procedures

Software Quality Principle Principle 2: Know What You Should be Doing

Software Quality Principle Principle 3: Know How to Measure the Difference Measurement? formal methods testing inspection metrics

Formal Methods What would you need? Formal methods include formal verification abstract interpretation state modeling other mathematical methods quality assurance E.g What would you need?

Software Testing : Definition “The process of devising a set of inputs to a given piece of software that will cause the software to exercise some portion of its code. The developer of the software can then check that the results produced by the software are in accord with his or her expectations.” www.reactive-systems.com/esda-glossary.msp

A Testing Cycle Subjective to organization Requirements Analysis: Design Analysis: Test Planning: Test Development: Test Execution: Test Reporting: Retesting the Defects

Question remains the same, “why would I need to test my software?”

Some Software Testing Objectives Find as many defects as possible. Find important problems fast. Assess perceived quality risks. Advise about perceived project risks. Advise about perceived quality. Certify to a given standard. Assess conformance to a specification (requirements, design, or product claims).

Testing Testing?? Types of Testing Acceptance testing, Functionality Testing Interface testing Regression testing Unit Testing GUI Testing Structural Testing And many more

Inspection and Metrics

Effectiveness of Removing Defects