Week # 4 Quality Assurance Software Quality Engineering 1

Principles of QA Three General Principles of QA – Know what you are doing – know what you should be doing – Know how to measure the difference 2

Three General Principles of QA Know what you are doing – Understand what is being built, how it is being built and what it currently does. – Follow a software development process with Management structure (milestones, scheduling) Reporting policies Tracking 3

Three General Principles of QA Know what you should be doing: – Having explicit requirements and specifications. – Follow a software development process with Requirements analysis, Acceptance tests, Frequent user feedback. 4

Three General Principles of QA Know how to measure the difference. – Having explicit measures comparing what is being done from what should be done. – Four complementary methods: Formal methods – verify mathematically specified properties. Testing – explicit input to exercise software and check for expected output. Inspections – human examination of requirements, design, code,... based on checklists. Metrics – measures a known set of properties related to quality 5

Testing The vast majority (over 99%) of software quality assurance uses testing, inspection and metrics instead of formal methods Testing – Testing includes a wide range of methods based on the idea of running the software through a set of example inputs or situations and validating the results – Includes methods based on requirements (acceptance testing), specification and design(functionality and interface testing), history (regression testing), code structure (path testing), and many more 6

Inspection Inspection includes methods based on a human review of the software artifacts Includes methods based on requirements reviews, design reviews, scheduling and planning reviews, code walkthroughs, and so on Helps discover potential problems before they arise in practice 7

Metrics Software metrics includes methods based on using tools to count the use of features or structures in the code or other software artifacts, and compare them to standards Includes methods based on code size (number of source lines), code complexity (number of parameters, decisions, function points, modules or methods), structural complexity(number or depth of calls or transactions), design complexity, and so on Helps expose anomalous or undesirable properties that may reduce reliability and maintainability 8

Quality in Context In order to understand the roles of quality assurance in software development, we must understand how software development works - we cannot discuss inspection, testing and metrics in a vacuum As background, therefore, we will begin by reviewing: – Major process models of the software development community – the ways software development efforts are organized – Some ways of assessing development process quality – Quality management standards for software processes 9

Big Band Model A huge amount of matter (people and money) is put together, a lot of energy is expended— often violently—and out comes the perfect software product…or it doesn’t. 10

Big Band Model It’s an ideal process if the product requirements aren’t well understood the final release date is flexible It’s also important to have very flexible customers, too, because they won’t know what they’re getting until the very end. 11

Testing in big bang model? there is little to no formal testing done under the big-bang model If testing does occur, it’s squeezed in just before the product is released 12

WaterFall Model 13

WaterFall Model There’s a large emphasis on specifying what the product will be. Note that the development or coding phase is only a single block! The steps are discrete; there’s no overlap. There’s no way to back up. As soon as you’re on a step, you need to complete the tasks for that step and then move on—you can’t go back. 14

Testing in Water Fall Model Advantage – Everything is documented From that, the test group can create an accurate plan and schedule Disadvantage – testing occurs only at the end, a fundamental problem could creep in early on and not be detected until days before the scheduled product release. 15

Spiral Model 16

Spiral Model Each time around the spiral involves six steps: 1. Determine objectives, alternatives, and constraints. 2. Identify and resolve risks. 3. Evaluate alternatives. 4. Develop and test the current level. 5. Plan the next level. 6. Decide on the approach for the next level. 17

Testing in Spiral Model Get a chance to influence the product early by being involved in the preliminary design phases. One can see where the project has come from and where it’s going. And, at the very end of the project, one will not feel as rushed to perform all the testing at the last minute. 18

Verification and Validation Verification: – Are we building the product right? – Performed at the end of a phase to ensure that requirements established during previous phase have been met. Validation: – Are we building the right product? – Performed at the end of the development process to ensure compliance with product requirements. Requirements Architecting Detailed Design Coding verification validation 19

Testing Axioms “rules of the road” or the “facts of life” for software testing and software development. It’s Impossible to Test a Program Completely – The number of possible inputs is very large. – The number of possible outputs is very large. – The number of paths through the software is very large 20

Software Testing Is a Risk-Based Exercise – software testers need to learn is how to reduce the huge domain of possible tests into a manageable set, and how to make wise risk-based decisions on what’s important to test and what’s not. 21

Testing Can’t Show That Bugs Don’t Exist – Software testing works exactly as the exterminator does. It can show that bugs exist, but it can’t show that bugs don’t exist. You can perform your tests, find and report bugs, but at no point can you guarantee that there are no longer any bugs to find. 22

The More Bugs You Find, the More Bugs There Are – Programmers have bad days. Like all of us, programmers can have off days. Code written one day may be perfect; code written another may be sloppy. – Programmers often make the same mistake. Everyone has habits. A programmer who is prone to a certain error will often repeat it. 23

The Pesticide Paradox – phenomenon that the more you test software, the more immune it becomes to your tests. – If you keep applying the same pesticide, the insects eventually build up resistance and the pesticide no longer works. Same happens in software 24

Not All the Bugs You Find Will Be Fixed – There’s not enough time. – It’s too risky to fix – It’s just not worth it. 25

What is Software Testing Testing is the process of executing software in a controlled manner, to answer the question: “does the software behave as specified?” Implies that we have a specification (or possibly the tests are it) Or implies that we have some property we wish to test for independently of the specification – e.g., “all paths in the code are reachable (no dead code)” Testing is often associated with the words “validation” and “verification” 26

What is Software testing ? Examination of a software unit, several integrated software units or an entire software package by running it. – execution based on test cases – expectation – reveal faults as failures 27

Types of Testing Source of Test Generation Black-box testing: Tests are generated from informally or formally specified requirements – Does not require access to source code – Boundary-value analysis, equivalence partitioning, random testing, pairwise testing White-box testing: Tests are generated from source code. – Must have access to source code – Structural testing, path testing, data flow testing 28

Types of Testing Life Cycle Phases 29