1. Chapter 4 Quality Assurance in Context
SE Software Testing and Quality Assurance
Chapter 4 Quality Assurance in Context 1

2. Handling Discovered Defect During QA Activities
Overview
Handling Discovered Defect During QA Activities
QA Activities in Software Processes
Verification and Validation Perspectives
Reconciling the Two Views 2

3. Introduction Quality assurance (QA) interpretation from last chapter
QA as dealing with defects
implicitly assumed that all discovered defects will be resolved within the software development process before product release
In this chapter we
Describe defect handling during the execution of specific QA activities
Examine how different QA activities fit into different software development processes
Examine the QA activities from the perspective of verification and validation (V&V) view 3

4. Defect Handling and Related Activities
QA aims to resolve each discovered defect before product release
Most important activity associated with defect handling is defect resolution
Ensures that each discovered defect is corrected or taken care of through appropriate actions e.g. reclassification, deferment for future fix
Two important activities to support defect resolution:
Defect logging: recording discovered defects
Defect tracking: monitoring defects up its final resolution 4

5. Defect Handling and Related Activities
What to record?
Various specific information can be recorded and updated through the defect handling process. Defect type, severity, impact areas, possible cause etc
Details in later chapter
To ensure proper collection and usage of defect data, we need to pay special attention to the following in the defect discovery and resolution activities
distinguish execution failures, internal faults and human errors
the specific problems need to be counted and tracked consistently
timely defect reporting 5

6. Defect Handling Process
Defect handling handled in similar ways as configuration management
A formalized defect handling process defines
Important activities and associated rules,
Parties involved, and their responsibilities
Multiple parties. Who could be?
Different states associated with individual defect status
Operating system example?
New, Working, Re-verify, Closed 6

7. We need the support of software for
Defect Handling Tools
We need the support of software for
implementation of the defect handling process
enforcement of various related rules
Commonly referred to as defect tracking tools or defect handling tools
Examples:
IBM: CMVC - configuration management and version control, an IBM Products was used for defect tracking
Open source projects: Tools such as
Bugzilla (
Issuezilla ( 7

8. Defect Handling Process
Bugzilla Bug Life Cycle 8

9. Defect Handling in Different QA Activities
Three classes of the QA activities:
defect detection and removal activities, such as testing and inspection,
are more closely associated with defect handling
inspector of a program may make an initial note about a possible problem in the program code. When it is confirmed during the inspection meeting, it is formally recorded as a defect
various defect prevention activities
do not directly deal with defects
rather deal with various ways to prevent the injection
consequently, there are little or no discovered faults
defect containment activities,
the focus is not on the discovery of underlying faults 9

10. QA Activities in Software Processes
QA activities - integral part of the overall software process
Maintenance stage, the focus of QA is on how:
Each problem reported from field operations is
logged
analyzed
resolved
Complete tracking record is kept so that we can learn from past problems for future quality improvement
Most of the core QA activities (defect prevention & reduction) are performed during software development instead of during in-field maintenance
So we focus on how different QA activities fit into software processes
First, we examine how QA activities fits in the Waterfall Process 10

11. QA in the Waterfall Process
Strictly sequential stages
Typical sequence includes, in chronological order:
product planning,
requirement analysis,
specification,
design,
coding,
testing,
release, and
post-release product support
Testing, a central part of QA activities,
is an integral part of the waterfall process,
forms an important link in the overall development chain 11

12. QA in the Waterfall Process
Other QA activities
not explicitly stated in the process description
can be carried out throughout other phases and in the transition from one phase to another
For example, part of the criteria to move on from each phase to the next is quality, typically in the form of checking to see if certain quality plans or standards have been completed or followed
Through reviews and inspection
Various defect prevention activities are typically concentrated in the earlier phases – why?
Because the experience tells us that the vast majority of faults are injected in the early development phases, particularly in detailed design and implementation phases. Therefore, effective defect prevention through error blocking needs to be carried out during these phases 12

13. QA in the Waterfall Process
Some detection and removal techniques like inspection can also be applied to early stages
e.g. inspecting requirement documents, product specs, product design etc.
But there are practical obstacles to the early fixing of injected defects
Dynamic problems may only become apparent during execution
e.g. component dependency issues cannot be detected & fixed till after implementation & execution
Defect containment (fault tolerance, safety assurance) in late phases
However, its planning/preparation is done at early stages 13

14. QA in the Waterfall Process14

15. QA in Other Software Processes
Incremental and iterative processes
consisting of several increments/iterations
each of them following more or less the same mini-stages corresponding to those in the waterfall process
The new increment has to be incorporated with the existing parts - Integration testing
Spiral process
Are similar to those performed in incremental/iterative processes
Minor difference is typically in the risk focus adopted in spiral process
Agile development method
can be treated as special cases of incremental, iterative, or spiral process models where many of their elements are used or adapted
Inspection & testing play an even more important role e.g. test driven development & inspection is integral part of extreme programming
Risk identification and analysis play an important role on the decision as to which part to work on next in the subsequent spiral iteration. This risk focus leads naturally to selective QA with a non-uniform effort applied to different parts of the software systems, with high-risk parts receiving more attention than other parts. 15

16. QA Activities in V&V Context
V & V → Validation and Verification
Validation activities
Related QA activities check whether a function needed and expected by the customers is present in a software product
Verification activities
Related QA activities to confirm the correct or reliable performance of these specified functions
Check the conformance of a software system to its specifications 16

17. QA Activities in V&V Context
Examples of QA activities that can be classified as Validation activities:
System testing focus is the overall set of system functions
Acceptance & beta testing with focus on the user
Usage based testing where the target environment is simulated during software testing before product release
Software fault tolerance for providing continued service even when local problem exists
Software safety assurance activities which focus on providing the expected accident free operations or reducing accident damage when an accident is unavoidable 17

18. QA Activities in V&V Context
When a expected feature is present
the activity to determine whether it performs or behaves expectedly is then a verification activity
therefore there are almost always accompanying verification activities
Therefore, all the QA activities we classified as dealing directly with faults, errors, or error sources can be classified as verification activities
E.g. proper working of algorithms, data structures, coding standards, process standards 18

19. V & V Based QA Activities in Software Processes
Validation checks whether the expected functions or features are present or not
validation deals directly with users and their requirements
Verification checks implementation against its specifications to see if it is implemented correctly
Verification deals with internal product specifications
Different processes involve users in different ways
Therefore validation and verification activities may be distributed in different processes differently
To check whether a reservation feature is there or not, is validation.
To confirm whether it is working correctly is verification 19

20. V & V Based QA Activities in Waterfall Process
Focus of validation activities
Direct involvement of users and their requirements at the very beginning & the very end
e.g. project planning, market analysis, requirement analysis, specification, beta testing, usage-based testing, acceptance testing, product release, post-release support and maintenance etc
Therefore, these are the phases where validation activities may be the focus
Focus of verification activities
No direct involvement of users in the middle part
e.g. inspection of design, formal verification of program correctness, unit testing, component testing etc
Therefore, these are the phases where verification activities may be the focus 20

21. The V Process Model
These verification and validation activities can be best illustrated by the V-model 21

22. V&V vs. DC View DC View - defect-centered view
Verification and validation activities included examples of specific QA activities
QA activities were also classified according to the generic ways of dealing with defects
Prevent
Reduce
Contain
Through this connection we can establish the relationship and the mapping between the V&V view and DC view 22

23. V&V vs. DC View Defect prevention deals with error blocking
verification and validation deal with failures and faults
Therefore, there is no direct connection, but only indirectly
e.g. if the target is eliminating ambiguity in the requirement, it is indirectly validation 23