1 Software Testing and Quality Assurance Motivation and Review of Software Verification & Validation (2)

2 Lecture Outline V & V Planning Static V &V techniques ◦ Walkthroughs. ◦ Software Inspections.

3 V & V Planning Careful planning is required to get the most out of testing and inspection processes. Planning should start early in the development process. The plan should have the balance between static verification and testing.

4 V-Model of Development Requirements specification System Specification System Design Detailed Design Acceptance Test System Integration Test Sub-System Integration Test Module, Unit Code & Test Acceptance Test Plan System Integration Test Plan Sub-System Integration Test Plan Service

5 Walkthroughs Usually done in a single meeting. Evaluate a software product to ◦ Find anomalies & improve the software product. ◦ Consider alternative implementations. ◦ Evaluate the conformance to standards and specification. Perform training of the project teams.

6 Software Inspection Formalized approach to document reviews. Intended explicitly for defect detection (not correction). Defects may be logical errors & anomalies in the code. For example, ◦ An un-initialized variable ◦ Non-compliance with standards.

7 Software Inspection Remove errors as near source as possible; hence, ◦ reducing costs of rework. It’s success depends on ◦ The inspection process, ◦ Checks applied; ◦ The diligence of the inspectors.

8 Inspection pre-conditions Precise specification must be available. Team members must be familiar with the organization standards. Inspections will increase costs early in the software process.

9 Software Inspections Time No. of Employees Planning Requirements DesignCoding Testing Without Inspection With Inspection

10 Inspection Process Planning Overview Individual Preparation Rework Follow-up Inspection Meeting

11 Inspection Steps Reader paraphrases design, describing how it will be implemented. Questions raised during discourse, only pursued until error recognized. Error noted (but not solution) and classified by severity. ◦ Written report of inspection prepared. Important to inspect modified products. ◦ Re-inspection to avoid ‘bad fix’ problems.

12 Software Inspection Overview Inspection Analysis Fix short term problems Error feedback Learning input for inspectors Feedback What error types to look for…

13 Feedback Inspection provide detailed real-time feedback to developers Process control using inspection ◦ Identification of error prone modules; and ◦ Distribution of error types Inspections should lead to Process Improvement

14 Feedback Inspection success depends on ◦ Management attitude ◦ Conduct of trained moderator Inspections should not be used for performance appraisals

15 Software Inspection Overview Operation 1Operation 2 I Analysis Fix short term problems Error feedback Error prone modules Error types distributions For Special Attentions Feedback Feed forward Learning input for inspectors What error types to look for… Better ways to find error types

16 Rate of Progress Rate of progress (man/hours) Process OperationsDesignCodeobjectives Overview500-communication preparation100125education inspection130150Find errors Rework2016Remove errors Follow-up--Ensure resolutions of errors

17 Error Analysis Finding errors is difficult ◦ Condition people to seek high occurrence, high cost error types. ◦ Take representative sample of code; Analyze errors ◦ Type and origin of error; ◦ cause and salient indicative feature

18 London Ambulance Service Case Study Computer Aided Dispatch Server Automatic Vehicle Location System Radio Communication Infrastructure Ambulance PRINCE Development Methodology was selected for the development

19 London Ambulance Service Case Study Throughout the project there are references to only functional testing ◦ No static V & V process. The draft project plan as provided by the supplier left no time for review and revision. As a result, …….. For example, ◦ Lack of formal clarification how the PRINCE methodology was to be applied.

20 Key Points Careful planning is required to get the most out of V &V processes. Identify error types to get maximum benefits out of static V & V techniques. Analyze inspection results and use it for process improvement.