1. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory1 Requirements Validation Section Four Version: 1.0 Mehr 1386

2. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory2 What is V&V? Purpose of V&V is to deliver a product that works properly and performs to customer expectations. Verification is the process of confirming deliverable hardware and software are in compliance with the functional, performance, and design requirements.( Have we built the system right?) Validation is the process of providing evidence that a systems meets the needs of the User.( Have we built the right system?) What we are trying to do is to establish confidence in the system, support equipment, test equipment, and people. What

3. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory3 Requirements validation Concerned with demonstrating that the requirements define the system that the customer really wants. (Define the right system) Pickup any problem before resources are committed to addressing the requirements. Requirements error costs are high so validation is very important –Fixing a requirements error after delivery may cost up to 100 times the cost of fixing an implementation error What

4. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory4 Requirements checking Validity. Does the system provide the functions which best support the customer’s needs? Consistency. Are there any requirements conflicts? Completeness. Are all functions required by the customer included? Realism. Can the requirements be implemented given available budget and technology Verifiability. Can the requirements be checked? What

5. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory5 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

6. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory6 Correct Each requirement statement accurately represents the functionality required of the system to be built. Example (of an incorrect requirement): Problem domain (real life) states that policeman ID numbers are in the range [10000…) and the requirements document specifies that each policeman has an ID number. When

7. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory7 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

8. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory8 Unambiguous The difficulty of ambiguity stems from the use of natural language which in itself is inherently ambiguous. There is one and only one interpretation for every requirement. Requirement statements should be short, explicit, precise and clear. A glossary should be used when a term used in a particular context could have multiple meanings (I.e. “the user”). Formal requirements languages help reduce ambiguity. When

9. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory9 Unambiguous (cont.) Example of ambiguity –The TVRS shall store changes made in the details of a traffic report as soon as the data is entered. Disambiguation: –The TVRS shall store changes made in the details of a traffic report if and only all input fields are valid and user approved saving of data How

10. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory10 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

11. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory11 Complete A requirements document is complete if it includes all of the significant requirements, relating to –functionality, – performance, – design constraints attributes – external interfaces. No sections are marked “To be determined” (TBD). Conforms to the company standards. When

12. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory12 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

13. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory13 Testable A requirements document is testable (verifiable) if and only if every requirement statement in it is testable. A requirement is testable if and only if there is some finite cost-effective way in which a person or machine can check to see if the software product satisfies that requirement. When

14. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory14 Testable (cont.) Example of a non-verifiable requirement: –The TVRS shall complete storage of data within a reasonable time of the user confirming a “Save” sequence. Example of a verifiable requirement: –The TVRS shall complete storage of data within 5 seconds of the user confirming a “Save” sequence, 80% of the time. How

15. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory15 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

16. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory16 Consistent Three types of conflicts: –Different terms used for the same object: F323 and a “Policeman Details Form” might be used to describe the same form. –Characteristics of objects: In one part of the requirements document: – “A policeman ID shall consist of decimal digits only”, while in another part – “Incase the policeman ID consists of non- alphanumerical characters, display an error message”. When

17. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory17 Consistent (cont.) –Logical or temporal faults: “A follows B” in one part, “A and B occur simultaneously” in another. “TVRS shall support removal of a policeman record from the personal database” vs. “TVRS shall support read-only access to policeman details”. Do clients know what a database is? When

18. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory18 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

19. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory19 Deal only with the problem Requirements should state “what” is required at the appropriate system level, not “how”. –In some cases, a requirement may dictate how a task is to be accomplished. Avoid telling the designer “how” to do this job, instead state “what” has to be accomplished. Requirements should be understood by the clients as well as the developers. When

20. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory20 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

21. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory21 Modifiable A requirements document is modifiable if its structure and style are such that changes can be made easily, completely and consistently : –Easy to use organization – table of contents, index and cross references. –No redundancy – a requirement should not be in more than one place. When

22. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory22 Characteristics of requirements Correct Unambiguous Complete Testable Consistent Deal only with the problem Modifiable Traceable What

23. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory23 Traceable Each requirement should be contained in a single, numbered paragraph so that it may be referred to in other documents: –Backward traceability - implies that we know why every requirement exists Each requirement explicitly references its source in previous documents. –Forward traceability – all documents to follow will be able to reference each requirement. When

24. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory24 Traceable (cont.) Example: –2.1 Functional requirements: 2.1.1 TVRS initialization: –… –2.1.15 TVRS shall display the user login window (see section 2.1.2.2) 2.1.2 TVRS user interfaces: –2.1.2.1 All user interaction with the TVRS shall occur by means of a graphical user interface. –2.1.2.2 User login window: »… How

25. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory25 Requirements validation techniques Requirements reviews Prototyping Acceptance tests Model Validation and Automated consistency analysisModel Validation and Automated consistency analysis What

26. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory26 Requirements reviews Requirements review is a systematic manual analysis of the requirements Regular reviews should be held while the requirements definition is being formulated Both client and contractor staff should be involved in reviews Reviews may be formal (with completed documents) or informal. What

27. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory27 General Guidelines for Review Good communications between developers, customers and users can resolve problems at an early stage Always conduct reviews in a meeting format, although the meeting participants might prepare some reviews on their own. Continuously check what is produced to make sure the product quality is as high as possible. Checkpoints are provided for this purpose; refer to the checkpoints for each analysis activity. What

28. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory28 Roles in Review Meetings a person has essential knowledge of the business or technology domain and detailed knowledge of the applied facilitation and modeling techniques: –Requirements ReviewerRequirements Reviewer –Requirements SpecifierRequirements Specifier –System AnalystSystem Analyst possibly at milestones such as the beginning or end of a Phase: –Stakeholders - customers and end-users (Where possible)Stakeholders - customers and end-users –Change Control Manager (Where reviewing Change Requests)Change Control Manager –Test Designer (Optional)Test Designer –Software Architect (Optional, usually in Inception and Elaboration)Software Architect –Project Manager (Optional, usually at Phase Start)Project Manager Who

29. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory29 Review checks Verifiability. Is the requirement realistically testable? Comprehensibility. Is the requirement properly understood? Traceability. Is the origin of the requirement clearly stated? Adaptability. Can the requirement be changed without a large impact on other requirements? What

30. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory30 RUP Checkpoints: Requirements Attributes –Has the correct set of requirements attributes been used as specified in the Requirements Management Plan? Requirements Management Plan –Have attributes been set up for each requirement type to account for the following, where applicable, for each requirement? Tracking status? Benefit? Rationale? Level of effort to implement? Type and amount of each type of risk involved in implementing? –Schedule risk? –Resource risk? –Technical risk? Stability of the requirement? Target release? Assignment? Marketing input? Development input? Revision history? Location? Reasons for change? Inconclusive requirements? –Have all traceabilities been set up as specified for the project in the Requirements Management Plan? What

31. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory31 Requirements validation techniques Requirements reviews Prototyping Acceptance tests Model Validation and Automated consistency analysisModel Validation and Automated consistency analysis What

32. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory32 System prototyping Prototyping is the rapid development of a system Using an executable model of the system to check requirements. In the past, the developed system was normally thought of as inferior in some way to the required system so further development was required Now, the boundary between prototyping and normal system development is blurred and many systems are developed using an evolutionary approach What

33. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory33 Uses of system prototypes The principal use is to help customers and developers understand the requirements for the system –Requirements elicitation. Users can experiment with a prototype to see how the system supports their work –Requirements validation. The prototype can reveal errors and omissions in the requirements Prototyping can be considered as a risk reduction activity which reduces requirements risks What

34. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory34 Prototyping benefits Misunderstandings between software users and developers are exposed Missing services may be detected and confusing services may be identified A working system is available early in the process The prototype may serve as a basis for deriving a system specification The system can support user training and system testing Why

35. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory35 Prototyping benefits Improved system usability Closer match to the system needed Improved design quality Improved maintainability Reduced overall development effort Why

36. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory36 Prototyping in the software process Evolutionary prototyping –an initial prototype is produced and refined through a number of stages to the final system Throw-away prototyping –a practical implementation of the system is produced to help discover requirements problems and then discarded. The system is then developed using some other development process What

37. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory37 Prototyping objectives Evolutionary prototyping –to deliver a working system to end-users. –The development starts with those requirements which are best understood. Throw-away prototyping – to validate or derive the system requirements. – The prototyping process starts with those requirements which are poorly understood What

38. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory38 Evolutionary prototyping Must be used for systems where the specification cannot be developed in advance e.g. AI systems and user interface systems Based on techniques which allow rapid system iterations Verification is impossible as there is no specification. Validation means demonstrating the adequacy of the system. What

39. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory39 Evolutionary prototyping What

40. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory40 Evolutionary prototyping advantages Accelerated delivery of the system –Rapid delivery and deployment are sometimes more important than functionality or long-term software maintainability User engagement with the system –Not only is the system more likely to meet user requirements, –they are more likely to commit to the use of the system Why

41. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory41 Evolutionary prototyping Specification, design and implementation are inter-twined The system is developed as a series of increments that are delivered to the customer Techniques for rapid system development are used such as CASE tools and 4GLs User interfaces are usually developed using a GUI development toolkit How

42. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory42 Evolutionary prototyping problems Management problems –If management processes assume a waterfall model of development –Specialist skills are required which may not be available in all development teams Maintenance problems –Continual change tends to corrupt system structure so long-term maintenance is expensive Contractual problems What

43. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory43 Prototypes as specifications Some parts of the requirements (e.g. safety- critical functions) may be impossible to prototype and so don’t appear in the specification An implementation has no legal standing as a contract Non-functional requirements cannot be adequately tested in a system prototype What

44. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory44 Throw-away prototyping Used to reduce requirements risk The prototype is developed from an initial specification, delivered for experiment then discarded The throw-away prototype should NOT be considered as a final system –Some system characteristics may have been left out –There is no specification for long-term maintenance –The system will be poorly structured and difficult to maintain What

45. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory45 Prototype delivery Developers may be pressured to deliver a throw-away prototype as a final system This is not recommended –It may be impossible to tune the prototype to meet non-functional requirements –The prototype is inevitably undocumented –The system structure will be degraded through changes made during development –Normal organisational quality standards may not have been applied Why not

46. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory46 Rapid prototyping techniques Various techniques may be used for rapid development –Dynamic high-level language development –Database programming –Component and application assembly These are not exclusive techniques - they are often used together Visual programming is an inherent part of most prototype development systems What

47. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory47 Component and application assembly Prototypes can be created quickly from a set of reusable components plus some mechanism to ‘glue’ these component together The composition mechanism must include control facilities and a mechanism for component communication The system specification must take into account the availability and functionality of existing components What

48. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory48 Visual programming Scripting languages such as Visual Basic support visual programming where the prototype is developed by creating a user interface from standard items and associating components with these items A large library of components exists to support this type of development These may be tailored to suit the specific application requirements What

49. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory49 Visual programming with reuse What

50. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory50 Problems with visual development Difficult to coordinate team-based development No explicit system architecture Complex dependencies between parts of the program can cause maintainability problems What

51. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory51 User interface prototyping It is impossible to pre-specify the look and feel of a user interface in an effective way. Prototyping is essential UI development consumes an increasing part of overall system development costs User interface generators may be used to ‘draw’ the interface and simulate its functionality with components associated with interface entities Web interfaces may be prototyped using a web site editor What

52. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory52 Requirements validation techniques Requirements reviews Prototyping Acceptance tests Model Validation and Automated consistency analysisModel Validation and Automated consistency analysis What

53. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory53 Acceptance Test Enable the customer to validate all requirements Conducted by end user Can range from informal “test drive” to a planned and systematically executed series of tests If software is developed as a product to be used by many customers a process called alpha and beta testing is used.

54. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory54 Alpha Testing Conducted at the developer’s site by a customer Software is used in a natural setting with the developer “looking over the shoulder” of the user and recording errors and usage problems.

55. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory55 Beta Testing Conducted at one or more customer sites by the end-user of the software. The developer is generally not present. Live application of the software in an environment that can not be controlled by the developer. Customer records all problems that are encountered and reports them to the developer. Developer modifies software and then prepare for release of the software product.

56. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory56 Requirements validation techniques Requirements reviews Prototyping Acceptance tests Model Validation and Automated consistency analysisModel Validation and Automated consistency analysis What

57. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory57 Model Validation Validating the quality of analysis models Is used to check the consistency of a structured requirements description E.g. Static analysis to verify that communication paths exist between objects that in stakeholder domain exchange data. If formal specification is used, using formal reasoning to prove properties of the requirements specification (e.g. completeness) What