Think about your view of QA The “contract” view of Requirements exhibit precise terms of the agreement not possible to get completely right but important to approach methods, tools, processes Wiegers “Rights and Responsibilities” (p 27) 5/23/2019 CSC 402, Fall 2000

Views of QA when we (developers) do not get it perfect: plan ahead by tighter control of scope in documents? defensive approach designed to keep the customer in line what motivates this approach? what is at stake? anecdotes? who is ultimately responsible for integrity of requirements (even if customer is flaky?) check Software Engineering Code of Ethics experience of requirements analysts and managers 5/23/2019 CSC 402, Fall 2000

Common Truth what is a reasonable approach to this reality? “fairness” not always the deciding factor what could possibly be more important? what is a reasonable approach to this reality? stick with strict interpretation of contract model similar to “freezing requirements,” waterfall approach prepare to litigate (make my day!) better QA built in? flexibility for scope built in (to the process itself!) allowance for “enough” QA to meet basic customer needs estimation based on methods and experience 5/23/2019 CSC 402, Fall 2000

Testing Throughout Lifecycle User Reqts Acceptance Test plan Acceptance Tests System Tests Functional Reqts System Test plan Architectural and Integration Test plans Integration Tests Detailed Design Unit Test plans Unit Tests Code 5/23/2019 CSC 402, Fall 2000

“Testing” during Requirements What is “testing?” add to your 402 journal list discover defects? show reliability (lack of defects)? must it be execution based? what are the implications of such a view? what is a “walkthrough,” really? 5/23/2019 CSC 402, Fall 2000

Testing Requirements Prototypes, analysis models, simulations good requirements specification is “testable?” what are we “testing” for? what “defects” can we possibly find? 5/23/2019 CSC 402, Fall 2000

value of “testing” requirements? ethics: do a “good job” planning tests (early) puts team in defect finding mode produce a knowledge store for later testing and analysis force deeper understanding of what your client wants increase “ownership” in the project by requirements team estimation of effort required to test (and other work) legal defense not so much “gotcha” for overbearing client :-) preparing for defense of other lawsuits or investigations (even insurance audits!) 5/23/2019 CSC 402, Fall 2000

Black Box Testing externally observable behavior Requirements to exhibit only: externally observable behavior system input / output system functionality what the user cares about in the domain of application 5/23/2019 CSC 402, Fall 2000

Design coverts a “black box” into a “white box” (or “clear box”) internals become visible (testable, analyzable) loop invariants for design and code dataflow analysis for design and code Black box tests project into the future requires knowledge, imagination, creativity you build a system that has not existed before! 5/23/2019 CSC 402, Fall 2000

Test Case Specification (See IEEE Standard 829) Generally: test case spec id test items what features, modules are being tested? include refs input specifications list inputs by value, range or name timing considerations 5/23/2019 CSC 402, Fall 2000

special procedural requirements inter-case dependencies output specs list output values and messages include response time if important environmental needs special requirements (h/w, s/w, staff...) special procedural requirements anything unusual that a tester needs to know inter-case dependencies any tests need to be executed before this one? (refs) 5/23/2019 CSC 402, Fall 2000

Test “Coverage” For code: statement, branch, path For requirements? implications? For requirements? similar concepts I think of the “state space” and how large that is is there some meaningful heuristic at this level? 5/23/2019 CSC 402, Fall 2000

choose a functional requirement trace functional reqt <- use case <- business reqt you must be able to do this to know the proper context possible to catch errors in this alone choose a “functional requirement” write a test case for normal use write a test case for abnormal, but expected use write a test case for a completely unexpected use check analysis models for consistency and accuracy and the document provides for acceptance tests present the document for early client review 5/23/2019 CSC 402, Fall 2000

Usability Testing “The validation of a system or a model through the use of prototypes and simulations by a user.” (Bruegge) Based on experiments looking at usability ease to learn to use system user time to accomplish a task rate of errors when user working on a task what about the election ballots? 5/23/2019 CSC 402, Fall 2000

Three types of typical usability tests: Scenario test developer presents user(s) with visionary scenario how quickly do users understand? what is their attitude towards the scenario? use paper mockups or simple prototyping envts 5/23/2019 CSC 402, Fall 2000

Prototype test Product test user(s) interact with a prototype vertical - implements a complete use case through the system horizontal - presents an interface for most use cases without providing much functionality Product test a functional version of the system 5/23/2019 CSC 402, Fall 2000

Basic Elements of Usability Test develop test objectives use of representative sample of end users use system in actual or simulated work environment involve end users controlled interrogation and probing of users by developer-experimenter collection and analysis of quantitative and qualitative results recommendations on how to improve system 5/23/2019 CSC 402, Fall 2000

Typical test objectives: compare two user interaction styles id best/worst features in scenario/prototype what help/training is needed for what users Problem: enrolling participants (Grudin90) project manager doesn’t want to lose control marketers don’t want to lose control end users have little time or don’t like to be studied 5/23/2019 CSC 402, Fall 2000