1. Think about your view of QA
The “contract” view of Requirements
exhibit precise terms of the agreement
not possible to get completely right
but important to approach
methods, tools, processes
Wiegers “Rights and Responsibilities” (p 27)
5/23/2019
CSC 402, Fall 2000

2. Views of QA when we (developers) do not get it perfect:
plan ahead by tighter control of scope in documents?
defensive approach designed to keep the customer in line
what motivates this approach?
what is at stake?
anecdotes?
who is ultimately responsible for integrity of requirements (even if customer is flaky?)
check Software Engineering Code of Ethics
experience of requirements analysts and managers
5/23/2019
CSC 402, Fall 2000

3. Common Truth what is a reasonable approach to this reality?
“fairness” not always the deciding factor
what could possibly be more important?
what is a reasonable approach to this reality?
stick with strict interpretation of contract model
similar to “freezing requirements,” waterfall approach
prepare to litigate (make my day!)
better QA built in?
flexibility for scope built in (to the process itself!)
allowance for “enough” QA to meet basic customer needs
estimation based on methods and experience
5/23/2019
CSC 402, Fall 2000

4. Testing Throughout Lifecycle
User Reqts
Acceptance Test plan
Acceptance Tests
System Tests
Functional Reqts
System Test plan
Architectural and
Integration Test plans
Integration Tests
Detailed Design
Unit Test plans
Unit Tests
Code
5/23/2019
CSC 402, Fall 2000

5. “Testing” during Requirements
What is “testing?”
add to your 402 journal list
discover defects?
show reliability (lack of defects)?
must it be execution based?
what are the implications of such a view?
what is a “walkthrough,” really?
5/23/2019
CSC 402, Fall 2000

6. Testing Requirements Prototypes, analysis models, simulations
good requirements specification is “testable?”
what are we “testing” for?
what “defects” can we possibly find?
5/23/2019
CSC 402, Fall 2000

7. value of “testing” requirements?
ethics: do a “good job”
planning tests (early) puts team in defect finding mode
produce a knowledge store for later testing and analysis
force deeper understanding of what your client wants
increase “ownership” in the project by requirements team
estimation of effort required to test (and other work)
legal defense
not so much “gotcha” for overbearing client :-)
preparing for defense of other lawsuits or investigations (even insurance audits!)
5/23/2019
CSC 402, Fall 2000

8. Black Box Testing externally observable behavior
Requirements to exhibit only:
externally observable behavior
system input / output
system functionality
what the user cares about
in the domain of application
5/23/2019
CSC 402, Fall 2000

9. Design coverts a “black box” into a “white box” (or “clear box”)
internals become visible (testable, analyzable)
loop invariants for design and code
dataflow analysis for design and code
Black box tests project into the future
requires knowledge, imagination, creativity
you build a system that has not existed before!
5/23/2019
CSC 402, Fall 2000

10. Test Case Specification (See IEEE Standard 829)
Generally:
test case spec id
test items
what features, modules are being tested?
include refs
input specifications
list inputs by value, range or name
timing considerations
5/23/2019
CSC 402, Fall 2000

11. special procedural requirements inter-case dependencies
output specs
list output values and messages
include response time if important
environmental needs
special requirements (h/w, s/w, staff...)
special procedural requirements
anything unusual that a tester needs to know
inter-case dependencies
any tests need to be executed before this one? (refs)
5/23/2019
CSC 402, Fall 2000

12. Test “Coverage” For code: statement, branch, path For requirements?
implications?
For requirements?
similar concepts
I think of the “state space” and how large that is
is there some meaningful heuristic at this level?
5/23/2019
CSC 402, Fall 2000

13. choose a functional requirement
trace functional reqt <- use case <- business reqt
you must be able to do this to know the proper context
possible to catch errors in this alone
choose a “functional requirement”
write a test case for normal use
write a test case for abnormal, but expected use
write a test case for a completely unexpected use
check analysis models for consistency and accuracy
and the document provides for acceptance tests
present the document for early client review
5/23/2019
CSC 402, Fall 2000

14. Usability Testing
“The validation of a system or a model through the use of prototypes and simulations by a user.” (Bruegge)
Based on experiments looking at usability
ease to learn to use system
user time to accomplish a task
rate of errors when user working on a task
what about the election ballots?
5/23/2019
CSC 402, Fall 2000

15. Three types of typical usability tests:
Scenario test
developer presents user(s) with visionary scenario
how quickly do users understand?
what is their attitude towards the scenario?
use paper mockups or simple prototyping envts
5/23/2019
CSC 402, Fall 2000

16. Prototype test Product test user(s) interact with a prototype
vertical - implements a complete use case through the system
horizontal - presents an interface for most use cases without providing much functionality
Product test
a functional version of the system
5/23/2019
CSC 402, Fall 2000

17. Basic Elements of Usability Test
develop test objectives
use of representative sample of end users
use system in actual or simulated work environment
involve end users
controlled interrogation and probing of users by developer-experimenter
collection and analysis of quantitative and qualitative results
recommendations on how to improve system
5/23/2019
CSC 402, Fall 2000

18. Typical test objectives:
compare two user interaction styles
id best/worst features in scenario/prototype
what help/training is needed for what users
Problem: enrolling participants (Grudin90)
project manager doesn’t want to lose control
marketers don’t want to lose control
end users have little time or don’t like to be studied
5/23/2019
CSC 402, Fall 2000