1. By Rick Clements cle@cypress.com
Software Testing 101
By Rick Clements

2. Overview Terminology Requirements Configuration control Test plan
Test cases
Test procedures
Bug tracking & ship decision
After the testing

3. Terminology
QA vs. QC
Quality assurance (QA) - The management of quality. The encouragement of best practices. QA is about preventing defects.
Quality control (QC) - Validation and verification of the product. This includes reviews, inspections and testing.

4. Terminology
Test Documents
Test plan (or validation plan) - The plan describing the general approach, the people required, equipment required and schedule for testing and other validation activities
Test cases - Specific data used to test the product.
Test procedures - The procedures to follow in testing the product. These may be manual or automated.

5. Terminology
Types of Tests
Black box testing - Testing by looking at the requirements to develop test cases. System level testing is often black box testing.
White box testing - Testing by looking at the structure of the program to develop test cases. White box testing often occurs in unit testing.

6. Terminology
Levels of Testing
Unit testing - The process of running small parts of the software. The design teams often handle unit testing.
Integration testing or Interface testing - The testing of pre-tested modules and hardware to determine that they work together.
System testing - The testing of the entire product to see that it meets its requirements. This occurs after the product has been integrated.

7. Software Development Process
Software Implementation & Debug
Software Design
Define Requirements
Software Test
Test Implementation & Debug
Test Design
Software Release

8. Requirements Why are they important? How are they documented?
What’s important?
What if you don’t have any requirements?

9. Why Are Requirements Important?
How do the designers know what to build?
How can you test that it was built correctly?
When you disagree, who’s right?

10. Documenting Requirements
Data sheet
Requirements specification
Functional specification
Data base
Traceability matrix

11. What’s Important They exist They are unambiguous and testable
Requirements
What’s Important
They exist
They are unambiguous and testable
Cover all of the customers (not just final customer)
They are under configuration control

12. No Documented Requirements
Ask the project manager
Ask the marketing representative
Has anything been sent to the customer?
Ask a domain expert
What are the designers building?
Write them down

13. Configuration Control
Why is it a testing issue?
What to track
Build & Version Number

14. Why Is It A Testing Issue?
Configuration Control
Why Is It A Testing Issue?
Ship the version that was tested
A single test system failing
Modules accidentally reverting to older version
Re-create software and tests

15. What To Track Requirements Software Hardware Tests
Configuration Control
What To Track
Requirements
Software
Hardware
Tests

16. Version & Build Number Simple to generate
Configuration Control
Version & Build Number
Simple to generate
Unique for each build or change
Readily visible and validated for correctness

17. Test Plan What will be tested and not tested?
What approach will be taken?
What resources are needed?
Which people are needed?
Schedule

18. Test Cases Test boundary conditions System interfaces
Where have other errors been found?
What will the users do most often?
What is the most serious if it fails?
Usability
What is unique to your product's environment?

19. Boundary Conditions Values at and away from the boundaries
Test Cases
Boundary Conditions
Values at and away from the boundaries
Valid and invalid values
Both numeric values and strings
Minimum-1, minimum, maximum, maximum+1, a good middle value

20. Where Have Errors Been Found?
Test Cases
Where Have Errors Been Found?
Errors tend to cluster
Can a common error be fixed?
Would a code review be useful?

21. Usability Often over looked First to see software outside of design
Test Cases
Usability
Often over looked
First to see software outside of design
The interface makes sense if you know the design
Need to know your users

22. Interfaces User interface Interfaces between software modules
Test Cases
Interfaces
User interface
Interfaces between software modules
Interfaces to other programs
Hardware / software interfaces

23. What Will Users Do Most Often?
Test Cases
What Will Users Do Most Often?
Frequently encountered errors impact the user more
Test heavily used areas more heavily
Less used areas can’t be ignored

24. What Failures Are Most Serious?
Test Cases
What Failures Are Most Serious?
Areas data could be lost
Errors with a large financial impact
Errors that could injure someone

25. Unique to Web Applications
Test Cases
Unique to Web Applications
Versions of browsers
Different operating systems
Server capacity
Multiple servers - one fails?

26. Unique to GUI Based Application
Test Cases
Unique to GUI Based Application
System configuration changes
Size of the screen
Number of colors available
Default font type and size

27. Unique to Database Applications
Test Cases
Unique to Database Applications
Compatible with existing data
Testing on a copy of real data
Server capacity

28. Unique to Embedded Applications
Test Cases
Unique to Embedded Applications
Can multiple simultaneous stimulus be handled?
Are hardware failures handled correctly?
Running out of supplies
Components breaking or degrading
Communications errors between components
Can temperature changes cause the system to behave differently?

29. Bug Tracking & Ship Decision
Bug states
Bug Information
Is the software ready?

30. Bug Tracking & Ship Decision

31. Bug Tracking & Ship Decision

33. After the Testing Known problems and solutions Test report
Defects not fixed
Shorten customer service leaning curve
Test report
Tuned to the audience
Introduction & conclusion
Major defects
What was run on which versions
What tests couldn’t be run