1. Best Practices By Gabriel Rodriguez
Test Design
Best Practices By
Gabriel Rodriguez

2. Agenda Test Design Test Data Test Matrix Test Cases Test Scripts
Test Scenarios
E2E Scenarios
Q&A
Reference

3. Test Design

4. Test Design Test design is the major function of a testing team.
This activity begins not when a software build is placed into the hands of the testing team, but as soon as the first requirements have been approved and baselined.
As requirements and system design are refined over time and through system-development iterations, so are the test procedures and test cases refined to cover the new or modified requirements and system functions.

5. Test Data

6. Test Data The concept of test data is a simple one:
Creating representative processing conditions using test transactions.
The complex part of creating test data is determining which transactions to make test transactions.
Experience shows that it is uneconomical to test all conditions in an application system.

7. Test Data Test File Design:
To design an adequate test file, the tester must be familiar with system flowcharts, operating instructions and other documentation.
A test file should use transactions having a wide range of valid and invalid input data:
Valid data for testing normal processing operations.
Invalid data for testing programmed controls.

8. Test Data Test of normally occurring transactions For Example:
Entering Alphabetic characters in text fields.
Entering Numeric Characters in Calendar Fields.

9. Test Data Test using invalid data For example:
Entering Alphabetic characters when numeric characters are expected and vice versa.
Using invalid account or identification numbers
Entering negative amounts when only positive amounts are valid and vice versa.
Entering transactions or conditions that will violate limits established by law or by standard operating procedures.

10. Test Matrix

11. Test Matrix
A Test Matrix is used to track a series of test cases that are essentially the same.
A Test Matrix is a tool to control the number of test cases created for the application, usually this document is used as an evidence of historical data.
A Test Matrix is a very effective way of displaying and tracking the progress of test.

12. Test Matrix The Structure of a general Test Matrix:
The Rows display the Functionality or (Objects) under test.
The Columns display the test to be run against the objects.

13. Test Matrix The process of defining the Test Matrix:
Enter the Functionality in the horizontal axis of the test matrix.
Enter the test to be run on the vertical axis of the test matrix.
Match up the test to be run with the functionality in the intersection cells enter notes that briefly. describe how the test to be run will be met
Review the test matrix at a high level to determine redundancy and coverage.

14. Test Matrix
The next slide will show an example of a test matrix, please have the following points in mind while reviewing the test matrix in the next slide:
Every organization has a way to structure a test matrix.
Due to the lack of knowledge of what a test matrix is, some organizations design this artifact not in a very effective way, causing rework for the Software testers.

15. Test Matrix

16. Test Matrix Description of the Test Matrix Example (Previous slide):
Top Columns:
Test To Be Run – This refers to the total of number of test planned to be run. The screenshot shows:
PC This was the name given to the future test case, in other words our nomenclature was established in that way.
Control testing - This is the testing type to be implemented

17. Test Matrix Description of the Test Matrix Example (Previous slide):
Left side
Functionality - These are the features to be tested, usually these features are gathered from Requirements doc or Functional specs. There is some test data specified under the features to be tested such as ‘Special Chars’, “Alphanumeric Chars’ this data is used to create different combinations of test cases.
Center side of the spreadsheet
These are all the combinations of test cases that can be designed, the ‘X’ means that the test case will have to be designed with the features on the left columns.

18. Test Matrix Granularity of the Test Matrix:
The granularity of the test matrix needs to be sufficient such that the QA team can answer the following questions:
Where did we test this piece of functionality?
What do we need to run in order to get the coverage we want?
What coverage did we get with this particular scenario?

19. Test Cases

20. Test Cases What is a Test Case?
IEEE Standard 610 (1990) defines test case as follows:
“(1) A set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.

21. Test Cases
Quality Assurance Institute (QAI) defines test case as follows:
Documents the actual values used for input along with the anticipated outputs.
Definition of Test Case from
A set of test data and test programs (test scripts) and their expected results. A test case validates one or more system requirements and generates a pass or fail.

22. Test Cases
What are we trying to learn or achieve when we run test cases?
Find Defects
Maximize bug count
Help Managers make ship/no-ship decisions
Assess conformance to specification
Verify correctness of the product
Assure Quality

23. Test Cases Characteristics of a good test case:
It has a reasonable probability of catching an error.
It is not redundant.
It’s the best of its breed.
It is neither too simple nor too complex.

24. Test Cases Blackbox Testing Technique
When creating test cases, the input data used is critical. Three successful techniques for managing the amount of input data required include:
Equivalence Partitioning
Boundary Analysis
Error Guessing

25. Test Cases Equivalence Partitioning
An equivalence class is a subset of data that is representative of a larger class. Equivalence partitioning is a technique for testing equivalence classes rather than undertaking exhaustive testing of each value of the larger class.
For Example: A program which edits credit limits within a given range ($10,000 - $15,000) would have three equivalence classes:
Less than $10,000 (invalid)
Between $10,000 and $15,000 (valid)
Greater than $15,000 (invalid)

26. Test Cases Boundary Analysis
A technique that consist of developing a test cases and data that focus on the input and output boundaries of a given function. In same credit limit example, boundary analysis would test:
Low boundary plus or minus one ($9,999 and $10,001)
On the boundary ($10,000 and $15,000)
Upper boundary plus or minus one ($14,999 and $15,001)

27. Test Cases Error Guessing
Based on the theory that test cases can be developed based upon the intuition and experience of the Test Engineer.
For example , in an example where one of the inputs is the date, a test engineer may try February 30, 2000 or 9/9/99

28. Test Cases
The following documents are needed in order to design test cases:
Requirements Document
Use Cases (When applicable)
Functional Specification Document
Design Specification Document
Test Plan
Test Matrix
This document is created to have better control on the combinations of test cases.

29. Test Cases
The following slide shows an example of a Test Case Template. Please have the following points in mind:
Every organization designs its test case template to meet its needs.
There are test case templates that are more effective than others because some of them have a more structure way to organize the test cases.

30. Test Cases

31. Test Cases Tips to write test cases: For Action Steps:
Use the following short words:
Click on (Buttons, Links)
Select (Radio Buttons, Options from lists)
Move to (Next Page, Next Window)
Navigate to (Next Page, Next Window)
Check ( Check Boxes)
Type in ( Text Fields, Message Boxes)

32. Test Cases Tips to write test cases: For Expected Results
Use the following statements:
Verify the following…
The following page will be displayed…
The following error message will be displayed…

33. Test Scripts

34. Test Scripts QAI Definition of a Test Script:
Note: Generally there is a confusion between a test script and a test case.
A Test script are step-by-step instructions that describe how a test case is to be executed.
Scripts should specify an order of actions that should be performed during a test session
The scripts should also contain expected results.
Test scripts may be manually prepared using paper forms, or may be automated using capture/playback tools or other kinds of automated scripting tools.

35. Test Scripts Definition of a test script from www.pcmag
The instructions in a test program. It defines the actions and pass/fail criteria. For example, if the action is "to enter a valid account number," the expected result is that the data are accepted. Entering an invalid number should yield a particular error message.

36. Test Scripts
Test Script Example

37. Test Scenarios

38. Test Scenarios
A set of test cases that ensure that the business process flows are tested from end to end.
They may be independent tests or a series of tests that follow each other, each dependent on the output of the previous one.
The terms "test scenario" and "test case" are often used synonymously, however there is a difference between them.

39. E2E Scenarios

40. E2E Scenarios E2E stands for End To End Goal of E2E Scenarios:
End-To-End (E2E) Testing is a QA activity whose primary goal is to verify interaction points between multiple features and/or systems using real-world user processes and data workflow.
By simulating these real-world scenarios, E2E Tests increase the quality of the applications in the areas most critical to our customers.

41. E2E Scenarios Goal of E2E Scenarios:
A secondary goal of E2E Testing is to find important usability flaws that affect the ability of the application to deliver its intended value.
By exercising the software in a manner that is similar to how customers will use the software, macro-level usability issues will surface and can be evaluated.

42. E2E Scenarios
The idea behind adopting E2E Testing is to help close gaps in the testing model.
Standard UI test cases focus on testing functionality of a particular feature area, but tend to overlook the interaction points between multiple features and/or systems. E2E Testing focuses more on a use-case based or data flow approach.

43. E2E Scenarios When is E2E a good fit:
E2E Testing can be a good fit when:
You have business use cases at the outset of the project.
It makes sense to focus on how customers will use the software.
You are taking a managed-risk approach and the focus is on making sure that the things customers will do will work appropriately.

44. E2E Scenarios When is E2E a good fit:
E2E Testing can be a good fit when:
The area under test cuts across multiple functional areas and it makes sense to test them according to a workflow.

45. E2E Scenarios
Example of an E2E Scenario

46. Q&A
Any questions…

47. Reference CSTE Study Guide 2002 by QAI CSTE Study Guide 2006 by QAI
Effective Software Testing: 50 Specific Ways to Improve Your Testing by Elfriede Dustin
QA Websites:

48. Thank you…