1. Software Testing Kobla Setriakor Nyomi Faculty Intern (Programming II)

3. Why test anyway?? $aves Money. Establish a degree of quality. Establish extent to which RS have been met. Identify areas of weakness. Provide objective reasons to deploy or not deploy. Proof of stability, reliability & usability.

4. Who does it? Users Customers Testers (S.E.T’s) Analysts Developers (Software Engineers) A good tester is a pessimist!

5. Interesting thoughts… “Testing is the process of executing a program with the intention of finding errors.” – Myers “Testing can show the presence of bugs but never their absence.” - Dijkstra

6. The SWE’s view of testing To use up spare budget. To provide good excuse for why project has delayed. Provide jobs for people who can’t code,

7. Considerations? How good does the product need to be? – How do we measure it? – How do we make a decision? Are there any legal or compliance issues? – What industry are you in? What will it cost?

8. Levels of testing Development Testing Unit Testing System testing Component testing Release Testing Requirements-based testing Scenario testing Performance Testing

9. Types of tests Functional tests Algorithmic tests (O(n) etc…) Positive tests Negative tests Usability tests Boundary tests Startup/shutdown tests Platform tests (Windows, Linux, mobile?) Load/stress tests

10. Unit Testing Incremental testing of small units of code. Often on a method by method basis. Criterion: each line of code must run at least once. Can be manual or automated Downside: does not work with static classes. Automation Tools: jUnit, Nunit, EasyMock…

11. What to test for Algorithms and logic Data structures (global and local) Interfaces (for consumers and producers) Independent paths(run each line @least once) Boundary conditions (Edge Cases) Error/Exception handling (Trying? Catching? Throwing?) Leaks (properly disposing objects?)

12. Mocks & Stubs ‘Fake’ objects, inputs and data for the purpose of testing. Used when ‘real’ producers are too large or impractical.  Mock components which are not yet available  Creating and using Mocks & Stubs can be as time consuming as using real objects.

13. Test-Driven Development Step 1 Define functionality of code to be tested Step 2 Write test for functionality & run to make sure that they all fail! Step 3 Add functionality to code & test to make sure they all pass. Step 4 Run the whole test suite to make sure newly added functionality does not break previous ones.

14. Component testing Components are made up of different objects which come together to perform a common function. Test discrete components that will eventually make of the system. Use Mocks & Stubs to test interfaces.

15. Systems-testing Integrating components to create a version and testing the integrated system. A necessity for large projects by large teams. May involve S.E.T’s who have no involvement in the development.

16. Why? One module can have an adverse effect on another Subfunctions, when combined, may not produce the desired major function Individually acceptable imprecision in calculations may be magnified to unacceptable levels

17. Why? (cont’d) Interfacing errors not detected in unit testing may appear Timing problems (in real-time systems) are not detectable by unit testing Resource contention problems are not detectable by unit testing

18. Requirements-based testing Determine if the software meets all of the requirements defined. Having written requirements is essential Regression testing is performed to determine if the software still meets all of its requirements in light of changes and modifications to the software Regression testing involves selectively repeating existing validation tests, not developing new tests

19. Scenario-Testing Devise use cases & Scenarios A short story that describes a way in which the software might be used. You cannot exhaust all possible scenarios.

20. Stress/ Load Testing How much data? How many connections? How much time? How many responses?

21. Test Methods White box or glass box testing Black box testing Top-down and bottom-up for performing incremental integration ALAC (Act-like-a-customer)

22. Alpha vs. Beta vs. Dogfooding Alpha: Users work with development team at developer’s site Beta: A release is made available to the user to experiment with and report bugs. Dogfooding: Developers use working version of application internally for extended periods before it is made available to the public

23. Best Practices A good test case is one that has a high probability of detecting an undiscovered defect, not one that shows that the program works correctly It is impossible to test your own program A necessary part of every test case is a description of the expected result

24. Best Practices (cont’d) Avoid non-reproducible or on-the-fly testing Write test cases for valid as well as invalid input conditions. Inspect the results of each test As the number of detected defects in a piece of software increases, the probability of the existence of more undetected defects also increases.

25. Personal Experiences Testing @ Google Defect Tracking Systems.