1. Software Engineering Lecture 11 Software Testing Presenter: Josef Hallberg 1

2. Overview Software testing in general Unit tests: What, how, why –Principles, frameworks, techniques –Benefits –Test Driven Development –Write unit tests! System tests –GUI test automation story time

3. Why test? Tests... (should) find bugs determine if SUT functions correctly –Tests don't increase quality by themselves. –Only if bugs are identified and fixed without introducing new bugs. specify expected behaviour of SUT document anticipated use of SUT SUT = System Under Test

4. Test sizes Small: Unit tests –usually white box tests, i.e. tests are aware of SUT's inner structure. Large: System tests –usually black box tests, i.e. tests don't know about SUT's inner structure. Medium: Integration tests Regression tests continuously as any changes are made to some piece of the infrastructure

5. Top down vs. Bottom up Top down –Test functionality of the system as a whole –Make stubs and mockups for emulating missing funconalities –Early demonstration of the main functionalities Bottom up –Test as components are completed –Need to create test-scripts –Tests errors deep down in the dependancy structure 5

6. Unit tests test functionality of smallest unit of software –a function –a method –grouped together: a class are easy to automate should be automated (should) run fast can give most precise error messages

7. System tests test complete system or product –Capacity, Stress-test, Usability, Security, Performance, Reliability, Compliance, etc. test end user functionality manual or automated manual: scripted or exploratory automated: –e.g. shell scripts for testing servers –e.g. capture-replay tools for GUI clients can be challenging to automate

8. Integration tests test collaboration of several classes, components, subsystems, interfaces usually automated can look like large unit tests can look like small system tests

9. Unit tests Basic concept: Small bits of code that –use function under test –verify output –verify behaviour –run easily and fast –are easy to understand Should issue precise failure messages

10. Unit test frameworks xUnit family: SUnit, JUnit, NUnit, pyUnit, Google C++ Testing Framework Test::Simple, Test::More, Test::Class, Test::Unit and many more see http://en.wikipedia.org/wiki/List_of_unit_testing_framew orks http://code.google.com/p/googletest/ http://www.junit.org/

11. Googletest #include "sample1.h" // Returns n! (the factorial of n). // For negative n, n! is defined to be 1. int Factorial(int n) { int result = 1; for (int i = 1; i <= n; i++) { result *= i; } return result; } #include #include "sample1.h" #include // Tests factorial of negative numbers. TEST(FactorialTest, Negative) { EXPECT_EQ(1, Factorial(-5)); EXPECT_EQ(1, Factorial(-1)); EXPECT_TRUE(Factorial(-10) > 0); } // Tests factorial of 0. TEST(FactorialTest, Zero) { EXPECT_EQ(1, Factorial(0)); } // Tests factorial of positive numbers. TEST(FactorialTest, Positive) { EXPECT_EQ(1, Factorial(1)); EXPECT_EQ(2, Factorial(2)); EXPECT_EQ(6, Factorial(3)); EXPECT_EQ(40320, Factorial(8)); }

12. Benefits: Unit tests... assess code quality make code easily runnable reduce need for debugger can be seen as “reproducible debug sessions” or as “extended compile time checks” create confidence in your code

13. Benefits: Unit tests... can serve as specification can serve as documentation are a precondition for refactoring improve design increase productivity

14. Test Driven Development (TDD) ‏ write failing test for bug or new feature write enough code to get new test compiling run all tests and see new test failing write enough code to make new test pass run all tests and see them pass refactor start over

15. TDD benefits Ensures that tests get written Spec is written before code Think about interface usage before interface design Design for Testability from the start Code can be run as soon as it's finished –no more dreading upcoming debugger sessions see http://googletesting.blogspot.com/2008/09/test-first-is-fun_08.html

16. Capture Replay Tools Common way to automate: “GUI level macro recorders” Scripting language – often Basic or Custom –GetEdit(), SetEdit(), ClickButton() etc. Often give quick and easy results Common problems: –Test maintenance –Test flakiness

17. GUI Test Automation as Feedback Control Problem Regard GUI application as a machine to control Get sensors in place Closely observe application for deviant behaviour –Provide corrective measure where reasonable –Fail test case otherwise Movies of GUI testingTest of an iPhone app using JUnit

18. SOME LIVE DEMOS And now… 18

19. Questions? Feel free to contact me: Josef Hallberg Mail: josef.hallberg@ltu.se josef.hallberg@ltu.se Phone: 0920 49 31 77 19