1. Testing Web Applications Willem Visser RW334

2. Overview Testing Theory Testing Web Apps Tools

3. The Basics A test consists of – Input – Expected output (also called the Oracle) White Box Testing – Considering the code when coming up with tests Black Box Testing – Only consider the input-output interface Automated Testing is the Holy Grail

4. Types of Testing Unit Testing Integration Testing Systems Testing Regression Testing Acceptance Testing – Alpha/Beta Testing – Usability Testing Performance Testing Robustness Testing Security Testing

5. Lower Level Code Testing Unit Testing – Test individual components of the code – Can be as little as one function – White box Integration Testing – Interface between components – White Box – Can be as much as the whole system (which case it becomes more black box) Code Coverage as Test Adequacy Measure

6. Functional Testing Testing that the systems meets the requirements Systems Testing – In the deployed environment Unlike Integration testing – Always black box

7. Cross Cutting Regression Testing – When code changes (some) old tests must still pass – Could be at unit, integration or systems level Very high cost involved in regression testing – Regression failure has high cost to fix

8. Customer Facing Acceptance Testing – Is the customer happy with the product? Alpha/Beta Testing – Let the system out to a few of your customers and see how they feel about it Usability Testing – Typically for GUI/Web/Mobile to not just check that the system is correct but also easy to use – Harder to create an Oracle

9. Non-Functional Testing Performance – Load See how the system behaves at peak load – Stress Push the system beyond its limits to see how far it will survive Security Testing – Check whether there are vulnerabilities that might lead to loss of privacy or other security issues

10. Web App Testing Full Gambit of Features Front End GUI – Usability issues – Hard to test – Browser Compatibility issues Server Side (including storage) – Performance issues Security issues

11. Unit Testing Utterly important! Finding bugs early saves money Makes regression testing much more effective – Write once, run often What do you test? – Business logic! – No need to test simple code or interactions with 3 rd party libraries Until they fail of course! Use Coverage tools to help you decide if you have tested enough

12. Stubs or Mocks In unit testing you are interested in local behavior and assume other things you might be using behave correctly Most unit testing frameworks provides stubs for these 3 rd party components Good example is datastore and memcache stubs provided by GAE Only problem is that sometimes these stubs don’t respect the behavior of the real thing!

13. Unit Testing Frameworks Java – JUnit – The most famous of them all PHP – PHPUnit Python – PyUnit – Actually just “import unittest” And many more XUnit tools for language X – For example GAEUnit, but it looks “dead” now Use coverage tools along side, examples… – Coverage.py for Python – (Ecl)Emma for (Eclipse) Java

14. Unit Testing and GUIs When doing Unit Testing you try and stay well clear of the GUI – Clicking and entering text through a GUI is not automated, although we will see later that with Replay Technology it can also be done Isolating the GUI to allow more efficient testing is the reason to use Model-View- Presenter rather than Model-View-Controller design pattern

15. Webapp2 example import unittest import webapp2 # from the app main.py import main class TestHandlers(unittest.TestCase): def test_hello(self): #input request = webapp2.Request.blank('/’) #output response = request.get_response(main.app) #oracle self.assertEqual(response.status_int, 200) self.assertEqual(response.body, 'Hello, world!’) import webapp2 class HelloHandler(webapp2.RequestHandler): def get(self): self.response.write('Hello, world!') app = webapp2.WSGIapplication([('/', HelloHandler)]) Code to Test Test

16. Integration Testing Thin line between unit and integration Strictly speaking when more than one component is used you are doing integration testing – For example if your web app uses a datastore then the test on the previous slide that came in via a GET request could be an integration test not a unit test This is not worth worrying about – Unit/Integration Testing is fine

17. System Testing Now the full round-trip is being tested, including the Browser component Unit/Integration Testing can be done on a local environment, but System Testing need to be in the deployed environment State-of-the-Practice – Record and replay tests

18. Selenium http://docs.seleniumhq.org/ The #1 tool in Web System Testing “automates browsers. That’s it!” – Much like the webapp2 example earlier, just many more options, including browser specific drivers IDE for record and replay – With scripts that can be edited WebDriver that allows one to run without a browser at all

19. Selenium Example from selenium import webdriver from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0 from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0 # Create a new instance of the Firefox driver driver = webdriver.Firefox() # go to the google home page driver.get("http://www.google.com") # find the element that's name attribute is q (the google search box) inputElement = driver.find_element_by_name("q") # type in the search inputElement.send_keys("cheese!") inputElement.submit() try: # we have to wait for the page to refresh, the last thing that seems to be updated is the title WebDriverWait(driver, 10).until(EC.title_contains("cheese!")) # You should see "cheese! - Google Search" print driver.title finally: driver.quit()

20. Java Example import com.thoughtworks.selenium.*; // This is the driver's import. You'll use this for instantiating a // browser and making it do what you need. public class NewTest extends SeleneseTestCase { public void setUp() throws Exception { setUp("http://www.google.com/", "*firefox"); } public void testNew() throws Exception { selenium.open("/"); selenium.type("q", "selenium rc"); selenium.click("btnG"); selenium.waitForPageToLoad("30000"); assertTrue(selenium.isTextPresent("Results * for selenium rc")); }

21. Usability Testing Based on user opinions Manual according to a list of tasks Observer records behavior Could use things like eye-tracking for more precise results Automated Usability Testing is still a research topic

22. Robustness Testing Try anything and see if something breaks Most famous of these is Android Monkey – Sends random keystrokes to your Android app gremlins.js does the same thing for web apps gremlins.createHorde().unleash();

23. Performance Testing System performance under specific loads – Concurrent users doing a certain number of transactions for a certain duration This is a very hard kind of testing to do – Needs lots of infrastructure – Often the performance bottleneck is your testing framework and not the system under test In the end we know where the problem is – THE DATABASE! – More generally where something has to wait for something else to finish

24. Web Performance Testing Measure throughput or transaction rate Server response time Rendering, but that might need additional scripts on the client side You might not know what is the expected performance, so often you profile the performance – You will quickly notice bad performance

25. Web front-end Performance Tools Google Pagespeed tools www.webpagetest.org Google Chrome Developer Tools Nice summary of tools related to performance can be found at http://samsaffron.com/archive/2012/03/23/s am-s-ultimate-web-performance-tools-and- resources http://samsaffron.com/archive/2012/03/23/s am-s-ultimate-web-performance-tools-and- resources

26. JMeter Load and Performance Testing Server side Not a browser, but can simulate some actions typically done by a browser – HTTP, etc. – No javascript execution

27. Security Testing This is HUGE! – A whole course or even degree can be devoted to it Nice checklist of things to do at https://www.owasp.org/index.php/Web_Application_Se curity_Testing_Cheat_Sheet https://www.owasp.org/index.php/Web_Application_Se curity_Testing_Cheat_Sheet Large part of security testing is manual Penetration Testing – Find security vulnerabilities that can be exploited Tools to find buffer overflows is one of the most effective automatic approaches Fuzz Testing – Semi-structured random testing

28. Zed Attack Proxy (ZAP) Automated Penetration Testing – Point it to URL and it does the rest Includes Fuzzing and many more Find it here https://www.owasp.org/index.php/OWASP_Z ed_Attack_Proxy_Project https://www.owasp.org/index.php/OWASP_Z ed_Attack_Proxy_Project