Assurance Leadership Forum 12th October 2019

How I learned to stop worrying and love load testing We chose this title for the presentation as… 1) Gordon thought it up and it was frankly better than all our other attempts. 2) Peter Sellers looks remarkably like a young Ali Rad – just look at that hair!

* Introductions Baz Sahathevan Ali Rad Gordon McKeown Sam Ferguson Who’s in the room – intros of each of us… Baz Ali Gordon Sam We are ButlerThing who have developed Turbo* - a Selenium/Appium/Winium WebDriver based family of plugins for eggplant Performance Eggplant performance provides a mature load testing tool that offers multiple protocol capabilities (As mentioned, Selenium WebDriver, HTTP, Web Services, Citrix, etc) Now audience can we get a quick show of hands for who has a good understanding of load and automation tools and technologies

Performance engineering innovation is an Investment Process Technology People Skills Team alignment Team development Collaboration Re-use Shift left UI capabilities Cloud CICD Performance Testing Innovation Successful innovation in performance engineering is the result of people, process and technology changes that enable a new level productivity and performance. People… Skills – Investing in new skills to bring load testing into the 21st Ways of working – Pair programming and the benefits it brings (resilience, etc) Team Alignment – Alignment of the people. Taking a generalist specialist approach Process… Collaboration – Closer working with developers and feature/sprint teams Re-use – Re-use of collateral and normal output for use in… Shift Left - … early performance and load testing Technology… UI (Selenium based) – Harnessing user centric and commonly used technologies Cloud – harnessing the power of cloud computing CICD – Integration into pipeline My colleagues will take you through a couple of case studies where we have used (or tried to use) this trinity approach to deliver successful load testing

Problem Statement Leading Legal Firm Go Live within 8 weeks Very Complex Application (Microsoft Dynamics AX) Previous test provider could not deliver a working solution after 3 months effort Protocol Level Testing not feasible (endangered project go-live date) Our partners at Spike95 engaged us to ensure system hit go live target Reviewed existing failing solution and a number of new options Chose TurboWinium and engaged ButlerThing to implement POC and end solution

What we did / How we did it Leading Legal Firm People Pair working Used Complimentary skillsets. Resilience Flexibility Process Best practice and principles Incremental testing Technology Leveraged TurboWinium for Windows UI automation Utilised Turbo* built in framework for faster implementation Limited ability to influence process for this engagement

Problem Statement Brewin Dolphin Very secure Infrastructure impeded implementation Highly dynamic website Previous attempts at automation had been deemed inadequate by Senior Management Our partners at nFocus engaged our solution to ensure system hit go live target

* Live demonstration of I’m going to stop now and hand you over to Ali for a real-time high level demo of TurboSelenium in action.

What we did / How we did it Brewin Dolphin People Pair programming Used Complimentary skillsets. Resilience Flexibility Process Best practice and principles Incremental testing Technology Leveraged TurboSelenium for Web Browser UI automation Limited ability to influence process for this engagement

How I learned to stop worrying and love load testing The techie bit

How I learned to stop worrying and love load testing IT landscape in 2018 User Experience A good user experience is paramount Automation Agile demands automation to test and re-test Limitations exist Lack of time and budget limit what is tested Performance testing is too hard to do well Constrained

How I learned to stop worrying and love load testing Modern software is much more complex than it was 20 years ago

Have load testing tools kept pace? Network layer recording Doesn’t capture time it takes to process on the screen Not capturing AJAX calls, Java script – and modern web technology that is executing on the client and impacting response time.

Why protocol level scripting has become painful… Data Correlation Dynamic data returned by server used for subsequent input. Asynchronous Data AJAX and render driven object and data creation. Client-side scripting Data originated by client-side code (e.g. JavaScript). Parameterisation Increasing variation of input data to make tests realistic.

How I learned to stop worrying and love load testing Moving testing to higher in the technology stack – How? Device emulation TurboAppium Windows UI TurboWinium Pure Image Based eggPlant Functional Browser automation TurboSelenium

Turbo* Virtual User(s) Product Architecture Error Handling Recovery Framework Turbo* Virtual User(s) 1. Mobile Device App 1. TurboAppium Control and Combine them! 1. Appium Scripts 2. Windows 10 App 2. TurboWinium 2. Winium Scripts 3. Website 3. TurboSelenium 3. Selenium Scripts Monitoring eggPlant Performance (Controller, Injectors & Analyzer) Data Tables (including test user credentials) Test Definitions Shared Data Server Reporting

Robustness & re-purposing functional scripts How I learned to stop worrying and love load testing Meeting the challenges to “shifting up” Robustness & re-purposing functional scripts Scaling Load injectors and client SUT instances Moore’s Law is still operating! Mix external Cloud and internal resources. Managing Resources Framework based Virtual machines, DevOps utilities. Scripting Compose using re-usable components (e.g. Page Objects) Robust test execution Enhanced framework for error recovery, failure management and debugging.

How I learned to stop worrying and love load testing Moving testing to higher in the technology stack – Benefits? Easier scripting Avoid protocol level traps. “Less technical”. Re-use functional test resources Increased test coverage Avoid “too difficult to script” gaps. Include full client-side stack. End to end timings User-centric and realistic response times.

How I learned to stop worrying and love load testing Q&A

How I learned to stop worrying and love load testing “What is right is not always popular, and what is popular is not always right.”

How I learned to stop worrying and love load testing

www.butlerthing.io info@butlerthing.io