From Design to Production Practicing what we preach at HP Shane Evans – Product Manager Oded Keret – Functional Architect

2 Design Develop Functional/UX Delivery Monitoring Security Performance Modern App Lifecycle Reuse Unit Tests throughout Automate test execution and analysis Reuse test scripts for monitoring Capture Real User behavior/conditions Virtualize Services for ease of deployment Leverage virtual services and network for realistic results

3 AUT #1 – HP TruClient A robust and intuitive recording and automation tool Designed for JS based Web 2.0 applications Easily extendable Built to scale Built to test performance

Demo

5 Challenges Object Identification Step Synchronization Keeping up with the Frameworks Keeping up with the Browsers

6 Strategies “What would a Human do?” HeuristicsBlackboard programming TruClient Elector system

7

8

9 Additional tools Trial and error“The Dow Jones” VoodooWorkarounds

10 How would you test such a tool? CoreNightlyAPIUII18NPerformance

11 CI System Flow 2. Report results Code Commit Build Build: 1.Get latest code 2.Compile 3.Run Unit tests 4.Verify 5.Deploy 6.Run Automation tests 1. Execute relevant test bundle Test TruClient Service detects new build available Report

12 Core Bundle – quick execution Core Execute a small bundle of scripts to test only core functionalities. Launching multiple browser processes (IE/FF/Chrome) to execute all scripts in a short time (200 scripts in under 3 minutes) We can easily scale up more browsers to reduce test time Bundle is executed for each code change

13 Core Bundle Core

14 Nightly Bundle Execute a large number of scripts to cover the entire product (UI, API,Core, and more). Bundle is executed every 3 hours (time interval can be configured). Can launch multiple browser processes (IE/FF/Chrome) to execute all scripts in a short time Coe Nightly

15 Nightly Bundle Nightly

16 Performance Bundle Execute 1 Vuser scenario for FF, IE and Chrome to take footprint measurements Script runs iteratively for 2H in each browser, to make sure we have no leaks. Repeat for 51 concurrent Vusers, to make sure performance scales up as expected. Performance

17 Performance Bundle Footprint Performance

18 Performance Bundle Performance

19 Single user performance is nice. But how would you scale up? Introducing StormRunner Load SimpleSmartScalable Web and Mobile Agile oriented Built for developers No hardware No Installation No configuration Just Load

20 AUT #2 – HP StormRunner Load Application Under Test Load Generators are provisioned per Load Test Results and Analysis are displayed in real time User creates and executes Load Test Load is Generated from multiple Geographies AUT is monitored by HP SiteScope

21 Simple. Fast. Scalable. Introducing: StormRunner

Demo

23 How would you test such a tool?

24

25

26 Monitoring production

27 Monitoring production SimpleSmartScalable Web and Mobile Agile oriented Built for developers No hardware No Installation No configuration

And now, a message from our sponsor:

"Build it, test it, and fix the things that go wrong. Repeat the process until the desired reliability is achieved. It is a feedback process and there is no other way." David Packard July 1972

Thank You