1. Agenda ● Performance testing Load Testing StressTesting
● What is JMeter
● JMeter basic elements
● Building a Script
●Running a Script
●Recording a Script (Bootstrapping)
Sam Varadarajan, for IEJUG

2. Performance Testing

3. What is Performance testing?
 Performance testing is performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It is typically non-functional, meaning, we only care about load on the system, scalability etc. Actual functionality is not tested in this phase. (That will be done in Functional Test phase).
The real goal is to find that maximum load that the system can handle without performance degradation or crashes.
There are several types of performance testing, but 2 are important:
Load/Capacity Testing
Stress Testing
We will be focusing mainly on Load Testing in this presentation.

4. Load / Capacity Testing
Load testing is the simplest form of performance testing. A load test is usually conducted to understand the behavior of the system under a specific (expected) sustained load.
The load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration.
In simple terms, it is the maximum load the horse can carry without it tripping or the carriage toppling!
During the tests we will collect the the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software.

5. Stress Testing
Every system has a capacity limit. When the load goes beyond the limit, the System starts responding very slowly and even produce errors. we have seen this in Databases, Network etc.
It is more visible on a Web site, as number of users can drastically (often unpredictably) increase, depending on content. We saw this recently with HealthCare.gov! Sometimes, websites crash due to attacks such as denial of service attack.
In simple terms, Stress on the system is that last straw on the camel's back that broke it.
The purpose of the stress testing is to find the capacity limit of the system. We do this by increasing the load on the system beyond a safe limit, so the system will eventually crash. With the findings, we can adjust the parameters to stretch this limit.

6. General Load Testing Script
Load Testing itself can broken down into following steps:
Create Test (Can be recorded using some tool)
Configure workload settings
Add measurement hooks
Data drive script (using variables and parameters to generalize it)
Add logging
Generate reports/charts from performance data collected.

7. Load Testing - Manually
This is how they did load testing before.
Just ak a bunch of users to come to a training room. The users will start using the application according to a script and run simultaneously!
We probably generated a very weak load and didn't really stress it either!

8. Load Testing Automated
All or most of these steps can be automated using several different programs/scripts.
There are testing tools that have integrated these into tools called Load Testing tools.
Mercury Load Runner and Rational Rose tools are commercial products that have been around for a long time.
With the advent of Web applications, several new tools have appeared in both commercial and open source arena. Following slide shows a list of Load testing tools available.

9. Load Testing Tools (Courtesy: Wikipedia) Tool Name Company Name Notes
Apache JMeter
An Apache Jakarta open source project
Java desktop application for load testing and performance measurement.
BlazeMeter
BlazeMeter Ltd.
BlazeMeter is a JMeter compatible, self-service, load testing platform.
Blitz
Spirent Communications
Blitz is a service for load and performance testing of websites, mobile, web apps and REST APIs in the cloud.
Gatling
Open Source
JVM application with scenarios as code and portable HTML reports.
Loader.io
SendGrid Labs
Cloud based load testing service
Load Impact
Load Impact - AB
Cloud based performance testing SaaS tool.
LoadRunner HP
Performance testing tool primarily used Licensed.
Load Test (included with Soatest)
Parasoft
Performance testing tool
loadUI
SmartBear Software
Cross-platform load testing tool
Login VSI
Login VSI, Inc.
Performance testing software for Windows based virtualized desktops. Licensed.
NeoLoad
Neotys
Load testing tool for web and mobile applications.
OpenSTA
Open System Testing Architecture
Open source web load/stress testing application, licensed under the Gnu GPL.
Rational Performance Tester
IBM
Eclipse based large scale performance testing tool
Silk Performer
Borland
Application performance tool with cloud and local virtual agents. Licensed
Test Studio
Telerik
Test Studio is a load testing tool
Visual StudioUltimate edition
Microsoft
Visual Studio Ultimate edition includes a load test tool
WebLOAD
RadView
Load testing tool for web and mobile applications Licensed.[3]
(Courtesy: Wikipedia)

10. JMeter

11. What is JMeter?
JMeter is a free, open-source performance measurement tool written in Java
JMeter is built by the reputed Apache project community
JMeter is available as a desktop Java application and can be run in GUI and batch modes. It can also be run in distributed modes.
It has a completely extensible architecture. It allows us to develops various elements that can be reused in different parts of a test plan. It also allows plugins to be developed, so the tool can be further extended.

12. Why JMeter? It is an Open source tool.
Can load and performance test many different server types:
Web - HTTP, HTTPS
SOAP
Database via JDBC
LDAP
JMS
Mail - POP3(S) and IMAP(S)
User friendly GUI Design. Can be run in batch and distributed modes to handle more load.
Full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
Can mimick Web browser functionality including Caching and Cookies.
Controllers are configurable and can monitor the server performance.
Test results can be captured in various format like summary report, Graph, Aggregate report, Aggregate graph, Results in tree and Results in Table. The results can be stored in files for later review as well.
Extensible architecture. Supports plugins, bean shell etc to add more functionality.

13. Installing Jmeter Install Java (1.5 or higher)
Download JMeter
Add path of java installation in environment path variables.

14. JMeter Test Plan
A test plan describes a series of steps JMeter will execute when run.
A complete test plan will consist of one or more Thread Groups, logic conrollers, sample generating controllers, listeners, timers, assertions, and configuration elements.

15. Elements of a Test Plan
The Test Plan is the wrapper for the entire test suite. It wraps around the following elements in an hierarchical fashion.
Thread Group
This is really the logic unit of a load test. There can be many thread groups in a Test Plan.
Setup number of thread
Set up ramp up period
No. of times test executes
Controllers
Sampler (Send Request to Server)
Logical Controller (Customize logic to send request); includes Loop Controller and One Time Controller.
Listener
Help to capture and display results in various formats. For e.g.,
Graph Result
View Results Tree or Table format and many more.
Timers
Add a delay to the next request
Continued….

16. Elements of Test Plans Assertions
Allow you to assert fact about responses received from HTTP request
Configuration Elements
Allow you configure settings
Preprocessor
Execute prior to sampler request
Post Processor
Execute some action after sampler request

17. Order of Execution 1. Config Element 2. Pre-Processor 3. Timer
4. Sampler
5. Post-Processor
6. Assertion
7. Listener

18. Building a Test Plan

19. Thread Group
Thread group elements are the beginning points of any test plan.
All controllers and samplers must be under a thread group.
Each thread represents a user  setting a thread group to 500 simulates 500 users.

20. Thread Group Contd...

21. Sampler
Samplers tell JMeter to send requests to a server and wait for a response. They are processed in the order they appear in the tree.

22. Sampler Contd...

23. Listener
Listeners can be used to display (or store) performance information JMeter collects during tests.

24. Listener Contd...

25. Listener Contd...

26. Logic Controller
Logic Controllers let you customize the logic that JMeter uses to decide when to send requests.
Logic Controllers can change the order of requests coming from their child elements. They can modify the requests themselves, cause JMeter to repeat requests, etc.

27. Configuration Elements
A configuration element works closely with a
Sampler. Although it does not send requests (except for HTTP Proxy Server ), it can add to or modifyrequests.

28. Configuration Element Contd...

29. Configuration Element Contd...

30. Assertions
Assertions allow you to assert facts about responses received from the server being tested.
Using an assertion, you can essentially "test" that your application is returning the results you expect it to.

31. Other Test Elements
● A Pre-Processor executes some action prior to a Sampler Request being made.
● A Post-Processor executes some action after aSampler Request has been made.
●A Timer will cause JMeter to delay a certain amount of time before each sampler which is in its scope.

32. Variables
JMeter allows parameters, properties and variables to keep the test scripts generalized.
Syntax for Variable:
${VARIABLE}
Variables can be declared and used in successive layers.

33. JMeter Test Plan Completed

34. JMeter Test Plan Completed
Once you completed, you can save the test plan with a meaningful name.
The script file generated is an XML file and will be saved with a .JMX extension.
You can run this script using command line mode as well, so you don't have to load GUI.

35. Running a Test Plan

36. Running in Batch Mode
On command line, in the JMeter\bin directory, you can type:
jmeter -n -t C:\sam\load_test\scripts\LT600_laweb2_test_syt6.jmx -j C:\sam\load_test\log\jmeter_1212.overnight.log -l C:\sam\load_test\log\jmeter_1212.overnight.jtl
Where,
LT600_laweb2_test_syt6.jmx is the script I saved in the completion step above.
-n makes it non-GUI
-t name of the test script you want to use
-j for the JMeter.log (I change this to match my log file naming above)
-l for the JTL file - in XML format, has the same details we get in listeners above. We may have some use for this, but for now I am just saving it.
You can put the above command inside a batch file to save typing!

37. Appendix

38. Non-Testing Elements

39. Non-Testing Elements Contd...
HTTP(S) Script Recording Element

40. Recording a Script
Setup a non-test element, HTTP(S) Test Script Recorder on the workbench.
Setup all the inclusion, exclusion lists and other properties.
Click on Start.
Now, JMeter is running as a local proxy server. If you adjust PC's proxy setting to use this as the proxy server, all requests will pass through JMeter.

41. Recording a Script Contd...
Next you need to setup the browser to use our JMeter listener as a local proxy

42. Recording a Script Contd...

43. Recording a Script Contd...
Now that we have captured all the pages to be visited, we can clean it all up. This is a good place to start.
Parametrize and add variables wherever possible and move the pages (samplers) to a Thread Group and build up our test plan.

44. Recording a script Contd...
An easier way is to use BadBoy software to capture the pages and export to JMeter. Download and install Badboy software from

45. Recording a Script Contd...

46. Real life Test Scenario
Our App Servers kept crashing on heavy load days. We had to constantly restart the servers, bringing down all the web users along with it. After making some changes to the application, had to simulate the huge load on the servers to test if it will sustain the next tax season. After reviewing various tools, we chose JMEter.

47. Real life Test scenario Contd...
When we upgraded our infrastructure* recently, we used JMEter to compare old environment and new environment for performance. We ran the same set of scripts against old and new environments to identify any bottlenecks.
* We upgrade to PowerBuilder 12.5, Oracle to 11g, Windows servers to Windows 2008.

48. Real life Test scenario Contd...
Our Web application did an overkill of logging in the database. We had a maximum concurrent user limit set to 300 in Oracle. We wanted to increase that.
To help with the tweaking, I ran JMeter scripts that pushed the EAServers to connect to the database continuously to make the database run out of handles - sort of stressing the database to its limits. We were able to tweak the setting and retest for the load condition.
Eventually, I refactored the code to reduce these loggings. Of course, JMeter came to the rescue again!

49. Demo

50. Q&A