1. Data Virtualization Demoette… JMeter Load Testing CIS SOAP
Hello, and welcome to the Demoette series for Cisco Information Server, or CIS. Demoettes are brief instructional videos that demonstrate specific features of CIS.
In this Demoette, we discuss using Jmeter to perform load testing on CIS SOAP resources.
Created 7/1/16, using CIS

2. Agenda What is it and why does it matter? A basic demo Summary
Here is our agenda.
We begin by defining JMeter load testing and outlining its importance for our customers.
Next we walk through a very basic demo of using JMeter with CIS SOAP resources.
Finally, we summarize the contents of this demoette.

3. Agenda What is it and why does it matter? A basic demo Summary
Let’s begin by discussing what JMeter is, and why it is important for our customers.

4. What is it? JMeter Apache open source project
Enables load testing for:
Functional behavior
Performance measurement
Supports many servers/protocols
Jmeter is an Apache open source project that offers a comprehensive load testing solution. It may be used to perform load testing for functional behavior as well as performance management. Jmeter supports many different servers and protocols. In this demoette, we concentrate on using Jmeter to load-test CIS SOAP resources.

5. Why does it matter? JMeter Understand CIS performance characteristics
Develop hardware requirements
Identify resources that require tuning
Identify key design considerations
Jmeter is important to our customers because they often need to understand the performance characteristics of CIS before purchasing the product, or before committing to large new projects. Jmeter load tests can help customers develop their hardware requirements. It can also help them identify project resources that may require tuning or other design considerations.

6. Agenda What is it and why does it matter? A basic demo Summary
Next, let’s walk through a very basic demo of using Jmeter with CIS SOAP resources.

7. Demo: Here is the business problem…
Here is the business problem that we illustrate in this demo. We have built a SOAP web service in CIS that functions correctly. Now we want to be sure that it will work with acceptable response times with many concurrent users.
<CLICK> We’ll use Jmeter to simulate concurrent user load.

8. Before you begin
Begin by going to jmeter.apache.org and learning about Jmeter. Jmeter is a very rich and sophisticated product, and we can only scratch its surface in this brief demo. You can download Jmeter from this site and install it. Installation is very simple; just extract the download to a directory of your choice. One important installation note is that Apache recommends that you install Jmeter in a directory path which does not contain any spaces.

9. Before you begin
You will need a CIS web service operation to test. Find the LookupProduct procedure in the CIS examples folder…
<CLICK> … and publish it to a web service endpoint called Demoette_JMeter_SOAP, as shown here.

10. Before you begin
You may also want to install SoapUI. We’ll be using this to make sure our web service runs correctly.

11. Plan and Scope: the Art of the Possible
Client machine limitations
LAN limitations
JMeter limitations: Coordinated Omission
CIS Query parameters
JMeter can handle all of the above, but do you have the time and resources to devote to it?
Don’t let the Perfect be the enemy of the Good
Best Practice: start small, grow incrementally as time and resources permit
When considering any load-testing, it is essential that everyone involved be aware of the need to plan and scope the effort. You can’t just fire up 10,000 simulated users on your laptop and expect to get realistic results. If you did this, you would simply surface limitations that are not related to what you are trying to test. For example, a load this big on a single client machine would likely overwhelm its available CPU, RAM, and disk resources.
In addition, this setup would probably cause artificial network problems. The client machine’s Network Interface Card would likely be overwhelmed. Even if you spread the load across several machines on a single LAN, you would likely have an artificial situation, because a LAN is a broadcast zone for IP messages, and would create a flood of packets that differs from a more real-world situation where users are spread across several LAN boundaries.
A heavy load on a single client can also surface Jmeter limitations. For example, Jmeter runs each simulated user on a thread, and it is susceptible to an issue known as “coordinated omission.” If a thread cannot complete a task before its next task is scheduled to begin, it simply skips the next task, which means that the total load will be smaller than expected.
It is also important to consider the design of the CIS resource requests to be tested. For example, if you have a SOAP resource that requires a filter parameter, you may get unrealistic results if every simulated user submits the same filter.
Jmeter has capabilities to deal with all of the above issues. You can design tests that use a fleet of remote client machines, and you can vary query parameters across simulated users. However, you could easily spend months in the design, construction, and execution of the perfect test. When time and resources are constricted, don’t let the Perfect be the enemy of the Good. Start small and take basic load measurements. Then make your test plans more sophisticated as time and resources permit.

12. Demo: Understand the CIS Web Service
As we begin the demo, let’s first examine the CIS SOAP resource we will use for load testing. It accepts a parameter that specifies a Product ID, and returns information for that product. Open the published endpoint in CIS, and double-click the WSDL URL for SOAP 1.2 in order to place it on the clipboard.

13. Demo: Understand the CIS Web Service
Open SoapUI, right-click Projects, and select New SOAP project. Paste the WSDL URL into the Initial WSDL field. SoapUI will create the operation.

14. Demo: Understand the CIS Web Service
Double-click Request 1 to open it.

15. Demo: Understand the CIS Web Service
Replace the question mark in the request body…
<CLICK> … with a Product ID value of 2.
<CLICK> Click the Run button to submit the request.
<CLICK> The request fails. That’s because we need to define an Authorization header.

16. Demo: Understand the CIS Web Service
Click Auth…
<CLICK> … Add New Authorization.
<CLICK> Select Basic…
<CLICK> … and enter the user name, password, and domain.

17. Demo: Understand the CIS Web Service
Execute again, and data is successfully returned.

18. Demo: Understand the CIS Web Service
As the Raw tab on the request shows, SoapUI has created the Authorization header, and changed the user name and password to BASE64. This will be an important consideration when we build our Jmeter test.

19. Demo: Build the Load Test
Now we’re ready to use Jmeter. To start it on Windows, run Jmeter.bat from the Jmeter installation directory.
<CLICK> A console window appears…
<CLICK> … followed by the user interface.

20. Demo: Build the Load Test
If you have watched the companion demoette for REST services with Jmeter, please note that we will be doing things a bit differently in this SOAP demoette. There are usually several ways to solve any problem in Jmeter, and we’ll introduce some interesting new techniques in this demo. For example, in this SOAP demoette…
<CLICK> … we’ll use a Counter component…
<CLICK> … a Response Assertion component…
<CLICI> … and a View Results Tree component.

21. Demo: Build the Load Test
The test plan is the highest level object. Begin by renaming it, as shown here. Note that Jmeter editors do not have a Save button. Your work is automatically saved as you make changes. However, you will need to Save your test plan project when you are all done, so that it can be reloaded later. You can do this from the File menu.
<CLICK> To build a test plan, we add a series of child objects to the top-level test plan object. Right-click the test plan…
<CLICK> … select Add…
<CLICK> … Config Element…
<CLICK> … User Defined Variables.

22. Demo: Build the Load Test
Define variables named host and port…
<CLICK> … and set the appropriate values. This is a useful way to make our test project more portable.

23. Demo: Build the Load Test
Now right-click the test plan…
<CLICK> … select Add…
<CLICK> … Threads…
<CLICK> … Thread Group. This will let us define our simulated end users.

24. Demo: Build the Load Test
Set the number of simulated users to 10…
<CLICK> … and tell Jmeter to ramp up all 10 users in a total time of 1 second.
<CLICK> Now set the Loop Count to Forever by checking the box. In the companion REST demoette, we told Jmeter to run a specific number of queries. In this SOAP demoette, we’ll use a different technique. We’ll tell Jmeter to run as many queries as possible in a given time period.
<CLICK> Be sure the Scheduler box is checked…
<CLICK> … and configure the Scheduler for a Duration of 10 seconds.

25. Demo: Build the Load Test
Next, right-click the Thread Group and select Add…
<CLICK> … Config Element…
<CLICK> … HTTP Authorization Manager.

26. Demo: Build the Load Test
We need the HTTP Authorization Manager to build the Authorization Header we looked at earlier in SoapUI. Enter the Base URL information, as shown here. Note that we are using the Host and Port User Defined Variables we built earlier. To use them, precede them with a dollar sign and enclose them in curly braces.
<CLICK> Now enter the user name, password, and domain.
<CLICK> Select BASIC_DIGEST from the drop-down in the Mechanism field.

27. Demo: Build the Load Test
Now right-click the Thread Group and select Add…
<CLICK> … Config Element…
<CLICK> … Counter.

28. Demo: Build the Load Test
The Counter object lets us build loops with variables that increment. We’ll use this to vary the filter parameter in the SOAP queries we define later. As you will recall, the filter in our SOAP web service is a Product ID.
<CLICK> Tell Jmeter to start with a value of 1…
<CLICK> … and increment by 1…
<CLICK> … until it reaches 20.
<CLICK> We’ll give the Counter a name of PROD_ID
<CLICK> … and specify that we want a separate instance of the counter for each of the simulated users in the Thread Group. Now let’s see how to put the counter into use.

29. Demo: Build the Load Test
Right-click the Thread Group and select Add…
<CLICK> … Sampler…
<CLICK> … HTTP Request.

30. Demo: Build the Load Test
Here we define the actual SOAP calls we will be making in the load test. Note that we are using our Counter object as part of the SOAP Request name field. To use it, precede it with a dollar sign and enclose it in curly braces. Since the counter iterates 20 times, we are actually defining 20 SOAP calls per user.
<CLICK> We also use the User Defined Variables we created earlier to specify Server and Port.
<CLICK> The rest of the Path goes here.
<CLICK> For SOAP requests, we must specify implementation details. Specify HttpClient 3.1, protocol HTTP, Method POST, and Content-Encoding utf-8.
<CLICK> Now paste in the actual request, which you created in SoapUI, into the Body Data area.
<CLICK> Replace the filter parameter value with the PROD_ID Counter name. Now the filters will vary between 1 and 20, giving us a wide variety of data results.

31. Demo: Build the Load Test
Now right-click the SOAP Request object and select Add…
<CLICK> … Assertions…
<CLICK> … Response Assertion.

32. Demo: Build the Load Test
The Response Assertion object checks for the presence or absence of specific data in the response, and is a useful way to verify success or failure of SOAP calls.
<CLICK> I know I have a successful response if a ProductName tag is present in the returned XML, so I check for the ProductName closing tag.
<CLICK> The Substring choice is a good pattern matching rule. It differs from the Contains pattern because Substring is case sensitive.

33. Demo: Build the Load Test
We are almost ready to run the load test. First, though, we need to define a Listener object. Listeners gather and report result data from the load test. Right-click the test plan and select Add…
<CLICK> … Listener…
<CLICK> … Graph Results.

34. Demo: Build the Load Test
Leave all the default values on the Graph Results object…
<CLICK> … and click the Run button.

35. Demo: Build the Load Test
The test runs, and results are rendered on the graph for all selected measures.
<CLICK> Summary results are shown at the bottom of the screen.

36. Demo: Build the Load Test
Let’s examine a different type of Listener. Right-click the Test Plan and select Add, Listener, View Results Tree.

37. Demo: Build the Load Test
Now run the test again. The View Results Tree Listener collects detailed data for each request.
<CLICK> Select any individual request…
<CLICK> … and examine the Sampler Result data…
<CLICK> … which appears both Raw…
<CLICK> … and Parsed.

38. Demo: Build the Load Test
You can also examine the individual Request data, in both Raw…
<CLICK> … and parsed formats.

39. Demo: Build the Load Test
Individual Response data is also available for viewing and searching. The two Listeners we have seen so far, Graph Results and View Results Tree, are great for functional testing of a Jmeter project. However, they are NOT recommended for actual load testing, because they are very resource intensive. Next, let’s look at a Listener that is more appropriate for actual load testing.

40. Demo: Run the Load Test
First, right-click the Graph Results Listener and disable it. Do the same for the View Results Tree Listener.
<CLICK> Now right-click the Test Plan and select Add, Listener, Summary Report.

41. Demo: Run the Load Test
The Summary Report is a good Listener for an actual load test, because it is not resource-intensive.
<CLICK> You can use the Browse button to define a CSV file to hold the test results.
<CLICK> This error message simply means that the file does not already exist. Don’t worry, Jmeter will create it when you run the test.

42. Demo: Run the Load Test
The Summary Report produces a useful summary of aggregated results.

43. Demo: Run the Load Test
It also records detailed information in the CSV file.
<CLICK> We can see that the tasks were successful, and we can learn about the timings.
In this presentation, we have only scratched the surface of Jmeter’s capabilities. However, we have shown how to create a simple but meaningful load test of CIS SOAP resources, and demonstrated several ways to view the test results. Our demo is complete.

44. Agenda What is it and why does it matter? A basic demo Summary
Let’s summarize what we have seen in this presentation.

45. Summary Apache open source project Enables load testing for:
Functional behavior
Performance measurement
Supports many servers/protocols
Understand CIS performance characteristics
Develop hardware requirements
Identify resources that require tuning
Identify key design considerations
Jmeter is an Apache open source project that offers a comprehensive load testing solution. It may be used to perform load testing for functional behavior as well as performance management. Jmeter supports many different servers and protocols.
Jmeter is important to our customers because they often need to understand the performance characteristics of CIS before purchasing the product, or before committing to large new projects. Jmeter load tests can help customers develop their hardware requirements. It can also help them identify project resources that may require tuning or other design considerations.
Thank you.

46. TOMORROW starts here.