Data Virtualization Demoette… JMeter Load Testing CIS JDBC Hello, and welcome to the Demoette series for Cisco Information Server, or CIS. Demoettes are brief instructional videos that demonstrate specific features of CIS. In this Demoette, we discuss using Jmeter to perform load testing on CIS Virtual Databases via JDBC. Created 7/1/16, using CIS 7.0.3.00.11

Agenda What is it and why does it matter? A basic demo Summary Here is our agenda. We begin by defining JMeter load testing and outlining its importance for our customers. Next we walk through a very basic demo of using JMeter with CIS JDBC resources. Finally, we summarize the contents of this demoette.

Agenda What is it and why does it matter? A basic demo Summary Let’s begin by discussing what JMeter is, and why it is important for our customers.

What is it? JMeter Apache open source project Enables load testing for: Functional behavior Performance measurement Supports many servers/protocols Jmeter is an Apache open source project that offers a comprehensive load testing solution. It may be used to perform load testing for functional behavior as well as performance management. Jmeter supports many different servers and protocols. In this demoette, we concentrate on using Jmeter to load-test CIS JDBC resources.

Why does it matter? JMeter Understand CIS performance characteristics Develop hardware requirements Identify resources that require tuning Identify key design considerations Jmeter is important to our customers because they often need to understand the performance characteristics of CIS before purchasing the product, or before committing to large new projects. Jmeter load tests can help customers develop their hardware requirements. It can also help them identify project resources that may require tuning or other design considerations.

Agenda What is it and why does it matter? A basic demo Summary Next, let’s walk through a very basic demo of using Jmeter with CIS JDBC resources.

Demo: Here is the business problem… Here is the business problem that we illustrate in this demo. Prospective customers of Cisco Data Virtualization often ask, “How much overhead does CIS add to a query?” To answer this question, we’ll take timings of queries against a SQL Server database table. Then we’ll publish a CIS view of the same table, and take timings of queries against CIS. <CLICK> We’ll use Jmeter to simulate concurrent user load. Let’s stop a minute, though, and ask if this is really a meaningful question. The answer depends on the customer’s use case. For most customers, the main value of CIS is its ability to federate data from multiple sources in an agile manner with data that is current. For these customers, the test above has little meaning. They use CIS because the alternative is either manual data wrangling, or traditional ETL processing. Manual data wrangling can cost days of human effort, with concomitant loss of data currency. ETL can also result in days of lost data currency. Both alternatives also suffer from poor agility in the face of ongoing change requests. For these use cases, it is much more meaningful to ask questions like, “How much time does CIS *save* compared to manual processing?” or “What is the value of serving up-to-the-second data as opposed to ETL data from last night?” or “How much money will I save if I can respond to change requests in days instead of months?” However, there are some CIS use cases where the test shown here *is* meaningful. Some customers use CIS as a central “front door” for enterprise data access. They may choose to expose data from physical sources unchanged. This approach can be useful for highly expert end users who are capable of designing and implementing their own joins. In this use case, CIS adds value by serving as a central point for authentication and authorization.

Before you begin Begin by going to jmeter.apache.org and learning about Jmeter. Jmeter is a very rich and sophisticated product, and we can only scratch its surface in this brief demo. You can download Jmeter from this site and install it. Installation is very simple; just extract the download to a directory of your choice. One important installation note is that Apache recommends that you install Jmeter in a directory path which does not contain any spaces.

Before you begin Next, you will need a physical data source. In this demoette, we will be using a SQL Server instance on a public web site. However, we do not make this database available for general use, because we do not want to expose the password. You can use a table from any physical database available to you. <CLICK> Next, create a CIS view using the table from your chosen database.

Before you begin You will need JDBC drivers for your chosen physical database, as well as for CIS. The CIS driver, csjdbc.jar, is found in the path shown here. <CLICK> Place both drivers in the lib directory of the Jmeter installation. Jmeter will automatically find them from this directory.

Plan and Scope: the Art of the Possible Client machine limitations LAN limitations JMeter limitations: Coordinated Omission CIS Query parameters JMeter can handle all of the above, but do you have the time and resources to devote to it? Don’t let the Perfect be the enemy of the Good Best Practice: start small, grow incrementally as time and resources permit When considering any load-testing, it is essential that everyone involved be aware of the need to plan and scope the effort. You can’t just fire up 10,000 simulated users on your laptop and expect to get realistic results. If you did this, you would simply surface limitations that are not related to what you are trying to test. For example, a load this big on a single client machine would likely overwhelm its available CPU, RAM, and disk resources. In addition, this setup would probably cause artificial network problems. The client machine’s Network Interface Card would likely be overwhelmed. Even if you spread the load across several machines on a single LAN, you would likely have an artificial situation, because a LAN is a broadcast zone for IP messages, and would create a flood of packets that differs from a more real-world situation where users are spread across several LAN boundaries. A heavy load on a single client can also surface Jmeter limitations. For example, Jmeter runs each simulated user on a thread, and it is susceptible to an issue known as “coordinated omission.” If a thread cannot complete a task before its next task is scheduled to begin, it simply skips the next task, which means that the total load will be smaller than expected. It is also important to consider the design of the CIS resource requests to be tested. For example, if you have a resource that is usually filtered with a WHERE clause, you may get unrealistic results if every simulated user submits the same filter. Jmeter has capabilities to deal with all of the above issues. You can design tests that use a fleet of remote client machines, and you can vary query parameters across simulated users. However, you could easily spend months in the design, construction, execution, and analysis of the perfect test. When time and resources are constricted, don’t let the Perfect be the enemy of the Good. Start small and take basic load measurements. Then make your test plans more sophisticated as time and resources permit.

Demo: Build the Load Test Now we’re ready to use Jmeter. To start it on Windows, run Jmeter.bat from the Jmeter installation directory. <CLICK> A console window appears… <CLICK> … followed by the user interface.

Demo: Build the Load Test The test plan is the highest level object. Begin by renaming it, as shown here. Note that Jmeter editors do not have a Save button. Your work is automatically saved as you make changes. However, you will need to Save your test plan project when you are all done, so that it can be reloaded later. You can do this from the File menu. <CLICK> To build a test plan, we add a series of child objects to the top-level test plan object. Right-click the test plan and select Add… <CLICK> … Threads… <CLICK> … Thread Group. This will let us define our simulated end users.

Demo: Build the Load Test The Thread Group Editor appears. You can rename the Thread Group, but this is optional. <CLICK> I have defined 5 users. <CLICK> I tell Jmeter to ramp up all 5 users in a total time of 1 second. <CLICK> I want each user to execute 10 iterations of whatever workload I define.

Demo: Build the Load Test Now right-click the thread group. Select Add… <CLICK> … Config Element… <CLICK> … JDBC Connection Configuration.

Demo: Build the Load Test Here we can specify the details of the JDBC connection. Choose any name you like for the connection pool. We’ll use this name later when we define specific queries. <CLICK> Provide the JDBC connection string… <CLICK> … driver class name… <CLICK> … user id… <CLICK> … and password.

Demo: Build the Load Test At this point, we have defined all the general information for the thread group. Now we can define the specific requests. Right-click the Thread Group and select Add… <CLICK> Sampler… <CLICK> … JDBC Request.

Demo: Build the Load Test For our first request… <CLICK> … enter the connection pool variable name we defined earlier… <CLICK> … and add a SQL Query. This query selects all our California customers… <CLICK> Then create a second JDBC Request. This one is identical to the first, except that it selects New York customers.

Demo: Run the Load Test We are ready to try out our load test. But first, we need to define a way to gather the results. Right-click the thread group and select Add… <CLICK> … Listener… <CLICK> … Graph Results.

Demo: Run the Load Test Leave the default settings for the Graph Listener, and click the Run button.

Demo: Run the Load Test Graph results are shown for all the selected measures. <CLICK> Summary information is provided at the bottom of the screen. Note that there are 100 samples. That’s because we have 5 users, each is running a query for California and another for New York, and each user is looping through the tasks 10 times.

Demo: Run the Load Test If you press the Run button repeatedly, the graph results accumulate. Here are the results after 1,000 samples. The Graph Results listener is great for initial functional testing, but it is not recommended for actual load testing, because this Listener is very resource-intensive. Let’s look at some other listeners that are more appropriate for load testing.

Demo: Run the Load Test First, right-click the Graph Results Listener and disable it. <CLICK> Now right-click the Thread Group and select Add, Listener, Simple Data Writer.

Demo: Run the Load Test The Simple Data Writer is a good Listener for an actual load test, because it is not resource-intensive. <CLICK> You can use the Browse button to define a CSV file to hold the test results. <CLICK> This error message simply means that the file does not already exist. Don’t worry, Jmeter will create it when you run the test.

Demo: Run the Load Test Run the test and then open the CSV file you defined. <CLICK> We can see that the tasks were successful, and we can learn about the timings.

Demo: Run the Load Test The Summary Report is another Listener choice that is also modest in resource consumption. It produces a useful summary of aggregated results. <CLICK> It also records detailed information in the CSV file, much like the Simple Data Writer.

Demo: Run the Load Test Now we can test access through CIS. Do a Save As, and rename the original test plan so we can use a new version for CIS Access. We’ll make just a few changes to the original plan. <CLICK> First, create a new variable name for the connection pool, to avoid any confusion. <CLICK> Next, change the connection details, as shown here.

Demo: Run the Load Test Now change the first JDBC request. <CLICK> Use the new connection pool variable. <CLICK> Change the query to name the CIS table in the FROM clause, and remove any square brackets that were used in the SQL Server query. <CLICK> Do the same for the second query.

Demo: Compare the Results Now we can run the test against CIS and compare the results. For this demo, we’ll simply look at the overall averages. <CLICK> The direct queries to SQL Server ran in an average time of 131 milliseconds each. <CLICK> The queries through CIS ran in an average time of 150 milliseconds each. That’s 19 milliseconds longer, on average, for queries running through CIS. This will be imperceptible to end users. In this presentation, we have only scratched the surface of Jmeter’s capabilities. However, we have shown how to create a simple but meaningful load test of CIS JDBC resources, and demonstrated several ways to view the test results. Our demo is complete.

Agenda What is it and why does it matter? A basic demo Summary Let’s summarize what we have seen in this presentation.

Summary Apache open source project Enables load testing for: Functional behavior Performance measurement Supports many servers/protocols Understand CIS performance characteristics Develop hardware requirements Identify resources that require tuning Identify key design considerations Jmeter is an Apache open source project that offers a comprehensive load testing solution. It may be used to perform load testing for functional behavior as well as performance management. Jmeter supports many different servers and protocols. Jmeter is important to our customers because they often need to understand the performance characteristics of CIS before purchasing the product, or before committing to large new projects. Jmeter load tests can help customers develop their hardware requirements. It can also help them identify project resources that may require tuning or other design considerations. Thank you.

TOMORROW starts here.