Using Apache JMeter Load Testing Tool

Running JMeter
Running Remotely – Server Mode
Configuring JMeter
Building a Test Plan
Test Plan Elements – Thread Group
Test Plan Elements – Controllers
Test Plan Elements – Listeners
Test Plan Elements – Timers
Test Plan Elements – Assertions
Test Plan Elements – Configuration Elements
Pre and Post Processors
Execution Order
Element Scope
Web Test

Running JMeter

The scripts used to run JMeter can be found in the JMeter/bin install directory:

windows: jmeter.bat
unix: jmeter

There are JVM HEAP settings in the scripts that can be overridden by the JVM_ARGS environment variable, or alternatively edit the scripts themselves.

Running behind a firewall/proxy server
run the jmeter.bat/jmeter file from a command line with the following parameters:

-H [proxy server hostname or ip address]
-P [proxy server port]
-N [nonproxy hosts] (e.g. *.apache.org|localhost)
-u [username for proxy authentication - if required]
-a [password for proxy authentication - if required]

Example : jmeter -H proxyServer -P 8000 -u username -a password -N localhost
Example : jmeter -H 172.16.15.43 -P 8080
Example : jmeter –proxyHost proxyServer –proxyPort 8000 –username username –password password -N localhost

Running Remotely – Server Mode

You can run JMeter on a remote server and also choose to control the server using the GUI. To start the server, run jmeter-server/jmeter-server.bat on the server host.

This option is for when you wish to run JMeter from 1 or more remote machines, generally because you dont have a single machine that can simulate the size of the load you need. One instance of the JMeter GUI client can control any number of remote JMeter instances and collect all their data.

Configuring JMeter

If you wish to modify the properties with which JMeter runs you need to either modify the jmeter.properties in the /bin directory or create your own copy of the jmeter.properties and specify it in the command line

Building a Test Plan

A test plan is a series of steps that JMeter executes when run and will consist of 1 or more:

• Thread Groups
• Logic Controllers
• Sample Generating Controllers
• Listeners
• Timers
• Assertions
• Configuration Elements

You can add elements to a test plan by right-clicking (windows) on tree-element in the GUI and choosing “add” or loading from a file. Elements can also be removed by right-clicking.

You can also save elements in the tree by right-clicking on them and then either choosing “save” or “Save As” from the Edit menu. the elements is saved along with any of its child elements. This allows you to save individual tree elements, the whole tree, or even the whole test plan.

Running a Test Plan
Choose “start” from the “Run” menu – a small green box will be shown top right of the GUI when JMeter is running, along with the number of active/total threads.

Stopping a Test Plan
There are 2 stop commands in the GUI:

1. Control + ‘.’ – stops threads immediately if possible and can interrupt samplers
2. Contol + ‘,’ – requests threads stop at the end of any current work but does not interrupt samplers and the modal shutdown dialog remains active until all threads have stopped
   

In non-GUI mode JMeter listens for commands on a specific port (default 4445 – see property jmeterengine.nongui.port):

1. Shutdown – graceful shitdown
2. StopTestNow – immediate shutdown

These commands are sent using the shutdown[.cmd|.sh] or stoptest[.cmd|.sh] scripts respectively (bin directory)

Test Plan Elements - ThreadGroup

ThreadGroup
The start of any test plan, these control the number of threads JMeter uses to execute your test. All controllers and samplers must be under a thread group but some elements eg. Listeners, may be placed directly under the test plan, in which case they apply to all thread groups.

Each thread is used to simulate concurrent connections to your application, and each executes the test plan in its entirety and independently of other test threads.

There are 3 controls:

• no of threads
• ramp-up period – time taken to start all test threads – nneds to be long enough so that the work-load is not too large at the start but that the last threads begin before the first ones have finished
• no of times to execute the test -

start with Ramp-up = no of threads and then adjust if necessary

Test Plan Elements - Controllers

Controllers
These drive the processing and there are 2 types:

• Samplers – tell JMeter to send requests to the server and wait for the response eg. you use a HTTP Request Sampler to send HTTP request. Other samplers – FTP, JDBC, LDAP, SOAP. If you wish to validate the response then you can add an Assertion to the sampler eg. the server may return a successful HTTP Response code but the page might have errors so you can assert for things like certain HTML elements or error strings.Remember to add a listener to your test plan to view and/or store the results
• Logical Controllers – allow you to customize the logic JMeter uses to decide when to send requests eg.example:
  - you can specify a One Only Controller to make a Login Request that is only executed on the first iteration
  - rather than having several individual HTTP request elements (eg. load search page, run search A, load search again, run search B), we can use the Interleave Controller which knows the ordering of its children and passes requests from one to the next

HTTP Request Defaults
This element can be configured for many test plan elements and are useful for abstracting default information that each request may use ie. if several (or all) of the requests under the controller use the same PATH information (domain, port, action, protocol etc) then we can leave these blank for each request element and the Interleave Controller fills in the blanks for each of those requests from the Configuration Element. If we setup a HTTP Request Defaults at the Thread Group level then these defaults will be used by all child HTTP Request elements in the test. This is very useful in web testing, for example, by leaving the DOMAIN field blank in all your HTTP Sampler elements and configuring this is the defaults, you can easily switch your test to a different server by changing the single field in the defaults.

Cookie Manager
Add to all web tests – otherwise JMeter ignores cookies. Add it at the Thread Group level to ensure all HTTP requests use the same cookies.

Test Plan Elements - Listeners

Listeners
These provide access to the data while JMeter runs. The Graph Results listener, for example, plots response times on a graph and the View Results Tree listener shows details of sampler requests and responses. Other listeners provide summary/aggregated information.

They can also direct data to a file for later use.

Listeners can be added anywhere in the test, including directly beneath the test plan. They collect data from elements at and below their level.

Test Plan Elements - Timers

Timers
By default, JMeter sends requests without pausing in-between -

? Thought – this seems a bit odd since you would surely want JMeter to wait after making a login request wouldnt you?

You can use a timer to instruct JMeter to wait before making each new request. It is recommended to specify a delay by adding a timer to the Thread Group or else you can overwhelm the server. You can add more than 1 timer in which case the sum of the timers is waited.

Timers can also be added at Sampler or Controller levl to restrict which components they are applied to.

Ramp-up – this controls how long JMeter waits before starting each thread, not how long each thread waits between each request.

Test Plan Elements - Assertions

Assertions
Allow you to test the results returned by the application and can use Perl-style regular expressions for the matches.

You can add an assertion to any Sampler eg. add one to an HTTP Request to check for markup or text in the response. You can view assertion results by adding an Assertion Listener to the Thread Group – failed assertions show up in the Tree View and Table Listners and count towards the percentage errors in the Aggregate and Summary reports.

Test Plan Elements - Configuration Elements

Configuration Elements
These work closely with Samplers and can add to, or modify, requests. They are only available to the elements at the same level and below the tree branch where you attach them.

User Defined Variables Element – this is different and is procesed at the start of a test no matter where it is placed – hence for simplicity it recommended to place it at the start of a Thread Group.

Pre and Post Processors

Pre-Processors
Executes some action prior to a Sample Request and most-used to modify the settings of the request before it runs, or to update variables not extracted from the response text

Post-Processors
same as above except after the sample runs, usually to extract some data from the response.

For both of the above – they are executed either before or after the Sampler element to which theyare attached.

Execution Order

1. Configuration Elements
2. Pre-Processors
3. Timers
4. Sampler
5. Post-Processors (unless sample result is null)
6. Assertions (unless sample result is null)
7. Listeners (unless sample result is null)

Rules for execution order:

• Timers, Assertions and Pre and Post-Processors are only processed if there is a sampler to which they apply
• Controllers and Samplers are processed in the order in whic they appear in the tree
• Other test elements are processed according to the scope in which they are found, and the type of test element they are within

Examples – see documentation http://jakarta.apache.org/jmeter/usermanual/test_plan.html

Element Scope

Test tree elements are either ordered or hierarchical.

Ordered
eg. Controllers, Samplers

When you create a test plan you create an ordered list of sample requests (via Samplers) that represent the steps to be executed. Often, these requests are organized within Controllers.

Hierarchical
eg. Listeners, Config Elements, Processors, Assertions, Timers

These elements are all affected by the hierarchy of the branch they exist in. An assertion, for example, if its parent is a request is applied to justthat request, whereas if its parent is a Controller it affects all the requests that are a descendant of that Controller.

Order is irrelevant for hierarchical elements. For example, in the following, Timer #1 is applied to Requests One and Two, but Assertion #1 is only applied to Request Two. Timer #2 is applied to all requests.

Thread Group
|
|—-Simple Controller
|       |
|       |—-Request One
|       |—-Timer #1
|       |—-Request Two
|                 |
|                 |—-Assertion #1
|
|—-Timer #2

Web Test Setup

Example of a basic web application test plan:

1. create Thread Group under the test plan
   - maybe give it a descriptive name
   - configure the number of threads (essentially the number of users we wish to simulate)
   - start with a Ramp-up period = No. on threads
2. add a HTTP Request Defaults element under Thread Group
   - fill in any default HTTP fields eg. Hostname/IP
3. add a HTTP Cookie Manager if your application uses cookies
   - add a HTTP Cookie Manager to the Thread Group, this ensures each thread gets its own cookie but shared across requests
4. add the HTTP Requests that you need for your test plan
   - give each a descriptive name
   - set the path for the request (no need to set the server since this was declared in the defaults)
5. finally, add a Listener to store the results

Dependency Injection

Inversion of control – IoC – is a concept, and an associated set of programming techniques. In the traditional interaction model of an application, the programmer specifies a series of events to happen by making a series of function calls. eg. a programme which takes a name from the command line and displays it back would follow this model, all the user can do is enter a string and then hit return ie. the flow will always be the same. In an application with GUI however we might have an input field and a submit button and although very simple the flow control will not necessarily be as straightforward:

eg.

• the user could enter several different strings and change them before submitting;
• they may repeatedly hit submit;
• they might close the window before doing anything;

In the second case, the programmer will set up some responses to particular events of interest ie. the submit button being pressed; the window being closed; and then explicitly gives up the control, usually by putting the code in an
infinite loop in which the application will wait for and react to events. The programme will usually be programmed to exit the loop when an event happens. This example illustrates one important aspect of inversion of control: because
the programmer no longer has direct control over the order of execution of individual functions, some things must change.

The example, despite its simplicity, hints at some very significant implications of inversion of control. In the traditional interaction style, it’s immediately obvious what will happen when the programme is run. In return, such a programme is inflexible: if the programmer wanted to allow the user to correct the name before it’s submitted, or to display the greeting twice on the command line, each such scenario would need to be coded for and supported separately. On the other hand, in the second event-driven example, infinitely many possible scenarios can be realised. The price paid for that additional flexibility is greatly increased complexity and the fact that the programme is no longer deterministic: it’s no longer possible to tell what functions will be called just by looking at the code; one needs to consider the code and the environment to be able to determine the order of events. The code is also much harder to write and reason about, and computations that are naturally sequential (A needs to happen before B, such as User name must be provided before the login can be successful) can be much harder to write and understand.

Consider the simple example below:

The crux of the matter is the finder object, or particularly how we connect the lister object with a particular finder object. What we really want is the moviesDirectedBy method to be completely independent of how all the movies are being stored. So all the moviesDirectedBy method does is refer to a finder, and all that finder does is know how to respond to the findAll method.

We put the code for the concrete MovieFinder class in the constructor. The name of the implementation class comes from the fact that we are getting the list from a comma delimited text file. What happens if someone else wants to use my code but their movies are not stored in a comma delimted file: a SQL database, an XML file, a web service, or just another format of text file? In this case we need a different class to grab that data. Now because we’ve defined a MovieFinder interface, this won’t alter my moviesDirectedBy method. But we still need to have some way to get an instance of the right finder implementation into place.

The MovieLister class is dependent on both the MovieFinder interface and upon the implementation. We would prefer it if it were only dependent on the interface, but then how do we make an instance to work with? We come to the concept of a plugin.

The implementation class for the finder isn’t linked into the program at compile time since we dont know what that finder is supposed to do ie. what sort of data store it needs to access. Instead we want the lister to work with any implementation, and for that implementation to be plugged in at some later point automatically. The problem is how can I make that link so that the lister class is ignorant of the implementation class, but can still talk to an instance to do its work?

A real system might have dozens of such services and components. In each case we can abstract our use of these components by talking to them through an interface (and using an adapter if the component isn’t designed with an interface in mind). But if we wish to deploy this system in different ways, we need to use plugins to handle the interaction with these services so we can use different implementations in different deployments. So the core problem is how do we assemble these plugins into an application? This is one of the main problems that the new breed of lightweight containers suach as Spring face, and universally they all do it using the concept of Inversion of Control.

For Spring the inversion is about how it looks up a plugin implementation. In the simple example the lister looked up the finder implementation by directly instantiating it. This stops the finder from being a plugin. The approach that containers such as Spring use is to ensure that any user of a plugin follows some convention that allows a separate assembler module to inject the implementation into the lister. For such a process, the term Inversion of Control is too generic – this is where the concept of the Dependency Injection pattern began.

Introduction to JProfiler

There are basically 3 types of profiling:

1. time – measures the execution paths of your application on the method level
2. space – gives you insight into the development of the heap, such as which methods allocate
   most memory
3. thread – these analyze thread synchronization issues

Some traditional profilers dump their profiling data after the application exits. JProfiler is an interactive profiler however, and it combines all the 3 above profilers into a single application and correlates the data from all 3 domains in real-time.

Before Java 1.5 the SDK offered a Java Virtual Machine Profiling Interface (JVMPI) but this varies between JVMs, cannot run with modern garbage collectors and has problems profiling large heaps. With Jave 1.5 the JVM Tool Interface was added to overcome these problems and JProfiler utilizes both.

These interfaces are event-based, allowing the profiling agent to register handler functions for different events. Disabling some events is important for reducing profiler overhead and JProfiler has object allocation recording switched off by default. When you enable allocation recording in the GUI the agent tells the JVMPI/JVMTI interface to enable the handler for object allocation events.

The Information vs Runtime Overhead Trade-off

Profiling adds overhead to the profiled application, causing it to run more slowly and consume more memory. If alot of objects are created and lots of method calls occur while the profiled application is running then this creates a significant overhead since the profiler has to do alot of record-keeping operations to keep track of all these operations in its internal data structures.

Optimizing the Data

Too much detail can make it more difficult to track the operations your interested in profiling. For example, most often you are not concerned at all with the internal call tree of framework classes, but are interested in what your own classes are doing. When you have no control over the implementation, the internal call structure is not very helpful and is likely clutter that you dont need. You can stillrecord this data and filter it out using the view filters eg. the CPU views allow you to remove call structures that do not match your filter from the tree, but this data is still being recorded and causing an overhead so its better practise to not record it.

When an application starts up alot of objects are created that you probably dont need to profile so its best to leave object allocation recording switched off during startup, the same goes for profiling method calls, or “CPU profiling” – this both increases performance and keeps the generated data focused on what your mainly interested in.

Some information, such as the references between objects as well as the data in them is not available from the events fired by the JVMPI/JVMTI interfaces. To get this level of detail on allobjects on the heap you need to have the profiling agent trigger a “heap dump”. This command takes a snapshot of the heapwalker.

Starting the Profiler

The profiling agent has to be started at the beginning of the JVM startup. This is done using JVM parameters eg. for Java 1.5+:

-agentlib:jprofilerti

The JProfiler GUI is started separately and runs in its own JVM, rather than being started by the profiling agent. This is so as not to disturb the profiled process and also to facilitate remote profiling. The GUI and agent communicate via a TCP/IP network socket – this is the case even with local sessions. The profiled data resides internally in the agent and only a subset is transferred to the GUI otherwise the data load on the socket would be too great for effective profiling, especially when the agent and GUI are remote from each other.

Configuring the Profiler

Successful profiling is about choosing the right profiling parameters that dictate to the agent what events to respond to and capture data for. There are filters that control the extent recorded call tree and these are configured in the GUI. Since these parameters are required at startup, the profiling agent stops the JVM and waits for a connection from the GUI before the profiled application is allowed to start.

Profiling Settings

These control the way profiled data is recorded and cannot be changed once a profiling session has begun.