Chapter 4. Reference Guide

Table of Contents

4.1. How - Generating load
4.1.1. DefaultMessageGenerator
4.1.2. ConstantSpeedMessageGenerator
4.1.3. CustomProfileGenerator
CsvProfile
4.1.4. RampUpDownGenerator
4.2. Where - Sending messages
4.2.1. CamelSender
4.2.2. CoapSender
4.2.3. CommandSender
4.2.4. DummySender
4.2.5. GroovySender
4.2.6. HttpSender
4.2.7. HttpsSender
4.2.8. ChannelDatagramSender
4.2.9. ChannelFileSender
4.2.10. ChannelSocketSender
4.2.11. JdbcSender
4.2.12. Jms[11]Sender
4.2.13. LdapSender
4.2.14. MqttSender
4.2.15. OauthHttpSender
4.2.16. PlainSocketSender
4.2.17. RequestResponseJms[11]Sender
4.2.18. ScriptSender
4.2.19. SslSocketSender
4.2.20. WebSocketSender
4.3. Receiving messages
4.3.1. HttpReceiver
4.4. Correlating messages
4.4.1. GenerateHeaderCorrelator
4.4.2. PrefixCorrelator
4.5. What - Messages
4.5.1. Filtering and templates
4.6. Sequences
4.6.1. PrimitiveNumberSequence
4.6.2. NumberSequence
4.6.3. RandomSequence
4.6.4. RandomUuidSequence
4.6.5. ThreadIdSequence
4.6.6. TimeStampSequence
4.6.7. FileLinesSequence
4.6.8. FilesContentSequence
4.7. Reporting
4.7.1. Reporters
ClassifyingReporter
GeolocationReporter
IterationsPerSecondReporter
MemoryUsageReporter
ResponseTimeHistogramReporter
ResponseTimeStatsReporter
ThroughputStatsReporter
WarmUpReporter
RawReporter
4.7.2. Destinations
ChartDestination
ConsoleDestination
CsvDestination
ElasticsearchDestination
InfluxDbDestination
Log4jDestination
4.8. Validation
4.8.1. Validators
DictionaryValidator
PrintingValidator
RegExpValidator
RulesValidator
ScriptValidator

4.1. How - Generating load

Generator is an object which uses Sender objects to send and receive messages. Each generator represents a method or technique how the load (sending the messages) is handled. It is capable of generating the load for a specified duration which can be an amount of time or a number of iterations using a specified number of concurrent threads. To define the duration and its length you have to configure a run of the specified type in the scenario.

Following table shows the run options:

Run type Value description
timeTime duration in milliseconds
iterationNumber of iterations

Table 4.1.  Run options


The generating is performed in so called iterations. In a single iteration generator takes a sender from the sender pool, uses it to send all the messages specified in the scenario (See Section 4.5, “What - Messages” ) and after that it returns the sender back to the sender pool. The reporting module (See Section 4.7, “Reporting” ) treats this as a single iteration.

Example 4.1.  An example of the run configuration in a scenario:

  1    <run type="time" value="10000"/>
  2    <generator class="..." threads="...">
  3       ...
  4       (properties)
  5       ...
  6    </generator>

In the example above a scenario is defined to run for 10 seconds.

When specifying the generator class, unless you enter a fully classified class name, the default package org.perfcake.message.generator is assumed.

The following sections describes the generators that can be used in PerfCake to generate the load.


4.1.1. DefaultMessageGenerator

The generator is able to generate a load using multiple threads for a long period of time (matter of days, weeks,...). The generator uses a thread pool of a fixed size that stores threads executing performance iterations. The thread pool uses an internal queue where tasks implementing the iterations are buffered and scheduled for execution. The size of the thread queue can be specified by the senderTaskQueueSize property. The generator regularly fills the queue with new tasks each period specified by the monitoringPeriod property.

This approach guarantees that the maximum number of thread instances (that is equal to the value of the senderTaskQueueSize property) exists in the memory during all the time the generator is working.

To configure the generator we need to specify a number of concurrent threads (using the threads attribute) and a duration (using the run discussed above).

During a shutdown, the thread queue is regularly checked every period specified by the shutdownPeriod for the threads finishing their work. If the same amount of threads keeps running for this period, they are forcefully stopped. If the shutdownPeriod is set to the value of -1 it's value is auto-tuned at the end of the scenario execution to the value of 5 times the average response time.

The following table shows the properties of the DefaultMessageGenerator:

Property nameDescriptionRequiredDefault value
monitoringPeriodA period in milliseconds by which the thread queue is filled with new tasks.No1000
shutdownPeriodA period in milliseconds by which the thread queue is checked for the threads finishing their work during a shutdown.No1000
senderTaskQueueSizeThe size of the task queue.No1000

Table 4.2. DefaultMessageGenerator properties


Example 4.2. An example of DefaultMessageGenerator configuration

  1    <run type="time" value="60000"/>
  2    <generator class="DefaultMessageGenerator" threads="100">
  3       <property name="senderTaskQueueSize" value="5000"/>
  4    </generator>

In the example above a DefaultMessageGenerator is defined to run for 60 seconds using 100 concurrent threads with the sender task queue with the size of 5000. The main goal of limiting the queue of sender tasks is to limit memory usage and improve performance of PerfCake.


4.1.2. ConstantSpeedMessageGenerator

This generator is based on DefaultMessageGenerator (seeSection 4.1.1, “DefaultMessageGenerator”) and inherits all its configuration properties. It tries to achieve given speed of messages per second. Uses the underlying buffer of sender tasks ofDefaultMessageGenerator. This buffer smoothens the changes in the speed. If you need the generator to change its speed more aggressively, configure senderTaskQueueSize property.

The following table shows the properties of the ConstantSpeedMessageGenerator:

Property nameDescriptionRequiredDefault value
speedDesired constant speed in messages per second. Setting the speed to -1 makes the generator run at maximum possible speed like DefaultMessageGenerator.No5000

Table 4.3. ConstantSpeedMessageGenerator properties


Example 4.3. An example of ConstantSpeedMessageGenerator configuration

  1    <run type="time" value="60000"/>
  2    <generator class="ConstantSpeedMessageGenerator" threads="100">
  3       <property name="speed" value="10000"/>
  4    </generator>

In the example above a ConstantSpeedMessageGenerator is defined to run for 60 seconds using 100 concurrent threads with the desired constant speed of 10000 messages per second.


4.1.3. CustomProfileGenerator

This generator allows the users to specify custom profiles to configure message generating. A custom profile is simply a function that for the given time period of the performance test returns the required number of threads and speed (number of generated messages per second). By such a function, the process of generating messages can be freely configured as needed.

The generator is based on DefaultMessageGenerator (seeSection 4.1.1, “DefaultMessageGenerator”) and inherits all its properties.

Each generator configuration has to specify the threads attribute. In the case of CustomProfileGenerator, this specifies the maximum number of threads used during the whole performance test. The profile function must not request more threads than that. Otherwise some tasks for sending message won't have threads available and some requests will be skipped while producing an error message. Please be careful about this.

When the profile function returns higher number of threads or higher speed then for the previous time period, the change is applied immediatelly. However, when it returns lower number of threads, the existing threads first need to finish their tasks before they are removed from the pool. When a slower speed is requested, the generator first finishes the tasks that are already in the task queue. The length of this queue is set by the senderTaskQueueSize property. When the queue is too short, it runs out of tasks frequently and this has a negative impact on the performance results. When it is too long, it takes longer time for the change in speed to be reflected.

For example, when sending messages to a system that is capable to response in 10 milliseconds with 20 threads, PerfCake can process 1000 / 10 * 20 tasks per second. If the queue length is set to 2000 (1000 / 10 * 20 = 2000), it would take 1 second for the speed to change.

The profile function can have its limits and simply do not provide any more values after a certain boundary. For this purpose, the generator has an autoReplay property which instructs it to replay the profile function from the beginning.

The profile function must implement the Profile interface and its class name is passed to the generator. By default, the package org.perfcake.message.generator.profile is assumed. The most important method is getProfile() that returns the profile (number of threads and speed) for the given time period. It must always return the correct value, despite the order of requests to the method. The period passed to the method specifies the type of time information (either an iteration number or milliseconds since test start) and time. The profile function must also properly handle the autoReplay property.

For an easier development, there is an AbstractProfile class where the only responsibility of a developer is to read all the time points where there is a change in either thread count or speed in the profile function. All these points are registered with the addRequestEntry() method. The class then handles all the correct behaviour.

It is important to note that the last value in the profile function either remains until the end of the performance test (when autoReplay is set to false), or it is immediatelly overriden by the first value. This means that the values for the last entry should be the same as for the first one when autoReplay is turned on.

The first entry in the profile function should always start with 0 to set the conditions at the very beginning of the performance test. When there is no such entry starting with 0, the first entry is taken for the starting conditions. The order of registered entries does not matter and they are automatically sorted by their time property.

Whenever the speed is set to -1, the generator runs at maximal possible speed as DefaultMessageGenerator.

The following table shows the properties of the CustomProfileGenerator:

Property nameDescriptionRequiredDefault value
autoReplayInstructs the generator to replay the profile function from the beginning when we hit its end.Notrue
profileClassThe class name of the custom profile to use.Yes-
profileSourceThe source where the profile is specified. This is passed directly to the profile. Some profiles might not use this property.No-
senderTaskQueueSizeDetermines the length of the queue for sender tasks. Influences how quickly the generator can react to slow downs in speed.No1000

Table 4.4. CustomProfileGenerator properties


Example 4.4. An example of CustomProfileGenerator configuration

  1    <generator class="CustomProfileGenerator" threads="100">
  2       <property name="profileClass" value="CsvProfile"/>
  3       <property name="profileSource" value="test-profile.csv"/>
  4    </generator>

The following sections describe the provided profile functions.

CsvProfile

This profile reads the points of profile function from a CSV file. The format of a single line in the file is <time>;<threads>;<speed>.

The following example shows a ramp-up function that keeps the maximum values between the last steps and then starts over (supposing autoReplay is set to true).

Example 4.5. Sample profile specified in a CSV file

0;10;100
250;20;100
500;30;100
750;30;200
1000;30;300
9999;10;100

4.1.4. RampUpDownGenerator

The generator is based on the DefaultMessageGenerator (SeeSection 4.1.1, “DefaultMessageGenerator”) and inherits all its properties. In addition to this functionality, RampUpDownGenerator is able to change the number of threads during the execution. The number of threads evolution during execution is illustrated in Figure 4.1, “RampUpDownGenerator time chart” .

Figure 4.1. RampUpDownGenerator time chart


The scenario starts (A) with the number of threads set to the value of the preThreadCount property (1) . It continues to execute for the duration set by the preDuration property (A) -> (B) , which is called the PRE phase. When PRE phase ends (B) , the RAMP UP phase starts.

In the RAMP UP phase the number of the threads is changed by the value of the rampUpStep property each period set by the rampUpStepPeriod until it reaches the number of threads set by the value of the mainThreadCount property (2) .

In that moment (C) MAIN phase starts and the execution continues for the duration set by the mainDuration property (C) -> (D) , when the RAMP DOWN phase starts.

In the RAMP DOWN phase (D) -> (E) the number of threads is again changed but this time in the opposite direction than in the RAMP UP phase. It changes by the value of the rampDownStep property each period specified by the rampDownStepPeriod property until the final number of threads (3) is reached. By that moment (E) the final phase called POST starts.

The POST phase ends by the end of the scenario (F) .

The outer borders of the number of threads and the duration is set by the maximum number of threads specified by the threads attribute of the generator and by the maximum duration set by the run element.

The following table describes all the properties of the RampUpDownGenerator:

Property nameDescriptionRequiredDefault value
senderTaskQueueSizeThe size of the task queue.No1000
preThreadCountInitial number of threads.NoGenerator's threads value.
preDuration A duration period in the units of run type of "PRE" phase. NoLong.MAX_VALUE
rampUpStepA number by which the number of threads is changed in the "RAMP UP" phase.No0
rampUpStepPeriod A period in the units of run type after which the number of threads is changed by rampUpStep value. NoLong.MAX_VALUE
mainThreadCountA number of threads in the main phase.NoGenerator's threads value.
mainDuration A duration in the units of run type for which the main phase lasts. No0
rampDownStepA number by which the number of threads is changed in the "RAMP DOWN" phase.No0
rampDownStepPeriod A period in the units of run type after which the number of threads is changed by rampDownStep value. NoLong.MAX_VALUE
postThreadCountFinal number of threads.NoGenerator's threads value.

Table 4.5. RampUpDownGenerator properties


Example 4.6. An example of RampUpDownGenerator configuration

  1    <run type="time" value="60000"/>
  2    <generator class="RampUpDownGenerator" threads="20">
  3       <property name="preThreadCount" value="10"/>
  4       <property name="preDuration" value="10000"/>
  5       <property name="rampUpStep" value="2"/>
  6       <property name="rampUpStepPeriod" value="1000"/>
  7       <property name="mainThreadCount" value="20"/>
  8       <property name="mainDuration" value="20000"/>
  9       <property name="rampDownStep" value="1"/>
 10       <property name="rampDownStepPeriod" value="1000"/>
 11       <property name="postThreadCount" value="15"/>
 12    </generator>

In the example above the scenario starts with 10 threads ( preThreadCount ). After 10s ( preDuration ) the number of threads starts to increase by number of 2 ( rampUpStep ) each second ( rampUpStepPeriod ) until the number of threads reaches 20 ( mainThreadCount ). Than after another 20 ( mainDuration ) seconds the number of threads starts to decrease by 1 ( rampDownStep ) each second ( rampDownStepPeriod ) until the number reaches 15 ( postThreadCount ) which remains until the end of the scenario. The whole scenario ends after 60s ( run ).