Table of Contents
Generator is an object which uses Sender objects to send and receive messages. Each
generator represents a method or technique how the load (sending the messages) is handled. It
is capable of generating the load for a specified duration which can be an amount of time or a
number of iterations using a specified number of concurrent threads. To define the duration
and its length you have to configure a run of the specified type in the scenario.
Following table shows the run options:
| Run type | Value description |
|---|---|
| time | Time duration in milliseconds |
| iteration | Number of iterations |
Table 4.1. Run options
The generating is performed in so called iterations. In a single iteration generator
takes a sender from the sender pool, uses it to send all the messages specified in the
scenario (See Section 4.5, “What - Messages” ) and after that it returns the
sender back to the sender pool. The reporting module (See Section 4.7, “Reporting” ) treats this as a single iteration.
Example 4.1.
An example of the
run
configuration in a scenario:
1 <run type="time" value="10000"/> 2 <generator class="..." threads="..."> 3 ... 4 (properties) 5 ... 6 </generator>
In the example above a scenario is defined to run for 10 seconds.
When specifying the generator class, unless you enter a fully classified class name, the
default package org.perfcake.message.generator is assumed.
The following sections describes the generators that can be used in PerfCake to generate the load.
The generator is able to generate a load using multiple threads for a long period of time
(matter of days, weeks,...). The generator uses a thread pool of a fixed size that stores
threads executing performance iterations. The thread pool uses an internal queue where tasks
implementing the iterations are buffered and scheduled for execution. The size of the thread
queue can be specified by the senderTaskQueueSize property. The generator
regularly fills the queue with new tasks each period specified by the
monitoringPeriod property.
This approach guarantees that the maximum number of thread instances (that is equal to the
value of the senderTaskQueueSize property) exists in the memory during all the
time the generator is working.
To configure the generator we need to specify a number of concurrent threads (using the
threads attribute) and a duration (using the run discussed
above).
During a shutdown, the thread queue is regularly checked every period specified by the
shutdownPeriod for the threads finishing their work. If the same amount of
threads keeps running for this period, they are forcefully stopped. If the
shutdownPeriod is set to the value of -1 it's value is auto-tuned
at the end of the scenario execution to the value of 5 times the average response time.
The following table shows the properties of the DefaultMessageGenerator:
| Property name | Description | Required | Default value |
|---|---|---|---|
| monitoringPeriod | A period in milliseconds by which the thread queue is filled with new tasks. | No | 1000 |
| shutdownPeriod | A period in milliseconds by which the thread queue is checked for the threads finishing their work during a shutdown. | No | 1000 |
| senderTaskQueueSize | The size of the task queue. | No | 1000 |
Table 4.2. DefaultMessageGenerator properties
Example 4.2. An example of DefaultMessageGenerator configuration
1 <run type="time" value="60000"/> 2 <generator class="DefaultMessageGenerator" threads="100"> 3 <property name="senderTaskQueueSize" value="5000"/> 4 </generator>
In the example above a
DefaultMessageGenerator
is defined to run for 60
seconds using 100 concurrent threads with the sender task queue with the size of 5000. The
main goal of limiting the queue of sender tasks is to limit memory usage and improve
performance of PerfCake.
This generator is based on DefaultMessageGenerator (seeSection 4.1.1, “DefaultMessageGenerator”) and inherits
all its configuration properties. It tries to achieve given speed of messages per second. Uses
the underlying buffer of sender tasks ofDefaultMessageGenerator. This buffer
smoothens the changes in the speed. If you need the generator to change its speed more
aggressively, configure senderTaskQueueSize property.
The following table shows the properties of the ConstantSpeedMessageGenerator:
| Property name | Description | Required | Default value |
|---|---|---|---|
| speed | Desired constant speed in messages per second. Setting the speed to -1
makes the generator run at maximum possible speed like
DefaultMessageGenerator. | No | 5000 |
Table 4.3. ConstantSpeedMessageGenerator properties
Example 4.3. An example of ConstantSpeedMessageGenerator configuration
1 <run type="time" value="60000"/> 2 <generator class="ConstantSpeedMessageGenerator" threads="100"> 3 <property name="speed" value="10000"/> 4 </generator>
In the example above a
ConstantSpeedMessageGenerator
is defined to run for 60
seconds using 100 concurrent threads with the desired constant speed of 10000 messages per second.
This generator allows the users to specify custom profiles to configure message generating. A custom profile is simply a function that for the given time period of the performance test returns the required number of threads and speed (number of generated messages per second). By such a function, the process of generating messages can be freely configured as needed.
The generator is based on DefaultMessageGenerator (seeSection 4.1.1, “DefaultMessageGenerator”) and inherits
all its properties.
Each generator configuration has to specify the threads attribute. In the case
of CustomProfileGenerator, this specifies the maximum number of threads used
during the whole performance test. The profile function must not request more threads than
that. Otherwise some tasks for sending message won't have threads available and some requests
will be skipped while producing an error message. Please be careful about this.
When the profile function returns higher number of threads or higher speed then for the
previous time period, the change is applied immediatelly. However, when it returns lower
number of threads, the existing threads first need to finish their tasks before they are
removed from the pool. When a slower speed is requested, the generator first finishes the
tasks that are already in the task queue. The length of this queue is set by the
senderTaskQueueSize property. When the queue is too short, it runs out of
tasks frequently and this has a negative impact on the performance results. When it is too
long, it takes longer time for the change in speed to be reflected.
For example, when sending messages to a system that is capable to response in 10
milliseconds with 20 threads, PerfCake can process 1000 / 10 * 20 tasks per
second. If the queue length is set to 2000 (1000 / 10 * 20 = 2000), it would take
1 second for the speed to change.
The profile function can have its limits and simply do not provide any more values after a
certain boundary. For this purpose, the generator has an autoReplay property
which instructs it to replay the profile function from the beginning.
The profile function must implement the Profile interface and its class name
is passed to the generator. By default, the package
org.perfcake.message.generator.profile is assumed. The most important method
is getProfile() that returns the profile (number of threads and speed) for the
given time period. It must always return the correct value, despite the order of requests to
the method. The period passed to the method specifies the type of time information (either an
iteration number or milliseconds since test start) and time. The profile function must also
properly handle the autoReplay property.
For an easier development, there is an AbstractProfile class where the only
responsibility of a developer is to read all the time points where there is a change in either
thread count or speed in the profile function. All these points are registered with the
addRequestEntry() method. The class then handles all the correct
behaviour.
It is important to note that the last value in the profile function either remains until
the end of the performance test (when autoReplay is set to false), or it is
immediatelly overriden by the first value. This means that the values for the last entry
should be the same as for the first one when autoReplay is turned on.
The first entry in the profile function should always start with 0 to set the conditions at the very beginning of the performance test. When there is no such entry starting with 0, the first entry is taken for the starting conditions. The order of registered entries does not matter and they are automatically sorted by their time property.
Whenever the speed is set to -1, the generator runs at maximal possible speed as
DefaultMessageGenerator.
The following table shows the properties of the CustomProfileGenerator:
| Property name | Description | Required | Default value |
|---|---|---|---|
| autoReplay | Instructs the generator to replay the profile function from the beginning when we hit its end. | No | true |
| profileClass | The class name of the custom profile to use. | Yes | - |
| profileSource | The source where the profile is specified. This is passed directly to the profile. Some profiles might not use this property. | No | - |
| senderTaskQueueSize | Determines the length of the queue for sender tasks. Influences how quickly the generator can react to slow downs in speed. | No | 1000 |
Table 4.4. CustomProfileGenerator properties
Example 4.4. An example of CustomProfileGenerator configuration
1 <generator class="CustomProfileGenerator" threads="100"> 2 <property name="profileClass" value="CsvProfile"/> 3 <property name="profileSource" value="test-profile.csv"/> 4 </generator>
The following sections describe the provided profile functions.
This profile reads the points of profile function from a CSV file. The format of a
single line in the file is <time>;<threads>;<speed>.
The following example shows a ramp-up function that keeps the maximum values between the
last steps and then starts over (supposing autoReplay is set to
true).
Example 4.5. Sample profile specified in a CSV file
0;10;100 250;20;100 500;30;100 750;30;200 1000;30;300 9999;10;100
The generator is based on the DefaultMessageGenerator (SeeSection 4.1.1, “DefaultMessageGenerator”) and inherits
all its properties. In addition to this functionality, RampUpDownGenerator is
able to change the number of threads during the execution. The number of threads evolution
during execution is illustrated in Figure 4.1, “RampUpDownGenerator time chart” .
The scenario starts (A) with the number of threads set
to the value of the preThreadCount property (1) . It continues to execute for the duration set by the
preDuration property (A) -> (B) , which is called the PRE phase. When
PRE phase ends (B) , the
RAMP UP phase starts.
In the RAMP UP phase the number of the threads is changed by the value
of the rampUpStep property each period set by the rampUpStepPeriod
until it reaches the number of threads set by the value of the mainThreadCount
property (2) .
In that moment (C)
MAIN phase starts and the execution continues for the duration set by the
mainDuration property (C) -> (D) , when the RAMP DOWN phase starts.
In the RAMP DOWN phase (D) ->
(E) the number of threads is again changed but this time
in the opposite direction than in the RAMP UP phase. It changes by the
value of the rampDownStep property each period specified by the
rampDownStepPeriod property until the final number of threads (3) is reached. By that moment (E) the final phase called POST starts.
The POST phase ends by the end of the scenario (F) .
The outer borders of the number of threads and the duration is set by the maximum number of
threads specified by the threads attribute of the generator and by the maximum
duration set by the run element.
The following table describes all the properties of the RampUpDownGenerator:
| Property name | Description | Required | Default value |
|---|---|---|---|
| senderTaskQueueSize | The size of the task queue. | No | 1000 |
| preThreadCount | Initial number of threads. | No | Generator's threads value. |
| preDuration | A duration period in the units of run type of "PRE" phase. | No | Long.MAX_VALUE |
| rampUpStep | A number by which the number of threads is changed in the "RAMP UP" phase. | No | 0 |
| rampUpStepPeriod | A period in the units of run type after which the number of
threads is changed by rampUpStep value. | No | Long.MAX_VALUE |
| mainThreadCount | A number of threads in the main phase. | No | Generator's threads value. |
| mainDuration | A duration in the units of run type for which the main phase
lasts. | No | 0 |
| rampDownStep | A number by which the number of threads is changed in the "RAMP DOWN" phase. | No | 0 |
| rampDownStepPeriod | A period in the units of run type after which the number of
threads is changed by rampDownStep value. | No | Long.MAX_VALUE |
| postThreadCount | Final number of threads. | No | Generator's threads value. |
Table 4.5. RampUpDownGenerator properties
Example 4.6. An example of RampUpDownGenerator configuration
1 <run type="time" value="60000"/> 2 <generator class="RampUpDownGenerator" threads="20"> 3 <property name="preThreadCount" value="10"/> 4 <property name="preDuration" value="10000"/> 5 <property name="rampUpStep" value="2"/> 6 <property name="rampUpStepPeriod" value="1000"/> 7 <property name="mainThreadCount" value="20"/> 8 <property name="mainDuration" value="20000"/> 9 <property name="rampDownStep" value="1"/> 10 <property name="rampDownStepPeriod" value="1000"/> 11 <property name="postThreadCount" value="15"/> 12 </generator>
In the example above the scenario starts with 10 threads ( preThreadCount
). After 10s ( preDuration ) the number of threads starts to increase by
number of 2 ( rampUpStep ) each second ( rampUpStepPeriod ) until
the number of threads reaches 20 ( mainThreadCount ). Than after another 20 (
mainDuration ) seconds the number of threads starts to decrease by 1 (
rampDownStep ) each second ( rampDownStepPeriod ) until the
number reaches 15 ( postThreadCount ) which remains until the end of the
scenario. The whole scenario ends after 60s ( run ).