PerfCake is a command line tool so it provides a CLI (command line interface) for executing a PerfCake scenario (see Section 2.2, “Performance scenario definition” ). The main script can be found in $PERFCAKE_HOME/bin/perfcake.sh for Linux OS or %PERFCAKE_HOME%\bin\perfcake.bat in case of Windows OS.

If the script is run without arguments, from the following help that is printed you can see the arguments that can be used to control the scenario execution.

[PERFCAKE_HOME]$ ./bin/perfcake.sh
=== Welcome to PerfCake 7.5 ===
usage: ScenarioExecution -s <SCENARIO> [options] [-D<property=value>]*
 -d,--debug                                start debug JMX agent for
                                           external monitoring
 -D <property=value>                       system properties
 -dn,--debug-name <AGENT_NAME>             debug agent name in the JMX
                                           tree
 -h,--help                                 prints help/usage
 -log,--log-level <LOG_LEVEL>              logging level
 -md,--messages-dir <MESSAGES_DIR>         directory for messages
 -pd,--plugins-dir <PLUGINS_DIR>           directory for plugins
 -pf,--properties-file <PROPERTIES_FILE>   custom system properties file
 -r,--replay <RAW_FILE>                    raw file to be replayed
 -s,--scenario <SCENARIO>                  scenario to be executed
 -sd,--scenarios-dir <SCENARIOS_DIR>       directory for scenarios
 -skip,--skip-timer-benchmark              skip system timer benchmark

The only mandatory argument is -s (or --scenario ), that specifies the name of the scenario to be executed. PerfCake will look for the scenario definition in the file called <SCENARIO>.xml that is placed under the directory that is located at $PERFCAKE_HOME/resources/scenarios . The path to the directory with scenarios can be specified by the -sd (or --scenarios-dir ) argument.

If the scenario is configured to send any message (see Section 4.5, “What - Messages” ), it will look under the directory that is by placed at $PERFCAKE_HOME/resources/messages . That location can be specified by the -md (or --messages-dir ) argument.

If you need to set one or more system properties for scenario (see Section 2.2.5, “Filtering properties” ), there are two ways. You can either use -Dproperty=value like arguments or specify a path to the property file using the -pf (or --properties-file ) argument.

Following table describes all the PerfCake CLI arguments.

Some useful properties can be configured using the -Dproperty=value parameter. These are not so common so there is no dedicated short parameter variant. The following table lists the other properties recognized by PerfCake. None of them is required.

PerfCake is provided with a Maven plugin which allows to run PerfCake scenario within Maven build. This makes perfomance test automation more easy and encourages you to run performance tests on a regular basis (e.g. within your favourite CI server). Running performance test on a regular basis allows you to spot performance drops very soon and thus makes it much easier to identify which commit has caused performance regression.

Currently, the plugin has only one goal scenario-run, which runs specified PerfCake scenario. By default, this goal is executed in integration-test phase (i.e. assumes, that you deploy/start your application in pre-integration-test phase and shut it down in post-integration-test phase).

The only mandatory parameter is <scenario>, which specifies the name of the scenario to be run. Optionally, you can also specify <scenarios-dir>, <messages-dir> and <plugins-dir>, which specify paths to directories containing scenarios, messages and plugins, respectively. If you don't setup these parameters, plugin assumes, that appropriate directories (scenarios, messages, plugins) are in src/test/resources/perfcake. This can be switched to src/main/resources/perfcake by configuring <use-test-resources> to false. If any of these directories does not exists, plugin will use src/test/resources (or src/main/resources) as a fallback value for missing parameter.

The PerfCake Maven plugin requires log4j2 configuration file for PerfCake to report properly. By default it assumes the file to be located in project root and named log4j2.xml. This can be changed by the <log4j2-config> tag. The PerfCake logging level can be configured with <log-level> tag and a file with additional configuration properties can be specified by <properties-file>.

The PerfCake version used with the plugin is always the same as plugin version. Both jar files are released together.

   <properties>
      <perfcake.version>7.5</perfcake.version>
      ...
   </properties>
   ...
   <build>
      <plugins>
         <plugin>
            <groupId>org.perfcake.maven</groupId>
            <artifactId>perfcake-maven-plugin</artifactId>
            <version>${perfcake.version}</version>
            <configuration>
               <scenario>my_perfcake_scenario</scenario>
               <log4j2-config>src/test/resources/log4j2.xml</log4j2-config>
            </configuration>
            <executions>
               <execution>
                  <id>perfcake-scenario-run</id>
                  <goals>
                     <goal>scenario-run</goal>
                  </goals>
               </execution>
            </executions>
            <dependencies>
               <dependency>
                  <groupId>org.perfcake</groupId>
                  <artifactId>perfcake</artifactId>
                  <version>${perfcake.version}</version>
                  <scope>test</scope>
               </dependency>
            </dependencies>
         </plugin>
      ...
      </plugins>
   </build>

By default, PerfCake comes configured for a production environment. Logging level is set to info and validation messages are logged in a separate file. You can change the default logging level for PerfCake by a command line parameter -log (see Section 2.3.1, “Command line parameters” for more details). Possible logging levels are: all, trace, debug, info, warn, error, fatal, off.

More tweaking can be done in the log4j2.xml file. It is out of the scope of this guide to describe this configuration as this is a standard Log4j2 configuration file and one can find its own documentation.

It is possible to record raw results data from a performance test run and later replay them. This can be useful when we use a remote machine to run the test and we need to process the data on a different location or when we want to fine-tune the reporting facilities and we cannot or don't want to rerun the real test many times.

To record the test data, you can use the section called “RawReporter”. This will create a single output file with the recorded data. The file can be of a significant size depending on the number of test iterations. From our experience, a test with 100,000 iterations results in afile of size approx 4-8MB (depending on test meta-data available, sequences used etc.).

The file can be later replayed by providing the corresponding scenario with a different configuration of reporting section and passing the recorded data file to PerfCake in the --replay parameter.

> $PERFCAKE_HOME/bin/perfcake.sh -s http-reporting -r perfcake-measurement-1465935794498.raw 
2016-06-14 22:37:20,359 INFO  {org.perfcake.ScenarioExecution} === Welcome to PerfCake 7.5 ===
2016-06-14 22:37:20,699 INFO  {org.perfcake.scenario.ScenarioBuilder} Scenario configuration: file:/home/perfcake/perfcake-7.5/resources/scenarios/http-reporting.xml
2016-06-14 22:37:20,660 INFO  {org.perfcake.ScenarioExecution} Replaying raw results recorded in perfcake-measurement-1465935794498.raw.
[0:00:00][1 iterations][3%] [319.848264 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 319.848264 ms]
[0:00:01][59 iterations][14%] [179.383135 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 207.6054971694915 ms]
[0:00:02][128 iterations][24%] [148.827863 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 179.01238675000008 ms]
[0:00:03][198 iterations][34%] [138.458243 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 167.52745448989901 ms]
[0:00:04][265 iterations][44%] [172.478225 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 161.4592350792453 ms]
[0:00:05][315 iterations][54%] [232.630756 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 167.65647934285724 ms]
[0:00:06][369 iterations][64%] [170.956762 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 170.2701377127373 ms]
[0:00:07][421 iterations][74%] [205.153517 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 173.20733971496443 ms]
[0:00:08][470 iterations][85%] [201.284832 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 176.6280006106384 ms]
[0:00:09][520 iterations][95%] [232.02807 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 179.5895076057694 ms]
[0:00:10][536 iterations][100%] [196.740767 ms] [warmUp => false] [Threads => 10] [ResponseSize => 0.00 B] [failures => 0] [RequestSize => 0.00 B] [Average => 181.79292323694048 ms]
2016-06-14 22:37:21,058 INFO  {org.perfcake.ScenarioExecution} === Goodbye! ===

The Debug Agent is an experimental feature suited for performance scenario development. It has a significant performance drawback and must not be used for real testing.

This agent publishes status of PerfCake components via JMX and can be read by utilities like jvisualvm (with MBean plugin).

The agent can be started by configuring the system property perfcake.debug to true or by passing the -d or --debug command line options. For the agent to work properly, the JDK's tools.jar must be on the classpath. This necessity will be removed in Java 9. The PerfCake startup scripts try to configure the classpath accordingly but they still might fail. As a workaround, you might copy the tools.jar file to ${PERFCAKE_HOME}/lib/ext.

When there are multiple PerfCake instances running in the same JVM, they can be differentiated by setting perfcake.debug.name property (or -dn, --debug-name command line options). The default agent name is perfcake-1.

The main purpose of the Debug Agent is for IDE plugin integration and debugging performance scenarios. The feature state will be claimed stable once there is enough testing with the IDE plugins and probably when we can remove the tools.jar dependency.

Figure 2.2. JMX tree with PerfCake debug information in jvisualvm