4.8. Validation

Validation can be used to check if the response received by a sender is valid. It is optional and if it is used, it is performed in a gentle way (only once in a 0.5 sec) not to add any significant overhead and not to affect the measuring.

The validation is configured in a scenario in two places. First there is a kind of validator pool where all validators available in the scenario are placed. The validator pool is configured in the validation section of the scenario. Each validator has a unique ID by which it can be referenced. Once at least one of the validators is configured, it can be referenced by a message 's sub-element validatorRef, which is the second place the validation is configured. By adding a validator refecence to the messages we tell PerfCake that the response to that particular messages should be validated by that particular validator. Any message can have multiple validator references and each validator must be passed in order to set the response valid. Any validator can be referenced from more than one message.

The validation as a whole can be enabled or disabled by setting the optional boolean attribute called enabled on the validation element to the value of true. By default the validation is disabled.

The validation thread has some sleep for to wait between validations not to influence the measured results. At the end, when there is nothing else to do, it goes through the remaining responses faster. This behavior can be controlled via the optional fastForward attribute on the validation element. If the value is true the sleep periods are ignored and the validation goes as fast as possible. However, that influences the measured results. For that reason the default value is false.

Example 4.66. An example of validation configuration:

  1    <messages>
  2       ...
  3       <message uri="...">
  4          <validatorRef id="validator1"/>
  5          ...
  6       </message>
  7       <message uri="...">
  8          <validatorRef id="validator1"/>
  9          <validatorRef id="validator2"/>
 10          ...
 11       </message>
 12       ...
 13    </messages>
 14    ...
 15    <validation fastForward="false" enabled="true">
 16       <validator id="validator1" class="..."> 
 17          ...
 18       </validator>
 19       <validator id="validator2" class="..."> 
 20          ...
 21       </validator>
 22    </validation>

In the example above there are two validators (validator1 and validator2) and two messages configured. The first message has just the validator1 attached so the response message received to the first message will be validated just by the validator1.

On the other hand there is a second message that has both validators attached. So the response to the second message will be validated by both of the validators.

The validation is enabled and the fast validation mode will be activated once the performance test finishes.

When specifying the validator class, unless you enter a fully classified class name, the default package org.perfcake.validation is assumed.

The rest of the chapter will present all the available validators that can be used in PerfCake.


4.8.1. Validators

DictionaryValidator

Dictionary validator can create a dictionary of valid responses and use this to validate them in another run. It is also possible to create the dictionary manually, however, this is too complicated task and we always recommend running the validation in record mode first. Any manual changes can be done later.

The validator creates an index file and a separate file for each response. A writable directory must be specified using the dictionaryDirectory property. The default index file name can be redefined. The response file names are based on hash codes of the original messages. Empty, null or equal messages will overwrite the file but this is not the intended use of this validator. Index file is never overwritten, if you really insist on recreating it, please rename or delete the file manually (this is for safety reasons). It is not sufficient to store just the index as it is likely that the correct messages will be manually modified after they are recorded.

The idea of usage is to first activate the record mode, run the whole test, switch the record mode off and then reuse the created dictionaryDirectory with future test runs.

Following table shows the properties of the DictionaryValidator:

Property nameDescriptionRequiredDefault value
dictionaryDirectoryThe directory where the dictionary is/will be store.Yes-
recordActivates the record mode.Nofalse

Table 4.51. DictionaryValidator properties


Example 4.67. An example of DictionaryValidator configuration

  1    <validator id="rimmerValidator" class="DictionaryValidator">
  2       <property name="dictionaryDirectory" value="/home/rimmer/work/tests" />
  3       <property name="record" value="true" />
  4    </validator>

PrintingValidator

Just prints out the original message and its response for the validation by human eyes. Please note that the validation output is normally stored in a separate log file and not printed to all other logging output. All messages controlled by this validator are stated valid.

There are no configuration properties available to this validator.

Example 4.68. An example of ScriptValidator configuration

  1    <validator id="print" class="PrintingValidator" />

RegExpValidator

The text validator treats each rule line as a regular expression and checks if the response message matches that regular expression. The response message must match all lines in order to be successfully validated.

Note

Please note that the value of pattern is passed through a template engine. The benefit of this is the ability to use standard properties in the pattern like ${prop1} and @{prop2}. The disadvantage is that each backslah character used to escape an RegExp character needs to be escaped as well. In reality this means that instead of writing \. to escape the dot character, one need to enter \\..

Example 4.69. An example of RegExpValidator configuration

  1    <validator id="rimmerValidator" class="RegExpValidator">
  2       <property name="pattern">
  3          <text><![CDATA[.*"I'm a [Ff]ish"!\\.*]]></text>
  4       </property>
  5    </validator>

RulesValidator

This validator uses Drools [17] engine to assert the validator rules. The rules for this validator have their own DSL [18] in a form of human readable sentences. The rules are listed in a following listing:

Example 4.70. RulesValidator rules

  1 Message body contains "{string}".
  2 Message body equals "{string}".

Example 4.71. An example of RulesValidator configuration

  1    <validator id="fareWellValidator" class="RulesValidator">
  2       <property name="pattern">
  3          <text><![CDATA[Message body contains "Farewell World!".]]></text>
  4       </property>
  5    </validator>

ScriptValidator

Validates messages using Java Script Engine and the provided script. The script engine must be installed in the extensions directory. The original message is passed to the script in the originalMessage property, the response is inserted as message, the attributes used for the particular message instance are passed as attributes, and logger is passed as log, all using script bindings. The script return value is evaluated for validation success (true for passed, false for failed).

Following table shows the properties of the ScriptValidator:

Property nameDescriptionRequiredDefault value
engineThe name of the Java Scripting Engine (please do not confuse with JavaScript).Yes-
scriptThe code of the validation script. Takes priority over scriptFile.Either script or scriptFile must be present-
scriptFileThe location of the validation script file.Either script or scriptFile must be present-

Table 4.52. ScriptValidator properties


Example 4.72. An example of ScriptValidator configuration

  1    <validator id="groovyValidator" class="ScriptValidator">
  2       <property name="engine" value="groovy" />
  3       <property name="script">
  4          <![CDATA[
  5             log.info("Be groovy")
  6             return message.payload.toString().contains('Pepa')
  7          ]]>
  8       </property>
  9    </validator>



[18] Domain Specific Language