26 September 2007 - 1.7.6 home user-guide eclipse jbossws intellij netbeans maven 1.X/2.X PDF files forums bugs sourceforge






LoadTest Execution

When running a LoadTest soapUI internally creates a complete copy of the underlying TestCase for each thread, allowing each TestCase to maintain its own state and properties. Depending on which limit and strategy has been selected, loadtesting will proceed as configured until the LoadTest terminates due to one of the following:

  • It has reached its configured limit
  • It has been cancelled by the user with the "Cancel" button on the LoadTest toolbar
  • It has been cancelled by a LoadTest Assertion when the maximum number of allowed errors for that assertion has been passed

During execution, the following statistics are periodically collected and displayed in the Statistics Table:

  • min - the shortest time the step has taken
  • max - the longest time the step has taken
  • avg - the average time for the test step
  • last - the last time for the test step
  • cnt - the number of times the test step has been executed
  • tps - the number of transactions per second for the test step, see Calculation of TPS/BPS below.
  • bytes - the number of bytes processed by the test step
  • bps - the bytes per second processed by the test step
  • err - the number of assertion errors for the test step

Collection/calculation of this data is performed asynchronously (ie independently from the actual TestCase executions) in soapUI, so it should not affect the actual loadtesting performance.

The Total row shows the statistics for the TestCase itself.

Step Time calculation

soapUI internally uses the System.nanoTime() method for determining the actual time taken by a test-step. The following table describes how these times are calculated for the available teststep types:

Step TypeDuration
Request StepThe request duration is calculated depending on the settings in the LoadTest Options dialog (as described there).
Groovy ScriptThe actual script evaluation
Properties StepThe time to load/save the properties (if configured)
Delay StepThe steps delay
Property TransferThe time it took to transfer all properties
Goto StepThe time it took to evaluate the goto-conditions

LoadTest Assertions

When running a LoadTest, the configured assertions are applied and will generate an error in the log if not met:

Assertion errors can be double-clicked and will show a view displaying the underlying TestStep state that failed the assertion. For example the following will be shown for Request that failed an assertion:

Calculation of TPS/BPS

Depending on the setting of the "Calculate TPS.." option in the LoadTest Options dialog, TPS and BPS are calculates as follows:

  • NOT based on actual time passed (default):
    • TPS : (1000/avg)*threadcount, for example avg = 100 ms with ten threads will give a TPS of 100
    • BPS : (bytes/cnt) * TPS, ie the average number of bytes per request * TPS. For example a total number of received bytes of 1000000 for 100 requests with a TPS of 100 would give (100000/100 * 100) = 1000000 BPS
  • Based on actual time passed:
    • TPS : Time passed in TestCase / CNT / 1000, ie a TestCase that has run for 10 seconds hand handled 100 request will get a TPS of 10
    • BPS : Bytes / Time passed / 1000, ie a TestCase that has run for 10 seconds and handled 100000 bytes will get a BPS of 10000.
    Please note the time-passed value used is the time for the entire testcase, not each step for itself. This can bias the TPS/BPS values quite a lot for those teststeps that take a small amount of the TestCases total time

The option to base on actual time passed can be usefull when setting a testcase delay using the Simple LoadStrategy, which will generally give a lov average, but the actual transactions per second will not be equivalently high (since there is a delay). Selecting this option will calculate TPS using (time-passed/cnt) instead.


Next: LoadTest Assertions