09 July 2009 - 3.0 user guide eclipse intellij netbeans maven download nightly forum bugs blog sf.net eviware


Eviware Logo

LoadTest Execution

When running a LoadTest soapUI internally creates a complete copy of the underlying TestCase for each thread, allowing each TestCase to maintain its own state and properties. Depending on which limit and strategy has been selected, load testing will proceed as configured until the LoadTest terminates due to one of the following:

  • It has reached its configured Load Test limit
  • It has been canceled by the user with the "Cancel" button on the LoadTest toolbar
  • It has been canceled by a LoadTest Assertion when the maximum number of allowed errors for that assertion has been passed

During execution, the following statistics are periodically collected and displayed in the Statistics Table:

  • min - the shortest time the step has taken
  • max - the longest time the step has taken
  • avg - the average time for the test step
  • last - the last time for the test step
  • cnt - the number of times the test step has been executed
  • tps - the number of transactions per second for the test step, see Calculation of TPS/BPS below.
  • bytes - the number of bytes processed by the test step
  • bps - the bytes per second processed by the test step
  • err - the number of assertion errors for the test step

Collection and calculation of this data is performed asynchronously (i.e. independently from the actual TestCase executions) in soapUI, so it should not affect the actual load testing performance.

The Total row shows the statistics for the TestCase itself.

LoadTest Step Time calculation

soapUI internally uses the System.nanoTime() method for determining the actual time taken by a test-step. The following table describes how these times are calculated for the available TestStep types:

Step TypeDuration
Request StepThe request duration is calculated depending on the settings in the LoadTest Options dialog (as described there).
Groovy ScriptThe actual script evaluation
Properties StepThe time to load/save the properties (if configured)
Delay StepThe steps delay
Property TransferThe time it took to transfer all properties
Goto StepThe time it took to evaluate the goto-conditions

LoadTest Assertions

When running a LoadTest, the configured assertions are applied and will generate an error in the log if not met:

LoadTest Assertion error log

Assertion errors can be double clicked and will show a view displaying the underlying TestStep state that failed the assertion. For example the following will be shown for Request that failed an assertion:

Request Properties for a failet Load Test Assertion

Calculation of TPS/BPS

Depending on the setting of the "Calculate TPS.." option in the LoadTest Options dialog, TPS and BPS are calculates as follows:

  • NOT based on actual time passed (default):
    • TPS : (1000/avg)*threadcount, for example avg = 100 ms with ten threads will give a TPS of 100
    • BPS : (bytes/cnt) * TPS, i.e. the average number of bytes per request * TPS. For example a total number of received bytes of 1000000 for 100 requests with a TPS of 100 would give (100000/100 * 100) = 1000000 BPS
  • Based on actual time passed:
    • TPS : Time passed in TestCase / CNT / 1000, i.e. a TestCase that has run for 10 seconds hand handled 100 request will get a TPS of 10
    • BPS : Bytes / Time passed / 1000, i.e. a TestCase that has run for 10 seconds and handled 100000 bytes will get a BPS of 10000.
    Please note the time-passed value used is the time for the entire TestCase, not each step for itself. This can bias the TPS/BPS values quite a lot for those TestSteps that take a small amount of the TestCases total time

The option to base on actual time passed can be useful when setting a TestCase delay using the Simple LoadStrategy, which will generally give a low average, but the actual transactions per second will not be equivalently high (since there is a delay). Selecting this option will calculate TPS using (time-passed/cnt) instead.


Next: Web Service Test Assertions