10 April 2007 - 1.7 |
When running a LoadTest soapUI internally creates a complete copy of the underlying TestCase for each thread, allowing each TestCase to maintain its own state and properties. Depending on which limit and strategy has been selected, loadtesting will proceed as configured until the LoadTest terminates due to one of the following:
During execution, the following statistics are periodically collected and displayed in the Statistics Table:
min
- the shortest time the step has takenmax
- the longest time the step has takenavg
- the average time for the test steplast
- the last time for the test stepcnt
- the number of times the test step has been executedtps
- the number of transactions per second for the test step, see
Calculation of TPS/BPS below.bytes
- the number of bytes processed by the test stepbps
- the bytes per second processed by the test steperr
- the number of assertion errors for the test stepCollection/calculation of this data is performed asynchronously (ie independently from the actual TestCase executions) in soapUI, so it should not affect the actual loadtesting performance.
The Total row shows the statistics for the TestCase itself.
soapUI internally uses the System.nanoTime()
method for determining the actual time taken
by a test-step. The following table describes how these times are calculated for the available teststep types:
Step Type | Duration |
---|---|
Request Step | The request duration is calculated depending on the settings in the LoadTest Options dialog (as described there). |
Groovy Script | The actual script evaluation |
Properties Step | The time to load/save the properties (if configured) |
Delay Step | The steps delay |
Property Transfer | The time it took to transfer all properties |
Goto Step | The time it took to evaluate the goto-conditions |
When running a LoadTest, the configured assertions are applied and will generate an error in the log if not met:
Assertion errors can be double-clicked and will show a view displaying the underlying TestStep state that failed the assertion. For example the following will be shown for Request that failed an assertion:
Depending on the setting of the "Calculate TPS.." option in the LoadTest Options dialog, TPS and BPS are calculates as follows:
The option to base on actual time passed can be usefull when setting a testcase delay using the Simple LoadStrategy, which will generally give a lov average, but the actual transactions per second will not be equivalently high (since there is a delay). Selecting this option will calculate TPS using (time-passed/cnt) instead.