10 April 2007 - 1.7 home user-guide eclipse jbossws intellij netbeans maven 1.X/2.X PDF files forums bugs sourceforge






Vote for soapUI at the WSJ Readers' Choice awards in the

'Best Web Services Utility' and

'Best Web Services Testing Tool'

categories

Welcome to soapUI 1.5

Overview

soapUI 1.5 contains a large number of small improvements and a small number of large ones, the latter mostly enhancing load/functional-testing of Web Services. This document outlines those few large ones, the small improvements will be covered in detail by the 1.5 documentation. If you have any problems please post a message to the mailing list or one of the sourceforge forums so we can fix it in the next beta.

soapUI 1.5 should work out of the box with 1.0.x projects, but you are advised to make backup of your projects to be on the safe side...

LoadTesting

Overview

soapUI 1.5 provides extensive load-testing functionality allowing you to perform:

Any number of LoadTests can be created for a TestCase (using the TestCases "New LoadTest" popup-menu action), each with different strategies, assertions, etc. to validate/assess a TestCases and its TestSteps performance under different circumstances. LoadTests can be run both interactively in soapUI or from the commandline, allowing scheduled loadtests to be used for performance surveillance (for example).

Right click on a TestCase and selected "New LoadTest" (you can create any number of loadtests), this will open the following LoadTest view after prompting for a title:

The view is divided into 3 parts (top-down):

  • A toolbar with general actions, test-limit, interactive thread-count and strategy settings
  • A statistics table showing the statistics for each step in the loadtest
  • A log showing general messages and any assertions errors

Assertions

The log at the bottom has an additional tab showing the configured loadtest assertions:

When running a LoadTest, the configured assertions are applied and will generate an error in the log if not met:

Assertion errors can be double-clicked and will show a view displaying the entire request/response that failed the assertion;

Limit, ThreadCount and Strategies

A LoadTest is controlled by its limit (by time or number of requests), threadcount and strategy. There are currently 4 available strategies:

  • Simple : allows the setting of a testcase delay and randomization factor. Use this for standard load/stress-testing, set both values to 0 for "full-throttle" testing.
  • Variance : varies the number of threads over time. Use this for monitoring behaviour under varying load
  • Burst : holds all threads for a specified time and runs them of a another specified time. Use this to simulate intermittent peak loads.
  • ThreadCount : steps the threadcount from a start to an end value over time. Use this to (for example) find which thread-count will give the best TPS (transactions per second) value

Since you can have multiple LoadTests for a TestCase, you can create several with different strategies / configurations. Also, you can run several tests simultaneously in soapUI so you can (for example) see how a Burst-strategy test affects the average time of a Simple strategy test

Diagrams

The toolbar actions include opening of 2 types of diagrams:

  • A statistics diagram showing all statistics for a selected teststep
  • A statistic value diagram showing one statistic value for all teststeps

Both these are meant to be used for behavioural analysis, ie seeing trends over time. Exact values/legends are not shown;

CommandLine Runner

A com.eviware.soapui.tools.SoapUILoadTestRunner is available for running loadtests from the commandline, the same arguments as described for the com.eviware.soapui.tools.SoapUILoadTestRunner are valid, with the introduction of -l<loadtest name>, -r for reporting and -f<folder name>. Failed results and error logs will be exported to text-files for viewing.

Functional Testing

Overview

Functional testing has been enhanced in a number of ways, most notably by the introduction of a Properties teststep which can be used for global properties and of a Grooy teststep which allows you to do more or less "anything" in a testcase, for example data-driven testing, notifications, external integrations, etc...

Minor enhancements include:

  • Much enhanced ValueTransfer teststep allowing transfer of properties between any other teststeps
  • TestCase options for setting session keeping, failure on error, etc.
  • A TestSuite viewer/runner (see below)
  • Possibility to loop a testcase continueously

Properties TestStep

The properties teststep allows you to define a number of properties which can further be loaded to/saved from a local file or URL (optionally specified by a system property). The defined properties can then be assigned/used from a groovy script or a ValueTransfer. The PropertiesStep editor is a simple table as follows:

Groovy Script TestStep

The GroovyScript teststep allows you to execute an arbitrary Groovy script in your testcase. The script has full access to the soapUI object-model, allowing it to modify other teststeps, get external data, etc.

The shown script sets the property "boss" of another teststep to a "random" value

TestSuite Runner

The TestSuite view (opened by double-clicking a TestSuite) allows you to run a selection of contained TestCases in sequence or parallel from within soapUI:

Other Notable Enhancements

Preferences

A preferences dialog invoked from the main menu allows you to set a number of general options:

Improved Logging

Logging has been color-coded and improved to include a HTTP wire log and a seperate log for groovy scripts: