10 June 2009 - 3.0-beta-2 user guide eclipse intellij netbeans maven download nightly forum bugs blog sf.net eviware


Eviware Logo

Test Request Assertions

One of the major quality assurance features of soapUI is the ability assert the content of response messages. soapUI comes with a number of different Assertions in order to cover the needs of high quality Testing.

Hopefully these assertions make a useful testing toolbox that will allow Agile testing of Web Services. Currently the following assertions are available:

TypeDescription
Schema Compliance *Validates the response message against its xml schema
Simple ContainsChecks for the existence of a token
Simple Not ContainsChecks for the non existence of a token
SOAP Fault *Checks that the response is a soap fault
Not SOAP Fault *Checks that the response is not a soap fault
SOAP Response *Checks that the response is valid SOAP Response
Response SLA *Checks the response time to be under a specified value
XPath MatchMatches the result of a specified XPath expression against a predefined value
XQuery MatchMatches the result of a specified XQuery expression against a predefined value
Script AssertionAllows a custom Groovy script for asserting the message exchange
WS-Security Status *Checks that incoming WS-Security processing was successful
WS-Addressing Response AssertionChecks that the response has valid WS-A header properties
WS-Addressing Request AssertionChecks that the request has valid WS-A header properties

Assertions marked with a (*) are "singular assertions", meaning that they can be added only once to a TestRequest

Start soapUI Testing
Did you know?

The XPath Match assertion is greatly simplified in soapUI Pro. The XPath Editor simplifies and improves your XPath Assertions.

soapUI Pro contains productivity enhancements as well as offers Professional World Class support.

Try soapUI pro to soap test

Schema Compliance Assertion

The Schema Compliance assertion checks that the response is compliant with the response messages XML Schema definition. If not, a list of validation errors as those displayed in the validation-tab of the request editors is displayed and the assertion fails. The list of errors is also displayed in the assertion tab of the TestRequest Editor; errors are double clickable and will highlight the row of the validation error if possible.

The assertion has only one configuration-parameter which is prompted for when creating/configuring a schema compliance assertion; the URL to the WSDL definition to use for validation. This defaults to the URL of the definition for the containing TestRequests' Operations' Interface.

Schema Compliance validation is currently targeted at Basic-Profile compliant WSDL's/messages and thus only supports for literal-encoded messages (both rpc and document), validation of SOAP-encoded messages is not supported and will result in a validation error in any case.

Be aware of the fact that a SOAP-Fault will only be a schema validation if the detail element contains a message part that is defined in the corresponding binding and not compliant with its corresponding schema definition (see the SOAP-Fault assertion below regarding validations of SOAP-Fault response messages).

Simple Contains Assertion

The "Simple Contains" assertion simply checks for the existence of a specified substring in the received response. No xml-parsing/validation is performed at all

The assertion has 3 configuration-parameter which is prompted for when creating/configuring a simple-contains assertion:

Simple Contains Assertion Editor

The options are:

  • Content - the content to look for
  • Ignore Case - ignores case (!)
  • Regular Expression - evaluates token as regular expression, Read More on supported Regular Expression syntax

Simple NotContains Assertion

The Simple NotContains assertion simply checks for the non-existence of a specified substring in the received response. No xml-parsing/validation is performed at all.

The assertion has the same configuration parameters as described for the Contains assertions above.

SOAP Fault Assertion

The SOAP-Fault assertion simply checks that the received response is a SOAP Fault.

This assertion has no configuration parameters.

Not Soap Fault Assertion

The Not SOAP-Fault assertion simply checks that the received response is not a SOAP-Fault. This should be used in conjunction with the Schema Compliance since a SOAP Fault does not get validated against any schema (unless there is a Fault Part defined in the WSDL and that Fault Part is present in the response) This assertion has no configuration parameters.

SOAP Response Assertion

Validates that the response is a valid SOAP Message. This is the minimal assertion that should be added to catch empty responses or HTTP error pages. This assertion has no configuration parameters.

Response SLA Assertion

The Response SLA assertion simply checks that the response time is under the configured limit.

WS-Security Status Assertion

The WS-Security Status assertion checks that the incoming message has valid WS-Security headers.

XPath Match Assertion

The XPath Match assertion allows specification of an XPath expression to be evaluated against the received response message and compare its result to a predefined value. Expressions can select everything from attribute values, make Boolean evaluations or select the entire response body. (XmlUnit is used internally for comparing xml elements, nodes or hierarchies)

soapUI internally uses the Saxon 9 XPath processor which has support for both XPath 1.0 and XPath 2.0.

The configuration dialog for the XPath-match assertion is divided into 2 areas (see below); the top XPath area which contains the desired XPath expression and the bottom result area which contains the expected result.

Creating an XPath expression in the soapUI Pro Xpath Match editor

The toolbar for the top XPath area contains the following actions (left to right):

  • Select XPath (soapUI Pro only) - opens the XPath Selector for the current response
  • Declare - adds namespace declaration for all namespaces currently defined in the underlying response message to the XPath expression. Saxon uses this syntax for namespace declarations so you can later use them in your XPath expression.

And the toolbar for the bottom content-match area contains the following actions (left to right):

  • Select from current - evaluates the specified XPath expression against the current response message (if available) for the underlying request. The result is written into the result area of the configuration dialog.
  • Test - evaluates the specified XPath expression against the current response message (if available) for the underlying request and compares that result with the value specified in the result area. This is essentially the same comparison that is performed during a "real" assertion.
  • Allow Wildcard - Allows '*' wildcard in attribute and element values, these will be skipped during comparison (see tutorial below).
  • IgnoreNamespace Prefixes (not shown in screenshot) - will ignore differing namespace prefixes in comparison.
  • Select Content (soapUI Pro only) - Allows one to select an existing property whose value must match that of the selected XPath value (as in screenshot above).

The bottom button bar shows the follows:

  • Help - Opens this page in a windows
  • Save - saves the current XPath/result values and closes the dialog.
  • Cancel - discards the current XPath/result values and closes the dialog.
  • The dialog is modeless allowing you to move focus back to soapUI and for example select values from the underlying response message.

    Creating XPath Assertions

    The recommended way to create an XPath assertion is as follows:

    1. Submit the TestRequest and await its response (so you have response to test)
    2. Create the XPath Assertion and begin by adding namespace declarations to the XPath expression with the "Declare" button
    3. Now add the actual XPath expression after the namespace declarations, count(//ns1:Item) in the screenshot above
    4. Use the "Select from current" button to evaluate the XPath expression against the available result and check that it returns what you expect; 10 in the screenshot above
    5. Double-check by pressing the "Test" button which should return an OK message since the test will compare the XPath selection result with the previously selected value

    With soapUI Pro the creation is greatly simplified through the Add Xpath Assertion Wizard which performs step 2 and forward through a simple right-click action.

    Creating XPath Assertions with Wild cards

    The wild card feature allows for assertion of large blocks of xml where selected attribute/element-values are ignored during comparison. Follow steps 1-4 above but specify an XPath expression that returns a block of xml. For example an Amazon Search Response returns a descriptive "OperationResponse" element which can be selected with the following XPath-expression:

    declare namespace ns1='http://webservices.amazon.com/AWSECommerceService/2006-11-14';
    //ns1:ItemSearchResponse/ns1:OperationRequest

    After using the "Select from current" button, the Expected Result field will contain the following:

    <OperationRequest xmlns="http://webservices.amazon.com/AWSECommerceService/2006-11-14" ...>
      <HTTPHeaders>
        <Header Name="UserAgent" Value="Jakarta Commons-HttpClient/3.0.1"/>
      </HTTPHeaders>
      <RequestId>*</RequestId>
      <Arguments>
        <Argument Name="Service" Value="AWSECommerceService"/>
      </Arguments>
      <RequestProcessingTime>*</RequestProcessingTime>
    </OperationRequest>

    Since the UserAgent@Value, RequestId and RequestProcessingTime will vary between responses, we want to ignore them in our assertion; check the "Allow Wildcards" checkbox and replace their values with a '*'. The Expected Result field should now look as follows:

    XPath Matching with wildcards in soapUI Pro

    and in following test-runs these values will be ignored.

    XPath Assertion Examples

    Here are some examples expressions for a search-result from the Amazon web-service returning 10 hits when searching on books with the word "Oxford" in their title. The following assertions could be created;

    Validate that the result contains 10 hits

    declare namespace ns1='http://webservices.amazon.com/AWSECommerceService/2005-07-26';
    count(//ns1:Item)

    which will return "10".

    Validate that the ProductGroup element always is "Book":

    declare namespace ns1='http://webservices.amazon.com/AWSECommerceService/2005-07-26';
    count(//ns1:ProductGroup[text()!='Book'])

    which will return "0".

    Validate that DetailPageURL always starts with 'http://www.amazon.com/exec/obidos':

    declare namespace ns1='http://webservices.amazon.com/AWSECommerceService/2005-07-26';
    count(//ns1:DetailPageURL) = count(//ns1:DetailPageURL[starts-with(text(), 'http://www.amazon.com/exec/obidos')])

    which will return "true"

    XQuery Match Assertion

    The XQuery Match assertions is configured exactly like the XPath Math assertion described above, the only difference being that the specified expression is evaluated as an XQuery 2.0 expression instead.

    XQuery MAtch Editorin soapUI Pro

    This assertion is useful when you want to assert a subset of your data and for example not be dependent on other data, the data order, etc. In the example in the screenshot, a list of ordered ids is created from the response and compared against a predefined list, resulting in an assertion that will not fail if the items come in another order and/or get more/other data "around" the id element.

    Script Assertion

    The Script Assertion allows for arbitrary validations (see examples below). When creating or double clicking a script assertion, a Groovy script editor is shown as follows;

    The assertion script can be run against the latest message exchange with the run button in the top left

    The script has access to the following objects:

    • messageExchange : the MessageExchange for the current request/response. Gives direct access to message content, HTTP Headers, Attachments, etc.
    • context : the TestRunContext running the current TestCase, this will for now always be an instance of WsdlTestRunContext
    • log : a standard log4j Logger object available for logging arbitrary information

    An assertion script should throw an Exception with the failure message to fail the assertion. One can also use the built in Groovy assert statement for an easy assertion syntax as shown in the examples below. If the assertion is valid, either return nothing or a status message that will be shown in the TestCase Log.

    Assertion Script Examples

    Validate a certain response time:

    assert messageExchange.timeTaken < 400

    Validate existence of a response HTTP Header:

    assert messageExchange.responseHeaders["x-amz-id-1"] != null

    Validate the existence of a specific element using GroovyUtils (although this would be easier with a standard XPath-Contains assertion):

    def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
    def holder = groovyUtils.getXmlHolder( messageExchange.responseContent )
    
    assert holder["//ns1:RequestId"] != null

    With soapUI Pro the creation is greatly simplified through the Add Script Assertion Wizard

    WS-Addressing Response Assertion

    Validates that the response has valid WS-A header properties. When creating or double clicking a ws-addressing response assertion, configuration panel is opened where you can choose which properties are to be validated.

    WS-Addressing Request Assertion

    Validates that the request has valid WS-A header properties. When creating or double clicking a ws-addressing request assertion, configuration panel is opened where you can choose which properties are to be validated.


    Next: soapUI Pro Request and Response Wizards