15 January 2008 - 2.0.1 home user-guide eclipse intellij netbeans maven PDF files forums bugs sourceforge eviware.com






WSDL Coverage

soapUI Pro introduces the unique concept of WSDL Coverage, which dynamically allows you to analyze how much of a given Interface/WSDL is being covered by your Functional Tests, MockServices, Mock Clients and SOAP Monitor clients and targets. This document aims at giving a brief overview of the concepts of WSDL Coverage, with links to the corresponding documentation for each Coverage point in soapUI. The best way to learn is to play around with it!

Coverage for Web Services

Traditionally in (for example) a Java environment, coverage is used to measure how much of the actual code-base that is being used during a given execution scenario (usually during unit-tests). The returned value tries to give an estimate of how much of the code-base that is actuality being used/tested.

Web Services in soapUI are defined by a WSDL contract, which specifies the operations, message, faults, etc... that make up the web services functionality. For each message there is generally a finite amount of defined elements/attributes that can be used to convey the message content, divided among message bodies, headers and faults.

The WSDL Coverage functionality analyzes each messages contract definition and then compares all corresponding messages during (for example) a TestRun to calculate how many of the defined elements/attributes that were actually used during the tests. The result is in soapUI presented as both a percentage and a visualization of the actual message; "Message Coverage"

Further, when running functional tests, soapUI analyzes the performed XPath assertions, and will try to calculate a corresponding value of how many of the defined elements/attributes that have actually been asserted. This result is added to the Message Coverage presentation and called "Assertion Coverage".

Coverage Visualization

In all situations where Coverage can be calculated in soapUI, a "Coverage Panel" is available with the following basic layout:

  • A toolbar at the top for enabling/clearing Coverage, setting Coverage Options and exporting a Coverage Report
  • A Coverage TreeTable to the left containing the same items as the containing project together with calculated coverage percentage/color bar for the item.
  • Two inspectors to the right containing either a Coverage Visualization for the selected operations request/response ("Message Coverage") or the actual Message Content for the selected request/response in the tree.
  • Optional Inspectors at the bottom, depending on measurement point:
    • Run Log - Shows a log of the
    • Assertion Results : available when running Test Coverage; contains a table of the performed XPath Assertions during the tests. Select a TestSuite, TestCase or TestStep to narrow down the assertions, or select an assertion in the table to navigate to that TestStep assertion in the tree.

The Coverage Tree

The Coverage Tree displayed to the left is divided into 2 parts:

  1. The Interface Part displays the current projects Interfaces and their Operations. For each operation the defined Request and Response header/body/faults/attachments messages are shown. The displayed Coverage values are aggregated "upwards" in the tree, for example in the image above, the search Operations' Coverage is 8/9, which is the aggregate of the corresponding Request and Response message Coverage values. The values show that the Response Faults was not covered during the Test Run (0/1)
  2. The TestSuite part displays the current projects' TestSuites, TestCases and relevant TestSteps. Under each TestStep that was the source of a Message Exchange during the Test Run, there will be "Message" nodes corresponding to the sent messages. Selecting a messages request or response node and then selecting the right "Message Content" inspector will show the actual content of the message (see image below). Selecting the "Message Coverage" inspector will instead show that messages' coverage value.

Double-click an item in the tree will try to open that items corresponding editor/viewer

Assertion Coverage

As mentioned above, when running functional Tests, all performed XPath assertions are analyzed and a coverage value measuring how large part of the WSDL contract that was actually asserted. Assertion Coverage is displayed in dark-green in both the Coverage Tree and the Message Inspector. In the image below, the Message Coverage Inspector shows that the item and contained values have been asserted by an XPath expression, the assertion can also been seen in the Assertion Results Inspector at the bottom.

Coverage Options

The Coverage Options dialog available from the toolbar allows you to set the following options:

  • Count Empty Values - select if empty elements/attributes should be regarded as covered
  • Count ? Values - select if elements/attributes containing '?' should be regarded as covered

Coverage Measurement Points

soapUI allows you to measure coverage in 3 distinct scenarios:

  1. For Functional Tests - for measuring how much of your WSDL contract is actually covered by your tests. The corresponding Coverage Panel is available at the Project, TestSuite and TestCase level.
  2. For MockServices - for measuring both how much of your WSDL is being mocked and how much of your WSDL is being utilized/sent by the Mocks clients.
  3. For SOAP Monitors - for measuring how much of your WSDL is being used in a real-life scenario.


Next: