Test suite: ets-security-client10

Scope

This executable test suite (ETS) verifies the conformance of the secure client behavior as the implementation under test (IUT) with respect to the following specification(s):

Conformance testing is a kind of “black box” testing that examines the externally visible characteristics or behaviors of the IUT while disregarding any implementation details.

Several conformance classes are defined in the principal specification; the ones listed below are covered by this test suite:

  • Conformance Class “Common Security” (abstract)
    • Validation of secure client HTTPS usage
  • Conformance Class “OWS Common”
    • Only affects Capabilities document presented to secure client
  • Conformance Class “WMS 1.1.1”
    • Only affects Capabilities document presented to secure client
  • Conformance Class “WMS 1.3.0”
    • Only affects Capabilities document presented to secure client

Test suite structure

The test suite definition file (testng.xml) is located in the root package, org.opengis.cite.securityclient10. A conformance class corresponds to a <test> element, each of which includes a set of test classes that contain the actual test methods. The general structure of the test suite is shown in Table 1.

Table 1 - Test suite structure
Conformance class Test classes
Conformance Level Common Security org.opengis.cite.securityclient10.levelCommonSecurity.*
Conformance Level OWS Common org.opengis.cite.securityclient10.levelOwsCommon.*
Conformance Level WMS 1.1.1 org.opengis.cite.securityclient10.levelWms111.*
Conformance Level WMS 1.3.0 org.opengis.cite.securityclient10.levelWms13.*

The Javadoc documentation provides more detailed information about the test methods that constitute the suite.

Test run arguments

The test run arguments are summarized in Table 2. The Obligation descriptor can have the following values: M (mandatory), O (optional), or C (conditional).

Table 2 - Test run arguments
Name Value domain Obligation Description
service_type String M

A string representing the service type to emulate for secure clients. Will affect the query parameters required to get capabilities. Valid values are:

  • wms111 for WMS 1.1.1 Conformance Class
  • wms13 for WMS 1.3.0 Conformance Class
  • wps20 for WPS 2.0.0 on the OWS Common Conformance Class
address String M Host interface on which to bind test server. When using IPv6, do not use square brackets, e.g. ::1
port Integer M Port on which to bind test server.
host String M Host name that will be advertised to clients. May be an IP address or domain name. Clients must be able to resolve this IP or name to the machine running the test server. If you are using a certificate from a Certificate Authority, this parameter must match the common name on that certificate. When using IPv6, use square brackets around the address, e.g. [::1]
path String O

URL Path at which the Test Server will listen. For example, test-session would result in the Test Server creating a servlet for https://host:port/test-session.

If left blank, a random string will be generated.

When used with TEAM Engine, the CTL script will automatically fill in a path such that the test session web page can inform the tester of the URL before starting the test session.

jks_path String M A Java KeyStore containing the X.509 certificates for the host address must be located at jks_path in order for the embedded Jetty server to provide HTTPS. Self-signed certificates are permitted, although the test client will have to trust that certificate manually.
jks_password String M The password required to unlock the Java KeyStore. Using a CDATA section is recommended to wrap passwords that have character data that may interfere with XML character entities.
authentication String O

Specify the authentication method that will be presented to Secure Clients via the capabilities document. Only one authentication method may be specified. Values are not case sensitive.

Enabling this will include an ows:Constraint for requiring an authentication from the secure client. For some authentication methods, additional test run properties may be mandatory.

If this element is omitted or is empty, then authentication will not be enabled.

For SAML2 authentication, specify saml2 and specify the idp_url test run property:

<entry name="authentication">saml2</entry>
idp_url String O

As part of the annotated capabilities document presented to the secure client, include Requirements Class "SAML2" (https://www.opengis.net/def/security/1.0/rc/authentication/saml2).

Enabling this will include an ows:Constraint for requiring SAML2 authentication from the secure client. The authentication test run property must be set to saml2, otherwise this test run property is ignored.

The value must be a URL that resolves to a SAML2 SSO resource. This value will then be passed to the secure client in the ows:Constraint. Note that it is up to the test user to validate the URL and the service at that URL, the ETS will do no verification and pass the URL "as-is" to the secure client in the capabilities document.

If this element is omitted or is empty, then SAML2 authentication will not be enabled.

A guide has been included for running SAML2 tests.
http_methods String O

As part of the annotated capabilities document presented to the secure client, include Requirements Class "HTTP Methods" (https://www.opengis.net/def/security/1.0/rc/http-methods).

Enabling this will include an ows:Constraint for listing all supported HTTP methods.

Only a value of true will enable this property, any other value will be evaluated as false.

If the w3c_cors property is set to true, then this property will be overridden to true as well.
w3c_cors String O

As part of the annotated capabilities document presented to the secure client, include Requirements Class "W3C CORS" (https://www.opengis.net/def/security/1.0/rc/cors).

Enabling this will include an ows:Constraint for the W3C recommendation"Cross Origin Resource Sharing".

Only a value of true will enable this property, any other value will be evaluated as false.

If set to true, then the http_methods test run property will also be set to true regardless of your configuration; "HTTP Methods" is required for this Requirements Class.
http_exception_handling String O

As part of the annotated capabilities document presented to the secure client, include Requirements Class "HTTP Exception Handling" (https://www.opengis.net/def/security/1.0/rc/http-exception-handling).

Enabling this will include an ows:Constraint for enabling HTTP error code mapping to OWS Common exception codes.

Only a value of true will enable this property, any other value will be evaluated as false.
http_post_content_type String O

As part of the annotated capabilities document presented to the secure client, include Requirements Class "HTTP POST Content-Type" (https://www.opengis.net/def/security/1.0/rc/content-type).

Enabling this will include an ows:Constraint for listing the mime-types permitted to be submitted by HTTP POST. At the minimum, the `application/x-www-form-urlencoded` MIME type must (and will) be supported and advertised by the service.

Only a value of true will enable this property, any other value will be evaluated as false.

Additional test run properties may be added in the future to support other authentication methods such as SAML 2.0 and OpenID Connect.

How to run the tests

The test suite is built using Apache Maven v3. Maven is necessary for self-hosting the test suite in your own test environment.

The test suite can be ran in the following environments:

  • Under an IDE such as Eclipse
  • As a self-contained JAR command-line application
  • As a module under TEAM Engine

1. Integrated development environment (IDE)

Use a Java IDE such as Eclipse, NetBeans, or IntelliJ. Clone the repository and build the project.

Set the main class to run: org.opengis.cite.security-client10.TestNGController

Arguments: The first argument must refer to an XML properties file containing the required test run arguments. If not specified, the default location at ${user.home}/test-run-props.xml will be used.

The TestNG results file (testng-results.xml) will be written to a subdirectory in ${user.home}/testng/ having a UUID value as its name.

2. Command shell (console)

One of the build artifacts is an “all-in-one” JAR file that includes the test suite and all of its dependencies; this makes it very easy to execute the test suite in a command shell:

java -jar ets-security-client10-0.1-SNAPSHOT-aio.jar [-o|--outputDir $TMPDIR] [test-run-props.xml]

This will require you to first compile and build the test suite as a Java application.

3. OGC test harness

Use TEAM Engine, the official OGC test harness. The latest test suite release are usually available at the beta testing facility. You can also build and deploy the test harness yourself and use a local installation.

For more details, please see the guide on setting up TEAM Engine with this ETS.

Debugging the ETS

If you need to debug a secure client connection and inspect the HTTP or HTTPS details, add these to your Java VM arguments:

-DDEBUG=true -Dorg.eclipse.jetty.LEVEL=DEBUG -Djavax.net.debug=ssl,handshake,data

This will print very detailed information about cipher suites and any TLS extensions available from the client.

About the included sample Java KeyStore

This repository contains a sample Java KeyStore with a self-signed certificate for testing purposes. The keystore can be created with the following command.

$ keytool -keystore src/main/resources/security.jks -storepass "ets-security-client" -genkey -alias dummy-key -keyalg RSA -sigalg SHA256withRSA -dname "cn=ETS Test Operator, ou=None, o=None, c=us"

This creates src/main/resources/security.jks with a single key (dummy-key) and protects the file with a password (ets-security-client). As this is a self-signed certificate, secure clients must allow insecure server certificates or install the certificate to their keystore.

When running the test suite from an IDE, this KeyStore will be used as it is specified in src/main/config/test-run-props.xml.

To create a PEM version of the file for secure clients to use:

$ keytool -exportcert -alias dummy-key -keystore src/main/resources/security.jks -storepass "ets-security-client" -rfc -file src/main/resources/security.pem

How to contribute

If you would like to get involved, you can:

  • Report an issue such as a defect or an enhancement request
  • Help to resolve an open issue
  • Fix a bug: Fork the repository, apply the fix, and create a pull request
  • Add new tests: Fork the repository, implement and verify the tests on a new topic branch, and create a pull request (don’t forget to periodically rebase long-lived branches so there are no extraneous conflicts)