Provider Verification

Pact JS supports both HTTP and non-HTTP verification (using plugins).

Contract Testing Process (HTTP)

Pact is a consumer-driven contract testing tool, which is a fancy way of saying that the API Consumer writes a test to set out its assumptions and needs of its API Provider(s). By unit testing our API client with Pact, it will produce a contract that we can share to our Provider to confirm these assumptions and prevent breaking changes.

The process looks like this:

diagram

The consumer writes a unit test of its behaviour using a Mock provided by Pact
Pact writes the interactions into a contract file (as a JSON document)
The consumer publishes the contract to a broker (or shares the file in some other way)
Pact retrieves the contracts and replays the requests against a locally running provider
The provider should stub out its dependencies during a Pact test, to ensure tests are fast and more deterministic.

In this document, we will cover steps 4-5.

HTTP Provider package

The provider interface is in the package: github.com/pact-foundation/pact-go/v2/provider.

Provider API Testing

Provider API

Once you have created Pacts for your Consumer, you need to validate those Pacts against your Provider. The Verifier object provides the following API for you to do so:

API	Options	Returns	Description
`verifyProvider()`	See below	`Promise`	Start the Mock Server

Start your local Provider service.
Optionally, instrument your API with ability to configure provider states
Then run the Provider side verification step

const { Verifier } = require('@pact-foundation/pact');
const opts = {
  ...
};

new Verifier(opts).verifyProvider().then(function () {
    // do something
});

Verification Options

Parameter	Required?	Type	Description
`providerBaseUrl`	true	string	Running API provider host endpoint.
`pactBrokerUrl`	false	string	Base URL of the Pact Broker from which to retrieve the pacts. Required if `pactUrls` not given.
`provider`	false	string	Name of the provider if fetching from a Broker
`consumerVersionSelectors`	false	ConsumerVersionSelector\|array	Using Selectors is a way we specify which pacticipants and versions we want to use when configuring verifications.
`consumerVersionTags`	false	string\|array	Retrieve the latest pacts with given tag(s)
`providerVersionTags`	false	string\|array	Tag(s) to apply to the provider application
`providerVersionBranch`	false	string	Branch to apply to provider application version (recommended to set)
`includeWipPactsSince`	false	string	Includes pact marked as WIP since this date. String in the format %Y-%m-%d or %Y-%m-%dT%H:%M:%S.000%:z
`pactUrls`	false	array	Array of local pact file paths or HTTP-based URLs. Required if not using a Pact Broker.
`providerStatesSetupUrl`	false	string	Deprecated (use URL to send PUT requests to setup a given provider state
`stateHandlers`	false	object	Map of "state" to a function that sets up a given provider state. See docs below for more information
`requestFilter`	false	function (Express middleware)	Function that may be used to alter the incoming request or outgoing response from the verification process. See below for use.
`beforeEach`	false	function	Function to execute prior to each interaction being validated
`afterEach`	false	function	Function to execute after each interaction has been validated
`pactBrokerUsername`	false	string	Username for Pact Broker basic authentication
`pactBrokerPassword`	false	string	Password for Pact Broker basic authentication
`pactBrokerToken`	false	string	Bearer token for Pact Broker authentication
`publishVerificationResult`	false	boolean	Publish verification result to Broker (NOTE: you should only enable this during CI builds)
`providerVersion`	false	string	Provider version, required to publish verification result to Broker. Optional otherwise.
`enablePending`	false	boolean	Enable the pending pacts feature.
`timeout`	false	number	The duration in ms we should wait to confirm verification process was successful. Defaults to 30000.
`logLevel`	false	string	not used, log level is set by environment variable

To dynamically retrieve pacts from a Pact Broker for a provider, provide the broker URL, the name of the provider, and the consumer version tags that you want to verify:

const opts = {
  pactBroker: 'http://my-broker',
  provider: 'Animal Profile Service',
  consumerVersionTags: ['master', 'test', 'prod'],
};

To verify a pact at a specific URL (eg. when running a pact verification triggered by a 'contract content changed' webhook, or when verifying a pact from your local machine, or a network location that's not the Pact Broker, set just the pactUrls, eg:

const opts = {
  pactUrls: [process.env.PACT_URL],
};

To publish the verification results back to the Pact Broker, you need to enable the 'publish' flag, set the provider version and optional provider version tags:

const opts = {
  publishVerificationResult: true, //generally you'd do something like `process.env.CI === 'true'`
  providerVersion: 'version', //recommended to be the git sha
  providerVersionTags: ['tag'], //optional, recommended to be the git branch
};

If your broker has a self signed certificate, set the environment variable SSL_CERT_FILE (or SSL_CERT_DIR) pointing to a copy of your certificate.

API with Provider States

If you have defined any states in your consumer tests, the Verifier can put the provider into the right state prior to sending the request. For example, the provider can use the state to mock away certain database queries. To support this, set up a handler for each state using hooks on the stateHandlers property. Here is an example from our e2e suite:

const opts = {
  ...
  stateHandlers: {
    [null]: () => {
      // This is the "default" state handler, when no state is given
    }
    "Has no animals": () => {
      animalRepository.clear()
      return Promise.resolve(`Animals removed from the db`)
    },
    "Has some animals": () => {
      importData()
      return Promise.resolve(`Animals added to the db`)
    },
    "Has an animal with ID 1": () => {
      importData()
      return Promise.resolve(`Animals added to the db`)
    }
  }
}

return new Verifier(opts).verifyProvider().then(...)

As you can see, for each state ("Has no animals", ...), we configure the local datastore differently. If this option is not configured, the Verifier will ignore the provider states defined in the pact and log a warning.

Provider State Setup and Teardown

Provider States can optionally take a setup and teardown function. These are useful in situations where you'd like to cleanup data specific to the provider state.

  'Whatever your state name is': {
    setup: (parameters) => {
      // do your setup here
      // return a promise if you need to
    },
    teardown: (parameters) => {
      // do your teardown here
      // return a promise if you need to
    },
  },

You can think of regular provider states as only being the "setup" phase.

Before and After Hooks

Sometimes, it's useful to be able to do things before or after a test has run, such as reset a database, log a metric etc. A beforeEach hook runs on each verification before any other part of the Pact test lifecycle, and a afterEach hook runs as the last step before returning the verification result back to the test.

You can add them to your verification options as follows:

const opts = {
  ...
  beforeEach: () => {
    console.log('I run before everything else')
  },

  afterEach: () => {
    console.log('I run after everything else has finished')
  }
}

If the hook errors, the test will fail. See the lifecycle of an interaction below.

Pending Pacts

NOTE: This feature is available on [PactFlow] by default, and requires configuration if using a self-hosted broker.

Pending pacts is a feature that allows consumers to publish new contracts or changes to existing contracts without breaking Provider's builds. It does so by flagging the contract as "unverified" in the Pact Broker the first time a contract is published. A Provider can then enable a behaviour (via enablePending: true) that will still perform a verification (and thus share the results back to the broker) but not fail the verification step itself.

This enables safe introduction of new contracts into the system, without breaking Provider builds, whilst still providing feedback to Consumers as per before.

See the docs and this article for more background.

WIP Pacts

NOTE: This feature is available on [PactFlow] by default, and requires configuration if using a self-hosted broker.

WIP Pacts builds upon pending pacts, enabling provider tests to pull in any contracts applicable to the provider regardless of the tag it was given. This is useful, because often times consumers won't follow the exact same tagging convention and so their workflow would be interrupted. This feature enables any pacts determined to be "work in progress" to be verified by the Provider, without causing a build failure. You can enable this behaviour by specifying a valid timestamp for includeWipPactsSince. This sets the start window for which new WIP pacts will be pulled down for verification, regardless of the tag.

See the docs and this article for more background.

Verifying multiple contracts with the same tag (e.g. for Mobile use cases)

Tags may be used to indicate a particular version of an application has been deployed to an environment - e.g. prod, and are critical in configuring can-i-deploy checks for CI/CD pipelines. In the majority of cases, only one version of an application is deployed to an environment at a time. For example, an API and a Website are usually deployed in replacement of an existing system, and any transition period is quite short lived.

Mobile is an exception to this rule - it is common to have multiple versions of an application that are in "production" simultaneously. To support this workflow, we have a feature known as consumer version selectors. Using selectors, we can verify that all pacts with a given tag should be verified. The following selectors ask the broker to "find all pacts with tag 'prod' and the latest pact for 'master'":

consumerVersionSelectors: [
  {
    tag: 'prod',
    all: true,
  },
  {
    tag: 'master',
    latest: true,
  },
];

NOTE: Using the all flag requires you to ensure you delete any tags associated with application versions that are no longer in production (e.g. if decommissioned from the app store)

Modify Requests Prior to Verification (Request Filters)

Sometimes you may need to add things to the requests that can't be persisted in a pact file. Examples of these are authentication tokens with a small life span. e.g. an OAuth bearer token: Authorization: Bearer 0b79bab50daca910b000d4f1a2b675d604257e42.

For these cases, we the ability to modify a request/response and modify the payload. The flag to achieve this is requestFilter.

Example API with Authorization

For example, to have an Authorization bearer token header sent as part of the verification request, set the verifyProvider options as per below:

let token
const opts = {
  provider: 'Animal Profile Service',
  ...
  stateHandlers: {
    "is authenticated": () => {
      token = "1234"
      Promise.resolve(`Valid bearer token generated`)
    },
    "is not authenticated": () => {
      token = ""
      Promise.resolve(`Expired bearer token generated`)
    }
  },

  // this middleware is executed for each request, allowing `token` to change between invocations
  // it is common to pair this with `stateHandlers` as per above, that can set/expire the token
  // for different test cases
  requestFilter: (req, res, next) => {
    req.headers["Authorization"] = `Bearer: ${token}`
    next()
  },
}

return new Verifier(opts).verifyProvider().then(...)

As you can see, this is your opportunity to modify\add to headers being sent to the Provider API, for example to create a valid time-bound token.

Important Note: You should only use this feature for things that can not be persisted in the pact file. By modifying the request, you are potentially modifying the contract from the consumer tests!

Lifecycle of a provider verification

For each interaction in a pact file, the order of execution is as follows:

BeforeEach -> State Handler -> Request Filter (request phase) -> Execute Provider Test -> Request Filter (response phase) -> AfterEach

If any of the middleware or hooks fail, the tests will also fail.

Publishing Verification Results to a Pact Broker

If you're using a Pact Broker, (e.g. a hosted one with our friends at PactFlow), you can publish your verification results so that consumers can query if they are safe to release.

It looks like this:

screenshot of verification result

To publish the verification results back to the Pact Broker, you need to enable the 'publish' flag, set the provider version and optional provider version tags:

const opts = {
  publishVerificationResult: true, //recommended to only publish from CI by setting the value to `process.env.CI === 'true'`
  providerVersion: 'version', //recommended to be the git sha eg. process.env.MY_CI_COMMIT
  providerVersionBranch: 'master', //recommended to be the git branch eg. process.env.MY_GIT_SHA
  providerVersionTags: ['tag'], //optional, recommended to be the git branch eg. process.env.MY_CI_BRANCH
};

Contract Testing Process (HTTP)​

HTTP Provider package​

Provider API Testing​

Verification Options​

API with Provider States​

Provider State Setup and Teardown​

Before and After Hooks​

Pending Pacts​

WIP Pacts​

Verifying multiple contracts with the same tag (e.g. for Mobile use cases)​

Modify Requests Prior to Verification (Request Filters)​

Lifecycle of a provider verification​

Publishing Verification Results to a Pact Broker​