Skip to main content

Step 4 - Create Provider Pact Test

Learning Objectives

StepTitleConcept CoveredLearning objectivesFurther Reading
step 4Verify the consumer pact with the Provider APIProvider side pact test
  • Understand basic Provider-side Pact concepts
  • Place provider side testing in a broader testing context (e.g. where it fits on the pyramid)
  • TODO LINK

As per the Consumer case, Pact takes the position of the intermediary (MQ/broker) and checks to see whether or not the Provider sends a message that matches the Consumer's expectations.

Verifying messages

Pact does this by using a proxy HTTP server in place of the message queue, some Pact DSL's will create one for you, in other languages you may need to create you own.

The verifier will read interactions in a pact file, and for asynchronous messages, will send POST request to the configured provider endpoint and expect the message payload in the response.

The POST request will include the description and any provider states configured in the Pact file for the message, formatted as JSON.

Example POST request:

{
"description": "Test Message",
"providerStates": [{ "name": "message exists" }]
}

Verifying metadata

Message metadata can be included as base64 encoded key/value pairs in the response, packed into the pact-message-metadata HTTP header, and will be compared against any expected metadata in the pact file.

The values may contain any valid JSON. Languages that include a message http proxy should automatically encode and decode the metadata for you.

For example, given this metadata:

{
"Content-Type": "application/json",
"topic": "baz",
"number": 27,
"complex": {
"foo": "bar"
}
}

The Pact framework, or the user would encode it into a base64 string, giving us ewogICJDb250ZW50LVR5cGUiOiAiYXBwbGljYXRpb24vanNvbiIsCiAgInRvcGljIjogImJheiIsCiAgIm51bWJlciI6IDI3LAogICJjb21wbGV4IjogewogICAgImZvbyI6ICJiYXIiCiAgfQp9Cg==.

The Pact framework, would them compare the provided metadata, with that contained in the pact interaction which has been replayed, ensuring that the metadata generated, matches the consumers exceptations. This may be important, dependant on your use case, in our example, we are ensuring that the message is sent to the correct topic.

Writing our test

  1. We require our system under test, which will be the API producer contains a function called create event which is responsible for generating the message that will be sent to the consumer via some message queue.
    1. We will use our Product domain model as we will use this to ensure the messages we generate comply with our Domain.
  2. Import the Pact DSL in the language of choice, and set up options
  3. We configure Pact to stand-in for the queue. The most important bit here is the messageProviders .
    • Similar to the Consumer tests, we map the various interactions that are going to be verified as denoted by their description field. In the JavaScript case, a product event update, maps to the createEvent handler.
    • We are using the providerWithMetadata function because we are also going to validate message metadata (in this case, the queue the message will be sent on).
    • The Python & Rust examples require the user to create the message proxy themselves, and the metadata is encoded and decoded manually. The user should inform the Pact framework of the location of the async-message proxy, with a hostname, path and port.
  4. We can now run the verification process. Pact will read all of the interactions specified by its consumer, and invoke each function that is responsible for generating that message.

in provider-js-kafka/src/product/product.pact.test.js:

// 1. Import message producing function, and Product domain object
const { createEvent } = require("./product.event");
const { Product } = require("./product");

// 2. Import Pact DSL
const {
MessageProviderPact,
providerWithMetadata,
} = require("@pact-foundation/pact");

const path = require("path");

describe("Message provider tests", () => {
// 3. Arrange

// Pact sources - here we are going to use a local file
const pactUrl =
process.env.PACT_URL ||
path.join(
__dirname,
"..",
"..",
"..",
"consumer-js-kafka",
"pacts",
"pactflow-example-consumer-js-kafka-pactflow-example-provider-js-kafka.json"
);

const opts = {
pactUrls: [pactUrl],
// Pact message providers
messageProviders: {
"a product event update": providerWithMetadata(
() => createEvent(new Product("42", "food", "pizza"), "UPDATED"),
{
kafka_topic: "products",
}
),
},
};

const p = new MessageProviderPact(opts);

describe("product api publishes an event", () => {
it("can generate messages for specified consumers", () => {
// 4. Run the pact verification
return p.verify();
});
});
});

Running the test

We can now run our test

> product-service@1.0.0 test
> jest --testTimeout 30000 --testMatch "**/*.pact.test.js"


RUNS src/product/product.pact.test.js
[21:15:59.007] INFO (36404): pact@13.1.4: Verifying message
[21:15:59.012] INFO (36404): pact-core@15.2.1: Verifying Pacts.
[21:15:59.013] INFO (36404): pact-core@15.2.1: Verifying Pact Files
RUNS src/product/product.pact.test.js
2024-10-22T20:15:59.196741Z INFO ThreadId(11) pact_verifier: Running setup provider state change handler with empty state for 'a product event update'
2024-10-22T20:15:59.196899Z INFO ThreadId(11) pact_verifier: Running provider verification for 'a product event update'
2024-10-22T20:15:59.196981Z INFO ThreadId(11) pact_verifier::provider_client: Sending request to provider at http://localhost:58571/
2024-10-22T20:15:59.196984Z INFO ThreadId(11) pact_verifier::provider_client: Sending request HTTP Request ( method: POST, path: /, query: None, headers: Some({"Content-Type": ["application/json"]}), body: Present(40 bytes, application/json) )
2024-10-22T20:15:59.206234Z INFO ThreadId(11) pact_verifier::provider_client: Received response: HTTP Response ( status: 200, headers: Some({"date": ["Tue, 22 Oct 2024 20:15:59 GMT"], "connection": ["keep-alive"], "keep-alive": ["timeout=5"], "pact_message_metadata": ["eyJrYWZrYV90b3BpYyI6InByb2R1Y3RzIn0="], "content-length": ["73"], "content-type": ["application/json; charset=utf-8"], "pact-message-metadata": ["eyJrYWZrYV90b3BpYyI6InByb2R1Y3RzIn0="], "x-powered-by": ["Express"], "etag": ["W/\"49-41p5fNWaTSGyF99I4ouOdCtiDE0\""]}), body: Present(73 bytes, application/json;charset=utf-8) )
2024-10-22T20:15:59.207511Z WARN ThreadId(11) pact_matching::metrics:

Please note:
We are tracking events anonymously to gather important usage statistics like Pact version and operating system. To disable tracking, set the 'PACT_DO_NOT_TRACK' environment variable to 'true'.

RUNS src/product/product.pact.test.js

Verifying a pact between pactflow-example-consumer-js-kafka and pactflow-example-provider-js-kafka

a product event update (0s loading, 185ms verification)
generates a message which
includes metadata
"contentType" with value "application/json" (OK)
"kafka_topic" with value "products" (OK)
has a matching body (OK)

PASS src/product/product.pact.test.js
Message provider tests
product api publishes an event
✓ can generate messages for specified consumers (657 ms)

Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: 1.233 s

Great, the test passed!

Breaking the test

Let's take a look at some failing situations.

Unmapped event handler

  1. Change the description mapping in the message provider, from a product event update to a product event updated
Verifying a pact between pactflow-example-consumer-js-kafka and pactflow-example-provider-js-kafka

a product event update (4ms loading, 196ms verification)
generates a message which
includes metadata
"contentType" with value "application/json" (OK)
"kafka_topic" with value "products" (FAILED)
has a matching body (FAILED)


Failures:

1) Verifying a pact between pactflow-example-consumer-js-kafka and pactflow-example-provider-js-kafka - a product event update
1.1) has a matching body
$ -> Actual map is missing the following keys: event, id, name, type, version
-{
"event": "UPDATED",
"id": "some-uuid-1234-5678",
"name": "Some Product",
"type": "Product Range",
"version": "v1"
}
+{}

1.2) has matching metadata
Expected message metadata 'kafka_topic' to have value 'products' but was ''

There were 1 pact failures
FAIL src/product/product.pact.test.js
Message provider tests
product api publishes an event
✕ can generate messages for specified consumers (466 ms)

● Message provider tests › product api publishes an event › can generate messages for specified consumers

Verfication failed

at node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.ts:52:20

Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 total
Snapshots: 0 total
Time: 1.172 s, estimated 2 s

Great, we can see a failure, where we don't have a mapping from our message interaction in the consumer pact, in our provider test. Change it back to a product event update.

You can expect Pact to fail, where there is no defined handler for a message, which ensures that the provider correctly handles each of these cases.

As a consumer generating contracts, one should work with the provider team, in order to ensure mapping can be agreed upon. There may be the opportunity to reuse existing mappings created by other teams.

Incorrect returned data

Change some data in the generated event, in your messageProviders. Lets try changing UPDATED to MODIFIED, and change the metadata key kafka_topic to topic

Run the test

Verifying a pact between pactflow-example-consumer-js-kafka and pactflow-example-provider-js-kafka

a product event update (4ms loading, 200ms verification)
generates a message which
includes metadata
"contentType" with value "application/json" (OK)
"kafka_topic" with value "products" (FAILED)
has a matching body (FAILED)


Failures:

1) Verifying a pact between pactflow-example-consumer-js-kafka and pactflow-example-provider-js-kafka - a product event update
1.1) has a matching body
$.event -> Expected 'MODIFIED' to match '^(CREATED|UPDATED|DELETED)$'

1.2) has matching metadata
Expected message metadata 'kafka_topic' to have value 'products' but was ''

There were 1 pact failures
FAIL src/product/product.pact.test.js
Message provider tests
product api publishes an event
✕ can generate messages for specified consumers (446 ms)

● Message provider tests › product api publishes an event › can generate messages for specified consumers

Verfication failed

Great, the test fails, both on the body content, and the returned metadata.

Here, Pact matchers restricted the value of $.event to be one of CREATED / UPDATED or DELETED, by way of a regular expression.

Our metadata is also checked, to ensure the correct value is generated.

Try reverting the metadata key topic back to kafka_topic, but change the topic name to product..

Running the test again will return a new error about the metadata, telling us the correct key was returned, but the incorrect value was. This will allow us not only to validate the body contents of our messages, but important data wthat will relate to our transmission protocol (or anything else we deem suitable).


1.2) has matching metadata
Expected message metadata 'kafka_topic' to have value '"products"' but was '"product"'

In our instance, if we were posting to a different queue, that the customer was listening to, it may be a while before anyone realises that messages will never be received. Pact gives you early feedback, long before requiring deploying each application, along side a queue and testing in an integration environment

Thats a wrap

We have now completed the workshop, and have a good understanding of how to use Pact to test asynchronous message contracts.

To take your Pact journey to the next level, why not apply the PactFlow CI/CD workshop fundamentals to your message contracts, and automate the publishing & verification of your message contracts as part of your CI/CD pipeline.