Skip to main content

Comparisons with other tools

How does Pact differ from Spring Cloud Contract?​

Spring Cloud Contract is another contract testing framework. It was originally written for the Spring/JVM ecosystem but can now also be used with non-JVM languages.

It has a Contract Definition Language, written in Groovy or YAML, which is used to produce:

  • JSON stubs for Wiremock, for testing consumer code
  • Acceptance tests, in Spock or JUnit, for testing against the provider
  • Messaging routes if you're using one

Both Pact and Spring Cloud Contract are essentially solving the same problem. The main difference between them is that Pact generates language-neutral acceptance contracts, in the form of JSON pact files. These pact files can be created, or tested, by anything that implements the Pact specification, whether the code is Ruby, Javascript, the JVM, or any other language. Even though it is possible to use SCC with non-JVM languages, it has no native support for them and requires that contracts are written manually in YAML and the use of an external stub runner (or Docker) to run the tests.

Another key difference is that in Pact the consumer code actually generates the contract. In Spring Cloud Contract the contracts are written by hand outside of the code base. This creates a potential for drift.

A third main difference is that Pact has always been a consumer-driven contract testing framework whereas Spring Cloud Contract started as provider-driven. It is possible to do consumer-driven contract testing with Spring Cloud Contract but the all contracts from all the consumers of one provider need to be stored in one centrally shared git repository whereas in Pact each consumer generates its contracts from its own repository.

Lastly, Pact has a series of features relating to provider states, which are Pact's answer to dealing with the traditional challenges of managing test data and sequencing tests in a dedicated test environment.

As stated above, Pact also has the Pact Broker which provides autogenerated documentation, network diagrams, and enables cross testing of the production and head versions of your consumer and provider, allowing you to decouple your consumer and provider release cycles. But, it is also possible to integrate Spring Cloud Contract with the Pact Broker!

In summary:

  • Pick the tool that you find the most natural to use - they're both good tools!
  • if you're tied to the JVM, and especially Spring, Spring Cloud Contract might be easier for you to integrate into your tests
  • if your workflow is inherently more provider driven, then Spring Cloud Contracts might suite you better (however, do read this blog post on how the Pact ecosystem is looking to support provider driven flows)
  • if you want increased flexibility with your choice of language, and to not be tied to a particular implementation, Pact might suit you better
  • if you want to make use of the Pact Broker, Pact might be the more natural choice

How does Pact differ from Accurest?​

Accurest is the predecessor of Spring Cloud Contract mentioned above. The differences to Pact are thus similar. It is no longer under active development.

How does Pact differ from Nock?​

Nock provides a way to mock out network requests in tests and verify that the consuming code interacts with the mocked interactions as expected.

Nock is great for functional testing, where Pact may not be the best tool to use for mocking.

However, unlike Nock:

  • Pact verifies that the interactions mocked are provided to the consuming service as expected by the providing service.
  • Pact provides a way to verify message based interactions behave as specified, not just HTTP interactions.
  • Pact runs a mock server on a real port, rather than overriding Node's http.request functionality.
  • When using the Pact Broker it will store the pacts (contracts) you make. This allows you to check either changes to the provider or consumer are safe to deploy, where assumed responses from Nock could drift from implementation and become out of date.

How does Pact differ from VCR?​

Pact is like VCR in reverse. VCR records actual provider behaviour, and verifies that the consumer behaves as expected. Pact records consumer behaviour, and verifies that the provider behaves as expected. The advantages Pact provides are:

  • The ability to develop the consumer (eg. a Javascript rich client UI) before the provider (eg. the JSON backend API).
  • The ability to drive out the requirements for your provider first, meaning you implement exactly and only what you need in the provider.
  • Well documented use cases ("Given ... a request for ... will return ...") that show exactly how a provider is being used.
  • The ability to see exactly which fields each consumer is interested in, allowing unused fields to be removed, and new fields to be added in the provider API without impacting a consumer.
  • The ability to immediately see which consumers will be broken if a change is made to the provider API.
  • When using the Pact Broker, the ability to map the relationships between your services.

How does Pact differ from Webmock?​

Unlike Webmock:

  • Pact provides verification that the responses that have been stubbed are actually the responses that will be returned in the given conditions.
  • Pact runs a mock server in an actual process, rather than intercepting requests within the Ruby code, allowing Javascript rich UI clients to be tested in a browser.

How does Pact differ from Pacto?​

Pacto is another Ruby implementation of a library that provides a mock service and provider verification using consumer driven contracts, it is no longer maintained. It differs from Pact in the following ways.

  • Pacto has the ability to create contracts by recording interactions with an existing service. This makes the contracts easy to set up.
  • Once the Pacto contracts are created, they are static, and are used to verify both the consumer and the provider. This would make it easy to determine whether a broken contract is due to a change in the consumer or a change in the provider.
  • Pact's contracts are dynamically generated artefacts. This makes them easier to maintain.
  • Pact allows you to make the same request with a different "provider state", allowing you to test different HTTP response codes for the same endpoint, or test the same resource in different states.
  • Pact allows you to do regular expression matching.
  • Pact has native support for Ruby, JVM, and .Net consumers, with a Javascript wrapper using the Ruby mock server.
  • Pact has the Pact Broker which provides autogenerated documentation, network diagrams, and enables cross testing of the production and head versions of your consumer and provider, allowing you to decouple your consumer and provider release cycles.

In summary:

  • The ability to record contracts would probably make Pacto a better choice than Pact for stubbing an existing 3rd party service (see their example for Github). The lack of provider states and regular expression matching would probably not matter in this scenario, as you are unlikely to be able to set up data on the provider without using the the very API you are testing.
  • Pact is probably a better choice for a new project where the provider service does not yet exist, where the consumer's functionality is driving out the requirements for the provider.

How does Pact differ from XXX?​

If you have experience with another mocking, stubbing or contract testing tool that you would like to write about, please click the EDIT button and submit a PR.