gaurav
2018-07-11 12:50
Had seen your Atlassian summit youtube video about pact some time back @bsayers, it was awesome! :slightly_smiling_face:
gaurav
2018-07-11 12:52
https://www.youtube.com/watch?v=-6x6XBDf9sQ&feature=youtu.be lot has changed since then but still, a really good one!
bsayers
2018-07-12 00:14
I?m glad you enjoyed the talk @gaurav, we took more time then I?d like to admit preparing it.
david.dyke
2018-08-15 04:30
Hey all, has anyone used the swagger validator using a pact broker with authentication?
gaurav
2018-08-17 10:09
@david.dyke It should be possible using `withPactsFrom` builder method for `PactProviderValidator`.
gaurav
2018-08-17 10:11
e.g.: ```PactProviderValidator validator = PactProviderValidator
.createFor(SWAGGER_JSON_URL)
.withPactsFrom(brokerUrl, username, password, providerName)
.build()```
bsayers
2018-08-30 10:12
There seems to be a bit of confusion about swagger request validator (a Java codebase maintained as a 20% project) vs swagger mock validator (a nodejs codebase we use in our CI and has a full time team behind it)
bsayers
2018-08-30 10:14
Anyway for those interested we have released OpenApi 3 support in swagger mock validator:
https://bitbucket.org/atlassian/swagger-mock-validator
bsayers
2018-08-30 10:17
It?s been well tested but isn?t being used in the wild inside Atlassian just yet so there might be some rough edges, I?d expect it to mature over the next month or two as we get lots of internal adoption and discover all the interesting edge cases we missed.
david.dyke
2018-08-30 10:19
This is the one I was looking for with auth for a pact broker, I made a bit of a hack to get it to work locally
bsayers
2018-08-30 10:22
@david.dyke can you raise an issue on the project asking for this feature and share how you?d like to pass the tool the credentials?
david.dyke
2018-08-30 10:25
sure will do, using flags might be the best option, assuming this will be part of a CI process the CI runner will need to use them easily
bsayers
2018-08-30 10:25
It?s not a feature we currently support but I have no objection, I?ll see if any of the team are interested in adding this during their 20% time. Someone asked for pact broker tagging support recently, this feels like a similar type of change.
david.dyke
2018-08-30 10:26
I've found the validator to be really useful at design time, e.g. you're updating your API's Swagger and you want to test that your new design doesn't break any consumers before you've even started coding it
bsayers
2018-08-30 10:27
Nice! Yeh the big driver for us investing in it is to get feedback as early in the dev cycle as possible.
bsayers
2018-08-30 10:28
Keep an eye on this project @david.dyke, we?re still working on it but you may also find it useful
https://bitbucket.org/atlassian/openapi-diff
gaurav
2018-08-31 01:46
Great! Thanks @bsayers for sharing this. We are looking for something similar to swagger-request-validator for non-java consumers. For java consumers, we can use swagger-request-validator (advantage: integrated as Junit rule) and for non-java consumers, we can use swagger-mock-validator as an additional step in CI.
matt.fellows
2018-08-31 10:31
I believe it can be a YAML or JSON format. YAML is the default. I don?t believe it has its own MIME type or anything
bethskurrie
2018-08-31 10:32
I tried to find the plain yaml mime type, and couldn't find a definitive answer.
bethskurrie
2018-09-04 10:18
Swagger validator users, can you give the validator an http url to a swagger document?
david.dyke
2018-09-04 22:30
You can yes, useful if you're storing all your swagger files in Swagger hub or similar
bethskurrie
2018-09-10 02:49
I'm looking for some users to try out the beta OAS support :party_parrot: that we're adding to the DiUS hosted broker. If you're a hosted broker customer (or would like to become one to try out this new feature) please let me know. I'm really excited about the potential that Pact+Swagger have to work together.
david.dyke
2018-09-10 03:16
Yep, although we're using it for some internal PoCs at the moment, so changing things might upset some of the devs
bethskurrie
2018-09-10 03:17
What I've done so far is just make endpoints to store and retrieve the oas doc.
david.dyke
2018-09-10 03:19
We've been using Swagger hub, or just source control at the moment, it's a bit of a pain
gaurav
2018-09-10 03:21
@bethskurrie I can also give it a try if there is a docker container for this beta. :slightly_smiling_face:
bethskurrie
2018-09-10 03:31
All the best features in Pact and the broker have come out because they've been developed to fix problems that we had ourselves, but unfortunately, I'm not working on a project with swagger at the moment, and it's been a while since I've used swagger hub.
david.dyke
2018-09-10 03:36
I'm happy to help throwing out suggestions for additional features. The trickiest part I'm finding so far is explaining how it works. And so far the best way to do this has been getting teams to write their own 'fake' providers to understand the process.
david.dyke
2018-09-10 03:38
Where I am at the moment involves multiple teams, so Swagger specs are important to aid communication between teams but also to validate that a consumer isn't going crazy and writing invalid pacts
bethskurrie
2018-09-10 03:38
"try it out yourself" is actually step 2 of the "Pact Nirvana" document https://docs.google.com/document/d/e/2PACX-1vRf1kSDccImNipOOm1G-bjcSs-ifbZjf1v54K-dIcq8BLKeFPAAm_bf_p71UKqkRMIx30QWWL-kN8TI/pub
bethskurrie
2018-09-10 03:38
I've got a good new "invalid pact protection" feature coming out soon too.
bethskurrie
2018-09-10 23:07
Cool. 2/3 I can give you the new 'pending pacts' feature (allows you to verify proposed new pacts without failing the builds)
david.dyke
2018-09-11 00:23
I guess pending makes it sounds like they have been finalised and will be there soon, whereas proposed gives the impression that they can be changed before being finalised?
bethskurrie
2018-09-11 01:32
The "pending/proposed" pacts are ones that are the latest for their tag, and haven't yet been verified successfully. Keeping in mind that verifications are attached to the pact content, not the version number, this means that they're the "newly changed" pacts. They could have changed because there is existing functionality that the consumer now wishes to to use, or because there is proposed new functionality that does not yet exist that they would like to exist.
bethskurrie
2018-09-11 01:33
These pacts will be run with "ignore failures" switched on (a new feature) so that the consumer can get feedback on them, without the provider's build breaking.
yousafn
2018-12-04 22:05
Hey up all,
I am working on a new greenfield application, integrating with some back end services.
I am reasonably familiar with micro-service architecture and consumer driven contract testing.
For around 5 years, I have advocated swagger use for documenting API's, especially in the initial start up phase, as it is always so hard to get people to agree to something concrete, especially in agile teams.
I was recently introduced to PACT a few weeks ago, and instantly bought into the concept. I thought it would be a great asset to our team.
I have built a POC which have received really great feedback.
Today I started looking into the tooling that can unify the swagger specs and our pact files and came across some great tools by atlassian.
I?ve now got a docker container with
- swagger-cli for its validate function, to check that the swagger specs we are working from, are resolvable
- swagger-mock-validator for its ability to validate pact mocks we are writing, match up with the swagger spec
Swagger-cli is a great tool for validating the swagger resolves, especially when its been hand-written. It has caught a few fields which specified as required, but didn?t exist in the object.
Swagger-mock-validator looks to be useful as it has highlighted several (68 issues so far!) - This looks to be mainly where the swagger has diverged from the pact expectations, however it has caught some failure states we are simulating but these are still yet to be defined in our swagger definitions.
With regards to swagger-mock-validator, and the output it sends out.
- Are there any decent formatters for the output. 68 errors filled my vscode terminal buffer instantly, so I?m just piping it into a file at the moment and will start working through the errors tomorrow)
- The docs mention analytics by means of `analytics.ts`, I am going to hook this up tomorrow, we don?t have anything for analytics at the moment and its on the board :slightly_smiling_face:
bethskurrie
2018-12-17 10:17
I'd recommend raising an issue on the actual codebase, as there aren't many of the actual swagger validator contributors on this channel.
wongso_a
2019-09-19 07:27
Hi everyone, we are currently spiking Pact/Swagger for a bank (https://bitbucket.org/atlassian/swagger-request-validator/src/master/swagger-request-validator-pact/) and are facing with and an issue similar to what is being described in https://bitbucket.org/atlassian/swagger-request-validator/issues/231/relative-refs-not-being-loaded-when . We modified our pom.xml file to override swagger-parser to version 2.0.14 and currently getting an error as shown in the attachment. Any idea? cc @bethskurrie #swagger-validator
matt.fellows
2019-09-20 02:16
It looks like a compatibility issue, but unfortunately you might need help from the atlassian authors
wongso_a
2019-09-20 02:31
@matt.fellows Do you know the best way to get in touch with the Atlassian authors?
prabha_kannusamy2
2019-12-10 20:03
Hi All,
I am trying to run pact validation with swagger.yaml file. In swagger file there is a security definition for my api like "security: - apiKey: []". But when i run the pact test it is complaining "[ERROR] GET on path '/api/v1/user-dashboard/user' requires security parameters. None found."
matt.fellows
2019-12-10 21:06
My guess is that your pact file doesn't have this apiKey or the definition in the swagger file is incorrect.. Can you share the two files? If you remove apiKey from swagger does it pass? Start with the smallest file you can and expand from there
prabha_kannusamy2
2019-12-11 15:09
Hi Matt, yes i have removed the apiKey Definition from swagger and the test are getting passed.
prabha_kannusamy2
2019-12-11 15:10
responses:
200:
description: "OK"
schema:
$ref: "#/definitions/AdvisorServiceResponse"
201:
description: "Created"
401:
description: "Unauthorized"
403:
description: "Forbidden"
404:
description: "Not Found"
security:
- apiKey: []
prabha_kannusamy2
2019-12-11 15:13
@Pact(provider = PROVIDER, consumer = CONSUMER)
@Validated
public RequestResponsePact createFragmentPutAdvisor(PactDslWithProvider provider) throws IOException {
Map<String, String> contentTypeHeader = contentTypeHeader();
contentTypeHeader.put("crmKey", "123456");
return provider.given("A valid advisor with a valid tenantId").uponReceiving("a request to update an advisor's CRM key")
.path(PUT_REST_URL).method("PUT").headers(contentTypeHeader)
.willRespondWith().status(200).toPact();
}
agarwalatrisha1212
2020-02-22 17:18
Can anyone point me a resource where i could see and find out how swagger validator helps ? Is this swagger-hub or swagger ? I will appreciate it :pray:
matt.fellows
2020-02-22 21:09
I'm going to write an article on this in the next week or so. But you could think of it as a way to avoid provider side pact tests, assuming you've already done your testing to ensure you API is compliant with the swagger spec
matt.fellows
2020-02-22 21:10
If you Google pact and swagger-mock-validator you'll find a few resources
agarwalatrisha1212
2020-02-23 17:07
Is based on sawgger or swagger-hub ? The json used is from swagger hub ?
matt.fellows
2020-02-23 21:25
The article I?ll be posting is about using OpenAPI (OAS) and Pact together. If you wanted to use SwaggerHub, you?d need to do the work to fetch from swagger hub also (I?ve done this before also)
heytaco
2020-08-25 03:57
Hi there! My name is HeyTaco!, and you can use me to give people tacos to show your appreciation. My tacos will spread joy through Slack!
jan.krejci
2020-10-20 12:52
Hello guys, anyone that already has used the CLI tool `swagger-mock-validator` of Atlassian and experienced the behaviour below?
1 - I have a *consumer* that publish a contract on PACT Broker like that below:
*Interactions*
Given get balance of accountId 1, upon receiving a request to /v1/accounts/1/balance/ from consumer, with:
```{
"method": "GET",
"path": "/v1/accounts/1/balance/"
}```
provider will respond with:
```{
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"accountId": 100,
"clientId": 100,
"balance": 100,
"credit": 100
}
}```
2 - and the in the *Provider* Side I have a Open API specification like that :
```{
"...": "...",
"paths":{
"/v1/accounts/{accountId}/balance/":{
"get":{
"...": "...",
"responses":{
"404":{
"description":"Account id not found"
},
"200":{
"description":"Return the balance for the account",
"content":{
"application/json":{
"schema":{
"$ref":"#/components/schemas/BalanceDTO"
}
}
}
}
}
}
}
},
"components":{
"schemas":{
"...": "...",
"BalanceDTO":{
"type":"object",
"properties":{
"clientId":{
"type":"integer",
"format":"int32"
},
"accountId":{
"type":"integer",
"format":"int32"
},
"balance":{
"type":"number"
}
}
}
}
}
}
}```
If I run the swagger-mock-validator CLI against the Pact Broker to verify the contract by the Provider, the contract between the participants is supposed to be invalid because I don't have a *credit field* into the Provider response BUT It runs without failures :
```0 error(s)
0 warning(s)```
yousafn
2020-10-20 22:50
@jan.krejci check out
https://bitbucket.org/atlassian/swagger-mock-validator/issues/78/response-parser-removes-required
For the reasoning as to why you are seeing this issue
jan.krejci
2020-10-21 21:39
I am struggling to generate this automatically with Spring doc OAS :disappointed:
tjones
2020-10-22 00:30
@matt.fellows: No, I don't think so. I'm about to need to validate some pacts against some swaggers, so I'll probably be filing an issue about the problem I experienced (which is that it the validator expects full coverage of the endpoints in the swagger)
matt.fellows
2020-10-22 00:30
ah fair enough, thought it might have been the same. keen to see how you go!
tjones
2020-10-22 00:31
I think the behaviour described on that ticket is the correct behaviour, since your contract should include minimal required
tjones
2020-10-22 01:59
@jan.krejci: there's a subtlety here that is worth pointing out:
> In reality, removing a required field from an API is a breaking change and will break my application.
This is often not actually the case. In a swagger-first land, there's a definition of breaking change which is something like "removing or renaming a field". However, removing or renaming a field that no one is using is not actually breaking change, since no one was using it anyway.
This is something that Pact helps with - it means you can have a more practically driven definition of breaking change which is "changes to the definitions of the fields that the consumer expects".
If you're in the case where you cannot unmarshal an incomplete response (even though you are not using the complete response), then I would argue that those fields should be in your contract.
Swagger mock validator is about testing that your contract would be fulfilled by a service that adheres to the swagger document. This is subtly different to checking that a service adheres to the swagger document, and the issue you've raised highlights one of the differences.
jan.krejci
2020-10-22 17:17
@tjones Thanks for your answer.
I understand completely that Pact has much more advantages than Design first approach and it is something we are trying to convince people here.
In the other hand we also are trying to figure out scenarios where it would be more difficult to implement PACT on the provider side that is where we started to look at swagger mock validator by Atlassian.
Accordingly to the issue *YOU54F* posted here there is a property that you can set up into OAS in order to strict that the provider's schema shouldn't accept anything else. This is a counter-measure since the mock validator removes all required properties.
If I was using PACT (*in both sides)* for the same scenario I prompted here, there would be an error when the provider performs a Contract verification due to the credit field does not exist there. It is the way I believed swagger mock validator would work.
tjones
2020-10-23 05:54
If you're wanting to validate that the provider's responses match the swagger, there might be better tools than the Atlassian Pact/Swagger tool. The atlassian tool is for confirming that a Pact contract could be fulfilled by a swagger file.
so, if your contract says "For <some request>, <some consumer> needs a response that returns `{ "a": "foo" }`", then a swagger document that says the response format is `{ "a": <string>, "b": <string>}` would be able to fulfil that contract, regardless of whether or not `a` and `b` are required.
The swagger validator isn't for validating whether the provider matches the swagger, it's for validating whether the Pact contract can be satisfied by a provider that implements the endpoints described in the swagger.
tjones
2020-10-23 06:05
If I'm understanding the problem you're describing, you're hitting the limitation of swagger where you have a pact contract that is something like:
* When I do GET "/someItem"
* in the state "someItem is named"
* I get the response `{ "name": "foo" }`
If the swagger says the format is:
```{
"name": <optional string>,
"value": <required string>
}```
You can use the swagger validator to check the expected response, and discover that it is *possible* to produce that response if the provider adheres to the swagger. This means that the swagger validator will pass (correctly), because the parts of the response you require are (possibly) present. However, you don't know for sure that it would be actually produced by the implementation.
Just because you require `name` for that particular request doesn't mean it would be present on all requests, so this is the best the validator can do for you. This is a limitation of swagger.
If the pact expectation was `{ "name": "foo", "otherProperty": "bar" }`, then the swagger validator would (correctly) fail, because it is not possible for `otherProperty` to be produced.
Did I understand your situation correctly?
jan.krejci
2020-10-23 13:55
@tjones My issue is almost that but the opposite you said and very close to that sentence you posted:
"If the pact expectation was `{ "name": "foo", "otherProperty": "bar" }`, then the swagger validator would (correctly) fail, because it is not possible for `otherProperty` to be produced."
However, it is just what is not happening when I perform the Swagger mock Validator.
So, in other words, the validator is not checking if the consumer is expecting a attribute(element) that is fulfilled by the Provider. As far as I got it this is some sort of side effect caused by the CLI that is better explained into the issue of
And that is something that just possible to solve adding `additionalProperties` to `false` since for some reason the Open API Specification defines it as true by default or in other words *accept everything even tough I don't know*
bethskurrie
2020-10-29 23:23
@lakitna just mentioned this to me in a separate conversation. Did you have a solution for it Sander?
tjones
2020-11-09 04:41
I've opened https://bitbucket.org/atlassian/swagger-mock-validator/issues/84/test-incorrectly-passes-when-mock-expects to chase this up
bsayers
2020-11-25 05:25
I answered in detail on the issue above but in summary the behaviour you are seeing is not a bug, we need to follow the validation rules of json schema quite strictly in order to avoid breaking the logical keywords (anyOf, oneOf, not and allOf)
matt.fellows
2021-12-16 20:32
Can you please elaborate on what you are trying to do? There are plenty of OSS tools that turn an OAS into a stub if that what you're asking?
diva.pant1
2021-12-16 22:39
I am trying to understand if we can use OAS just as pacts. I read about OAS *contracts.* Till now difference between schema and contracts is clear. But when we say OAS contracts, then its confusing. So we are trying to incorporate concept of contracts with OpenAPI schemas. To simply understand, if we use OAS, it will generate schemas. Now we want to create mock, say we use ReadyAPI for that (mocking and visualization tool), and now we want to create a contract for this response. How will we do it, using pact or pactflow?
tjones
2021-12-17 00:41
> I am trying to understand if we can use OAS just as pacts
You can't do this with any real confidence.
tjones
2021-12-17 00:44
If your provider team don't want to use pact, and have a swagger document then the bi-directional contracts feature from pactflow automates the checking for you.
matt.fellows
2021-12-17 04:24
> I am trying to understand if we can use OAS just as pacts
it?s better to think of Pact as a different thing to the OAS/bi-directional conversation
matt.fellows
2021-12-17 04:24
Pact is Pact - OAS is nothing to do with Pact itself, nor the Pact tool chain
matt.fellows
2021-12-17 04:24
they are related, but different (we have some articles on our http://docs.pact.io website about that)
matt.fellows
2021-12-17 04:25
OAS can be used _as_ a contract, either in your own DIY approach or using something like Pactflow?s bi-directional contracts feature. In this case, it _can_ be used as a contract (specifically, the provider contract).
From a contract testing perspective, you still must reliably produce a contract for the consumer to capture what it needs of a provider.
matt.fellows
2021-12-17 04:26
This guide talks about the ways to https://docs.pactflow.io/docs/workshops/bi-directional/contracts/pact#strategies-to-capture-consumer-contracts a consumer contract in that mode
diva.pant1
2021-12-17 12:24
Okay thanks, the distinction is clear! But I am still wondering how can I generate OAS contracts, not in theory, but practically. So is my understanding correct, that first we create schema using OAS, then its mock using some OSS tool, and then the coded contract? And by that time we will already have a consumer mock to create a contract with, right?
matt.fellows
2021-12-17 13:38
You don't need a mock for it on the provider wide. Your job is to make sure your provider is compatible with it and publish that as a *provider contract* to Pactflow
matt.fellows
2021-12-17 13:40
The consumer contract can be generated by using Pact, or by converting exisiting mocks to a contract (specifically in the format of a pact file, as shared in our bidirectional docs previously)
matt.fellows
2021-12-17 13:42
The primary use case here is to rapidly get some contract testing coverage on exisiting systems, where there is already decent test coverage on both sides and you want to "upgrade" them into a basic contract testing workflow. This is going to be a lot less effort than Pact testing, but with the tradeoffs previously shared
