neonmd
2019-08-14 16:30
Hi, I was looking at the comparison table on pactflow's site: https://pactflow.io/pricing/
What do you mean by Swagger integration in the business column?
matt.fellows
2019-08-14 20:38
Hi Greg, we are still figuring out exactly how it will work, but the general challenge is articulated here: https://blog.pactflow.io/p/9b0db099-cadb-4c36-b123-0586b1e87998/ (still a WIP before we fully publish it to the world)
matt.fellows
2019-08-14 20:39
We have some workflows planned out to support it and have performed technical validation (spikes etc.) on some options, but are interested in feedback on the problem before we go too far down the implementation path.
neonmd
2019-08-14 21:51
For my company we require swagger generation to provide to large companies we contract with and their request is usually along the lines of "email our dev team your swagger". We however happily use Pact internally as Swagger is no match for up-to-date and accurate docs and contract testing.
I've also had to deal with a whole pile of misleading or patchy Swagger myself from other vendors.
Any attempt at marrying the Swagger and Pact world together so that Swagger is forced to be validated like Pact is would be a boon to the whole industry.
It sounds like your direction would be take Swagger and extend it to form request and response pairs then convert that into a Contract. Currently it's easier to make a Pact than Swagger, might there be a workflow to extend a Pact into Swagger?
matt.fellows
2019-08-14 21:57
> Any attempt at marrying the Swagger and Pact world together so that Swagger is forced to be validated like Pact is would be a boon to the whole industry.
Agree!
> Currently it?s easier to make a Pact than Swagger, might there be a workflow to extend a Pact into Swagger?
As in, generate a Swagger from a Pact file?
matt.fellows
2019-08-14 22:00
One of the approaches (and there are a few), is to enable provider teams to continue using Swagger and providing a mechanism to verify that their swagger implementation is correct during their local tests (akin to provider verification today). They then publish to the Broker (which basically supports this already, we just don?t really advertise it) and we would provide a mechanism for the consumer to verify that they are compatible with the provider. This inverts the _consumer-driven_ workflow - because it?s _not_ consumer driven, but it does provide the same sets of guarantees
matt.fellows
2019-08-14 22:00
So basically, one side of the contract is a Swagger document, the other is a Pact test - and we ensure they are compatible
matt.fellows
2019-08-14 22:01
This has another nice side-effect - for other provider-driven workflows that aren?t Swagger, we can still use the same mechanisms to prove compatibility
matt.fellows
2019-08-14 22:01
or put another way - Swagger and Pact are just types of general _contracts_
matt.fellows
2019-08-14 22:02
> ?email our dev team your swagger?.
This is exactly what we see over here, but you can extend that to internal teams also ?I sent you the latest swagger last week?
matt.fellows
2019-08-14 22:03
The best version of this was a colleague who was notified of an API change via Yammer - unsurprisingly, the notification wasn?t received and the API change broke a lot of things
matt.fellows
2019-08-14 22:04
(just saw your signup btw, let us know if we can help you get started in any way!)
brent.foley
2019-08-22 20:32
hello. I'm testing the trial for pactFlow and while my consumer can publish pacts just fine, I am getting connection timeouts when the provider attempts to call it. I used the /docs/troubleshooting example for junit5
brent.foley
2019-08-22 21:05
it's likely a corporate proxy issue but i don't know where I'm supposed to tell pact about the proxy config. i was hoping it would implicitly use env vars like http_proxy if set
brent.foley
2019-08-22 21:05
and again, the consumer side talks to pactflow just fine from same machine
matt.fellows
2019-08-22 21:39
@uglyog what?s the process for using a proxy in Pact JVM. The issues in the repo don?t indicate the solution (except one that states you can set the client)
brent.foley
2019-08-22 21:53
and consumer has pact settings in gradle task while provider is java annotations, if that is relevant
uglyog
2019-08-22 22:49
Should be the same, they just need to be set on the test JVM (Gradle will spin up a separate process to run the tests). You do this in the `test` config using `systemProperty`. See https://github.com/DiUS/pact-jvm/issues/690#issuecomment-394054857 for an example (of setting different properties though)
brent.foley
2019-08-22 23:24
@uglyog what are the names of the systemProperties you're referring to in the test { } section of gradle file?
I had tried setting http.proxyHost and http.proxyPort as environment variables but that didn't help
uglyog
2019-08-22 23:29
Let me just double check for you, but they should be the properties as documented here: https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/SystemDefaultHttpClient.html
uglyog
2019-08-22 23:35
Ok, can you raise a defect for that, it is not enabling the default system properties for the HTTP client
uglyog
2019-08-22 23:49
Sorry, the system properties should be enabled (I just checked the source code). Can you try adding the following to your Gradle?
uglyog
2019-08-22 23:51
```
test {
useJUnitPlatform()
// Show test results.
testLogging {
events "passed", "skipped", "failed"
}
systemProperty('http.proxyHost', 'forwardproxy')
systemProperty('http.proxyPort', '3211')
systemProperty('https.proxyHost', 'forwardproxy')
systemProperty('https.proxyHost', '3211')
}
```
brent.foley
2019-08-24 22:45
I'm trying to use pact broker cli against pactFlow broker. I followed example that wants a username and password but nothing I tried works; I get "Authentication failed".
I verified it's not a proxy issue cause I can talk to example test broker with same command. I think I'm confused as to what I use for username and password. For my java tests it recommended using api token for username and blank password for example. That doesn't work for pact-cli though. I tried my actual username and password and that also failed. Tried my username and token and that also failed.
I am following example here: https://hub.docker.com/r/pactfoundation/pact-cli
bethskurrie
2019-08-24 23:32
@brent.foley you just use a bearer token for Pactflow API access. Your username/password are for the UI.
bethskurrie
2019-08-24 23:36
Set `PACT_BROKER_TOKEN` instead of the username and password variables. You can get your token from the settings page.
bethskurrie
2019-08-24 23:37
@uglyog is this right? `For my java tests it recommended using api token for username and blank password for example. `? I thought we added a proper field for the token in all the places. Did we miss one?
matt.fellows
2019-08-25 12:42
Reminds me about needing to update your Avatar. We all want to see it happen. The only issue is, no one will recognize you after
brent.foley
2019-08-25 22:53
Aha! Yep that PACT_BROKER_TOKEN var worked. I missed that in the docs. Thanks @bethskurrie @uglyog @matt.fellows
brent.foley
2019-08-27 17:00
hello. Does the pact-cli docker tool allow a proxy to be passed in?
When I run the command on our build agents I am seeing "OpenSSL::SSL::SSLError - SSL_connect returned=1" ... "tlsv1 alert access denied". However, I don't get these errors when I run on my machine so I am wondering if it's certificate related?
brent.foley
2019-08-27 17:08
ah, i passed http_proxy and https_proxy as -e params and it worked :slightly_smiling_face:
tritorto
2019-09-24 04:19
Hey there, im testing webhook integration with our TeamCity atm, and have run into a problem when Testing the webhook in PactFlow: `An unexpected error has occured. This error has been sent to the Pacflow team and will be investigated ASAP.`
I have the curl working locally which is triggering a build ok - so far our devops have enabled added the IP address for our CI to access as per the pactflow faq and also made TC accept all CORS. Any help?
bethskurrie
2019-09-24 04:52
Hi @tritorto. Can you describe what you're seeing. I'm just looking at our systems to work out why I can't see an error on our side.
nathan.deamer
2019-09-26 09:41
Hi,
Just signed up for pactflow team - is there anyone available to help add google authentication? :slightly_smiling_face:
bethskurrie
2019-09-26 09:43
Just saw your email come through. Can you send the domain that you would like to use in another email?
bethskurrie
2019-10-14 02:33
added an integration to this channel: https://pact-foundation.slack.com/services/BPBESL6AG
2019-10-14 02:34
Notification from *Pactflow Technical Blog* :ghost:
2019-10-18 05:30
Notification from *Pactflow Technical Blog* :ghost:
2019-10-22 01:59
Notification from *Pactflow Technical Blog* :ghost:
matt.fellows
2019-11-10 03:16
FYI our publish maintenance window is underway. See http://status.pact.dius.com.au/
matt.fellows
2019-11-10 12:48
vitaliy.grigoruk
2019-11-21 14:25
Hi there, couple of questions / requests:
? we?re using github authentication and I see that I need to re-login quite frequently. Is it possible to make session last longer?
? would be nice to have a link to ?download? a JSON version of pact contract from a page which shows contract contents & verification results.
bethskurrie
2019-11-21 22:37
Ron will chat to you about the authentication. I'll put the download request into our backlog. Can you expand on that use case further?
uglyog
2019-11-21 22:45
How often are you having to log in? Every hour? Are you closing the browser, or leaving it open?
uglyog
2019-11-21 22:50
We have a change to our authentication that we may be releasing today that may fix that for you.
vitaliy.grigoruk
2019-11-22 10:35
@beth sometimes it is very convenient to download a contract JSON locally for test troubleshooting (on provider side). E.g. when we implement new pact tests and both teams haven?t settled on exact state handlers that needs to be added to provider. In this case provider team:
? downloads a pact json file,
? implement state handlers
? and manually modifies `providerState` field in contract JSON.
? run provider verification tests using PACT_URL=/path/to/local/contract.json
? when tests are green -> they commit new state handlers and give a heads up to consumer team to update he contract with new states.
vitaliy.grigoruk
2019-11-22 10:37
@uglyog usually, I?m asked to login when I?m closing browser tab and try to open broker URL after some time (probably ~1h). I?ll check if this is also reproducible with browser tab being opened for some time.
bethskurrie
2019-11-22 10:38
We discovered a race condition when refreshing the cognito token. If you press refresh once again, you should still be logged in.
bethskurrie
2019-11-22 10:40
As a quick fix, I might make the API browser button go straight to the Pact JSON in the HAL browser.
bethskurrie
2019-12-05 23:39
Today's Pactflow feature release: you can now paste the verification results URL into the browser, and see the pact/results together.
vasile.pop
2019-12-23 14:44
had a great experience with Spring Cloud Contracts when I was working in a JVM collection of microservices. My current company uses Python, Java and TypeScript for microservices, so PACT is an obvious choice
vasile.pop
2019-12-23 14:45
however, a lot of messaging between services is done via Kafka and RabbitMQ. I noticed PactFlow supports Kafka out-of-the-box
vasile.pop
2019-12-23 14:46
can you point me towards some documentation related to this? would gladly pay for a service which would allow us to create event-=based contracts without intricate setup
uglyog
2019-12-23 17:06
@vasile.pop I wrote a series of blog posts on that. See https://dius.com.au/2017/09/22/contract-testing-serverless-and-asynchronous-applications/
vasile.pop
2019-12-24 08:51
Service A does a POST request to Service B, sending a payload with some metadata. Service B receives the POST request, responds with 200 and as a side-effect it will send an email. Obviously, A is a consumer, B is a producer (even though most engineers I work with are outraged by this inversion on classic terminology)
vasile.pop
2019-12-24 08:52
Service A sends a Kafka message (as Kafka producer) on a topic on which Service B listens to (as Kafka consumer). As soon as an event is received, Service B sends an email, as side effect. Is it the same terminology? Is B still the producer?
uglyog
2019-12-24 09:01
No, with messages it's the reverse. Service A produces the messages, service B consumes it
matt.fellows
2019-12-25 22:26
In HTTP interactions we don't call service B a producer, but a _provider_. A is a _consumer_ of the service, which B provides
vasile.pop
2019-12-30 06:36
Got it, thanks. I still cannot find details about how http://pactflow.io supports Kafka
matt.fellows
2019-12-30 07:05
It doesn't support Kafka specifically (albeit how we could do that better for you would be useful for us to know)
matt.fellows
2019-12-30 07:06
I've forgotten. Which language are you using? Pact JVM had pretty good messaging support
vasile.pop
2019-12-30 07:06
```Support for Microservices, GraphQL, Message Queues, Websockets and serverless functions```
matt.fellows
2019-12-30 08:05
Cool. Yeah JS supports it https://github.com/pact-foundation/pact-js/blob/master/README.md#asynchronous-api-testing
bethskurrie
2020-01-06 23:46
@pavel.azarau as Matt said, it should be, but we have found that there were unexpected changes from AWS in the past. It was updated a couple of months ago because of this.
pavel.azarau
2020-01-07 00:16
thanks! please let me know when the list is updated. I?m trying to enable using of webhook (to trigger a job). But at the moment it looks like the problem is on our side :slightly_smiling_face:
pavel.azarau
2020-01-07 00:36
@bethskurrie So I believe I?ve fixed the issue on our side. But I need to whitelist the Pactflow IPs. So please let me know when the faqs page has a correct list
2020-01-10 03:04
Notification from *Pactflow Technical Blog* :ghost:
wesleythomaswilliams
2020-01-15 09:27
Can anyone give guidance of publishing pact verification results? I'm using Pact Flow, the pact is published and verified against with no problem, but I can't get the this property working `pact.verifier.publishResults`. Our tests are Java/Spring and driven by Maven and I've tried three different methods based on official docs I've read and blogs:
1. pom.xml adding to surefire plugin
``` <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<systemProperties>
<property>
<pact.verifier.publishResults>true</pact.verifier.publishResults>
</property>
</systemProperties>
</configuration>
</plugin>```
2. pom.xml adding to pact plugin
```<plugin>
<groupId>http://au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven</artifactId>
<version>4.0.4</version>
<configuration>
<pactBrokerUrl>https://wesley.pact.dius.com.au</pactBrokerUrl>
<pactBrokerToken>...</pactBrokerToken>
<pactBrokerAuthenticationScheme>Bearer</pactBrokerAuthenticationScheme>
<serviceProviders>
<serviceProvider>
<name>sain_service</name>
<protocol>http</protocol>
<host>localhost</host>
<port>1337</port>
<path>/</path>
</serviceProvider>
</serviceProviders>
<systemProperties>
<property>
<pact.verifier.publishResults>true</pact.verifier.publishResults>
</property>
</systemProperties>
</configuration>
</plugin>```
3. In the tests spring annotation
```@SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT,
classes = {
SainSearchControllerV1.class
},
properties = {
"server.port=1337",
"pact.verifier.publishResults=true"
})```
bethskurrie
2020-01-15 09:30
I have a feeling Ron mentioned this to me the other day. The verification step spawns another JVM instance, and the property needs to be passed through to it. Or something.
bethskurrie
2020-01-15 09:31
I'd ask in the pact-JVM channel. It's unlikely to be related to it being Pactflow (rather than the OSS Broker).
wesleythomaswilliams
2020-01-15 09:35
Skipping publishing of verification results as it has been disabled (pact.verifier.publishResults is not 'true')
bethskurrie
2020-01-15 09:37
Yeah, I remember there's some subtlety with this. It's because there's another JVM instance. But I don't know how to fix it.
bethskurrie
2020-01-15 09:38
I'm not a JVM dev. But I remember Ron talking about it. Someone in pact-jvm should know!
wesleythomaswilliams
2020-01-15 11:51
I think my pact has started failing to be published to pact flow, but it doesn't seem to be reporting that to me:
```[DEBUG] (f) pactBrokerAuthenticationScheme = Bearer
[DEBUG] (f) pactBrokerToken = <token>
[DEBUG] (f) pactBrokerUrl = https://wesley.pact.dius.com.au
[DEBUG] (f) pactDirectory = /Users/williamw/IdeaProjects/<provider>/target/pacts
[DEBUG] (f) projectVersion = 0.0.4-SNAPSHOT
[DEBUG] (f) settings = org.apache.maven.execution.SettingsAdapter@6aa648b9
[DEBUG] (f) skipPactPublish = false
[DEBUG] (f) trimSnapshot = false
[DEBUG] -- end configuration --
Publishing '<consumer>_service-<provider>_service.json' ... HTTP/1.1 200 OK```
In Pact Flow, the Pact is showing:
```<consumer>_service (consumer) version: 0.0.4
Pact publication date: 6 days ago
Pact specification version: 3.0.0```
I'll try deleting with https://github.com/pact-foundation/pact_broker/wiki/Deleting-resources and report back
wesleythomaswilliams
2020-01-15 12:24
Deleting all versions of the pact and then publishing again worked ok. Just trying to nail down when/why versions don't get published.
wesleythomaswilliams
2020-01-15 12:28
Ok, I think this must be related to versions. So I get a 200 back regardless, but the pact is only published if there's a version change, is that correct?
bethskurrie
2020-01-15 21:46
@wesleythomaswilliams if you republish with exactly the same consumer version and pact content, it's a no-op. The fact that you're getting a 200 OK instead of 201 Created says to me that's what is happening in your case.
bethskurrie
2020-01-15 21:49
I can see that you're using semantic version numbers. Can you have a read of this document? https://docs.pact.io/getting_started/versioning_in_the_pact_broker
wesleythomaswilliams
2020-01-15 21:59
Thanks @bethskurrie I've not dug into versioning yet, but that's one of the next things on my list. :slightly_smiling_face:
bethskurrie
2020-01-15 22:00
There are subtleties in the versioning that are good to understand. Let me know if you have any questions after reading the docs.
wesleythomaswilliams
2020-01-16 16:23
Had our dev engineers looking at the versioning side of things. I'll provide feedback as I get it, but first thing that was mentioned:
```Point 2:
Ensure that application version numbers are unique. For example, a feature branch that changes the contract should not be able to have the same version number as any other branch - an application version should always identify one specific instance of application code.```
DEV: We don't work quite how that point requires. Our feature branches do not have their own versions
We've been discussing it more and we don't think we can (or necessarily want to) move to having versions per feature branch. However, we think we've identified why we could get into problems.
We're curious though, is this a reasonable description of that type of versioning that Pact would ideally work best with? https://medium.com/@simon.maxen/branch-based-versioning-5ebf6ca2bccb
tjones
2020-01-20 04:53
@wesleythomaswilliams If you?re doing versioning as described in that link, you will want to also tag your pacts with the feature branch, otherwise your `latest` pact might point to any branch
bethskurrie
2020-01-20 04:55
@wesleythomaswilliams what do you mean that your feature branches don't have their own versions? Their git shas would be different wouldn't they?
bethskurrie
2020-01-20 04:59
You don't have to use the exact same string for you application version as what you publish to the broker, but you do have to be able to reverse engineer one from the other.
bethskurrie
2020-01-20 05:00
If you want to use can I deploy, you have to be able to take the app version you're about to deploy and work out what version the broker is using
bethskurrie
2020-01-20 05:01
If you're publishing pacts with versions like `1.4.0` from multiple branches, you're definitely going to get corrupt data in the broker though.
tjones
2020-01-20 05:03
:point_up: Yep. The version you tell Pact and your contracts need to have a 1:1 mapping.
A side point that I?ve been thinking that Pact could provide (or link to) some easy to use autoversioning tools for different ecosystems
tjones
2020-01-20 05:05
Also, I?ve never tried this, but I think it would work if just provide a different version to pact when publishing (eg `"${Version}-${Git SHA}"`). It doesn?t *have* to be your app version.
bethskurrie
2020-01-20 05:07
`x.(y+1).0-featureName-SNAPSHOT` is close, but you'd be overriding the pact with each build, and this can lead to race conditions as your data is then mutable.
bethskurrie
2020-01-20 05:17
I would recommend publishing the pacts and verifications with the git sha appended as metadata (`+abcd123`) as Tim mentioned before, and tagging (broker tagging) with the git branch name. When you use can-i-deploy, you'll need to do the same thing to make the versions match.
wesleythomaswilliams
2020-01-20 09:25
@bethskurrie @tjones I'm quite new to the team I'm in and branches and versioning etc aren't something I've had a deep understanding of before, so I'll feed all this back to the team. We've all read the Pact docs on versioning, but I've not looked at can-i-deploy yet.
As I understand it, we're using semantic versioning, but based on your comments above I think my team may be underestimating problems we could encounter. The only issue we could think we might have is when two people create a branch in the same repo and they have the same version number e.g. 0.0.20-SNAPSHOT and those engineers make changes that result in the pact file being changed.
bethskurrie
2020-01-20 09:27
So you would have very unreliable results if even one of them made a change (and you do tend to make changes to pacts on a branch, so you don't break your main line of development when adding new features)
wesleythomaswilliams
2020-01-20 09:27
Yeah, that was our understanding. The likelihood of that occurring in our team is very slim and we'd planned to mitigate it through education, but if adding the SHA to things resolves that problem, we may be able to do that.
davidmolinero.com
2020-01-26 10:19
Hello, I?ve been sending multiple emails in the last couple of weeks with regard to creating users in http://pactflow.io and I have not received any response so far, can you take a look into it or let me know if there is an issue of any kind? Thanks
matt.fellows
2020-01-26 10:26
I can see 2 of the users haven?t yet set their passwords, but there are no issues I?m aware of.
wesleythomaswilliams
2020-01-28 16:15
If in my provider test I have the `@PactBroker` annotation configured to point to my pact flow account and `pact.provider.publishResults` set to true, am I correct in thinking my provider should then be verifying against pact flow when the test is executed?
If that's correct and with that in mind, I'm trying to work out under what conditions I'd want to run `mvn pact:verify` and thus whether I even need the maven plugin specified in my pom.
matt.fellows
2020-01-28 21:23
mmmm might need @uglyog to confirm but it sounds like it. My understanding is that the annotation will pull down contracts from Pactflow, your test framework (JUnit I assume?) would then do the verification and at the end of it all, publish it back out. What I?m not so sure about is the publishing property
bethskurrie
2020-01-28 21:24
@wesleythomaswilliams there's some black magic with getting the publish property to work!
bethskurrie
2020-01-28 21:26
I think you use the annotations or the mvn config, but I'm just guessing.
uglyog
2020-01-28 22:26
`mvn pact:verify` is used for execution the verification test against a running provider. It is unnecessary if you are using a JUnit test with the Pact annotations to do the same thing.
uglyog
2020-01-28 22:27
You don't need the Maven plugin, just the Pact JUnit libs on your test classpath.
wesleythomaswilliams
2020-02-05 12:59
I published a pact to pact flow today and my provider is reporting it has published the result of the verification, but it's not showing up on pact flow:
```2020-02-05 12:55:00.666 INFO 74045 --- [ main] a.c.d.p.p.DefaultVerificationReporter : Published verification result of 'http://au.com.dius.pact.core.pactbroker.TestResult$Ok@492e988c' for consumer 'Consumer(name=tu_hybris_product_adapter)'```
wesleythomaswilliams
2020-02-05 13:01
Integrations shows:
``` {
"consumer": {
"name": "tu_hybris_product_adapter"
},
"provider": {
"name": "sain_service"
},
"verificationStatus": "never",
"_links": {
"pb:dashboard": {
"title": "BETA: Pacts to show on the dashboard",
"href": "https://wesley.pact.dius.com.au/dashboard/provider/sain_service/consumer/tu_hybris_product_adapter"
},
"pb:matrix": {
"title": "Matrix of pacts/verification results for tu_hybris_product_adapter and sain_service",
"href": "/matrix/provider/sain_service/consumer/tu_hybris_product_adapter"
},
"pb:group": {
"href": "https://wesley.pact.dius.com.au/groups/tu_hybris_product_adapter"
}
}```
Verification has worked for this provider previously with another consumer.
bethskurrie
2020-02-05 21:19
Is there a debug output that gives you the URL of the published verification?
ryan.dens
2020-02-10 21:09
Anyone have recommendations for managing pactflow?s webhooks via an ansible playbook?
bethskurrie
2020-02-10 21:11
@ryan.dens there's an open PR to support webhook upsets that should help with this.
ryan.dens
2020-02-10 21:13
Awesome! We?re not super tied to ansible, just looking for a way to automate the configuration of it and will use the best tool for the job :thumbsup:
matt.fellows
2020-02-10 21:15
But eventually things like users, roles, groups, tokens should all be automatible
ryan.dens
2020-02-10 21:16
Great! Thanks Beth and Matt! Is there a PR I can watch so I know when this feature is available?
matt.fellows
2020-02-10 21:16
Out of interest, how would you plan to test the changes? I'm wondering if we need to provide a test environment as part of provisioning
ryan.dens
2020-02-10 21:37
> Out of interest, how would you plan to test the changes? I?m wondering if we need to provide a test environment as part of provisioning
I?ve been testing on our production account because currently we had relying on automated contract publishing with manual contract verification. Messing with the webhook isn?t messing with anyone?s workflows right now. I was planning on introducing some tests which utilized the https://hub.docker.com/r/dius/pact-broker image to make sure that the webhook is registered successfully
ryan.dens
2020-02-10 21:38
a test environment would probably help with that, but I don?t see us investing time in the tests for this in the near future, mainly because I don?t anticipate it to be a pain point
matt.fellows
2020-02-10 22:52
Thanks Ryan. That?s good feedback. As you know though, the OSS code base doesn?t have all of these administration-y features - the kinds of things you?d probably _want_ to manage as code. Webhooks are one of the exceptions obviously. So keen to see how you go over time.
matt.fellows
2020-02-10 22:52
We won?t jump into creating a sandbox environment yet, but it?s something that?s been floated around for a bit
ryan.dens
2020-02-10 22:54
Yeah, I?m not terribly worried. If we ever decide to undergo a complex change to our webhooks, we would probably just add a second webhook and see how it behaves while letting the original webhook continue its usual behavior
ryan.dens
2020-02-10 22:54
So the test environment isn?t a huge point of need for us, but an interesting idea
matt.fellows
2020-02-10 23:28
FYI just created this https://pact.canny.io/feature-requests/p/create-a-terraform-provider
matt.fellows
2020-02-10 23:28
If you?d rather ansible, feel free to duplicate but replace Terraform with Ansible - and we?ll let the community help us prioritise
ryan.dens
2020-02-11 01:26
Great! Terraform actually works better for us generally, I was guessing there was more likely to be someone who had used an ansible module to accomplish something like this
ryan.dens
2020-02-18 22:39
Any schedule to be aware of for pactflow upgrading the underlying pact broker? I?m on a saas hosted pactflow pact broker. I was attempting to use some new functionality in the CLI to create-or-update a webhook via a preset uuid. I got this error:
```This version of the Pact Broker does not support creating webhooks with a specified UUID. Please upgrade your broker to >= 2.49.0 or use the create-webhook command.```
When I do a GET on my broker?s index page, I see the following header:
```x-pact-broker-version: 2.48.0```
ryan.dens
2020-02-19 00:23
It?s a fantastic feature coming just in time for my team! I feel very lucky that it was already in progress when I started looking for it :slightly_smiling_face:
ryan.dens
2020-02-19 16:07
update > re-ran my build this morning and everything worked as expected :slightly_smiling_face: infrastructure is now merged in
ryan.dens
2020-02-19 16:13
Something I noticed was that the client always responds with the message:
```Webhook "POST http://my.ci.provider.com" created```
I might find useful for the client to give some kind of indication about whether the resource was updated or whether no change was made, but I?m unsure if that makes sense in the context of the rest of the pactflow api.
bethskurrie
2020-02-19 21:07
Thanks @ryan.dens. We could change the message based on the http status code that comes back. It'll be a 201 for created, and 200 for updated.
ryan.dens
2020-02-19 21:08
cool :thumbsup: not a big deal, if it ever becomes one I?d be happy to make the pr myself :slightly_smiling_face:
bethskurrie
2020-02-19 21:08
It should be a 2 minute job, just looking at it while I'm thinking about it
bethskurrie
2020-02-19 21:10
```action = webhook_entity.response.status == 201 ? "created" : "updated"
CommandResult.new(true, "Webhook #{webhook_entity._link('self').title_or_name.inspect} #{action}")```
bethskurrie
2020-02-19 21:18
```pact-broker create-or-update-webhook https://foo.com -X POST --uuid 3aeb3ffc-0ff3-4c97-8d95-a0ef2d86ec0a --contract-content-changed --description "my webhook"
Webhook "my webhook" updated```
paul.davies
2020-02-20 23:34
Good morning guys. I?m hoping someone (looking at @bethskurrie and @uglyog in particular) can help out with my issue. I?m having problems with verifying my tests against my pactflow broker. Publishing is fine and I can hit the verify endpoint using postman. The endpoint I?m hitting is <my broker>/pacts/provider/my-provider/consumer/my-consumer/latest and no issue. It seems that my problem is that no matter what configuration I use in my provider test (directly in java test file, or in my build.gradle), the endpoint I hit is just the broker host, not the full path, as I?ve indicated above. I can connect ok, but I don?t get my contract back, which then throws an exception in the HalClient.kt (line 277: ContentType.getOrDefault(response.entity)) as my response.entity doesn?t exist. Any ideas what I could be doing wrong? More than happy to send across my build.grade and test file if needed. Thanks!
paul.davies
2020-02-21 01:01
I?ll send you both my gradle and test files. Incidentally. My test is an async one using the latest junit5 classes. I?m guessing that the PactVerificationInvocationContextProvider has a problem resolving the consumer/provider url?. Anyway, i?ll send you these items privately shortly. Thanks for responding btw!
matt.fellows
2020-02-21 01:59
:wave: anybody using Pact or Pactflow keen on showcasing their company on our Pactflow website? We?re in the process of putting together some case studies now. If you are interested, we?ll be happy to pick up as much of the legwork as we can - e.g. interview you or send you a Q&A and then can author the post. DM me if you?re keen!
wesleythomaswilliams
2020-03-05 13:51
I've hit a bit of a roadblock with getting Pact Flow whitelisted for access to our AWS hosted Jenkins. So I'm pondering running our own Broker inside our firewall. I just wondered if there are any differences in the features?
matt.fellows
2020-03-05 19:44
Bugger! If you're talking about Pactflow enterprise, it will have the same features as our hosted offering. The OSS broker is obviously more basic.
We are considering deploying via the AWS marketplace which does have options for the use of private link into your VPC. We could also discuss how that could solve your problems if if interest?
matt.fellows
2020-03-05 19:45
P.s. I saw your other emails, we'll respond this morning (in meantime please check spam folder just in case)
2020-03-10 20:31
Notification from *Pactflow* :ghost:
matt.fellows
2020-03-13 11:18
Feedback welcome - it?s still beta so will need some ironing out. But I have acceptance tests now that run against the real thing(s) so that gives me a level of confidence
bernardoguerr
2020-03-13 11:19
Really busy today, but going to try to give it a go this weekend! Will lyk if anything comes up!
matt.fellows
2020-03-13 11:33
I?m thinking for Webhooks, we could create some custom resources that could automate common scenarios - e.g. all of the triggers documented here: https://docs.pact.io/pact_broker/advanced_topics/webhooks/template_lib
tjones
2020-03-23 03:57
Hi hi - it would be cool if the UI had 404s or nice redirects for subsets of the path - for example, I tried to use
`/overview`, but it just loaded a blank page
vitaliy.grigoruk
2020-03-25 10:21
Hi there. It looks like `readonly` bearer token can?t be used to query /pacts/provider/{provider}/for-verification endpoint (I?m getting http 401). With read/write token everything works fine.
vitaliy.grigoruk
2020-03-25 10:23
This actually becomes an issue for us, as developers in our team have access to `readonly` token only.
bethskurrie
2020-03-25 20:20
It's our first non-write POST request! We'll update our auth code @vitaliy.grigoruk. There's actually a GET version of this endpoint, but the structured query parameters become a bit conceptually difficult to understand, so we switched it to POST with a json body.
vitaliy.grigoruk
2020-03-25 20:25
I?m fine with get, if you can give me an example how to write query params in this case.
bethskurrie
2020-03-25 22:06
@vitaliy.grigoruk the query string looks like this `consumer_version_selectors[][tag]=prod&consumer_version_selectors[][latest]=true`
bethskurrie
2020-03-25 22:07
With 2 selectors it would look like `consumer_version_selectors[][tag]=prod&consumer_version_selectors[][latest]=true&consumer_version_selectors[][tag]=master&consumer_version_selectors[][latest]=true`
bethskurrie
2020-03-25 22:10
You can use this repl to build up the query in a structured way and output the query string https://repl.it/@bethesque/PreciousMassiveTree
vitaliy.grigoruk
2020-03-29 13:19
Hi there. Just found a bug in pactflow UI (webhooks editing). When I?m trying to edit existing webhooks:
1) select a webhook between `consumer` and `Provider` and click edit
2) set consumer=ALL
3) save
Expected: webhook is updated
Actual: webhook is not updated
vitaliy.grigoruk
2020-03-29 20:55
Np. Please let me know if I should use another place to report issues specific to pactflow next time :slightly_smiling_face:
tjones
2020-04-01 23:52
It would also be cool if the UI tokens page had snippets for setting them into the appropriate env variables (including pact broker host name, etc)
matt.fellows
2020-04-01 23:54
We will be changing it so that a) users can have multiple tokens (with descriptions) and eventually b) tokens can be attached to the account level (so they aren?t tied to users)
nathan.deamer
2020-04-14 13:49
We have google authentication turned on for pactflow.
Is it possible to have a CI user? Without having to set up another google user?
uglyog
2020-04-14 22:25
Hi @nathan.wilson, you can have a normal user to use for your CI. Just send an email to with the details (it will have to have a working email address), and we will create it for you.
matt.fellows
2020-04-16 02:23
Next week, @uglyog @bethskurrie (http://Pactflow.io) and I will be presenting a lunch and learn session at Pactflow?s parent company (DiUS) on ?Lessons learned whilst building a SaaS business?.
If you?re interested in hearing a bit about our journey, we?d love for you to join us. We?ll cover a bit about our origins, the challenges of balancing OSS and commercial interests and talk about the mistakes we?ve made along the way.
The event starts at 12:30 AEST on Tuesday 21st April:
* 22:30 EDT (Monday evening)
* 19:30 PDT (Monday evening)
* 04:30 CEST (sorry :disappointed: !)
Sign up here https://docs.google.com/forms/d/e/1FAIpQLSfBb7UBVoIfW4ioOyiWTG2ha0zhOpsV5Xs_ie3py-znWYdXUw/viewform
safiyya
2020-04-17 08:59
Is there a way to delete a contract from Pactflow ? I cant see anything in the UI or a DELETE route
matt.fellows
2020-04-17 09:44
Ah, yes. There is a delete API (see http://docs.pact.io). You can delete from the old UI and we'll add it to the new soon.
matt.fellows
2020-04-22 00:56
For those of you who missed the session yesterday, here is the recording: https://www.youtube.com/watch?v=w50VOIRHFcQ&feature=youtu.be
Thanks again for those who attended and asked us difficult questions :)
If you have any feedback questions, you know where to find us!
vitaliy.grigoruk
2020-05-27 13:25
Hi here. I?m unable to add new webhooks via new UI. Both ?create? and ?test? buttons are disabled. We?re on ?team? plan and there should be no limitation to the number of webhooks. Any ideas what?s happening?
matt.fellows
2020-06-09 23:56
FYI as per the several emails over the past month, the TLS1.2 upgrade is going out this morning (night for most of you probably).
matt.fellows
2020-06-09 23:57
No outage is expected, but if you?re using Powershell or really old http clients and haven?t updated the way you interact with our systems, you may experience issues. See https://docs.pactflow.io/docs/powershell-guide
a.smith
2020-06-17 14:06
How can I delete the example pact (`/overview/provider/Example%20API/consumer/Example%20App`) from my Pactflow organisation? I can't see anything obvious in either the UI or the HAL API browser...
uglyog
2020-06-17 22:59
You can do it from the old UI (button on the top), then select the button next to the interaction.
matt.fellows
2020-06-17 23:13
And also, you can issue an HTTP `DELETE` to pretty much any resource and bam, gone
bethskurrie
2020-06-18 10:34
@matt.fellows is our new UI expert, I can sense it's about to be released any day.
nathan.deamer
2020-06-19 14:33
I need to view the subscription portal on pactflow - but don't seem to have access. Can anyone confirm the email of our subscription manager?
2020-06-25 05:17
Notification from *Pactflow* :ghost:
2020-07-07 06:31
Notification from *Pactflow* :ghost:
matt.fellows
2020-07-07 07:13
we are currently working on the design of our Teams and permissions management feature in Pactflow (see https://pactflow.io/pactflow-feature-roadmap/), and would love to get some feedback from teams and organisations about how you use Pact and how we can best ensure it fits your needs. If you?re interested in contributing to the discussion, could you please DM me and we can line up a time to chat?
2020-07-09 07:47
Notification from *Pactflow* :ghost:
bernardoguerr
2020-07-10 19:44
This is really cool!!
The only thing is I still think the Stubbing suffers from the issue that you usually want to stub error codes that might not necessarily be things you want/have easily in a contract. Setting up a 500 error is not necessarily easy on the provider, so you may have some of the other parts of the contract in place to be able to verify but not that. I've seen some teams basically created dummy contracts just so they could use the pact stubbing features with it, but then they didn't publish it.
To that end, maybe it would be nice to accompany this feature with another feature to be able to tag specific interactions of a contract to block verification on the provider (and make that obvious in the UI)? I think it would be a good way to enable teams to have stubs even if they don't necessarily want/have the ability to verify that yet against a provider.
matt.fellows
2020-07-13 06:31
Minor UI change that?s been requested by a few - you can now copy some useful URL templates and delete integrations (or individual pact files) directly via the UI
matt.fellows
2020-07-14 04:15
The other challenge with the current stubbing is that you may want to be able to have more flexible stubs. Right now, you have to have exact matches on each of the contract paths, but that won?t work with provider states, for example
matt.fellows
2020-07-14 10:54
June edition of the Pactflow POST is out https://mailchi.mp/pactflow.io/pactflow-post-july-2020
2020-07-17 03:46
Notification from *Pactflow* :ghost:
2020-07-29 00:44
Notification from *Pactflow* :ghost:
vitaliy.grigoruk
2020-07-30 07:42
Hi there, looks like there is an issue with pactflow pact broker (our provider verification job started to fail today):
```error: INFO: Fetching pacts for api from https://******.http://pact.dius.com.au with the selection criteria: latest for tag master, latest for tag release
/app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-1.49.3/lib/pact/hal/entity.rb:102:in `assert_success!': Error retrieving https://******.http://pact.dius.com.au/pacts/provider/api/for-verification status=500 {"error":{"message":"An error has occurred. The details have been logged with the reference pYEwtxIazJ","reference":"pYEwtxIazJ"}} (Pact::Hal::ErrorResponseReturned)
from /app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-1.49.3/lib/pact/hal/link.rb:65:in `post!'
from /app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-1.49.3/lib/pact/pact_broker/fetch_pact_uris_for_verification.rb:68:in `pacts_for_verification_entity'
from /app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-1.49.3/lib/pact/pact_broker/fetch_pact_uris_for_verification.rb:54:in `pacts_for_verification'
from /app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-1.49.3/lib/pact/pact_broker/fetch_pact_uris_for_verification.rb:39:in `call'
from /app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-1.49.3/lib/pact/pact_broker/fetch_pact_uris_for_verification.rb:33:in `call'
from /app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-1.49.3/lib/pact/pact_broker.rb:18:in `fetch_pact_uris_for_verification'
from /app/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.84.0/pact/lib/vendor/ruby/2.2.0/gems/pact-provider-verifier-1.30.1/lib/pact/provider_verifier/aggregate_pact_configs.rb:45:in `pacts_for_verification'```
matt.fellows
2020-07-30 08:16
Thanks David and sorry folks. There was a recent dark launch with some preparations for our teams and permissions feature
2020-08-11 01:46
Notification from *Pactflow* :ghost:
denitsa.kulezich
2020-08-19 13:41
Hi, we have an issue with Pactflow: when trying to delete a given contract is says: Unable to delete resource, as it was not found (the API returned 404).
The contract exists and is visible when I query it in HAL browser.
matt.fellows
2020-08-19 22:53
I need to check the code, I think you'd get a 403 if you tried to delete with a read only token. How are you trying to delete it? Using API, UI or HAL browser?
heytaco
2020-08-25 03:57
Hi there! My name is HeyTaco!, and you can use me to give people tacos to show your appreciation. My tacos will spread joy through Slack!
alecgerona
2020-09-01 06:20
Looking into using Pactflow for our company seeing that setting up pact-broker, with the various tools it'd need to work (CI changes, hack to make verifiers or work, setting up stub server) might be a bit too much to maintain going forward.
alecgerona
2020-09-01 06:21
So I saw the pricing is at $99 for 10 devs. And a big jump to more than double the price for 25. We're not a big company by any means and the price might be too steep going in.
2020-09-04 02:56
Notification from *Pactflow* :ghost:
2020-09-04 03:22
Notification from *Pactflow* :ghost:
2020-09-04 03:22
Notification from *Pactflow* :ghost:
2020-09-22 07:39
Notification from *Pactflow* :ghost:
scwang
2020-09-23 05:27
I am evaluating pactflow by creating a webhook for slack, but the webhooks fails to executed
uglyog
2020-09-23 05:33
That's to do with the slack APIs, can you try that request in something like Postman?
bethskurrie
2020-09-23 05:34
can you post a screenshot of your webhook and the command in your curl (without auth tokens)
bethskurrie
2020-09-23 05:34
302 redirect usually happens when you are hitting a UI, not an API, which hints to me that you haven't set the content type accept header properly.
scwang
2020-09-23 05:41
this is the curl I am using `
```curl -X POST -H 'Content-type: application/json' --data '{"text":"Hello, World!"}' https://hooks.slack.com/services/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```
scwang
2020-09-23 05:41
webhook `
```{
"request": {
"method": "POST",
"url": "https://hooks.slack.com/services/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",
"body": {
"channel": "automation",
"username": "Quality Team Bot",
"text": "a new pact file is created",
"icon_emoji": ":ghost:"
}
}
}```
bethskurrie
2020-09-23 05:45
can you just show me a screenshot of the webhook as it looks in the UI please?
bethskurrie
2020-09-23 05:49
it's possible that that example is out of date as well - I know slack's notification API changed sometime in the last few years.
bethskurrie
2020-09-23 05:51
here are the slack docs https://api.slack.com/messaging/webhooks#posting_with_webhooks
bethskurrie
2020-09-23 05:57
well, I've replicated the issue! I can do the request with curl, but I get a 302 in Pactflow.
bethskurrie
2020-09-23 06:05
If you send an email to I'll be able to let you know when we've identified the cause and fixed it if necessary
scwang
2020-10-01 01:03
I am evaluating pactflow and cypress, I am wondering how I can upload pact file from within Cpress tests ? is there any plugin avaliable
uglyog
2020-10-01 01:06
The Pact files can all be uploaded after the test run. This doesn't have to have anything to do with Cypress. You can use the standard Pact tools to do that.
scwang
2020-10-01 01:10
thanks , I am wondering can I call pact API from cypress and generate pact files ?
uglyog
2020-10-01 01:15
You don't need to call anything, the pact file will be generated if the test succeeds.
scwang
2020-10-01 01:18
sorry I did not get this , I have a bunch of cypress tests as my consumers , how I can make cypress generate pact files ?
uglyog
2020-10-01 01:24
There is no plugin, yet. Cypress has a task mechanism that allows you to execute code on the Node environment. See https://github.com/pactflow/example-consumer-cypress for an example
matt.fellows
2020-10-01 03:34
I saw you booked a meeting to chat tomorrow @scwang , i?ll ping you offline to get a bit more info
xandebianchi
2020-10-05 13:12
Hi guys! Could you help me please ?
I?m starting with http://pact.io and pactflow, using Android, but it?s a generic doubt.
I could create the classes and unit test, and generate the pact file.
I published it using Gradle task.
1. My doubt is about the Provider. Do I need to create the provider part or this is a task for the Microservices responsibles?
2. Also, how can I test to Verify a Pact? Where can I add the URL from Microservice for pactflow to analyze the results ?
3. For pactflow to execute correctly, do I need to put the tasks in CI ?
xandebianchi
2020-10-05 16:17
Is it possible to publish verification through Pactflow simulating the Provider ?
bethskurrie
2020-10-05 23:31
Hi @xandebianchi. I recommend you watch this animation https://pactflow.io/how-pact-works/#slide-1 and read this document https://docs.pact.io/pact_nirvana
bethskurrie
2020-10-05 23:32
Also, if you have time, this workshop should make everything clear https://docs.pactflow.io/docs/workshops/ci-cd/
bethskurrie
2020-10-05 23:32
A verification should be against a real provider - otherwise, there's no point to it.
bethskurrie
2020-10-05 23:33
> Do I need to create the provider part or this is a task for the Microservices responsibles?
Who creates the provider is up to your organisation, and nothing to do with Pact.
bethskurrie
2020-10-05 23:34
> For pactflow to execute correctly, do I need to put the tasks in CI ?
Yes.
bethskurrie
2020-10-05 23:34
You'll find instructions on how to set up your CI in both the "effective pact set up guide" and in the workshop
matt.fellows
2020-10-05 23:35
> 1. My doubt is about the Provider. Do I need to create the provider part or this is a task for the Microservices responsibles?
correct, the team looking after the microservice needs to verify your pact on the provider side. What language are they doing it in?
phil.endsley
2020-10-07 03:43
Is there a page somewhere that outlines differences between pactflow and oss? Looking for premium features, and other stuff to sell me on "why should we pay for this"
(I don't make that decision, but I can get it to the people who do :slightly_smiling_face: )
bethskurrie
2020-10-07 03:45
At the moment, the differences are mostly around user management, but the very next thing off the rank is to start focussing on provider driven workflows, and making it easier to get started with contract testing.
campellcl
2020-10-09 00:15
Hey all, I was trying to get started with the http://pactflow.io pact broker but I am about to give up and host my own non-commercialized version in Docker. I keep getting `HTTP 405 "Method Not Allowed for URL"` when trying to issue either POST or PUT requests to our PactFlow broker endpoint. I am including the bearer token in the HTTP headers with the call, and am using OpenSSL > v1.0.0 which supports TLS v1.2. Does anyone know why I would continuously get this response when attempting to publish a pact to PactFlow? I can't find anything in the docs, except for the bit on using TLS v1.2. I am using Python 3.8 and the `requests` module to handle the HTTP communication. I haven't done any setup steps in PactFlow, except for copying the authorization credentials and the "pact template URL" in order to attempt to publish a pact to the "pact template URL".
matt.fellows
2020-10-09 00:17
If you?re getting a `405` then you?ll get the same on the OSS broker (the underlying code is the same), so it?s something else
matt.fellows
2020-10-09 00:17
If you can please share the request, i?m sure we can get to the bottom of it
campellcl
2020-10-09 00:21
@matt.fellows Sure, let me grab it. Yup, of course! Thanks. The path probably is wrong, I didn't have a clear idea from the docs what the appropriate path was, if not the "pact template URL" available via the GUI. Do you need to register the Provider or Consumer prior to the PUT call to the broker? Or does the initial PUT call register a new Provider and/or Consumer?
campellcl
2020-10-09 00:42
@matt.fellows Turns out it is somewhat non-trivial to get the formatted HTTP request object from `requests`. Sorry about that. Here is a string version that is identical except for the carriage returns and newline escape characters. But of course, that shouldn't matter for this. I replaced secret info with `<some_text>` , and the `b\'` is serialized python for `b'` which is a flag representing the start of the binary data which contains the JSON serialized pact.
```'-----------START-----------\nPOST https://<our_org>.http://pactflow.io/pacts/provider/TaskOrchestratorV2/consumer/JobManagementServiceV3/latest/dev\r\nContent-Type: application/json\r\nAuthorization: Bearer <bearer_token>\r\nContent-Length: 1016\r\n\r\nb\'{"consumer": {"name": "TaskOrchestrationV2"}, "provider": {"name": "JobManagementServiceV3"}, "interactions": [{"description": "an initialization request from the TaskOrchestrator", "providerState": "An existing JobConfig_0 item", "request": {"method": "post", "path": "/"}, "response": {"status": 200, "headers": {}, "body": {"Service": "JobManagementServiceV3", "Id": "fc763eba-0905-41c5-a27f-3934ab26786c", "Result": "Success", "Result_Params": {"StateMachine_Event": {"State": {"State_UUID": "fc763eba-0905-41c5-a27f-3934ab26786c"}, "Event": {"Event_Name": "Success", "Event_Context": {"Job_Id": "Job_0"}}, "Event_Source": {"ARN": "someArn"}}}}, "matchingRules": {"$.body": {"match": "type"}, "$.body.Id": {"match": "regex", "regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"}, "$.body.Result_Params.StateMachine_Event.State.State_UUID": {"match": "regex", "regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"}}}}], "metadata": {"pactSpecification": {"version": "2.0.0"}}}\'' ```
campellcl
2020-10-09 00:44
@matt.fellows The response I get is:
``` requests.exceptions.HTTPError: 405 Client Error: Method Not Allowed for url: https://<our_org>.http://pactflow.io/pacts/provider/TaskOrchestratorV2/consumer/JobManagementServiceV3/latest/dev```
matt.fellows
2020-10-09 00:45
OK cool yep, the format of the URL is wrong. It should be:
```/pacts/provider/PROVIDER/consumer/CONSUMER/version/CONSUMER_VERSION_NUMBER```
matt.fellows
2020-10-09 00:45
Change format to this:
```/pacts/provider/TaskOrchestratorV2/consumer/JobManagementServiceV3/version/CONSUMER_VERSION_NUMBER```
matt.fellows
2020-10-09 00:46
Personally, I?d just use https://docs.pact.io/implementation_guides/cli to publish pacts (and run other useful pact commands)
campellcl
2020-10-09 00:49
Then why does the http://PactFlow.io broker produce the following URL when you click "Copy pact template URL":
```https://<our_org>.http://pactflow.io/pacts/provider/Example API/consumer/Example App/latest/%3Cinsert%20tag%20here%3E```
That url is
```/pacts/provider/PROVIDER/consumer/CONSUMER/latest/SOME_TAG```
not:
```/pacts/provider/PROVIDER/consumer/CONSUMER/version/CONSUMER_VERSION_NUMBER```
I get that you can manually specify a version number if you don't want to use the `latest`. Does `latest` only work if you already have registered a Pact?
bethskurrie
2020-10-09 00:51
it just returns whatever is latest. but you don't publish to latest (unlike docker). you publish with a specific consumer version.
bethskurrie
2020-10-09 00:51
This is a good doc to read https://docs.pact.io/getting_started/versioning_in_the_pact_broker
bethskurrie
2020-10-09 00:52
There is a bug in that template URL that we need to fix, thanks for reminding me.
bethskurrie
2020-10-09 00:52
You can read about the URLs here https://docs.pact.io/pact_broker/publishing_and_retrieving_pacts
bethskurrie
2020-10-09 00:54
You'll have the same issue with the OSS broker. The client code is wrong.
bethskurrie
2020-10-09 00:57
As Matt said, it's best to use the pact-cli to publish, as there are some subtleties around publishing that will not work as expected unless you do it right.
campellcl
2020-10-09 00:57
You mean the display of the encoded `%20` instead of the space character ` `? Eh, most devs can figure that one out haha. But sure, I'm glad to help. I will give those documents you linked a proper read. I only skimmed it originally. Thank you for mentioning them.
bethskurrie
2020-10-09 00:58
Here's an example of publishing https://github.com/pact-foundation/pact-ruby-cli#publish-pacts
bethskurrie
2020-10-09 00:59
Many implementations have a native interface for publishing, but they're mostly calling that same code under the hood.
campellcl
2020-10-09 01:01
I need to read more about the integration with CI/CD pipelines (we aren't quite there yet). But right now we are doing the publishing from the code that generates the actual pact file. Hence, not using the CLI. Eventually a CI/CD system will be driving the scripts that create and upload the pacts automatically, when the content of the pact changes. Although we could get the CI/CD system to do it via CLI if need be.
campellcl
2020-10-09 01:02
That is what I was using for reference, except for the HTTP Auth, because you all use tokens instead.
bethskurrie
2020-10-09 01:03
the rest of the code is probably fine, but don't manually upload the pact
campellcl
2020-10-09 01:04
Ah, I figured that is what you are supposed to do. Generate the pact file, then upload it to http://PactFlow.io (or a custom Pact Broker).
bethskurrie
2020-10-09 01:06
Typically, you generate a pact and use the CLI or one of the native language wrapper interfaces to publish. I'm not sure why python hasn't done this yet.
bethskurrie
2020-10-09 01:08
Let me know which route you decide to go down. If you must use the API directly, I'll have to explain how to create the tags. But it would be a lot simpler to use the cli. If you're not using docker, you can also use this https://docs.pact.io/implementation_guides/cli#pact-ruby-standalone
bethskurrie
2020-10-09 01:09
this is what the python tool wraps anyway. you already have it running on your machine now.
campellcl
2020-10-09 01:09
Yeah, if you google "pact python publish pact" or some combination of that, that `pact-foundation` example is the first to come up. The docs do recommend using a CLI though. But normally that is for ease of user experience (e.g AWS CLI)
bethskurrie
2020-10-09 01:10
I had a feeling the python interface was mostly done and just needed to be tweaked and documented.
campellcl
2020-10-09 01:15
I would be using Docker. Let me read up on how you recommend doing the CI/CD integration. I saw you have some documentation on it, but I haven't read it yet. So I won't bother you with questions you have already answered in those docs. We would be hosting the Pact broker in a dockerized container.
bethskurrie
2020-10-09 01:16
sorry to throw lots of docs at you but here's the guide to setting up CI https://docs.pact.io/pact_nirvana
campellcl
2020-10-09 01:18
Ah, right. To clarify, I mean we will be running the pact broker and pact verifier in Docker. Right now, with my limited knowledge of how this integrates into a CI/CD process... I believe our CI/CD system would also be executing the code that generates the pacts, and uploading or sharing them with the Pact Broker. Which means the CI/CD system would have to run the CLI in order to generate the pacts. If I can't use the native API to do it, which still doesn't appear to work with an arbitrary UUID for a version tag.
campellcl
2020-10-09 01:33
@bethskurrie So, we are already using Dockerized Jenkins, which in turn is running Dockerized build agents to mimic deployment environments that are different from our local machines. If we were to use the CLI (which is provided as a Docker container) we would need our Dockerized build agents to access Docker in order to spin up the CLI Docker container. That is three levels of Docker inception, which I would rather avoid. Two is bad enough as it is.
bethskurrie
2020-10-09 01:48
It's a PUT to `/pacticipants/{pacticipant}/versions/{version}/tags/{tag}` with a content-type of `application/json` (but you can use an empty body).
bethskurrie
2020-10-09 01:50
You mentioned that you were were planning on hosting your own Pact Broker. Would you mind sharing what factors make that a preferable option to Pactflow for you?
matt.fellows
2020-10-09 02:12
> If we were to use the CLI (which is provided as a Docker container) we would need our Dockerized build agents to access Docker in order to spin up the CLI Docker container. That is three levels of Docker inception, which I would rather avoid. Two is bad enough as it is.
Yes, that sounds painful. The CLI is also available as a standalone as Beth mentions. It is actually packaged by the Python implementation under the hood already - so if pact-python is installed, you already have it. I can?t recall exactly where it gets stored, but something like a `find <path to where python puts libs> -name pact-broker` should get you close enough
campellcl
2020-10-09 02:20
Sure! Primarily, this experience haha. Not being able to just copy an endpoint URL, swap out the version number, and publish a pact file to the broker. PactFlow is advertised as plug-and-play to be able to test out Pact without having to host our own broker. It hasn't been plug-and-play at all. Maybe it is for people who know what they are doing, but not for someone who is new to Pact. I read all the Pact docs (and have a vague concept of how it all should work), I have that python example I linked, and that's all the prior knowledge I've got. I expect a bit more hand holding from a plug-and-play solution, and less passing the buck to the original Pact documentation. The Pact documentation doesn't tell me how to get up and running with PactFlow, it tells me the role that PactFlow is supposed to fulfill in the Pact ecosystem.
You and Matt were very helpful on here, but I would never have gathered that I was not supposed to be issuing a PUT call to the API directly, at that template link you can copy out of the PactFlow GUI. I don't want to have to know everything about pact versioning, and tagging intricacies, from the original docs to use a supposedly turn-key solution such as the PactFlow broker. So having an interactive getting started guide is really a necessity. Right now PactFlow just kinda dumps you on the dashboard with a few sample pacts and says: good luck! How about explaining how to publish a pact to the broker, and how to create a tag, and if the provider and consumer need to exist prior to publishing a pact, and the template URL that should be used as a default URL to publish an example Pact to in PactFlow. That is all useful information for getting up and running in PactFlow that is not really anywhere to be found.
We don't really have access to the error logs of the PactFlow broker, and can't really figure out why it keeps just HTTP 405-ing all the requests, even with the changes Matt recommended. If it was our own hosted version, we would be able to debug the logs of the OSS broker much easier (I would presume).
campellcl
2020-10-09 02:22
So basically it comes down to, not enough visibility into the logs, not being turn-key enough for a productized solution, and not enough materials explaining how to actually get up and running with PactFlow irrespective of Pact overall. If I have to read all the original Pact docs and understand them to this degree, I might as well just deploy my own broker.
campellcl
2020-10-09 02:29
I would expect a getting started walkthrough on PactFlow which references the original Pact documentation at certain points. I'm not opposed to documentation reading. It's just way more documentation reading than I would like for what is advertised as a turn-key solution. At that point, it's no longer plug-and-play. I've read all the docs several times to get the thing almost working (it still doesn't work yet). So there isn't really a benefit to using a commercialized solution, when a prerequisite is understanding enough of Pact itself, that the benefit in terms of decreased implementation time and ease of use is negligible.
bethskurrie
2020-10-09 02:30
Improving our onboarding process is actually something we're working on at the moment, and your feedback is very relevant.
bethskurrie
2020-10-09 02:33
Have you seen the example python consumer and provider btw? https://github.com/pactflow/example-consumer-python
bethskurrie
2020-10-09 02:34
Would that have helped? We're considering capturing the relevant technologies at sign up so we can send out the example for the relevant languages.
campellcl
2020-10-09 02:36
Of course! Happy to help any way I can. Thank you to both you and Matt for taking the time to help. It's well past the end of the working day here, so I'm going to pick this up tomorrow. We are not against using PactFlow, we will still probably keep at it, and try and get it working again tomorrow. We just have a tight dev schedule, and if a turn-key solution doesn't get up and running within a certain window of dev time, we move on to the next approach (e.g. hosting it ourselves). Otherwise, we will end up with no contract testing infrastructure at all.
campellcl
2020-10-09 02:37
I'm not sure I had seen that example, I've made note of it to reference tomorrow, that does look helpful. The main one I had seen was https://github.com/pact-foundation/pact-python/tree/master/examples/e2e
bethskurrie
2020-10-09 02:38
There are a lot of docs and code spread throughout the OSS and commercial product, so it's hard to know where to find them.
campellcl
2020-10-09 02:41
Yeah, agreed. Which is why having some PactFlow specific documentation (even if it is just appropriately placed links to other docs in a getting-started kind of ordering) would be very helpful. We were looking at https://github.com/reecetech/pactman as well.
bethskurrie
2020-10-09 02:42
The joys of OSS. You can't stop people creating and publishing their own solutions and then walking away from them.
campellcl
2020-10-09 02:42
Yeah, we noticed that. Which is why we went with your version, after seeing it was/is actively maintained.
matt.fellows
2020-10-09 03:08
I know you?re probably off to bed Chris, but thanks heaps for this feedback - it?s something we know and are (painfully) aware of, but getting honest feedback like this is really appreciated!!
ruth
2020-10-09 13:14
This is probably a realy silly question, but I am confused on how to download schema from pactflow to the provider. I?ve been googling it and most articles I found says I need to do it but doesnt give me guidance on how.
ruth
2020-10-09 13:16
https://pactflow.io/blog/contract-testing-using-json-schemas-and-open-api-part-3/
^^ was reading the blog post and realized there is a difference between verification and testing.
I want to download the schema so that I can test against it in the provider to make sure that we are still doing whats expected by the consumer.
wesleythomaswilliams
2020-10-09 13:18
In our use of Pact Flow with Java, we provide details about the PactBroker at the top of the Provider test class:
```@PactBroker(
host = "${pact.broker.host}",
scheme = "https",
port = "443",
authentication = @PactBrokerAuth(token = "${pact.broker.token}"))```
wesleythomaswilliams
2020-10-09 13:21
And we setup values in our maven pom:
``` <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables>
<pact.broker.host>http://MYTEAM.pact.dius.com.au</pact.broker.host>
<pact.broker.token>MYTOKENGOESHERE</pact.broker.token>
<pact.verifier.publishResults>false</pact.verifier.publishResults>
<pact.provider.version>${pact.provider.version}</pact.provider.version>
<pact.provider.version.trimSnapshot>false</pact.provider.version.trimSnapshot>
</systemPropertyVariables>
</configuration>
</plugin>```
wesleythomaswilliams
2020-10-09 13:25
In our jenkins pipeline, we then do this:
```' mvn -B -f pom.xml test '
' -Dpact.provider.version=' + env.GIT_COMMIT +
' -Dpact.verifier.publishResults=true' +
' -Dpact.provider.tag=' + env.GIT_BRANCH```
How you tag and version things will probably be different, but you can see that when we execute our provider tests in our build process we override the default `publishResults` flag to set it to true. We default it to false so we don't accidentally publish results on local builds.
wesleythomaswilliams
2020-10-09 13:26
Btw, not a silly question. This stuff twisted my head in knots for quite a while.
tritorto
2020-10-20 05:40
Hey guys, just a question - do you support MFA with pactflow logins at all or plan to?
sklein
2020-10-21 14:11
What's the easiest way in the UI to get the url that would fire from the webhook integration for a newly added pact? I'm putting together an internal tutorial/walkthrough and am struggling to find an easy way to find this.
sklein
2020-10-21 15:06
FYIses... I know that I can grab the url from the pact overview screen, but I noticed that the url sent via the webhook is actually different (looks like its using an actual id for the pact)
bethskurrie
2020-10-23 05:57
`${pactbroker.pactUrl}` . You can't hardcode that URL or the same pact will be verified each time!
bethskurrie
2020-10-23 05:57
If you click on the ? mark next to the template body, you'll see a list of available template parameters.
bethskurrie
2020-10-23 05:58
Here are the docs for webhooks https://docs.pact.io/pact_broker/webhooks/debugging_webhooks
bethskurrie
2020-10-23 05:59
This page needs to be moved, it's too hard to find where it is: https://docs.pact.io/pact_broker/advanced_topics/api_docs/webhooks
sklein
2020-10-23 13:28
Understood about the template. This was explicitly for use in a demo just to simulate the webhook in Jenkins. My question was more around what Id is being used here. For example, this is the url sent in the webhook `https://my-company.pactflow.io/pacts/provider/example-python-provider/consumer/example-python-consumer/pact-version/3751957d917d2b51a1e24c1ab8918d6c4d328fd5/metadata/Y29uc3VtZXJfdmVyc2lvbl9udW1iZXI9NjQ0NjdhYzM4MGNmJmNvbnN1bWVyX3ZlcnNpb25fdGFnc1tdPWZlYXR1cmUlMkZwcm9kdWN0LXRlY2gtZGVtbw` The version listed here is `3751957d917d2b51a1e24c1ab8918d6c4d328fd5` but the actual tagged version of this contract is `64467ac380cf` . I was just trying to understand why the difference in version exists.
bethskurrie
2020-10-27 07:57
@sklein one is the *consumer* version and one is the *pact* content version.
bethskurrie
2020-10-27 07:58
The consumer version is what it gets published with. You can read about the URLS here https://docs.pact.io/pact_broker/publishing_and_retrieving_pacts
bethskurrie
2020-10-27 07:59
The version that gets used in the webhook is the *pact* version (that is, the sha of the underlying pact JSON content).
bethskurrie
2020-10-27 08:00
If you're just trying to simulate a webhook in jenkins, you can just use the same URL that the pact gets published with.
bethskurrie
2020-10-27 08:03
The easiest way to get the URL is to click on the "view pact" button, and then copy the URL in the browser, discarding any query parameters.
bethskurrie
2020-10-27 08:11
Pactflow users - I'm after a bit of feedback regarding the use of the API tokens. Most companies start using a Pactflow account in "evaluation" mode, by doing a proof of concept, and then transition into "production" mode, when they integrate it for real into their CI/CD pipelines. If this is a reflection of your experience, can you please tell me if anyone (apart from the CI/CD user) uses the *write* token from their local machines once in "production" mode? If the evaluation -> production model I described doesn't match how you used it, how did your experience differ?
tjones
2020-10-27 08:23
this isn't actually a direct answer to the question, but I do remember it being a pain to share the tokens around.
I just realised that what I *really* want is the ability to add ssh keys to pactflow, and have that just work with all the tooling. Bonus points if Pactflow can read my ssh keys from github when I sign in.
I also realise that this would be a substantial change to the way things work right now.
tjones
2020-10-27 08:27
The more I think about this, the more I think it's a good idea. Authorisation would "just work" in many cases. Also it would be fun to implement, which is definitely bonus points
matt.fellows
2020-10-27 08:59
SSH keys is a terrible idea, not because it's inherently bad, but for no other reason than API tokens confuse people enough. So SSH keys would result in many heads exploding
matt.fellows
2020-10-27 09:04
But on a more serious note, we now have system users with API tokens tied to them (not real/human users)
matt.fellows
2020-10-27 09:06
SSH tokens would make the API itself harder to communicate with though ergonomically (think HMAC etc). You now essentially must use a client lib to talk to the API
kong
2020-10-27 09:29
hm.. SSH keys as a form of auth&auth with APIs? does require some investigation. I am partially with @matt.fellows there, it is definitely a lot more work to get it working on clients and Windows can be a very different beast again.
matt.fellows
2020-10-27 09:30
I wouldn't be surprised if a customer asked for HMAC (or MASSL) at some point tho, which certainly isn't simple
kong
2020-10-27 09:30
Not impossible, though. Would wonder how much confusion will ensue for both API token and SSH key support
matt.fellows
2020-10-27 09:31
I can see it making some sense for developer tools. Are there any examples out there that come to mind we can model from?
wesleythomaswilliams
2020-10-27 09:40
I don't think we ever formally went from evaluation to production, beyond upgrading from free pactflow to paid for my team's account and basically everything was done by me and isn't yet complete. At the time, all the services we were testing were owned by my team and the API key was embedded in the jenkinsfile and pom for every microservice. I'd rather put it in one place and somewhere secure, but that wasn't a high priority. Is that helpful at all?
sklein
2020-10-27 13:22
Ah it's the sha. That makes total sense. I learned something! Thanks @bethskurrie!
sklein
2020-10-27 13:32
The only person that uses that write token locally is me (usually during some form of demo), but normal workflow is as you describe here. Only the CI/CD system uses the R/W token and everyone else grabs the RO token for local development purposes.
tjones
2020-10-28 02:53
@matt.fellows: I'm not proposing that you can put an ssh key in the header (especially since your public key is ... well... public). I'm proposing that you could also accept signed payloads in some way, and that the ssh public keys could be automatically (or one click) propagated to pact-flow from github.
The solution I'm proposing would be less config from the user, could work alongside the existing solution, and if done well people wouldn't even realise they were authenticating automatically.
nasir.amin
2020-10-28 11:13
Hi all,
We are in the middle of experimenting with pactflow with GitHub Actions before adoption.
We are following the CI/CD example and at the stage where we want a webhook trigger the provider build.
However, the docs talk about other CI systems but nothing on how to do this using GitHub actions.
I would really appreciate some guidance on this please
matt.fellows
2020-10-28 11:20
The https://t.sidekickopen80.com/s1t/c/5/f18dQhb0S7kF8bWHGMW58WrZP2zGCwVN8Jbw_8QsRtKVngSwj5w6K98W16gGDH8BNVxm101?te=W3R5hFj4cm2zwW3H8_MV1JDWTLW4fH0kN4m8k_SW1LwtWL4cgBJ5W49N0jC43Vs87w3F7TGD48H2&si=8000000002979937&pi=f67a3602-bc68-4ab0-d44a-b7ff1da6555f has recently been updated to work with GH actions, so I think you should be able to use that as a basis. In summary:
Step 1: create a webhook to trigger a custom Pact-only step of the provider build. It will use a custom event of your choosing to be the source of the trigger (in this case pact_changed). See https://github.com/pactflow/example-provider/blob/master/Makefile#L83-L93 for how to programmatically create that webhook (or use it as a basis to do via the UI)
Step 2: create a workflow (provider build) to receive and act on this GH event https://github.com/pactflow/example-provider/blob/master/.github/workflows/verify_changed_pact.yml.
Step 3: update your provider https://t.sidekickopen80.com/s1t/c/5/f18dQhb0S7kF8bWHGMW58WrZP2zGCwVN8Jbw_8QsRtKVngSwj5w6K98W16gGDH8BNVxm101?te=W3R5hFj4cm2zwW3P28X24hCPvhW43T4P43z8p26W3ZZpwX3Kchm7W45Rjxp45TRgYW3SZ8dl1LvFJtW3BLhfh4cQgTgW1LDLhp1LCtTLW3HcvCH1LCtTLW3HcvCH1JDWTLW4fdKsf4cQb1nn4c9xbz113&si=8000000002979937&pi=f67a3602-bc68-4ab0-d44a-b7ff1da6555f to verify the _specific_ pact that was changed (instead of fetching by tag as you would a standard build). You can see that on https://t.sidekickopen80.com/s1t/c/5/f18dQhb0S7kF8bWHGMW58WrZP2zGCwVN8Jbw_8QsRtKVngSwj5w6K98W16gGDH8BNVxm101?te=W3R5hFj4cm2zwW3P28X24hCPvhW43T4P43z8p26W3ZZpwX3Kchm7W45Rjxp45TRgYW3SZ8dl1LvFJtW3BLhfh4cQgTgW1L4_zX4fHT3sW1LGnX-3XX4Q5W4mHZGs3K8Qc8W4rtQLv3zdZ6pW3H2mj53Fdr-Jw3_Wzdx4Br2&si=8000000002979937&pi=f67a3602-bc68-4ab0-d44a-b7ff1da6555f it converts a field in the incoming webhook (initiated from Pactflow) to the environment variable PACT_URL.
I hope that helps! If you haven?t already, i?d highly encourage running through that CI/CD workshop. Beth?s done a fantastic job of it.
nasir.amin
2020-10-28 11:24
However, are you sure that the GitHub actions guidance is published in the CI/CD workshop as it is still talking about Travis CI. I can't see anything about GH Actions in the guide there.
The other links you have mentioned is exactly what we were thinking of doing i.e. repository dispatch. However, I was hoping for a more neater way.
matt.fellows
2020-10-28 11:26
> However, are you sure that the GitHub actions guidance is published in the CI/CD workshop
hm, the content may not yet reflect GH actions, but I believe the code has been done for it. @bethskurrie will be able to elaborate more (I?ve not tested it personally, but it? looks like it should work!)
The principles in that workshop are the same regardless though, so even if you don?t do the steps, reading the workshop will explain why things have been done the way they have. The specific CI system is kind of irrelevant then
tritorto
2020-10-29 20:31
Not locally - for our team, any local development is done against a local docker pact broker on each devs machine, only our CI/CD interacts with pactflow
bethskurrie
2020-10-29 21:38
@tritorto you can use local file URLs rather than a local broker if that's a hassle.
bethskurrie
2020-10-29 21:48
@nasir.amin the docs are still be updated. I've got them on a branch, but haven't tested and merged them yet.
tritorto
2020-10-29 21:50
Thanks for the suggestion! we actually find it pretty straightforward using the docker container locally... good to know there are options though
bethskurrie
2020-10-29 21:51
I tend to have two verification tasks if I'm working locally. One that uses the broker, and one that uses the file system.
nasir.amin
2020-10-30 17:30
@bethskurrie Thanks a lot. You have done a lot of good work already. We are trying to establish an end to end pact workflow between a consumer and a provider.
? Consumer publishes a newer version of pact to the broker
? Consumer waits for pactflow to trigger the provider build.
? Provider verifies the contract
? Consumer build gets the result on a PR in GH
? Consumer can now deploy or not depending on the above result.
? However, we are a bit unsure about consumer waiting for the provider to verify the contract automatically and get the result as part of Can-i-deploy in one build rather than having to run a new build for the consumer. I think there might be a bit of confusion on our part here
deepika.krishnakumar
2020-10-30 19:43
Hi. Is there a way to set http proxy before accessing pactflow from my provider. I?m trying from a java application. Any way of setting http.proxy. I was able to set it and publish to pactflow but the provider is unable to connect to it
deepika.krishnakumar
2020-10-30 22:34
I actually did paste the question there but did not get a response. Let me paste it again
silverton.gimenes
2020-11-03 18:11
hey everyone, I would receive an email when invited to http://pact.io, but It didn't happen, who can help me on that ?
matt.fellows
2020-11-03 21:18
Drop us a line at with your registration info and we'll help you out. Did it end up in spam perhaps?
cluu
2020-11-06 15:08
Hi, I?m at my wit?s end here with trying to PoC PactFlow for my organization. I am able to publish my contract to PactFlow, but I cannot verify due to 401 error. I have set the token to the ?PACT_BROKER_TOKEN? env without any success. Does anyone has any suggestion for the next course of action?
matt.fellows
2020-11-06 21:42
Also confirming you're definitely using the read/write token as described https://docs.pactflow.io/#configuring-your-api-token
matt.fellows
2020-11-06 21:42
Lastly, not all libraries automatically pickup that Env vat, which language are you using?
matt.fellows
2020-11-07 00:50
Thanks. For future people, it was an unfortunate copy/paste error. We all have those days :slightly_smiling_face:
2020-11-10 04:20
Notification from *Pactflow* :ghost:
bethskurrie
2020-11-12 23:23
@nasir.amin if you put a `can-i-deploy` between your test phase and your deploy phase that polls for the result to come back (see the can-i-deploy CLI docs for this) then you can make it wait for the result to come back and then go straight into a deployment if it passes.
bethskurrie
2020-11-12 23:24
Or, you can make the 'verification passed' webhook trigger another consumer build to do the deployment.
bethskurrie
2020-11-12 23:24
2020-11-23 05:53
Notification from *Pactflow* :ghost:
nerea.tamayo
2020-11-24 12:42
Hi all!
I'm trying to create webhook in PactFlow for triggering Jenkins pipeline but I don't know how to start....
In PactFlow, clicking on "add webhook", there are some parameters...
? url, is the provider pipeline I want to trigger with in Jenkins?
? About credentials....Which are the credentials I need to use in the PactFLow's webhook? Is the service account I'm using for running the pipeline in Jenkins?
Sorry for this type of questions, but I'm lost on this
thanks!!!
bethskurrie
2020-11-24 22:39
Hi @nerea.tamayo I recommend looking up the jenkins documentation on how to trigger a build first, and understand how that works.
bethskurrie
2020-11-24 22:42
Here's a template for how to make a jenkins webhook through the API https://docs.pact.io/pact_broker/webhooks/template_library#jenkins---trigger-build
bethskurrie
2020-11-24 22:43
Here are the docs for creating a webhook through the Pactflow UI https://docs.pactflow.io/docs/user-interface/settings/webhooks
bethskurrie
2020-11-24 22:44
Here's some advice on creating webhooks https://docs.pact.io/pact_broker/webhooks/debugging_webhooks#creating-webhooks
bethskurrie
2020-11-24 22:44
"The best process to create a new webhook is to first get the API call that the webhook will execute working outside the Pact Broker, using a tool like curl or Postman."
pavel.azarau
2020-11-25 18:30
Hello folks.
I?m using Gradle plugin ?http://au.com.dius.pact? version ?4.1.9". I?ve configured it to access our company?s pactflow acc. And when I run `pactVerify` task I?m getting
```Failed to fetch the root HAL document
au.com.dius.pact.core.pactbroker.RequestFailedException: Request to path '/' failed with response 'HTTP/1.1 400 Bad Request'
at au.com.dius.pact.core.pactbroker.HalClient.handleHalResponse(HalClient.kt:301)
at au.com.dius.pact.core.pactbroker.HalClient.access$handleHalResponse(HalClient.kt:140)
at au.com.dius.pact.core.pactbroker.HalClient$getJson$1.invoke(HalClient.kt:284)```
However in Postman I can successfully perform a GET request using the same url and token I used to configure plugin
matt.fellows
2020-11-25 21:02
Have you tried upgrading? There was an issue with auth in a previous version where multiple duplicate headers were being sent and the load balancer was rejecting the request.
pavel.azarau
2020-11-25 21:06
oh, thanks! 4.1.11 worked for me :slightly_smiling_face: didn?t see that the plugin was updated recently
wilkinsweiqiangliang
2020-11-27 04:24
#pactflow Hey guys, does pactflow always trigger the `HEAD` commit? i follow the event template from docs, and find it sometime send `HEAD` commit, sometime send a different commit.
matt.fellows
2020-11-27 04:24
What do you mean by triggering the HEAD commit? We don?t have the concept of a commit in Pactflow
matt.fellows
2020-11-27 04:26
The configuration of you webhook may be to trigger the `HEAD` commit in your CI, but Pactflow knows nothing about that
wilkinsweiqiangliang
2020-11-27 04:27
```{
"branch": "master",
"commit": "HEAD",
"message": "Build all the things from pact! :rocket:",
"env": {
"PACT_URL": "${pactbroker.pactUrl}",
"CONSUMER_NAME": "${pactbroker.consumerName}",
"PROVIDER_NAME": "${pactbroker.providerName}"
}
}```
matt.fellows
2020-11-27 04:27
Pactflow sees a contract, and will issue that request to (in your case) buildkite
matt.fellows
2020-11-27 04:27
your webhook template specifies the HEAD commit, so presumably it will always trigger the HEAD of master
matt.fellows
2020-11-27 04:29
that?s a buildkite thing though, we can?t control what commit your CI system picks up when the webhook comes in
matt.fellows
2020-11-27 04:30
if your CI hasn?t picked up the latest commit yet, not sure what we can do about that
wilkinsweiqiangliang
2020-11-27 04:30
yea thats what i think as well, and i believe we specify to use HEAD
matt.fellows
2020-11-27 04:33
In any case, the BK team is pretty responsive in their slack channel - I?d recommend hitting them up there!
fealaer
2020-12-02 11:52
Hey everyone! We're experiencing an unexpected behaviour running `can-i-deploy` . We see that pact is verified in http://pactflow.io UI but CLI response says `There is no verified pact between version <version> of Client and the latest version of API with tag <tag> (<version>)`. We had that message for several PRs this morning now most of them are green but a few are still failing this check. http://status.pactflow.io/ states `All systems are go` but it seems like something is not working correctly. Could you please help to resolve this issue? Thanks!
uglyog
2020-12-02 21:51
http://status.pactflow.io/ gives the status of the platform, which is not currently experiencing any outage. Can you provide the consumer and tags you are using?
bethskurrie
2020-12-02 23:25
@fealaer the way to debug can I deploy is to turn on --verbose in the cli. You'll see the matrix URL that is being requested. Copy that, then go into pactflow, and switch to the "old UI" using the butting at the top of the screen.
bethskurrie
2020-12-02 23:26
This will show you all the rows that are used in the query. You can adjust the parameters in the form to investigate.
tjones
2020-12-08 04:36
hi friends, I think I may have found a bug with sending credentials out - I signed up for the free account level today. I got the "welcome, here's your login URL" email, but I haven't had the temporary creds yet (in 10 minutes)
When searching my email to confirm that I haven't received the email, I found out that I've already got an account (which I forgot about) with the same email address. This might be related
tjones
2020-12-08 04:42
Also, the reset password flow UX is a little surprising -
Go to login, hit reset password, get an email
The email has a code and a link to the login page(s)
However, the code doesn't work as a password - you have to click "reset password" again, and it then asks for the code
I reckon it would be better if the email linked to the reset password screen that asked for the code (or warns you that you'll need to click the thing)
tjones
2020-12-08 04:42
If I remembered the creds, I might have remembered that I already had one :wink:
bethskurrie
2020-12-08 04:44
Matt's been moving some of the default Cognito communications into custom handlers, so this feedback is timely.
matt.fellows
2020-12-08 05:00
I remember some idiot doing the Cognito integration. They had no idea what they were doing.
gayatree.eee
2020-12-11 15:46
Consumer 1, Consumer 2, Consumer 3 is making use of Provider1 API to receive the list of Students.
Sample Student List Response Format:
[{
"id":1,
"name": "John Doe",
},
{
"id":2,
"name": "John Smith",
}
]
Scenario1 :
Suppose Consumer1 wants ?name? field to be converted into 2 separate fields like firstName and lastName.
Now the required response for Consumer looks like below:
[{
"id":1,
"firstName": "John",
?lastName?: ?Doe?
},
{
"id":2,
"firstName": "John",
?lastName?: ?Smith?
}
]
Now if Consumer makes the necessary changes to the pact contract and publish the pending contract. As discussed in the pact document (http://blog.pact.io/2020/02/24/how-we-have-fixed-the-biggest-problem-with-the-pact-workflow/), provider should make the necessary changes (to separate name field into 2 different fields) to satisfy the contract published by Consumer1.
In this scenario, how the provider will communicate the required changes to Consumer2 and Consumer 3? As the changed code will break the contract of other Consumers (Consumer 2 and Consumer 3), it will fail the provider build. So how should we handle this scenario?
matt.fellows
2020-12-11 21:54
So there are a few things to consider here:
1. The provider may choose to deny that change - aka say ?no, that?s not something we?re going to do?. That?s always there prerogative
2. Just because `Consumer` wants `firstName` and `lastName` doesn?t mean the provider should remove `name` altogether.
> As the changed code will break the contract of other Consumers (Consumer 2 and Consumer 3), it will fail the provider build.
So the answer is to not break the API!
Until all consumers move to the new fields, the provider should return something more like this (often referred to as the ?expand and contract? approach):
```[{
"id":1,
"firstName": "John",
"lastName": "Doe",
"name": "John Doe",
},
{
"id":2,
"firstName": "John",
"lastName": "Smith",
"name": "John Smith",
}
]```
Extra fields present in the response do not break the contract for the consumers that don?t need it.
The contract that Provider must satisfy is the union of all interactions of Consumer 1,2,3 (where there is no conflict).
In the case of conflicts, there are other things to consider - how did we get to a conflict? Is the API boundary correct? Do we need a narrower, use-case specific interface (e.g. BFF) for our consumer?
writetojoshma
2020-12-17 06:30
in `require': cannot load such file -- pact/mock_service/request_handlers/base_administration_request_handler (LoadError)
writetojoshma
2020-12-17 06:30
node_modules/@pact-foundation/pact-node/standalone/win32-1.88.3/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.6.2/lib/pact/mock_service/request_handlers/interaction_post.rb:1:in `require': cannot load such file -- pact/mock_service/request_handlers/base_administration_request_handler (LoadError)
matt.fellows
2020-12-17 06:31
thanks for asknig in a public channel. It looks like you?re using #pact-js (so best to ask these questions in the language specific channel)
matt.fellows
2020-12-17 06:32
I think you?re running into https://github.com/pact-foundation/pact-node/issues/100 though
matt.fellows
2020-12-17 06:33
Alternatively, you can try out our new https://github.com/pact-foundation/pact-js#pact-js-v3 which does not suffer this issue, and has other benefits (new matchers etc.)
matt.fellows
2020-12-17 06:37
If you click the link above (beta branch) it has the instructions on how to install and use it
writetojoshma
2020-12-17 06:43
I am still getting the same error even after insalling the beta version
writetojoshma
2020-12-17 06:45
Pact Binary Error: C:/Users/**/node_modules/@pact-foundation/pact-node/standalone/win32-1.88.3/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.6.2/lib/pact/mock_service/request_handlers/interaction_post.rb:1:in `require': cannot load such file -- pact/mock_service/request_handlers/base_administration_request_handler (LoadError)
from C:/Users/**/node_modules/@pact-foundation/pact-node/standalone/win32-1.88.3/pact/lib/vendor/ruby/2.2.0/gems/pact-mock_service-3.6.2/lib/pact/mock_service/request_handlers/interaction_post.rb:1:in `<top (required)>'
brendan.donegan
2020-12-17 09:56
Hi, I had a quick question. Will there ever be any plans to offer Pact Flow in an on-premises version (as opposed to the open-source Pact Broker)?
bethskurrie
2020-12-17 09:58
There's some marketing info in that page. The technical docs are here: https://docs.pactflow.io/docs/on-premises
bethskurrie
2020-12-17 10:01
You can book a demo with Matt if you're interested @brendan.donegan, using the "book a demo" button on https://pactflow.io/
brendan.donegan
2020-12-17 10:02
Thanks @bethskurrie something for after the holidays :slightly_smiling_face:
gayatree.eee
2020-12-17 10:07
Hi, Need some clarity on Can-I-Deploy in Provider Pipeline. As per my understanding a breaking change in provider for any consumer can be caught in Provider Testing phase, then what is the purpose of Can-I-Deploy in provider pipeline.
bethskurrie
2020-12-17 10:08
1. the consumer may have deployed between when you ran the verification and when you deployed.
bethskurrie
2020-12-17 10:09
2. If you are using pending pacts, your build may NOT fail when a pact fails verification, so it might need to be caught by can-i-deploy.
gayatree.eee
2020-12-17 10:40
Is enabling pending pacts a provision to pass provider build irrespective of contract failure? Is there any point of just passing the building for provider by enabling pending pact?
matt.fellows
2020-12-17 10:45
No, pending pacts only prevents a failing build for new, unseen contracts for that provider tag. Basically, a new request by a consumer. Once the provider has successfully verified a pact for a given provider tags, it will fail a build going forward
thanuxxxx
2020-12-21 23:59
Hi there, I am using PactFlow as my pact-broker. From consumer side, I am able to publish a contract to the server(PactFlow), and I am able to import the contract to my provider and all tests are passing as I expected. But in the dashboard (PactFlow), it says `Provider version never verified`
matt.fellows
2020-12-22 00:00
This almost certainly means you haven't published the verification results
thanuxxxx
2020-12-22 00:01
Java with
``` <groupId>au.com.dius.pact.provider</groupId>
<artifactId>junit5</artifactId>
<version>4.1.12</version>
<scope>runtime</scope>
</dependency>```
thanuxxxx
2020-12-22 00:03
``` @TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider.class)
void pactVerificationTestTemplate(PactVerificationContext context) {
String providerName = context.getProviderInfo().getName();
String interactionDescription = context.getInteraction().getDescription();
http://LOGGER.info(String.format("testTemplate called: %s, %s", providerName, interactionDescription));
context.verifyInteraction();
}```
thanuxxxx
2020-12-22 00:05
I thought after verifying all tests in provider side, pact automatically publish the results back to Pactflow
thanuxxxx
2020-12-22 00:07
Maybe I don't have any setting to send the verification results back to Pactflow
gayatree.eee
2021-01-04 10:25
In pending contract doc the sentence *"This means that any subsequent verification failures for a pact with identical content will cause the provider build to fail, as the failure can now only be due to a change in the provider code." -* With enablePending value true, when does the provider build fails? We noticed that the build never fails when the enablePending is true on provider end even if the provider code is changed.
matt.fellows
2021-01-04 11:29
It has to first accept the contract by successfully verifying it for the configured provider tag
matt.fellows
2021-01-04 11:29
> It achieves this by treating the first successful verification of a pact version by a particular branch of the provider as an implicit acceptance of the contract. Thereafter, if verification of that pact version fails, it can only be because the provider has made a backwards incompatible change.
matt.fellows
2021-01-04 11:30
It will fail once it's first verified the new pact and then breaks the contract
kjayachandra2000
2021-01-04 12:03
Hi All, any help how to publish repository url to the network graph, :thinking_face:
matt.fellows
2021-01-04 13:26
Issue a POST/PUT to create or update a pacticipant and include the `repositoryUrl` property in the payload.
kjayachandra2000
2021-01-04 13:32
is there a way i provide the repositryUrl while `pactPublish` or `pactVerify`?
matt.fellows
2021-01-04 13:38
There is a terraform provider. It's just a once off API call per pacticipant to do it tho
alan.hanafy
2021-01-05 21:13
Is there some feature in pactflow that will clear the webhook execution logs somehow? i've been trying to setup a webhook to github and at some point the logs are all gone. i've checked console, network logs in browser and not finding anything. logged out and back in, still no logs
matt.fellows
2021-01-06 04:03
I don?t think we log webhook executions from the ?test? button on the webhook screens though, so that?s the only thing I can think of
matt.fellows
2021-01-06 04:03
once it?s been created, it should log all triggers and related executions (a trigger may cause multiple executions e.g. if the first attempt failed for example)
bethskurrie
2021-01-06 04:14
@alan.hanafy you can raise an issue by sending an email to and we can have a look in the database for you. The webhook log screen is new, so there may be a bug.
alan.hanafy
2021-01-06 04:25
@matt.fellows @bethskurrie thanks. The execution logs were not from the test button (yes those didn't show up which I expected) but from "real" contract publish events. I'll send an email to support to ask someone to look into it. Thanks for the clarification.
alan.hanafy
2021-01-06 18:14
I'll have to wait for support to get back to me to give them an update, but in the mean time, i was able to troubleshoot my webhook setup to the point that it now works successfully.
alan.hanafy
2021-01-06 18:15
On the first successful webhook execution, the log was populated again (with this single new execution).
alan.hanafy
2021-01-06 18:15
For whatever reason it looks like it stopped showing the execution logs after a certain amount of failed ones.
alan.hanafy
2021-01-06 18:15
I even intentionally went back and broke the webhook settings to see if the failed webhook logs would show, and they did.
thanuxxxx
2021-01-08 16:01
Hey guys, I am getting this error whenever I publish a pact to pactflow, why I am getting this error
```$ node pactio/publish_pact.ts
[2021-01-08T15:27:59.248Z] INFO: pact-node@10.11.1/3274 on MacBook Pro: Publishing Pacts to Broker
[2021-01-08T15:27:59.249Z] INFO: pact-node@10.11.1/3274 on MacBook Pro: Publishing pacts to broker at: http://my.pactflow.io
[2021-01-08T15:28:03.780Z] ERROR: pact-node@10.11.1/3274 on MacBook Pro:
Could not publish pact:
Tagging version 1.0.0.1 of WebOrder as "dev"
Publishing WebOrderA/Service pact to pact broker at http://my.pactflow.io
The latest version of this pact can be accessed at the following URL (use this to configure the provider verification):
Pact contract publishing failed: Error:
Tagging version 1.0.0.1 of WebOrder as "dev"
Publishing WebOrder/Service pact to pact broker at http://my.pactflow.io
The latest version of this pact can be accessed at the following URL (use this to configure the provider verification):
at ChildProcess.<anonymous> (/Users/user/src/web-order/node_modules/@pact-foundation/pact-node/src/publisher.js:72:40)
at Object.onceWrapper (events.js:422:26)
at ChildProcess.emit (events.js:327:22)
at maybeClose (internal/child_process.js:1048:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
? Done in 5.04s.
```
gayatree.eee
2021-01-13 12:28
PendingPact: If we use ConsumerVersionSelector, then the changes in consumer contract on specific feature branch will not fail any provider build. Then where can we use PendingPact effectively?
matt.fellows
2021-01-13 23:44
Pending pacts relate to when a provider has never seen a particular version of a contract (for a given provider tag).
matt.fellows
2021-01-13 23:45
consumer version selectors are a way of defining which contracts to verify. Some, all or none of those contracts may deemed to be pending
sklein
2021-01-14 14:32
Is there a way to retrigger a webhook for a published contract or verification result? I noticed the shiny new Webhooks tab which is very nice. It would be great if I could trigger a webhook for a specific event from that page!
phil.endsley
2021-01-14 15:06
Just adding on in case you were intending to ask about pending pacts and where to use them effectively.
> If we use ~ConsumerVersionSelector~ pending pacts, then the changes in consumer contract on specific feature branch will not fail any provider build.
Correct. When verification runs on a provider, if the pact is pending, and verification fails, the provider build will not fail. It will still publish a failed verification result.
Using pending pacts makes it so a breaking change committed to the consumer does not block progress on the provider. Imagine a scenario where the consumer and provider are maintained by separate teams. In this scenario, there is an existing pact that is verified.
The consumer team makes a change to the existing pact, and that change is merged. Provider verification fails.
Without pending pacts, the provider team is now blocked by this changed contract. Their builds will fail until the new contract is fulfilled. This blocks them from new feature development, deploying hotfixes, and anything else until that contract is satisfied.
In some organizations, this causes frustration for the provider team, because the status of their build is now being dictated by consumers. If a consumer team merges a contract that can't be verified, the provider team is blocked until that is resolved.
If pending pacts are enabled, verification will still fail, but the build will pass. If you're using can-i-deploy, this means the consumer cannot deploy the latest version, because there is not a successful verification. Since the provider's build is still "green", they can continue development and deployments as long as they don't break existing contracts.
This shifts the burden of responsibility to fix the issue back to the consumer team, because they cannot deploy with a pact that has not been verified. For some organizations, this makes sense, since the consumer team was responsible for introducing the new contract before the provider team was ready.
bethskurrie
2021-01-14 20:11
@phil.endsley you explained that better than I could. I need to put that somewhere in the docs.
bethskurrie
2021-01-14 20:11
@sklein no there isn't but it would be pretty easy to add the feature. Can you raise it in https://pact.canny.io?
matt.fellows
2021-01-27 01:33
For posterity, you can definitely issue a `DELETE` to that specific resource in the API (via HAL browser or otherwise)
matt.fellows
2021-01-27 01:33
You can also navigate directly to the pact version through the UI, and choose to delete it from the drop down (note that the user I?m logged into below can?t do that, because of permissions)
matt.fellows
2021-01-27 06:33
That?s going to delete the pacticipant though and all related data - so be careful running that one if you care about history!
artur
2021-01-29 05:48
I'm playing currently with pactflow. I can upload and tag pacts also the verification from the provider seems to work. the dashboard shows me a successfully verified pact
but when I browse in the details the verification status for every interaction is "unknown" -what does that mean?
uglyog
2021-01-29 06:01
Ah, that is probably an issue with the published results. It doesn't have the result for each interaction. I recently fixed it in Pact-JVM, but need to do the same fix in the Rust codebase.
uglyog
2021-01-29 06:27
The whole set of results has a successful status (so they have all passed), but the individual interactions don't have a result.
uglyog
2021-01-29 06:30
Because some have failed, the main status is failed, but the individual ones only have results for the failed ones
artur
2021-01-29 06:30
yes, the second screenshot is a different example. but yes the issue seems to be that the overall result is published correctly, for the interactions the result is only published if they fail
steven.beaver.retail
2021-01-29 15:39
is http://pactflow.io working today? Yesterday I was able to push to and pull ... today I am getting 302 errors. Is anyone else seeing this?
steven.beaver.retail
2021-02-04 14:48
Hello Matt, the issue has resolved itself and now I am getting 200 status as expected. When I was receiving the 302 errors, I was simply trying to access the pacts as I am able to do now.
matt.fellows
2021-02-04 21:56
Thanks for getting back. You didn?t happen to be on an internal network/proxy did you? Sometimes reverse / transparent proxies do funny things
matt.fellows
2021-02-04 21:56
(I?m going to check our external load balancers, because when you first raised that ticket, I couldn?t see any `302`s - not just for your account, but on the platform at all!)
steven.beaver.retail
2021-02-09 18:20
Hello @matt.fellows I am able to reproduce the issue with the 302 with another account I had to create on http://pactflow.io which is http://bottomline.pactflow.io
I was working with the pact broker without issue, until I deleted a Pact Integration using the UI provided with http://bottomline.pactflow.io
After this, all interactions I have with http://bottomline.pactflow.io end with http errors.
From curl I see http status 302.
I am getting `Unable to contact API` on http://pactflow.io in the browser
steven.beaver.retail
2021-02-09 18:22
to work around this issue, I have had to switch back to the original account which originally was giving me the 302 status and is now working as expected ... :flushed:
matt.fellows
2021-02-10 00:21
Can you please share (or DM me, send to ) with some more details so we can investigate?
adam_figgins
2021-02-16 08:05
Hi guys. I'm experiencing some unexpected behaviour with verifying WIP / Pending pacts on the provider side. Using our current config in http://Pact.Net, I deleted an integration from the Pactflow UI while developing (to clean house while testing), and assumed this would also remove all the contracts that were part of that integration. When I tried the workflow again, I hadn't modified the consumer side contract code in any way, published the Pact using the same integration details, and couldn't get the provider side to pick up my 'new' WIP Pact. The UI in Pactflow also states it was still in an unverified state. It wasn't until I modified the consumer contract code that the provider would see this as an unverified contract and pick it up at a WIP pact. I assume that there is still some record behind the scenes that holds on to the old contracts from the deleted integration, which is why this happened. Thought I would raise this here instead of creating a new issue in case you already knew about this / I missed something in the docs that mentions this behaviour.
wesleythomaswilliams
2021-02-16 10:57
Any folks here using Pact Flow Enterprise? I have a question around how teams work in respect to publishing pacts so that they appear in the team space.
matt.fellows
2021-02-16 12:58
Hmmm I'll confirm this behaviour tomorrow when I'm back online. I think it should work as described
matt.fellows
2021-02-16 12:59
So the application needs to be added to the team also, otherwise it lands in the default team.
matt.fellows
2021-02-16 12:59
You can use our Terraform provider if helpful to configure teams, perms, applications etc
matt.fellows
2021-02-16 13:00
Otherwise an API call to create the pacticipant and then assign via the team's page should work
wesleythomaswilliams
2021-02-16 16:53
Matt, it looks like the API only supports GET, PATCH, DELETE for Pacticipants, no POST. So is the only way to do this to publish a pact for each of our services and then move them over?
matt.fellows
2021-02-16 22:53
```curl -X POST -H"Content-Type: application/json" -v -H"Authorization: Bearer $PACT_BROKER_TOKEN" $PACT_BROKER_BASE_URL/pacticipants/ -d '{"name":"WiffleWaffle"}'```
bethskurrie
2021-02-17 00:23
@wesleythomaswilliams we've got plans to redesign the Pactflow UI with an application centric (rather than pact centric) focus soon (if provider driven contracts gets good feedback). When that happens, we'll be adding screens for pacticipant CRUD.
adam_figgins
2021-02-17 08:30
Hi @bethskurrie. I used the 'Delete Integration' option from the Pactflow UI:
adam_figgins
2021-02-17 08:32
I'm just doing some more digging into this at the moment, as I notice that actually the provider side is no longer running against WIP / pending Pacts at all. I've noticed that we upgraded the http://Pact.Net nuget packaged to 2.61 from 2.5.5, so rolling back to check this isn't the issue
matt.fellows
2021-02-17 08:33
It definitely deletes all of the pacts, because I regularly do a demo and have to use that feature to show the ?how to do it from scratch? flow
matt.fellows
2021-02-17 08:33
So it was more the WIP/Pending behaviour I wanted to confirm, just to clarify. But Beth is here, and anything she says is definitive :stuck_out_tongue:
wesleythomaswilliams
2021-02-17 09:10
That curl command appears to work, thanks :slightly_smiling_face:
matt.fellows
2021-02-17 09:10
I think it?s probably intuitive that any publish inherits the team permissions, but there is likely a good reason it doesn?t
adam_figgins
2021-02-17 09:12
I've managed to find the root cause. It was a case of too much code changing at once, so it was hard to determine the problem. Turns out it was the way we were implementing the dispose pattern in the xUnit fixture. We switched to using the IAsyncDisposable interface and were stopping the pact webhost in DisposeAsync, rather than in the synchronous Dispose method. I guess the DisposeAsync calls were running before the webhost had chance to finish fetching and running all the WIP pacts, cutting it off prematurely.
gabe707
2021-02-17 18:26
```PactBroker::Client::PublicationTask.new do | task |
task.consumer_version =1.0
task.pattern = /spec/pacts/*.json' # optional, default value is 'spec/pacts/*.json'
task.pact_broker_base_url = 'https://testbasic.pactflow.io'
task.tag_with_git_branch = true|false
PACT_BROKER_TOKEN='dsfdfsdyK693Dhh7kjg'
# task.pact_broker_token = 'dsfdfsdyK693Dhh7kjg' # Bearer token
end```
_*I have a blocker. when i run*_ -- >rake pact:publish
_*I get an undefined method for pact_broker_token*_
NoMethodError: undefined method `pact_broker_token=' for #<PactBroker::Client::PublicationTask:0x00007ff836a143f8>
Did you mean? pact_broker_base_url=
when I comment out *# task.pact_broker_token = 'dsfdfsdyK693Dhh7kjg'*
*I get an error of authentication:*
Publishing pact to pact broker at https://testbasic.pactflow.io
Failed to publish l pact due to error: PactBroker::Client::Error - Authentication failed
One or more pacts failed to be published
artur
2021-02-19 06:43
is it possible to make the results to be visible publicly on pactflow?
Working on a open-source project I want the contributors to be able to browse the results
matt.fellows
2021-02-19 06:47
It?s not currently supported, albeit we?d like to make that possible also for additional reasons
matt.fellows
2021-02-19 06:48
for example, http://test.pactflow.io is our ?legacy? Pactflow account that uses basic auth (so we can easily share the creds), but it?s not a very nice experience and doesn?t really show off what we do very well. If you head to http://testdemo.pactflow.io it?s the same underlying data with the better experience - but we don?t have a way (yet) of making it public
matt.fellows
2021-02-19 06:48
I believe now that our RBAC work has been completed, it?s not a big leap to make it read-only visible, at least
artur
2021-02-19 06:55
:+1: there is already a read-only token, so a developer could get the results on the local machine, or see them in text-form in CI. so its not a must-have for us, but having the results in a nice web UI would be just an extra.
wilkinsweiqiangliang
2021-02-19 07:09
@matt.fellows @uglyog Hey guys, do we have read timeout issue in pact?
matt.fellows
2021-02-19 10:05
On closer inspection, it looks like maybe it?s a docker issue? Did you get to the bottom of it?
wilkinsweiqiangliang
2021-02-20 01:13
you are right, seems its Buildkite issue, It work after retry today.
matt.fellows
2021-02-20 01:28
Was worried - two days in a row issues on your account would not be good!
bethskurrie
2021-02-21 04:15
I do actually have code for "public" brokers that allow anyone from github or google to log in, but it's on a branch and hasn't been released yet. cc: @matt.fellows
wesleythomaswilliams
2021-02-24 10:31
So I have a small problem. I'm migrating our services from one Pact Flow instance to another. In the majority of cases, the services are both consumers and providers and run both sets of pact tests as part of the unit test phase of our pipeline. This all works fine, until I redirect my pipeline to the new instance of Pact Flow. I'm doing this one service at a time, so if we assume Service 1 is both a consumer and a provider, the pipeline fails when redirected to the new Pact Flow instance because provider tests fail (because there are no contract(s) from it's consumer(s) in the new instance). This then also prevents the publication of the consumer pacts for this service as that stage of the pipeline comes after the tests have run (and passed, which they don't).
The workaround seems to be to disable my provider tests, but I wondered if Pact provides a simple solution beyond me commenting out tests in each service until all the initial versions of the contracts have been published?
ben.kirberger
2021-02-24 20:30
Hi team, I?m currently working through the CI/CD tutorial and I?m having an issue at step 5 here: https://docs.pactflow.io/docs/workshops/ci-cd/workshop/verifying-feature-pacts
> 5. On your local machine, run `TRAVIS_BRANCH=master make test` - you will now see that the `feat/new-field` pact is not included, as it is no longer a work in progress pact.
I show the `feat/new-field` tag in pactflow has changed to verified (from step 4), however, when running `TRAVIS_BRANCH=master make test` now, I _still_ see it listed as unverified in logs. If I click on the example URL it shows in the logs, it takes me to the pactflow `feat/new-field` pact, and shows as verified! How can this be?
```DEBUG: The pact at <EXAMPLE URL HERE> = is being verified because it is a 'work in progress' pact (ie. it is the pact for the latest version of pactflow-example-consumer tagged with 'feat/new-field' and is still in pending state). Read more at https://pact.io/wip```
wesleythomaswilliams
2021-02-24 20:37
I've realised I do actually have a service which is effectively just a consumer and I'm working backwards from this point through our services, which is working so far.
bethskurrie
2021-02-25 03:46
The pending state is based on the provider branch (for which the tag is used as a proxy). Unfortunately, for the sake of simplicity, I didn't include the provider tag name for which the pact was pending in the output.
ben.kirberger
2021-02-25 03:53
Agh, I?m afraid I just continued the tutorial and tried to just carry on. Unfortunately I didn?t take down the state in CI at the time, sorry. :disappointed:
matt.fellows
2021-02-25 07:47
I think one approach is to query in advance if there are consumers and only do the check if that's true.
I think there is a flag that says "don't fail if no consumers found"
wesleythomaswilliams
2021-02-25 09:06
It's ok, I'm a demon, I've migrated them all :smile:, however, a don't fail if no consumers flag would be awesome.
matt.fellows
2021-02-25 10:39
When I'm at my desk I'll do a little search back thru slack history to see if I can find
zhoutianli1234
2021-02-25 21:15
Hi, we are using Azure pipelines for our development. From what we tried, we get the verification results using can-i-deploy. However, to see the details we have to go into the url and see it from the Pactflow UI. Is there a way to get the detailed verification results? For example, something similar to the CLI like can-i-deploy but gives u detailed results, or a way to export a test results report? Ideally a way to display the details on the azure pipeline.
bethskurrie
2021-02-25 21:16
@zhoutianli1234 no, but you can request a feature at http://pact.canny.io
bethskurrie
2021-02-25 21:16
Or you can run the can-i-deploy query and output it as JSON, then fetch the results endpoints yourself to print them out.
bethskurrie
2021-02-28 22:51
@wesleythomaswilliams @matt.fellows when I had to migrate brokers at a previous client, I wrote a little script which queried the latest pacts endpoints and copied the pacts over.
bethskurrie
2021-02-28 22:52
I'm sure there's a "no fail" annotation for the JVM, and I don't think the ruby standalone fails if there are no pacts.
bethskurrie
2021-02-28 22:59
@wesleythomaswilliams there's some missing documentation for creating pacticipants. I've got that PR open for adding new fields to the pacticipant, so I'll make sure I update it.
bethskurrie
2021-02-28 23:00
We can't automatically assign a pacticipant to a team, because a user may be a part of many teams, and we don't know which team the pacticipant should belong to @wesleythomaswilliams
bethskurrie
2021-02-28 23:01
When it gets created, we assign the "created_by" user, and until the pacticipant gets assigned to a team, the creator user has the admin privilege for the pacticipant.
kmckeever
2021-03-04 14:50
Hey there - I am using pactjs/Jest to run my tests, and publishing to Pactflow from within the project using `Publisher(opts).publishPacts()`. We're using GitLab as our CI provider, and have a stage to execute and publish the Pact tests.
While the tests execute successfully (indicating success for all tests and suites), I see the following error after each test run:
`Pact Binary Error: /builds/myProj/myProjDir/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.88.35/pact/lib/ruby/lib/ruby/gems/2.2.0/gems/bundler-1.9.9/lib/bundler/shared_helpers.rb:78: warning: Insecure world writable dir /builds/myProj/myProjDir in PATH, mode 040777`
The failure comes when we attempt to publish to Pactflow. While I am able to publish from my local (Windows) dev machine, I am seeing the following error when publishing through GitLab:
``` Could not publish pact:
/builds/myProj/myProjDir/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.88.35/pact/lib/ruby/lib/ruby/gems/2.2.0/gems/bundler-1.9.9/lib/bundler/shared_helpers.rb:78: warning: Insecure world writable dir /builds/myProj/myProjDir in PATH, mode 040777
Error making request - Net::HTTPServerException 403 "Forbidden" /builds/myProj/myProjDir/node_modules/@pact-foundation/pact-node/standalone/linux-x64-1.88.35/pact/lib/vendor/ruby/2.2.0/gems/pact_broker-client-1.35.0/lib/pact_broker/client/base_client.rb:109:in `put', attempt 1 of 3```
I attempted to resolve this by running the command `chmod go-w /builds/myProjc/myProjDir`, but to no avail. Does anyone have any possible insight?
kmckeever
2021-03-04 15:39
`sudo` does not appear to be available on our gitlab runner. I was hoping to avoid having to do command gymnastics to get the Pact tests to run successfully, if possible. :confused:
uglyog
2021-03-04 22:05
The warning is probably a red herring, `403 "Forbidden" ` seems to be the actual problem. Check you have a valid token.
kmckeever
2021-03-04 22:21
Hey there, @uglyog - I had the GitLab job print out the token value pulled from my environment variable and verified it's the same token I use locally with success.
uglyog
2021-03-04 22:24
You'll probably need to enable debug logs to see what is going on with the GitLab runner. @matt.fellows might have some ideas to help.
matt.fellows
2021-03-04 22:48
The insecure world writable dir is a red herring indeed, you can ignore that
matt.fellows
2021-03-04 22:52
`403` is an error that means you?re _authenticated_ but not _authorized_ to perform the action
matt.fellows
2021-03-04 22:52
so that means you could be using a read-only token (so you can?t publish) or the role associated with the token is not allowed to publish
matt.fellows
2021-03-04 22:52
if you would kindly share debug logs to me offline, or drop it to we can take a look
kmckeever
2021-03-05 00:16
Hey there @matt.fellows thanks for the follow-up! I verified it's the read/write token we're using in CI. I can try and enable the debug logs, but may be limited in what I can do at the moment.
I am happy to share the account name here if you are able to delete the message after receiving it, or I can shoot you a message at the above email. Whichever is easiest!
wesleythomaswilliams
2021-03-08 12:09
Is there any reason why the create button for webhooks in pact flow wouldn't be working? Our enterprise instance won't allow me to create them (when it previously did). I've also checked the pact hosted instance we still have a license for and the button is disabled there too. @bethskurrie @matt.fellows
alan.hanafy
2021-03-08 19:41
Is there some documentation or example of a webhook for when a consumer publishes a contract that will trigger a verify against all providers?
thanuxxxx
2021-03-08 21:19
I think, you need one webhook for every consumer provider set. If you have 1 consumer and 10 providers, I think you still need 10 webhooks since every provider has different POST URL
matt.fellows
2021-03-08 21:52
You can definitely create a webhook that triggers for a given consumer, but the target of that webhook would need to know how to trigger all providers.
matt.fellows
2021-03-08 21:53
Usually you create one webhook per provider, omitting the consumer name (so then, any consumer of that provider will trigger the verification build)
matt.fellows
2021-03-08 21:54
hmmm that sounds like a bug perhaps. What does the browser console say? Are there any errors there?
matt.fellows
2021-03-08 21:55
Just wondering if perhaps there?s an error at the API or some bug in the UI
matt.fellows
2021-03-08 21:57
(FYI as a backup next time I know it?s not ideal, but you could create it via the API or CLI. The `pact-broker create-webhook` command could be used to circumvent it)
bethskurrie
2021-03-08 21:57
Matt's right, you usually create one webhook per provider, because the verification build will be the same for each consumer.
alan.hanafy
2021-03-08 22:05
Right I've done that so far but I'm having trouble when a new consumer contract is published. If I understand correctly I need to run the provider verification after the new contract is published?
alan.hanafy
2021-03-08 22:07
The consumer pipeline is currently running tests, publishing contract and then tagging version but I see afterwards that The contract isn't verified and have to manually run provider verification afterwards (for any/all providers that the new contract changes)
wesleythomaswilliams
2021-03-08 22:46
Oh god, what a muppet. Whenever I see that URL section I always read it like a Title and totally miss that URL and Headers are a separate thing.
bethskurrie
2021-03-08 22:58
You're not the only one - it trips people up a lot. We need to do something to make it more obvious.
bethskurrie
2021-03-08 22:59
It's the way the UI library we use styles the text input. We can probably add a custom style to make it more obvious.
bethskurrie
2021-03-08 23:00
@alan.hanafy have you read this? https://docs.pact.io/pact_nirvana/step_4
raghavendra.kalakonda
2021-03-11 23:22
Hello Everyone,
Has anyone used Pact with Azure event hub please?
2021-03-12 21:28
Notification from *Pactflow* :ghost:
2021-03-15 10:57
Notification from *Pactflow* :ghost:
alan.hanafy
2021-03-15 19:30
assuming this is the right channel for this question. besides the HAL browser in pactflow, and this blog post: https://pactflow.io/blog/audit-api/ is there anything else on using the audit api? is there any way to view/retrieve the most recent logs? is there any way to GET the audit logs reverse chronologically? Is there any way to filter on timestamp for say only things that happened today (or on a particular year-month-day etc)? i understand that the idea for this endpoint is for some siem to ingest it but I was hoping to be able to review the logs for certain events without having to setup a siem. if i _do_ have to setup a siem, is there any examples on how to set it up with say ELK or something free? I dont mean how to setup ELK, i just mean the configuration examples for logstash or something
2021-03-16 12:03
Notification from *Pactflow* :ghost:
2021-03-17 21:30
Notification from *Pactflow* :ghost:
2021-03-18 04:00
Notification from *Pactflow* :ghost:
2021-03-18 04:58
Notification from *Pactflow* :ghost:
matt.fellows
2021-03-19 11:47
Hi Alan, apologies, I saw this and forget (and then it got lost down the pile)
matt.fellows
2021-03-19 11:50
We created that endpoint on the request of a specific customer who wanted to do it that way (and to be honest, it?s not used all that much from the data I can see). so if you had specific needs for your use case, we?d consider enriching the model
matt.fellows
2021-03-19 11:51
there is the `from` parameter, but that only sets the start point from the UUID, not by date
matt.fellows
2021-03-19 11:53
Filtering by date, type and other properties should be fairly doable. Would you kindly raise a feature request here: https://github.com/pactflow/roadmap/issues?
2021-03-23 12:30
Notification from *Pactflow* :ghost:
wesleythomaswilliams
2021-03-23 19:17
Interested about this @matt.fellows, our PRs in github are dependent on the build status. So if pact verification fails, the build will fail and block the PR merge. Trying to work out any benefit from getting pact verification status separately.
matt.fellows
2021-03-24 00:14
Yeah, so you would need to consider the use of it. That?s a common strategy (checks must be green before a merge can happen). So if you curretnly have a flow that would allow a Pact contract to be broken before a merge, this would break that process.
matt.fellows
2021-03-24 00:14
For most cases, feature branches shouldn?t be merged into main until the pact is passing, but there are of course other flows here. This post is really for the ?standard? scenario
anbu.pandian
2021-03-24 13:45
is there someone who can assist me? I am trying to run pact-python from the gitrepo.
matt.fellows
2021-03-24 23:13
invalid SSL smells like me like you have an internal network proxy that is man-in-the-middle-ing the request, presenting an invalid certificate, whilts trying to download a dependency
bethskurrie
2021-03-26 22:41
@gayatree.eee I'm not sure if I understand your question. Do you mean, a list of the endpoints that are available in Pactflow? We don't currently document them for public use, though there is a terraform provider here https://docs.pact.io/pact_broker/terraform_provider/
matt.fellows
2021-03-26 23:25
The API follows a hypermedia based design which is somewhat self documenting, so if you issue a GET to the root resource requesting the HAL media type, you can follow the resources from there. What are you trying to do?
gayatree.eee
2021-04-06 11:14
Hi, what all roles have permission to create system account in pactflow. Please share any doc having these details. Thanks.
matt.fellows
2021-04-06 11:50
Were adding a feature where a team admin could create system accounts scoped to their team
matt.fellows
2021-04-06 11:51
Any role with the following permissiom: https://docs.pactflow.io/docs/permissions/permissions#system_accountmanage
matt.fellows
2021-04-06 11:51
Note you can change which role has what permission or create customer roles via the API or the Terraform provider
matt.fellows
2021-04-06 12:23
Were those docs helpful? If not, we?ll get them updated to be more so. Let me know what you were expecting and i?ll share with the team
nathan.deamer
2021-04-08 09:41
:wave: My username and password hasn't come through yet (~ 1h). Is it because I selected the "Use github organisation" option and something needs configuring?
matt.fellows
2021-04-08 10:15
Ah, it's the same email address as your other account so the creds are the same. I thought we updated the registration email to indicate this but it seems not.
nathan.deamer
2021-04-08 12:20
Feel free to delete the old ones - setting it up for real and about to start spiking with the teams.
Can you help enable github on the free tier? I am not seeing any of these options
https://pactflow.io/blog/github-and-google-authentication-now-included-in-our-free-tier/
c.pavan1986
2021-04-09 08:55
Do we have environment variable to specify postgres port for pactflow docker image. Actually we are running postgres on other than default port 5432. Using PACTFLOW_DATABASE_URL is exposing the password in env variable as plain text, we are thinking of using kubernates/openshift secret object. I have tried using PACTFLOW_DATABASE_PORT, assuming it would similar to the one thats present for pactbroker (https://docs.pact.io/pact_broker/docker_images/pactfoundation/ : PACT_BROKER_DATABASE_PORT). Please suggest
matt.fellows
2021-04-09 08:59
Assuming you're using our enterprise docker container via http://quay.io, right?
matt.fellows
2021-04-09 08:59
We don't seem to expose a port only option: https://docs.pactflow.io/docs/on-premises/environment-variables#database
matt.fellows
2021-04-09 09:00
Given that you'd need to use a secret for the password anyway, is it a big issue for the secret to just be the full URL?
c.pavan1986
2021-04-09 09:01
yes we are using enterprise docker, we could try with whole url being secret. But is there any plan to add environment variable for PORT?
matt.fellows
2021-04-09 09:23
No plans I'm aware of. @bethskurrie might know why it's not currently exposed. You could raise an issue on our roadmap requesting it if you need
matt.fellows
2021-04-09 09:24
When I'm in front of a computer later I'll check the code base, it might just be undocumented
c.pavan1986
2021-04-09 09:46
So, if we wanted to connect to postgres running on other than default port (5432) we shall be using the PACTFLOW_DATABASE_URL right? :password@machinename:port/databasename
c.pavan1986
2021-04-10 05:19
We would like to have a environment variable exposed. As we cannot use the default port or the full connection string as secret. Can you please do the needful..
matt.fellows
2021-04-10 08:24
How will you secure the password if you can't secure the full connection string?
bethskurrie
2021-04-10 08:27
Also, there is some logic to hide the password in the logging. Someone recently added the logic to hide the password in the OSS DATABASE_URL, so we can port that to Pactflow.
c.pavan1986
2021-04-10 09:21
We can secure the full connection string (using kubernates secret), we would not be doing it as it does not make sense to secure the entire connection string. I think its logically better to decouple the have separate variable serving each of its purpose.
2021-04-15 11:11
Notification from *Pactflow Contract Testing Platform* :ghost:
2021-04-17 00:49
Notification from *Pactflow Contract Testing Platform* :ghost:
nathan.deamer
2021-04-20 14:18
Are system accounts available on the free pactflow broker?
https://pactflow.io/blog/system-accounts/
matt.fellows
2021-04-20 22:06
I don't think so ( see https://pactflow.io/pricing/) but drop us a line and we'll give you the ability
matt.fellows
2021-04-21 05:04
We?ve just removed the limit for all free tier users - you should now have the ability to create a system user, as per the Team plans
aparnachaudhari
2021-04-23 20:40
are there more resources that talk/walk through bi-directional contract testing that is coming down the pipeline?
matt.fellows
2021-04-24 02:11
We haven't officially launched the feature yet, as it's still in developer preview and we're shaping and tweaking based on feedback. If you like, we can setup a session with the team to go through it in more detail and provide support if you'd like to try it out
wesleythomaswilliams
2021-04-27 14:37
Just noticed that there's no protection when deleting a webhook, which I accidentally just did in Pact Flow Enterprise. Any chance of a confirmation box pls to stop morons like me from being morons like me :smile:
bethskurrie
2021-04-29 02:08
At least in the OSS, you have to type the word `DELETE` in the hal browser to delete a webhook :stuck_out_tongue:
c.pavan1986
2021-04-29 14:53
We have deployed pactflow to kubernates environment with basic auth. It shows up the signon screen with username and password. And once I keyin username and passowrd, it redirects to https://encironment/auth/saml.
bethskurrie
2021-04-30 02:23
@c.pavan1986 From the docs: "PACTFLOW_BASIC_AUTH_ENABLED: Whether or not to enable basic authentication. For demo purposes only - not to be set to `true` for production use. Cannot be set to `true` at the same time as PACTFLOW_SAML_AUTH_ENABLED."
bethskurrie
2021-04-30 02:24
@c.pavan1986 the basic auth is only for trying out the docker image locally using docker compose. You cannot have the basic auth on if you are deploying to a real environment with SAML.
c.pavan1986
2021-05-03 05:39
Spyder mesh is for additional security we have, we were using the basic auth only. PACTFLOW_BASIC_AUTH_ENABLED is true and PACTFLOW_SAML_AUTH_ENABLED is false.
c.pavan1986
2021-05-03 05:41
We have deployed pactflow with basic auth on aws eks and postgres as service from aws rds. All was working fine.
c.pavan1986
2021-05-03 05:49
After this we introduced spyder mesh (istio-proxy). This changes introduced 2 problems
1) pactflow (on aws eks) was not able to connect to rds. Because the outbound traffic from aws eks via spyder mesh was not allowing. Solution: we created a serviceentry kubernates object with (aws postgres details: host, post, protocol).
2) pactlow was not accessible, we have ingress via which we were able to access pactflow application (before introducing spyder mesh).
Solution: We added 2 additional atrributes in ingress metadata to reference the pactflow service. This started showing up the ui for keyin username and password, but after keying in username and password it redirect to https://encironment/auth/saml
c.pavan1986
2021-05-03 06:01
Below are the 2 kubernates attributes we hav used in ingress metadata.annotations
http://ngnix.ingress.kubernetes.io/service-upstream = true
http://ngnix.ingress.kubernetes.io/upstream-vhost
matt.fellows
2021-05-03 06:09
We don?t run EKS/Kube here or test Pactflow against it, but if you have a config we could test locally (e.g. run against minikube or something) that could help us sanity test certain things
bethskurrie
2021-05-03 06:30
@c.pavan1986 I've run the app locally with the following env vars
``` export PACTFLOW_BASIC_AUTH_ENABLED=true
export PACTFLOW_BASIC_AUTH_USERNAME=user
export PACTFLOW_BASIC_AUTH_PASSWORD=pass
export PACTFLOW_SAML_AUTH_ENABLED=false```
bethskurrie
2021-05-03 06:31
That's running the native app. I'm just building the docker container to see if there's any difference in the behaviour.
bethskurrie
2021-05-03 06:33
Got the docker container running, and I'm still seeing the expected behaviour. There must be some difference that we aren't aware of between your environment and mine.
c.pavan1986
2021-05-03 07:44
After keying in the username and password (in basic auth promt) I use to see a screen with below details
Login to pactflow
?-?------Login with?----------
ID hyperlink
On click of hyperlink it use to get redirected to /auth/saml
bethskurrie
2021-05-04 00:32
It should just go straight to the logged in page. Can you try a hard refresh for me?
c.pavan1986
2021-05-04 14:42
On a saml enabled pactflow instance can we call can-i-deploy using pact_broker_username and pact_broker_password instead of pact_broker_token. I am using a docker based implementation of pactbroker cli
chris.williams
2021-05-04 16:45
Wondering if there's a problem with new password codes / password resets - as I can't get passwords changed :confused:
aparnachaudhari
2021-05-05 01:58
I want to make sure my understanding of the bi-directional pact testing is correct. Please point out any gaps.
? With bi-directional pact testing, the provider service will upload its swagger spec (OpenApi v2 is compatible) to the pact broker.
? The consumer service writes its contract and expectations for this spec.
? The big difference with doing this vs the things are done now is - the provider side of things are spun up easily -
? i.e. just upload the oas spec and we can have validations vs provider service needing to be spun up with mocking any dependencies.
? The consumer still writes the contract it expects from the provider. - I am unclear how this might differ from the way we write the contract now?
? I am proposing the use of this against an api gateway - consumer .. which aggregates the oas from multiple provider services. Does this sound like a use-case that might be supported by this.
matt.fellows
2021-05-05 02:04
> With bi-directional pact testing, the provider service will upload its swagger spec (OpenApi v2 is compatible) to the pact broker.
Yes. The caveat to add here, is that *you* are responsible for ensuring your provider actually is compatible with the spec, and the functional tests.
> The consumer service writes its contract and expectations for this spec.
Yep. You can use Pact to do this (so same as standard flow) or convert existing mocks into contracts (currently this would be manually done from your side, as we build out adapters). We also envisage the community build out adapters for things they see as useful - e.g. @yousafn has actually just created one for MSW for example: https://twitter.com/YOU54F/status/1389653091927142400
> The big difference with doing this vs the things are done now is - the provider side of things are spun up easily -
> i.e. just upload the oas spec and we can have validations vs provider service needing to be spun up with mocking any dependencies.
Yep, as per above, this testing is very much delegated to you (for now, we will also be building tools in this space over time)
> The consumer still writes the contract it expects from the provider. - I am unclear how this might differ from the way we write the contract now?
see above.
> I am proposing the use of this against an api gateway - consumer .. which aggregates the oas from multiple provider services. Does this sound like a use-case that might be supported by this.
definitely one of the core use cases
aparnachaudhari
2021-05-05 02:27
The caveat you speak of.. can you please elaborate..how is the provider verifying this compatibility?
matt.fellows
2021-05-05 02:41
So basically just becase you have an OAS doesn?t mean it actually represents the behaviour of your provider, right? Especially if made by hand. So in the bi-directional mode, you have two choices:
1. Generate the OAS from code (e.g. in Golang, that could be struct tags/generators)
2. Test that your provider is compatible with the spec (there are heaps of tools for this - e.g. Dredd, Postman, Prism and so on)
matt.fellows
2021-05-05 02:41
We?ll be investing in our own tooling for (2), but that?s a little while away, and also may not be something people want to adopt - especially if they have already built their own suite of tests
c.pavan1986
2021-05-05 06:12
We just wanted the connectivity to go through regular authentication process if possible. As the tokens wont expire in pactflow (unless user regenerates the token), incase if we have username and password based connectivity the authentication process can be in sync with password expiration policy of the saml provider.
c.pavan1986
2021-05-05 06:16
For now we have removed the spydermesh, will give it a try when we add it back
ssorcnafets
2021-05-05 07:43
Hello, Im hoping someone can help me out here as im at a bit of a dead end, I have the following failure with a pactflow verification, can anyone help direct me in which way to go?
```java.lang.AssertionError: Pact between onboarding-changefeed-messages (bcd51dfe4259a56c3ae045142678e262094b7863) and onboarding-messages - Generates message 'an accepted invitation change message'
Failures:
1) an accepted invitation change message
1.1) Failed to invoke provider method 'Verify Provider For Accepted Invitation'
at au.com.dius.pact.provider.junit5.PactVerificationContext.verifyInteraction(PactVerificationContext.kt:61)
at com.sedex.connect.onboarding.pact.MessageProviderPactTest.verifyInteraction(MessageProviderPactTest.kt:67)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)```
Seems to be somewhat intermittent, and only on our CI pipeline, I can not reproduce locally. Any suggestions on how to get any more insight as to the nature of `Failed to invoke provider method`? There is some more stack trace but left it out for the sake of brevity. More then happy to give more details or equally let me know if this is more suitable to ask in the JVM channel?
matt.fellows
2021-05-05 08:10
There should be a log file that your Junit test spits out - anything helpful in there?
matt.fellows
2021-05-05 08:10
flakiness on CI usually indicates timing issues. So perhaps the provider is not spinning up in time for the tests to run (depending on how you start it)
ssorcnafets
2021-05-05 08:17
I only have the junit test reports, I can look to see if I can enable additional logging if thats a possibility
ssorcnafets
2021-05-05 08:18
the rest of the stack trace is?
```at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:212)
at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:192)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:139)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$2(TestTemplateTestDescriptor.java:107)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:107)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:42)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy5.stop(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.base/java.lang.Thread.run(Thread.java:832)```
matt.fellows
2021-05-05 09:42
hopefully somebody else can help. But interestingly, there are no pact classes in there - so it looks to be not a Pact issue
ssorcnafets
2021-05-05 09:54
no worries, you did get me thinking about the provider timing/availability issues. Ive popped some more try catches around the verify method and the provider method to see if we can get some insight as to the underlying exception which i think is getting swallowed? this investigation continues :male-detective:
aparnachaudhari
2021-05-05 22:41
Question - has any one actually generated a pact contract using an existing swagger spec i.e. given this swagger spec generate a contract for it.. and then use that contract to test for any new changes in the spec
yousafn
2021-05-05 23:13
a swagger spec will define a aset of required and optional fields for all operations, and doesn?t define a contract, so your output pact files would potentially contain far more than the consumer needs and would make the contract incredibly brittle
yousafn
2021-05-05 23:14
we use a swagger spec from a provider to validate that our consumer generated pact contracts are achievable, but they will read the docs from a feature branch if it exists, otherwise fall back to master
bethskurrie
2021-05-05 23:15
@aparnachaudhari have a read of https://docs.pact.io/faq/#can-i-generate-my-pact-file-from-something-like-swagger
aparnachaudhari
2021-05-05 23:18
```Manually writing or generating a pact file from something like a Swagger document would be like marking your own exam, and would do nothing to ensure that the code in the consumer and provider are compatible with each other.```
I do agree with ^^..
aparnachaudhari
2021-05-05 23:19
I got asked by a developer who works on the API Gateway about this actually.. the idea was - if we have a known good spec on the gateway for all the aggregated providers.. can pact enable us to validate any breaking changes that happen after this.
aparnachaudhari
2021-05-05 23:21
yeah.. it was actually when we were reviewing a possibility of using bi-directional contract testing using pactflow
aparnachaudhari
2021-05-05 23:22
but the consumer in this case - per my understanding is still written in the standard way - i.e. hand written consumer tests? what he wanted to know was if I have a known good spec can I just use that as a consumer contract
bethskurrie
2021-05-05 23:59
I don't think this is released in the on prem version yet though. Are you on prem or saas @c.pavan1986?
tjones
2021-05-06 00:47
You may also be interested in this discussion:
https://pact-foundation.slack.com/archives/C5F4KFKR8/p1618548682066100
matt.fellows
2021-05-06 01:04
true, but it will be soon so it?s worth knowing, in case you try and build your own tooling around it
agarwal.akash333
2021-05-07 03:25
Hello. I?ve been looking into using pact-cli for verifying a Springboot provider and publishing the results back to pactflow. To use `pact-cli verify` , does it mean that I would need to have the Spring server up and running for Pact CLI to run the verification on? In essence, does that mean the junit setup becomes useless?
bethskurrie
2021-05-07 03:28
The Junit setup would be useless if you used the CLI verifier. You'd have to stand up the entire application on a port with a real process.
agarwal.akash333
2021-05-07 03:30
Yes, I tried using provider 4.2.4 and it apparently broke the Junit setup quite a bit. On the consumer side, 4.2.4 just didn?t work on 2 different machines using Gradle 6.5 and java 11. This led me to consider using CLI for publishing pacts and verification results
bethskurrie
2021-05-07 03:31
Yes you can use the CLI. It's not ideal, and you lose the ability to dynamically mock things inside the process, but it can be done.
agarwal.akash333
2021-05-07 03:33
> The Junit setup would be useless if you used the CLI verifier. You?d have to stand up the entire application on a port with a real process.
Thanks! This confusion is exactly what has been crunching my brain for quite a while now :sweat_smile:
> Yes you can use the CLI. It?s not ideal, and you lose the ability to dynamically mock things inside the process, but it can be done.
Yeah, I was wondering about state management, and I?m guessing one would have to do that via API calls as in https://github.com/pact-foundation/pact-provider-verifier/blob/master/examples/api.rb.
bethskurrie
2021-05-07 03:35
Depending on which verifier you're talking about (there's a ruby based one and a rust one).
bethskurrie
2021-05-07 03:35
The ruby one requires that you create a "change provider state" endpoint, either on the running app, or in a separate endpoint.
agarwal.akash333
2021-05-07 03:37
> either on the running app, or in a separate endpoint.
How would you be able to do that on a seperate endpoint and affect the main provider? Some ruby world magic?
agarwal.akash333
2021-05-07 03:37
(I don?t mean to diss ruby haha, I?ve been quite a rails enthusiast)
bethskurrie
2021-05-07 03:37
the provider state set up call usually modifes somethign in the database.
bethskurrie
2021-05-07 03:37
YOu can either add an extra "test only" endpoint to your app, or make a separate "state change" app.
agarwal.akash333
2021-05-07 03:39
That makes a lot of sense, thanks for explaining it so well, Beth! Time for me to reconsider using CLI :slightly_smiling_face:
matt.fellows
2021-05-07 05:12
> The ruby one requires that you create a ?change provider state? endpoint, either on the running app, or in a separate endpoint. (edited)
the rust one is also the same
matt.fellows
2021-05-07 05:12
Maybe you could have a separate sub-project or something Akash, that doesn?t have dependency conflicts but can pull in the same source? Just thinking out loud :thinking_face:
agarwal.akash333
2021-05-07 05:14
I?m leaning towards getting the JUnit stuff working so far :slightly_smiling_face:
ssorcnafets
2021-05-07 08:29
Yes, thanks! for the benefit of others reading and trying to debug such things?
added `./gradlew pactProviderTest -Ppact.showStacktrace=true --info` and also a try catch around the offending method
this yielded the following?
```MessageProviderPactTest > Pact between onboarding-changefeed-messages (9a48634e26aa894bed722b4ed956b83e6b1da70b) and onboarding-messages - an accepted invitation change message STANDARD_ERROR
java.lang.NullPointerException: Cannot invoke "com.fasterxml.jackson.databind.JsonNode.asText()" because the return value of "com.fasterxml.jackson.databind.JsonNode.get(String)" is null
at com.sedex.connect.onboarding.pact.MessageProviderPactTest$Verify Provider For Accepted Invitation$1.test(MessageProviderPactTest.kt:98)```
We eventually found that the underlying test on a kafka topic was reused across different test suits and as such was not getting the message it was expecting, so theres some json parsing error message there :point_up_2:
We seems to have stabilised things now by randomising our topics names for tests where required.
c.pavan1986
2021-05-07 12:19
Is there any other way other than using pactflow token, like a bearer token if producable by saml provider
matt.fellows
2021-05-07 12:37
The short answer is no, only tokens generated by the app itself may be used as bearer tokens
c.pavan1986
2021-05-07 12:42
We want to publish contracts/verification results, can i deploy. Using something else other than pactflow generated tokens. We want to have single flow of access to pactflow for user login and api calls (publish contracts/verification results etc)
matt.fellows
2021-05-07 12:47
The tokens are still associated with a user principal, and therefore the federated IdP
matt.fellows
2021-05-07 12:47
If the user is deactivated in the IDP and deprovisioned their tokens will cease to work
matt.fellows
2021-05-07 12:48
I'd need to double check if there is a deprovisioning hook, but there is most definitely an API you can call to disable a user
thomas.rosati
2021-05-10 23:21
Specifically, we're wondering if we can use a basic Access Token obtained via OAuth Resource Owner Password Credentials flow instead of a Pactflow API Token to make api-to-api calls from our CI pipeline to Pactflow?
thomas.rosati
2021-05-10 23:23
Pactflow currently authorizes human operators through a standard OAuth flow supported by our enterprise security, but just wondering if we could do the same for our non-human functional accounts as well..
robbert.van.der.zon
2021-05-11 11:40
Hi all!
I am not sure if this is asked before, but I could not find it.
We started using contract testing with pactflow within our own organisation.
But now we would like another company which uses one of our services to use contract testing too.
That means that they need to upload their client contracts to our pactflow environment.
For their CI/CD, they need an API Token. Since we can only create one read/write token, we need to give them the same token as we use internally. I am wondering if that is a security issue since they can see all our other internal contract also with that token.
Also, I think it might be useful for them to be able to logon to pactflow user interface to see if their contracts are verified.
I tried creating a user with 'Viewer' permissions and assign this user to a team that has only one team-application configured.
I was hoping that this user would only see the contracts that belong to his team, but they can see all contracts (and also all teams and users).
My main question is: How must we deal with the situations where we would like a contract test with an external company.
matt.fellows
2021-05-11 11:43
great question!
First up, might be worth looking at this specifically: https://github.com/pactflow/roadmap/issues/35
matt.fellows
2021-05-11 11:43
Currently, there is no way to hide contracts from another team, but obviously we can lock down who can do what to them
matt.fellows
2021-05-11 11:44
We haven?t really focussed on this use case as you can guess, but it?s something we?ve known about as a secondary type use case
roy_collings
2021-05-13 10:00
Hi! I just came across this: My 'can-i-deploy' (CID) check doesn't seem to work for provider (with OAS):
1. publish matching consumer (pact) and provider(using oas), and tag both as 'prod'
Provider CID = ok
Consumer CID = ok
2. Remove an endpoint completely in provider OAS
Provider CID = *ok*
3. Publish new provider and tag as 'prod'
Consumer CID now fails (as expected)
Why doesn't the _provider_ CID check fail at step 2? Seems to _never_ fail, no matter what I change in it.
roy_collings
2021-05-13 10:08
in case it matters, the result from the provider 'can-i-deploy' is always `There are no missing dependencies`
roy_collings
2021-05-13 10:18
... also, in case it matters, when I publish my provider oas, this is the payload:
```{
"content": "$OAS",
"contractType": "oas",
"contentType": "application/yaml",
"verificationResults": {
"success": true,
"content": "$RESULTS",
"contentType": "text/plain",
"verifier": "verifier"
}
}```
matt.fellows
2021-05-13 11:14
Thanks Roy. I have forwarded your email to our support address and we?ll look into it
roy_collings
2021-05-13 11:45
By the way @matt.fellows, should I be using `record-deployment` instead of tagging like this? Until I noticed that endpoint I thought that's how to mark something as 'test'/'prod'.
matt.fellows
2021-05-13 11:57
I don't think so yet, albeit it is now in the system and will eventually be the new way.
@bethskurrie may have thoughts on it tho
matt.fellows
2021-05-13 11:57
I'd like to ensure all docs/tutorials etc are using it instead of tagging when we pull the pin on it
roy_collings
2021-05-13 12:05
Cool - I think I've figured out my issue btw: I was assuming that since the _consumer_ can do can-i-deploy without specifying other pacticipants, that tthe provider could do that too (`can-i-deploy --pacticipant 'test-provider' --version '1.0.0' --to 'prod'`). If I specify the consumer in there though, then it works as expected (I'll update the gihub action to enable that).
Would be nice if the provider didn't need to explicitly know which consumers used it though.
matt.fellows
2021-05-13 12:06
That?s certainly an option to work around the current problem, but it _should_ discover consumers by default
bethskurrie
2021-05-13 20:28
My initial thought was that it was hitting the bit of logic that allows providers to deploy into a new environment without the consumer being present.
bethskurrie
2021-05-13 20:29
But if they're both tagged prod, then it should be discovering them. You should not need to explicitly list both consumer and provider. That does seem like a bug. I'll try and get a reproduction today.
bethskurrie
2021-05-14 07:04
@roy_collings the record-deployment API is not finished yet, but should be out soon.
matt.fellows
2021-05-14 07:49
Sorry for the delay, had a late phone call this afternoon and wanted to do a few checks/balances before I signed it off
matt.fellows
2021-05-14 07:57
It?s looking good to me. I?ve tested a few scenarios, including the negative one that triggered this conversation, and it correctly detects the backwards incompatible change
roy_collings
2021-05-14 08:42
Oh weird, it's reliably not working for me :disappointed: I'll look into it more next week hopefully.
matt.fellows
2021-05-14 09:05
You may need to rerun the consumer build and then return the provider. If that doesn't work, should he delete the integration and start again?
bethskurrie
2021-05-14 09:06
When you do get a moment @roy_collings please do let me know your steps to reproduce the problems you're seeing.
matt.fellows
2021-05-14 09:12
I'll have a look later Beth. I may have been confused, but I found that when I ran the provider verification the first time after the fix it didn't show the result
matt.fellows
2021-05-14 09:13
I thought because even though the consumer was tagged prod, the contract may have been pending (obviously a guess, I don't know how that works with the new BDC flow).
roy_collings
2021-05-14 15:45
@bethskurrie without getting into the low-level details the steps were ('CID' = 'can-i-deploy' to 'prod'):
1. Published matching consumer (pact) and provider (OAS), and tag both as 'prod', then ran CID on both:
Provider CID = ok
Consumer CID = ok
2. Remove an endpoint completely in provider OAS and publish it again (new version, but tagged as 'prod'):
Provider CID = *ok*
Consumer CID = failed (as expected)
abdel.akkoub
2021-05-21 22:07
Hello, we are facing an isssue with PactFlow's webhooks that trigger the corresponding Azure DevOps CICD provider pipeline. In the body of the webhook's POST, we are sending the following JSON:
`{`
`"definition": {`
`"id": <AZURE PIPELINE DEFINITION ID>`
`},`
`"parameters": "{'consumerVersionTags': '${pactbroker.consumerVersionTags}'}",`
`"sourceBranch": "main"`
`}`
However, when we face a situation where 1 branch on the consumer side breaks provider, and a 2nd code fix is committed to the same consumer branch, the webhook trigger seems to skip the consumerVersionTag on the 2nd go:
```[2021-05-21T21:58:37Z] DEBUG: Webhook context {"base_url":"https://myCompany.pact.dius.com.au","consumer_version_tags":[],"event_name":"contract_published"}
[2021-05-21T21:58:37Z] INFO: HTTP/1.1 POST https://dev.azure.com/myCompany/Overcast/_apis/build/builds?api-version=5.
[2021-05-21T21:58:37Z] INFO: accept: */*
[2021-05-21T21:58:37Z] INFO: user-agent: Pact Broker v2.79.0
[2021-05-21T21:58:37Z] INFO: authorization: [REDACTED]
[2021-05-21T21:58:37Z] INFO: content-type: application/json
[2021-05-21T21:58:37Z] INFO: host: http://dev.azure.com
[2021-05-21T21:58:37Z] INFO: content-length: 91
[2021-05-21T21:58:37Z] INFO: {"definition":{"id":1580},"parameters":"{'consumerVersionTags': ''}","sourceBranch":"main"}
[2021-05-21T21:58:37Z] INFO: HTTP/1.1 200 OK
.......```
This causes the provider to run against main instead of running against the correct consumer branch:
```
Description of differences
--------------------------------------
* Expected a Hash (like {"activeSowTemplatesV2"=>{"totalCount"=>1, "items"=>[{"sowTemplateId"=>"1231-1ab23f", "name"=>"Development", "projectType"=>"Development", "businessJustification"=>"1238", "industry"=>"IT", "geography"=>"United States", "currency"=>"USD", "sowCreationCallback"=>"ServicesProcurement/ProjectAgreementBuilder.aspx?TemplateID=1231-1ab23f", "__typename"=>"SowTemplateV2"}], "__typename"=>"SowTemplatePageV2"}}) but got nil at $.data
2 interactions, 1 failure
Failed interactions:
* A request to get sow templates given user has sow templates and is authenticated (to re-run just this interaction, set environment variable PACT_BROKER_INTERACTION_ID="b20ce3f669130aeba7da505067299fa048ed7ae5")
INFO: Tagging version a64c2a71747accf8bc98a596ff00046129ec947a of DigitalExperienceApollo_Overcast as "main"
INFO: Verification results published to https://myCompany.pact.dius.com.au/pacts/provider/DigitalExperienceApollo_Overcast/consumer/UI/pact-version/2f2dd87919908660defc04e4b9f4b51b961279bd/verification-results/12107
at ChildProcess.<anonymous> (node_modules/@pact-foundation/pact-node/src/verifier.ts:272:58)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 total```
We have seen this issue on multiple occasion in the past few months. However, replicating it has been tricky given the circumstances under which the issue surfaces. Please advise how to go about resolving this. Thx!
abdel.akkoub
2021-05-21 23:59
It turns out the webhook is triggered a fraction of a second before the tagging task runs in the consumer pipeline. I also noticed that we configured the webhook to run only when new contracts are published. I added the option the to trigger webhook when a new contract is published with a content change or tags that way the webhook is triggered under both conditions.
The downside with this approach is having the provider pipeline run twice with different consumer tags which may give false negatives under some circumstances. Please advise if you recommend following a different approach.
bethskurrie
2021-05-23 23:16
@abdel.akkoub ideally, you should be passing through the Pact URL, not the consumer version tags.
bethskurrie
2021-05-23 23:17
Here is the documentation for setting up the verification build https://docs.pact.io/pact_nirvana/step_4#e-configure-pact-to-be-verified-when-contract-changes
bethskurrie
2021-05-23 23:17
The reason that it's important to use the URL is that there is extra information encoded in that URL that helps the broker line up all the events and results properly.
bethskurrie
2021-05-23 23:22
> It turns out the webhook is triggered a fraction of a second before the tagging task runs in the consumer pipeline. I also noticed that we configured the webhook to run only when new contracts are published. I added the option the to trigger webhook when a new contract is published with a content change or tags that way the webhook is triggered under both conditions.
I can see that there is some flakey behaviour here, but I'm not sure if this is the cause. The "create tag" response comes back before the pact publishing request is sent. That means that the tag must have been committed to the database before the publish pact request is sent. The webhook is not triggered until the pact has been published (it's not triggered on the tagging, it's triggered on the pact publication).
bethskurrie
2021-05-23 23:24
There is a new pact publishing API that is coming out soon which does the tagging/pact publishing all in one, so if the problem is caused by timing issues, they should be fixed.
bethskurrie
2021-05-24 00:26
@abdel.akkoub can you try using the pact URL instead of the consumer version tags, and let me know how you go? If that's not possible, or doesn't fix the problem, then I'll let you know how you can try out the new publishing API to see if that fixes the issue.
bethskurrie
2021-05-24 00:27
Let me know which language you're doing the verifications in, so I can get that client updated.
roy_collings
2021-05-24 10:25
@bethskurrie @matt.fellows just fyi: this seems to be working now (finally got some time to figure it out ... and it just works!). No idea what happened there, but happy that it's working now - apologies, since no-one else was complaining I can only assume it was something I did wrong the last time :man-shrugging: .
2021-05-24 11:51
Notification from *Pactflow Contract Testing Platform* :ghost:
abdel.akkoub
2021-05-24 15:16
Hi @bethskurrie, I appreciate the response. I am going through the documentation to replace existing parameter with the URL one. As for the provider, it is in Node. Will update you shortly. The reason why I am thinking the tagging is an issue from our end, is the fact that we have two separate tasks in the consumer CICD pipeline where the contract is first uploaded then is tagged. I am guessing the webhook is triggered between upload and tagging given that the broker is configured to fire whenever a new contract is uploaded regardless of tagging.
2021-05-24 15:35
Notification from *Pactflow Contract Testing Platform* :ghost:
bethskurrie
2021-05-24 22:50
@abdel.akkoub why is the contract tagged after uploading? What are you trying to achieve?
abdel.akkoub
2021-05-24 23:58
I believe the team was trying to adhere to pact standards by assigning the feature branches to tags. After that, consumer pipeline runs can-I-deploy using the uploaded contract's version (commit hash) against latest prod branch on the provider. The webhook is also triggered to run provider against latest consumerVersionTags passed in via the webhook. If no consumerVersionTags are found when the provider test is running, the provider defaults to the main branch on the consumer side.
abdel.akkoub
2021-05-25 00:04
Now here is the interesting part, This issue appears to be related to the provider verification. We are publishing two contracts that appear to be identical to one another with the only difference being the order of interaction objects in the contracts. When replaying against contract A, provider passes, when run against contract B, provider fails. I initially thought the broker not passing in the tags was the cause of issue. However, I am now able to replicate the issue locally using each contract. I attached both contracts for you to compare.
bethskurrie
2021-05-25 00:05
> assigning the feature branches to tags
Are you doing this manually, or in the pact publishing configuration?
bethskurrie
2021-05-25 00:06
If you're doing it in the pact config, the tagging happens first, as it goes through the officially supported pact broker client, that knows to do this in the right order.
bethskurrie
2021-05-25 00:06
> the only difference being the order of interaction objects in the contracts
bethskurrie
2021-05-25 00:07
Order is ignored in the broker. If you publish a pact with the same interactions in a different order, it thinks it is the same pact.
bethskurrie
2021-05-25 00:13
Open both of the pacts in the API browser (open them in Pactflow, then click on the "API") button. On one of them, scroll down to the `pb:diff` relation. Click on the green "?" button. Go to the other pact, and grab the SHA that you'll see next to the `pb:pact-version` relation and put that in the `pactVersion` value in the form. Click "Follow URI"
bethskurrie
2021-05-25 00:32
Ok, I've parsed them using the broker code, and you're right, they are identical once all the interactions and keys are sorted.
bethskurrie
2021-05-25 00:35
```def prepare_content json_content
if options[:raw]
json_content
else
SortContent.call(Parse.call(json_content)).to_json
end
end```
bethskurrie
2021-05-25 00:35
Those two pacts you sent me are the same, but that diff endpoint is showing an actual difference
bethskurrie
2021-05-25 00:37
I'm not sure that the two pacts you sent me are the same as the two that are being diffed.
bethskurrie
2021-05-25 00:37
Either way, ordering differences should not cause failures because there should be no dependencies between the interactions.
abdel.akkoub
2021-05-25 00:39
I've gone through each interaction to make sure I am not losing my sanity lol :slightly_smiling_face: What is really puzzling here is that I looked at the consumer PR that got merged into main when the contract version was bumped, and I didnt see anything committed to the contract files that are in question
abdel.akkoub
2021-05-25 00:41
The ones I sent you are the local ones vs whats in PactFlow. Not sure what would cause a state leak if the given strings is all what would impact that in the contracts. Correct?
bethskurrie
2021-05-25 00:43
Eg. a database. Is that being cleared completely between interactions? That's the most common cause of failures due to ordering.
abdel.akkoub
2021-05-25 00:45
We do. We have override methods that basically inject hardcoded data in the return objects rather than making db calls and mapping everything back in the response
bethskurrie
2021-05-25 00:46
I've run out of suggestions. If you can reproduce it locally with those two pacts, then it seems unrelated to Pactflow.
2021-05-25 02:30
Notification from *Pactflow Contract Testing Platform* :ghost:
abdel.akkoub
2021-05-25 13:51
Thank you Beth. It turned out to be related to the provider states. Some of the states were not being reset. I guess that would make sense that the order of the interactions was breaking the provider. Anyway, thank you so much for your help!
c.pavan1986
2021-05-26 08:09
We have deployed patflow to aws eks, enabled basic auth for testing purpose. But after keying in basic auth username and password it gets redirected ui which shows Login the Pactflow
?---------login with-----------
c.pavan1986
2021-05-26 08:10
On developer toolbar on chrome, network tab shows "Uncaught (in promise) Error: User is not authenticated at api.js:20
c.pavan1986
2021-05-26 08:13
On patflow container logs below is output
Time D -- Basic auth check is beong performed, but no valid auth credentials er provided. This might be just because the client does not send credentials until ot receives the challenge
Time D : Begin
Time D : Insert into audit_events ....
bethskurrie
2021-05-26 22:43
Hi @c.pavan1986 I remember you had this issue a few weeks ago, and I was unable to reproduce it locally. Can you see if you can reproduce the issue by taking this docker compose file and using the same environment variables you're using for your kubernetes set up https://docs.pactflow.io/docs/on-premises/docker-compose-example
bethskurrie
2021-05-26 22:43
The other alternative is to add the simplesaml container to your kubernetes set up until you're ready to use your real IDP.
vishnuprakash323
2021-05-31 04:08
Hi folks, I am new to pact and pactflow. I was going through the feature comparison between Pact broker and pactflow in pactflow website. I could see pactflow supports contract types other than pact and multiple testing modes. Can someone help me to understand what are the other contract types and testing modes supported by pactflow? Any input will be helpful. Thankyou
matt.fellows
2021-05-31 05:42
It?s currently in developer preview. If you drop us a line to I can send you some more material
matty_o_connor01
2021-06-02 13:21
I don't know whether this is language binding specific but I'll try here first.
How do I update pactflow with the result of my provider verification?
I've got a passing provider test that appears to be successfully communicating with pactflow to do the verification, however nothing happens after the test finishes. The pact remains in an unverified state.
This is the C# code I'm using for reference
`var pactUriOptions = new PactUriOptions()`
`.SetBearerAuthentication("<Read/Write Token>");`
`IPactVerifier pactVerifier = new PactVerifier(config);`
`pactVerifier`
`.ServiceProvider("OconnorEvents.EventCatalog", "http://localhost:5000")`
`.HonoursPactWith("OconnorEvents.React")`
`.PactBroker("https://xxxxx.pactflow.io", pactUriOptions)`
`.Verify(description: "a request for events", providerState: "");`
And this is my log output
`INFO: Reading pact at https://xxxxx.pactflow.io/pacts/provider/OconnorEvents.EventCatalog/consumer/OconnorEvents.React/pact-version/a82e5668a80c716c76c87a50366b56ff86c3e192/metadata/c1tdW2xdPXRydWUmc1tdW2N2bl09MQ`
`DEBUG: The pact at https://xxxxx.pactflow.io/pacts/provider/OconnorEvents.EventCatalog/consumer/OconnorEvents.React/pact-version/a82e5668a80c716c76c87a50366b56ff86c3e192 is being verified because the pact content belongs to the consumer version matching the following criterion:`
`* latest version of a consumer that has a pact with OconnorEvents.EventCatalog (1)`
`DEBUG: Filtering interactions by: {:description=>/a request for events/, :provider_state=>nil}`
`Verifying a pact between OconnorEvents.React and OconnorEvents.EventCatalog`
`A request for events`
`with GET /api/events`
`returns a response which`
`has status code 200`
`has a matching body`
`includes headers`
`"Content-Type" which equals "application/json; charset=utf-8"`
`1 interaction, 0 failures`
bethskurrie
2021-06-02 20:39
@matty_o_connor01 there will be a "publish verification results" flag that you need to turn on.
bethskurrie
2021-06-02 20:39
I don't know what it is off the top of my head, but from memory, it's documented so where near the bottom of the Pact Net readme.
matty_o_connor01
2021-06-02 21:45
Yep it was hiding at the bottom :slightly_smiling_face:. All sorted thanks.
i.pinzari
2021-06-07 18:19
does anyone experience any issues triggering webhook from pactflow? My webhook is not being triggered by the events
bethskurrie
2021-06-07 20:28
Hi @i.pinzari can you send an email to with the details of what is being published, and I can look at the logs for you.
matt.fellows
2021-06-08 00:08
And more generally, the events themselves: https://docs.pact.io/pact_broker/webhooks#events
matt.fellows
2021-06-08 00:08
In Pactflow, if you navigate to the webhooks you can see the latest executions for them, to see if there are any issues firing the event
i.pinzari
2021-06-08 18:47
The issue that we are having now is that the webhook is invoked all the time a contract gets published regardless if conent is changed or not. Even tho I selected `Contract published with changed content or tags`
gayatree.eee
2021-06-09 10:58
Hi Team, does pactflow APIs are following SCIM standard to be consumed by identity management system like Okta.
c.pavan1986
2021-06-09 18:34
We are trying to input the license file for our onprem deployment onto aws eks. We have created a kubernates secret with the license file, and mounted the secret onto the deployment. When I get into the pod (running instance) the license file is located in /home/pactflow-onprem.lic/license.lic
Is this expected or the license file should be in /home/pactflow-onprem.lic? Is there a way (env variable, or someother config), which will point to license file in the pod.
https://docs.pactflow.io/docs/on-premises/license
bethskurrie
2021-06-09 23:00
Hi @c.pavan1986, the licence should definitely be at /home/pactflow-onprem.lic It seems like the kubernetes volume mounting config is not correct. There is no way (currently) of changing where it should look for the licence file.
uglyog
2021-06-10 01:16
Looks like it has to be in either `/`, `/home` or `/home/pactflow` and the file has to be named `pactflow-onprem.lic`. I've raised a support ticket with a third party supplier to see if we can get it to use an environment variable.
uglyog
2021-06-11 02:05
@c.pavan1986 there is an environment variable to change the directory, but the file name must be `pactflow-onprem.lic`
c.pavan1986
2021-06-11 02:09
Thank you. Can you please share the environment variable name? not able find it in documentation https://docs.pactflow.io/docs/on-premises/environment-variables
c.pavan1986
2021-06-11 02:26
Thank you, from which version of onprem release is this variable available?
matt.fellows
2021-06-14 08:23
Drop a line to with your business account details and SAML metadata (as per docs) when you're ready and we'll enable it for you!
gayatree.eee
2021-06-14 11:20
Can you please share the pactflow on-premise release version where custom role feature is available
bethskurrie
2021-06-16 07:21
@i.pinzari please do send an email with the details of the pact, because I can look at the logs to see what Pactflow thinks has changed.
c.pavan1986
2021-06-17 11:52
If the password of the postgres has been changed is there any to make pactflow use the updated password without restarting the container. Is there any other way other than environment variable to supply postgres password.
matt.fellows
2021-06-17 11:53
Not at this stage. I believe we discussing the possibility of AWS IAM access for you folks which would address this
dawoud.sheraz
2021-06-17 13:31
Hello. Not sure if this is the right place but what are the deployment options for pactflow? Are both options, on-premise/self-hosted and managed by Pactflow, available? Or is it just self-hosted/on-premise? Are there difference in pricing?
matt.fellows
2021-06-17 23:55
Drop us a line at if you?d like to know more or be sent some more details
dawoud.sheraz
2021-06-18 05:00
thanks for sharing the link. I remember seeing this page before. I was trying to find the info on pricing page.
michael.branders
2021-06-22 06:31
I got a question about the users on pactflow. You have 2 subscription team and business on is 10 user the other 25. I work for a company that has over 100 IT personel. How would this work if we get pactflow? Would only 10 or 25 people be able to login to pactflow? This would have information for all developers not just some. Would love some more information on this.
matt.fellows
2021-06-22 06:33
We have plans that go well beyond it. We just stop at 50 online because usually people want to talk to us aver that
matt.fellows
2021-06-22 06:34
If you drop a line to we can share pricing and/or have a chat about how we can support you
michael.branders
2021-06-22 06:34
Thats fine i was just wondering becous online it stopt at 25 and that was odd.
michael.branders
2021-06-22 06:37
I've tryed pact last week and now i have to make a presentation compairing you to spring cloud contract. But i want to be prepaired becouse if they choose pact then we have to get a pact broker. community or pactflow. But if it was limited to 25 that would def be an issue
michael.branders
2021-06-22 06:44
SCC i got working and pact on there own. I know there are alot of options to make pacts but the imput that i got from the dev community is lets choose a framework and work with it. So there not looking to combine right now :slightly_smiling_face:
matt.fellows
2021-06-22 06:45
Obviously we're biased, but I think there is some material floating about
michael.branders
2021-06-22 06:46
and pact offers more then just jvm. Its nice to be able to make pacts with node (the frontends we make) and amqp what we use too. SCC is limited in that. So there you def win points :slightly_smiling_face:
joaoproenca
2021-06-22 09:32
hey folks I have a colleague who is doing a ramp up project where he will be evaluating pactflow. He signed up for a new free account to start looking at pactflow but somehow he can't get his password to work. Can anyone help me out in:
1. Unblocking his pactflow user
2. Inviting him to this slack space so that he can talk to you folks :slightly_smiling_face:
matt.fellows
2021-06-22 09:45
Check spam, and if there is a trailing space / character is the password
joaoproenca
2021-06-22 09:48
according to him the problem is that he gets a temporary password in the email (with special characters) and then the login form looks like this
varun.patil
2021-06-23 12:49
Hi All, I am conducting a POC and pactflow, i just wanted to know if we can publish pacts to pactflow using maven?
varun.patil
2021-06-23 19:33
I am using pactflow for one of our POC's trying to verify the pacts published by the consumer
varun.patil
2021-06-23 22:01
Hello Team, facing the below issue when trying to verify the pacts published by provider ..
varun.patil
2021-06-23 22:03
@RunWith(SpringRestPactRunner.class)
@Provider("account-service")
@PactBroker(
host="http://myorg.pactflow.io",
scheme = "https",
authentication = @PactBrokerAuth(scheme = "basic", username = "sername", password = "password")
)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT,
classes={AccountBridgeApplication.class})
@ActiveProfiles("dv")
public class AccountSeviceProviderContractTest {
@TestTarget
public final Target target = new SpringBootHttpTarget();
@State("Ban 3232 exists")
public void accountExists() {
// nothing to do, real service is used
}
}
matt.fellows
2021-06-23 22:28
Varun - can you please wrap future code in code gates using the backtick (```) and also try to get in a thread?
matt.fellows
2021-06-23 22:33
either that, or you?re not using an API token to get the pacts from pactflow
matt.fellows
2021-06-23 22:33
> authentication = @PactBrokerAuth(scheme = ?basic?, username = ?sername?, password = ?password?)
you cannot use username/password here
matt.fellows
2021-06-23 22:34
see this example for more: https://docs.pactflow.io/docs/examples/java/provider-springboot/readme
matt.fellows
2021-06-23 22:34
Maven spring boot workshop: https://github.com/pact-foundation/pact-workshop-Maven-Springboot-JUnit5
matt.fellows
2021-06-23 22:34
Gradle spring boot: https://github.com/pact-foundation/pact-workshop-jvm-spring
varun.patil
2021-06-23 22:46
@PactBroker(host = "${pactbroker.url:http://telus.pactflow.io}",
authentication = @PactBrokerAuth( password = "", username = "zFaJyhilT-Ht_ATj8Jwy5Q"))
matt.fellows
2021-06-23 22:51
it?s token based auth, not user password: https://github.com/pactflow/example-provider-springboot/blob/master/src/test/java/com/example/springboot/ProductsPactTest.java#L21
varun.patil
2021-06-24 20:20
Hi Matt .. below is my class now ..
```package com.telus.starter.springboot.service;
import org.junit.runner.*;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;
import com.telus.accountbridge.AccountBridgeApplication;
import au.com.dius.pact.provider.junit.Provider;
import au.com.dius.pact.provider.junit.State;
import au.com.dius.pact.provider.junit.loader.PactBroker;
import au.com.dius.pact.provider.junit.loader.PactBrokerAuth;
import au.com.dius.pact.provider.junit.target.Target;
import au.com.dius.pact.provider.junit.target.TestTarget;
import au.com.dius.pact.provider.spring.SpringRestPactRunner;
import au.com.dius.pact.provider.spring.target.SpringBootHttpTarget;
@RunWith(SpringRestPactRunner.class)
@Provider("account-service")
@PactBroker(host = "http://telus.pact.io",
scheme = "https",
authentication = @PactBrokerAuth(token = "zFaJyhilT-Ht_ATj8Jwy5Q"))
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT,
classes={AccountBridgeApplication.class})
@ActiveProfiles("dv")
public class AccountSeviceProviderContractTest {
@TestTarget
public final Target target = new SpringBootHttpTarget();
@State("Ban 3232 exists")
public void accountExists() {
// nothing to do, real service is used
}
}```
varun.patil
2021-06-24 20:21
```org.junit.runners.model.InitializationError
at au.com.dius.pact.provider.junit.PactRunner.initialize(PactRunner.kt:93)
at au.com.dius.pact.provider.junit.PactRunner.getChildren(PactRunner.kt:140)
at org.junit.runners.ParentRunner.getFilteredChildren(ParentRunner.java:426)
at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:351)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createUnfilteredTest(JUnit4TestLoader.java:91)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createTest(JUnit4TestLoader.java:76)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.loadTests(JUnit4TestLoader.java:49)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(```
matt.fellows
2021-06-25 02:12
Sorry I don?t have enough information here to resolve it Varun. Perhaps you could attach a full debug trace as an txt file, as well as a reproducable code repo? (Need the gradle/maven deps and all) and the commands you?re using to repro it. Ideally, a GH issue
matt.fellows
2021-06-25 02:17
Also I have just regenerated your API token. You just pasted credentials to your account in a public forum which could lead to your account being compromised
c.pavan1986
2021-06-25 14:27
Trying to invoke the healthcheck apis is returning no response.
https://host/diagnostic/status/heartbeat
https://host/diagnostic/status/dependencies
Tried with http as well. We are using pactflow 1.8.0 onprem
varun.patil
2021-06-25 23:25
Publishing 'pq-app-account-service.json' ... FAILED! 401 Unauthorized - Unauthorized. Please see https://docs.pactflow.io/docs/login-help for more information.
matt.fellows
2021-06-25 23:39
What does the log say with verbose on? Positive you're using the read write token?
varun.patil
2021-06-26 00:24
Hi Matt, when i run a contract test to verify the pacts ..
`@RunWith(SpringRestPactRunner.class)`
@Provider("account-service")
@PactBroker(host = "http://myorg.pactflow.io",
scheme = "https",
authentication = @PactBrokerAuth(token = "testtesttest"))
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
classes={AccountBridgeApplication.class})
@ActiveProfiles("dv")
public class AccountSeviceProviderContractTest {
@TestTarget
public final Target target = new SpringBootHttpTarget();
@State("Ban 3232 exists")
public void accountExists() {
// nothing to do, real service is used
}
}
varun.patil
2021-06-26 00:24
`java.lang.NoSuchMethodError: com.google.gson.JsonParser.parseString(Ljava/lang/String;)Lcom/google/gson/JsonElement;`
at http://au.com.dius.pact.core.pactbroker.HalClient.handleHalResponse(HalClient.kt:320)
at http://au.com.dius.pact.core.pactbroker.HalClient.getJson(HalClient.kt:312)
at http://au.com.dius.pact.core.pactbroker.HalClient.fetch(HalClient.kt:273)
at http://au.com.dius.pact.core.pactbroker.HalClient.fetch(HalClient.kt:268)
at http://au.com.dius.pact.core.pactbroker.HalClient.navigate(HalClient.kt:261)
at http://au.com.dius.pact.core.pactbroker.PactBrokerClient.fetchConsumers(PactBrokerClient.kt:94)
at http://au.com.dius.pact.provider.junit.loader.PactBrokerLoader.loadPactsForProvider(PactBrokerLoader.kt:140)
at http://au.com.dius.pact.provider.junit.loader.PactBrokerLoader.load(PactBrokerLoader.kt:92)
at http://au.com.dius.pact.provider.junit.PactRunner.initialize(PactRunner.kt:85)
at http://au.com.dius.pact.provider.junit.PactRunner.getChildren(PactRunner.kt:140)
at org.junit.runners.ParentRunner.getFilteredChildren(ParentRunner.java:426)
at org.junit.runners.ParentRunner.getDescription(ParentRunner.java:351)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createUnfilteredTest(JUnit4TestLoader.java:91)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createTest(JUnit4TestLoader.java:76)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.loadTests(JUnit4TestLoader.java:49)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:526)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:770)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:464)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
matt.fellows
2021-06-26 00:29
The dependencies endpoint is protected by authorization (so it will need a bearer token)
matt.fellows
2021-06-26 00:29
I wouldn?t recommend using that endpoint for HA checks, just the heartbeat
dimundo
2021-06-29 11:08
hello, is this valid one ?
``` % ping http://somename.pactflow.io
PING http://prod.pactflow.io (54.79.255.137): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3```
dimundo
2021-06-29 11:09
and time to time i?m getting
```Call to fetch pacts from Pact Broker failed with an exception
java.lang.RuntimeException: Call to fetch pacts from Pact Broker failed with an exception
at au.com.dius.pact.provider.ProviderInfo.hasPactsFromPactBrokerWithSelectors(ProviderInfo.kt:95)
at pact4s.ProviderInfoBuilder.applyBrokerSourceToProvider(ProviderInfoBuilder.scala:93)
at pact4s.ProviderInfoBuilder.toProviderInfo(ProviderInfoBuilder.scala:65)
at pact4s.PactVerifyResources.providerInfo(PactVerifyResources.scala:27)
at pact4s.PactVerifyResources.providerInfo$(PactVerifyResources.scala:27)```
but this maybe for @kyle.florence
matt.fellows
2021-06-29 11:37
For no other reason than we don?t use it at Pactflow, and by default we only open network protocols/ports etc. for things we need
dimundo
2021-07-01 18:01
hi!
i have pact between `A` and `B`
also
i have pact between `B` and `A`
Pact between `B` and `A` is not verified yet, but between `A` and `B` - many times
If i open Matrix in pactflow for `B` and `A` , i see validations between `A` and `B` only
any ideas ? bug?
bethskurrie
2021-07-02 07:48
Yes, you have A-B and B-A pacts. You are seeing the verifications for the direction that has verifications and not seeing the verifications for the direction that does not have any.
bethskurrie
2021-07-02 07:50
you are seeing the rows for where one app has the role of consumer and the other has the role of provider
bethskurrie
2021-07-02 07:51
you are not seeing the rows (because they do not exist) for where they are in the reverse roles.
dimundo
2021-07-02 07:51
but person, selecting direction in sidebar expects only that direction only ( as i could expect )
bethskurrie
2021-07-02 07:53
We actually have an entirely new application version focussed UI designed, but not yet built.
bethskurrie
2021-07-02 07:54
At the moment, the Pactflow UI has the contract as the top level item, but really, it should be the application.
zhaoyi0113
2021-07-05 12:26
Hi There, a question about how to test API for mutation via pact. I have a rest API which is to create a customer. But the customer email is required and must be unique. If I create a `create customer` pact and publish it to the broker, the provider side can only run the test once. It will fail the second time because the same email already exists. How can I make a contract test for this mutation?
2021-07-05 12:28
Here are a number of useful hands-on labs that teach all of the key concepts https://docs.pact.io/implementation_guides/workshops
zhaoyi0113
2021-07-05 22:10
@matt.fellows thanks for your response.
I think the example works because you can launch the application in your local. And use the provider url: `providerBaseUrl: "http://localhost:8080"`. In that way, you can set up all the mock data easily.
But my application is deployed to AWS apigateway and a few lambdas behind it, I can?t launch it in a local environment.
What is the best practise to verify pact against AWS apigateway?
zhaoyi0113
2021-07-05 22:36
how do you handle the deployment on AWS and run pact verify? Do you deploy first then run pact verify?
matt.fellows
2021-07-05 22:37
well, because you?re deploying to an environment thins are a bit different. so you _can_ deploy, but it has to be a test environment obviously
zhaoyi0113
2021-07-05 22:38
ok, basically I need to deploy a temporary version of my backend, then verify, then deploy to the real environment
zhaoyi0113
2021-07-05 22:41
actually I have mixed environment, some projects are using serverless while some are using terraform
matt.fellows
2021-07-05 22:42
Sure, but a lambda is still a lambda, and should be able to be run like a unit test.
matt.fellows
2021-07-05 22:42
I would highly recommend you have an approach that enables unit testing of it if you can
zhaoyi0113
2021-07-05 22:44
what is the pacts I should publish from consumer side? The pacts should be rest based format right? like have `path`, `body`, etc. How can I use `pact` to verify something not match to what consumer published?
zhaoyi0113
2021-07-05 22:50
sure, the consumer is rest based client. So the pacts is in rest format. how can I verify the provider bypass api gateway?
matt.fellows
2021-07-05 22:52
right, so you?re saying because a lambda is not an HTTP endpoint, how to test it?
matt.fellows
2021-07-05 22:52
One option is to run a basic HTTP server in test to pass through the requests
zhaoyi0113
2021-07-05 22:55
alright thanks. I think I have to deploy it to a temporary environment for pact testing. After pass pact, it can be deployed to real environment.
Put a mock http endpoint before lambda doesn?t test the resource mapping in api gateway.
matt.fellows
2021-07-05 22:58
It?s up to you. I don?t know about terraform tools for running gateway locally, but if they exist i?d want to know about it. Having to deploy is OK, but slows down feedback cycles (ideally you want to know prior to commit).
zhaoyi0113
2021-07-05 22:59
sure, what I am thinking is to deploy the tempoaray environment when a PR is open, then run pact test on the PR environment, if it passes, it can be merged to master branch, then tear down the PR environment.
steve.etherington
2021-07-08 15:01
Hi
I am new to pactflow. I am working on a POC for OAS bi-directional contracts. We have some Dredd tests already so I have been trying to run this
https://docs.pactflow.io/docs/examples/bi-directional/provider/dredd/readme
steve.etherington
2021-07-08 15:01
I can successfully run `make fake_ci` but running `make ci` fails, with a 500 error from pactflow.
(I had to fake the two TRAVIS env vars)
Is this project still expected to be working? I have just set up a new free pactflow account for this rather than pollute my company account. Was there any set up I needed to run before expecting this to work?
```error publishing contract + results Error: Request failed with status code 500
at createError (/Users/steve/depop/example-provider-dredd/node_modules/axios/lib/core/createError.js:16:15)
at settle (/Users/steve/depop/example-provider-dredd/node_modules/axios/lib/core/settle.js:17:12)
at IncomingMessage.handleStreamEnd (/Users/steve/depop/example-provider-dredd/node_modules/axios/lib/adapters/http.js:260:11)
at IncomingMessage.emit (node:events:377:35)
at endReadableNT (node:internal/streams/readable:1312:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
config: {
url: 'https://setherdredd.pactflow.io/contracts/provider/pactflow-example-provider-dredd/version/91a4000126ca47947e01191824efb16e8b9bc7ac',
method: 'put',```
steve.etherington
2021-07-08 15:43
I have solved the 500 error. But does this example show the latest/best way to to do OAS testing?
matt.fellows
2021-07-08 22:36
The docs are also here: https://docs.pactflow.io/docs/workshops/bi-directional
sams.prashanth
2021-07-11 10:40
Is there any time limit for the pactflow auth token expiry? or no expiry date?
bethskurrie
2021-07-11 10:41
matt.fellows
2021-07-11 10:41
You can configure the expiration at the account level via system preferences.
sams.prashanth
2021-07-12 06:25
Crazy question: is it possible to self-host pactflow system for the paid enterprise version?
sams.prashanth
2021-07-12 06:28
Just asked as we need to open our internal network to trigger jobs from the pactflow using webhooks
matt.fellows
2021-07-12 06:29
but also thinking about https://pact.canny.io/feature-requests/p/support-a-polling-endpoint-for-a-webhookless-workflow
matt.fellows
2021-07-12 06:51
I?m hoping it could mean that customers that currently have those kinds of restrictions could work around it this way
jayr.motta
2021-07-12 14:01
Hi there :wave:
I have a service that requires a state to be setup before I can observe if the behavior is working as expected, how can I verify a contract if the simulated consumer will only replay a simple request but not setup such data?
steve.etherington
2021-07-12 16:06
Hi
I have been trying to set up an OAS provider test using one of our existing consumer tests. Is this a ?normal? verification failure, or is it an error within the verification process
```{"error":{"message":"Error executing pact/OAS verification. Error reference sBldFsWvJA."}}```
steve.etherington
2021-07-12 16:08
```$> ${PACT_CLI} broker can-i-deploy --pacticipant mobile-api-sellerhub --version c06b16d --to dev
Computer says no ¯_(?)_/¯
CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
---------|------------|----------------------|-----------|----------|--------
iOS-app | cc1d3c9... | mobile-api-sellerhub | c06b16d | false | 1
VERIFICATION RESULTS
--------------------
1. https://depop.pactflow.io/hal-browser/browser.html#https://depop.pactflow.io/contracts/provider/mobile-api-sellerhub/version/c06b16d/consumer/iOS-app/pact-version/b8e788b9bde35c9a9bbd1500db1863628391623e/verification-results (failure)
The cross contract verification between the pact for the latest version of iOS-app with tag dev (cc1d3c9c7d9599b68b86782aa6830663667c9440) and the oas for version c06b16d of mobile-api-sellerhub failed```
steve.etherington
2021-07-12 16:08
```$> curl -H 'Authorization: Bearer <secret>' https://depop.pactflow.io/contracts/provider/mobile-api-sellerhub/version/c06b16d/consumer/iOS-app/pact-version/b8e788b9bde35c9a9bbd1500db1863628391623e/verification-results
{"summary":{"success":false},"crossContractVerificationResults":{"success":false,"results":{"error":{"message":"Error executing pact/OAS verification. Error reference sBldFsWvJA."}},"verificationDate":"2021-07-12T15:10:25.418+00:00","verifier":"atlassian-swagger-mock-validator","verifierVersion":"10.0.0"},"providerContractVerificationResults":{"success":true,"content":"VE9ETwo=","contentType":"text/plain","verifier":"verifier"},"_links":{"self":{"title":"Cross contract and provider contract verification results","href":"https://depop.pactflow.io/contracts/provider/mobile-api-sellerhub/version/c06b16d/consumer/iOS-app/pact-version/b8e788b9bde35c9a9bbd1500db1863628391623e/verification-results"}}}```
steve.etherington
2021-07-12 16:09
? the report content published in the provider contract is made up ?
matt.fellows
2021-07-12 22:31
You could definitely use injected provider states, but just standard provider states should be fine in most cases
matt.fellows
2021-07-12 22:31
e.g. with JS, you can do this: https://github.com/pact-foundation/pact-js/#api-with-provider-states
2021-07-12 22:32
Here are a number of useful hands-on labs that teach all of the key concepts https://docs.pact.io/implementation_guides/workshops
bethskurrie
2021-07-12 23:53
@steve.etherington I'm looking into the error, but I can't see what has caused it. There was a runtime error when comparing the oas and the pact, and that error result has now been saved. Can you make a change to either the pact or the oas and try again please?
bethskurrie
2021-07-13 03:11
@steve.etherington there are actually two issues. one is a bug, and one is that the verifier only works for oas 3.0, not swagger 2.0 files.
bethskurrie
2021-07-13 03:26
No, I'm wrong, sorry.
> A command line tool that confirms mock request and responses conform to the schema specified in a Swagger 2 or OpenApi 3 specification.
matt.fellows
2021-07-13 04:01
Steve - so there is definitely a bug on our side we need to fix, as Beth has alluded to. We also have a backlog item to fix some validation response that would have made it easier to diagnose your problem
matt.fellows
2021-07-13 04:02
There were also a few problems with your spec:
1. Duplicate oauth security configs
2. Missing absolute links in the security definitions of authorizationUrl and tokenUrl
3. use of `anyOf` in a swagger 2.0 format (only available in OAS 3)
I?d recommend using tools like https://editor.swagger.io/ if you?re authoring the file by hand, to ensure it?s valid first.
steve.etherington
2021-07-13 08:22
Thanks for this @bethskurrieand @matt.fellows. I was generating the swagger yml from an Apiary API blueprint file using LucyBot api-spec-converter. Our apis are currently documented with apib files and validated with Dredd. I was hoping to fit pact into that workflow. You have given me a few pointers to try to find workarounds or solutions.
sorin_balbae
2021-07-13 08:50
hi, new guy here with a question :-)
I ran locally a set of consumer tests that generated and successfully published a contract to pactflow.
the contract is not visible under the "overview" section of the consumer-provider interaction, but I could find it via direct link and I also see it in the "matrix" (it was also verified by the provider).
running the same way (with the same version and tag) worked a week ago and the contract was visible under "overview"; other colleagues running the exact same code today get the contracts published and visible.
since my last successful publication, someone else cleaned up old interactions and contracts using the same version and tag.
browser cache cleanup did not change the situation.
anyone experienced something similar? any suggestions or explanations?
thanks!
matt.fellows
2021-07-13 08:54
Can you show a screen of what you're expecting to see Sorin? Is it a new integration, or just a new version of a consumer?
dimundo
2021-07-13 08:56
then it could be something like https://pact-foundation.slack.com/archives/CLS16AVEE/p1625162501118900
sorin_balbae
2021-07-13 09:11
the screenshot won't help, as we have a lot of interactions listed from different branches and local runs
the initial contract was published under sorinTestTag/sorinTestVersion
I had several versions of it, verified by the provider
then a cleanup was performed - interactions deleted, contract deleted (not 100% sure all verification results were deleted before the contract)
now I published the same contract under the same tag and version, but I cannot see it in the overview tab
I can see it in the matrix and I can see the webhook was triggered, though
*UPDATE:* I changed the tag and version to sorin_Tag/sorin_Version and the contract is also visible in the "overview"
james_fraser
2021-07-13 11:39
Hey there :wave: does anyone know if Pactflow has a status page or anything to notify of potential outages or problems?
james_fraser
2021-07-13 11:55
No worries - all good :smile: I?m just adding checks for external tooling at work so we can catch any issues
steve.etherington
2021-07-13 12:53
I tried using apib2swagger for the swagger generation. It also creates swagger v2 with an `anyOf` field. The resulting error message was
```{
"error": {
"message": "Error executing pact/OAS verification. Error reference niIkecxGoe."
}
}```
james_fraser
2021-07-13 12:56
Nothing obvious for RSS that I can see @paulorochag, but it does have a Slack integration
steve.etherington
2021-07-13 13:43
I also tried on another service, with swagger that does not include anyOf. This also fails with
``` "error": {
"message": "Error executing pact/OAS verification. Error reference JvqTfXjoPD."
}```
matt.fellows
2021-07-13 13:49
Thanks, if you could please drop a support ticket to with the OAS file we can look tomorrow. There is another issue which needs to be fixed and released which is causing the error reference (we've identified and have created a patch that needs to be verified)
steve.etherington
2021-07-13 15:29
No I haven?t checked them in the swagger editor - I need to be able to auto-generate them from our existing apib files. If I can?t do that we will just go with scalapact (or pact-jvm) instead.
steve.etherington
2021-07-13 15:41
? all 3 files have errors reported by the swagger editor :disappointed: I wonder if there is some other apib to swagger converter I can try :thinking_face:
matt.fellows
2021-07-14 01:29
Yeah, I can?t speak for the generator you?re using, but you may need to do some post-processing (scriptable) to fix any problems.
sorin_balbae
2021-07-14 06:46
the pact is in only one direction.
seems related to how pactflow keeps and re-uses past verifications.
steve.etherington
2021-07-14 09:15
As mentioned in the support ticket, https://www.apimatic.io/ generates valid swagger v2 for my apib files. (It?s not free, though.)
bethskurrie
2021-07-15 00:34
@sorin_balbae I'm not sure what would be causing this issue, but please do let us know if you see it again, so we can do some debugging.
abatan.k
2021-07-15 03:52
hello, i just register for free account on Pact flow, but i cannot access to my space, i did receive an email which told me that i could login with my github credentials, but when i try it doesn't work.
I tried to reset my password, but i never get the mail with the code in order to set a new password...
Do you have any idea how i can access my space ??
sorin_balbae
2021-07-15 14:37
At this point I gave up the old tag and using a new one (no issues here). But if I experience this again, I'll let you know.
efemgy
2021-07-15 14:53
Hi, I'd like to know if I can create a webhook to be only triggered when a contract content changes and not when the tag does. I only see the option for both. Thanks
bethskurrie
2021-07-15 21:52
@efemgy no, but this is coming soon https://github.com/pact-foundation/pact_broker/discussions/434
steve.etherington
2021-07-16 11:03
I have been able to verify an OAS provider against a consumer, but have not been able to get the verification results published to http://pactflow.io UI. Can anyone help?
matt.fellows
2021-07-16 11:04
(we?re still in developer preview, the roadmap can be seen here: https://docs.pactflow.io/docs/workshops/bi-directional/#high-level-roadmap)
matt.fellows
2021-07-16 11:05
We would have liked to progress it a bit faster, however it?s taken a little longer to hire than we?d hoped. We have a new starter on Monday and are close to signing on another - stay tuned!
2021-07-19 03:56
Notification from *Pactflow Contract Testing Platform* :ghost:
c.pavan1986
2021-07-19 11:16
The database connections in pactflow are active always. Say there is no activity on pactflow for more than 24hrs, would the database connections created when the container comingup seem to remain active. Or would they get closed?
bethskurrie
2021-07-19 22:24
@c.pavan1986 I don't know the answer to that, but you'll probably find it in the Sequel documentation http://sequel.jeremyevans.net/
bethskurrie
2021-07-19 22:27
The underlying database specific library is this one https://deveiate.org/code/pg/
matt.fellows
2021-07-21 06:31
:wave: Hi folks, one final round for our feedback request *- I?d like to know how we can improve Pactflow* (and by extension, also Pact)*.*
_Can you spare a few minutes to tell me how you use Pactflow, what you like and how we can improve?_
_Your feedback is important to the team and me to help improve aspects of our product, such as:_
? _New features - for Pactflow and for the OSS Pact Broker_
? _Support and documentation_
? _The new user onboarding experience_
_The survey will take between five and seven minutes time to complete._
_As a thank you, we?re pleased to *offer you an upgrade* to your current Pactflow plan. See all the details via the link_ https://dius.typeform.com/to/XSlrZYAB
matt.fellows
2021-07-23 09:05
I don't think so, we only talk about Pact here, in fact, the topics have exclusively come from questions that come from those forum
tjones
2021-07-23 09:08
I think the document is written as if it's in the pactflow docs.
> Pactflow runs regular monthly AMA's
then
> You can watch all of our previouss AMA's
(also I just noticed there's a doubled s there)
matt.fellows
2021-07-23 09:08
Ideally we have maintainers join us (hence your invite!) But we've been wary of people's time
tjones
2021-07-23 09:10
I think there are several options.
1) We could correct the document so it's written from a Pact perspective
2) We could move the document as-is to Pactflow (and link it here - "Pactflow runs regular monthly AMAs, read about them at <link>")
3) We could make them the "Pact AMA hosted by Pactflow"
4+) Other options that I haven't thought of
I don't really have an opinion on which is better, but I think in the current state the document doesn't feel like it fits in the pact docs.
matt.fellows
2021-07-23 09:11
It's an example of where I wanted to make it clear that Pactflow ran it, but it's currently completely OSS focused
tjones
2021-07-23 09:13
For option 3, what about swapping:
"Pactflow runs regular monthly AMA's"
for
"We have regular monthly AMAs, currently hosted by Pactflow"?
matt.fellows
2021-07-23 09:13
I think having agreed guidance on those language will help, but would love a shared session where we can collaboratively work out a longer term solution
tjones
2021-07-23 09:13
The meaning is the same for people who already know there's a difference, and it's less likely to mislead those who don't.
tjones
2021-07-23 09:20
> So only Pactflow customers should know about it?
I 100% think that everyone should know about the AMAs, and I think they're a great service to the community :blue_heart:. I'm just defending the OSS docs from accidental "we" as pactflow.
matt.fellows
2021-07-23 09:28
Will look later tonight if I'm back online (minor cold but energy levels low)
tjones
2021-07-23 09:57
> energy levels low
Tell me about it. I?ve been struggling all week. So many things I meant to do.
tjones
2021-07-25 02:39
@kamoljan spotted this repeated `are` in the pactflow docs:
https://pact-foundation.slack.com/archives/CAN147DFD/p1627178501067100
c.pavan1986
2021-07-27 08:14
The person who logs into pactflow is becoming an admin user is there a way to change this, like make someother user administrator and make the first loggedin user to nonadmin user
matt.fellows
2021-07-27 08:34
whoever the first user is can make a new admin, and the new admin can remove the first users admin privilege
c.pavan1986
2021-07-27 08:50
From the settings > users. I am not able to see a checkbox for making first login user non admin user
c.pavan1986
2021-07-27 12:25
The users logging in after the first user are by default getting assigned test maintainer role. Is there a way by default assign Viewer role.
matt.fellows
2021-07-27 12:38
In the next version you?ll be able to configure this via the UI/API (it?s just been added to our hosted version)
c.pavan1986
2021-07-28 09:16
PACTFLOW_MASTER_ENCRYPTION_KEY/PACTFLOW_MASTER_SECRETS_ENCRYPTION_KEY, PACTFLOW_API_TOKEN_ENCRYPTION_ENABLED, PACTFLOW_API_TOKEN_IV. Looks like configuring these env variables will encypt the saas_api_tokens. We are currently using 1.8.0, will be moving to 1.11.0. Just want to know what is the encryption algorithm used and we can still get the decrypted token from settings>apitokens>copy token right?
c.pavan1986
2021-07-28 09:17
Can you also please share any documentation related to PACTFLOW_ADMIN_API_KEY
bethskurrie
2021-07-28 23:58
@c.pavan1986 it uses aes-256-cbc encryption. There are some technical reasons why that particular algorithm is used (it has to be reversible so you can get the token back, amongst other reasons).
bethskurrie
2021-07-28 23:58
> we can still get the decrypted token from settings>apitokens>copy token right?
Yes.
bethskurrie
2021-07-28 23:59
You shouldn't need to use the PACTFLOW_ADMIN_API_KEY Pavan, as you are using federated authentication. That's only required for setting up the initial admin user.
sams.prashanth
2021-07-29 09:46
Is there any documentation for handling PactMismatchesException?
```au.com.dius.pact.consumer.PactMismatchesException:
The following requests were not received:
method: POST
path: /v1/process
query: {}
headers: {xyz-xyz-Id=[xyz-xyz-xyz-xyz-xyz], xyz-Request-Id=[xyz], xyz-Accept-Version=[v1]}
matchers: MatchingRules(rules={})
generators: Generators(categories={})
body: PRESENT([])```
matt.fellows
2021-07-29 09:51
Each language is different, but reading that it seems your test didn't make the call to `/v1/process`
matt.fellows
2021-07-29 09:52
Logs might show if any requests were made, usually if a request is made you also see an unexpected event for things it wasn't expecting
todd.lemmonds
2021-07-29 16:24
Hello Everyone...in all the examples we've seen, contract testing has mainly been with JSON message payloads. Does Pactflow support XML based message payloads? @eric.tang1
todd.lemmonds
2021-08-02 16:34
@matt.fellows :point_up:We are looking for this in the JS implementation
matt.fellows
2021-08-02 22:22
todd.lemmonds
2021-08-03 05:13
@matt.fellows curious what it would take to build support for that. Is it on the roadmap at all?
matt.fellows
2021-08-04 01:18
Probably not a huge amount of effort. We haven?t added message support into the v3 JS yet, but it?s not too far away. And adding XML support _should_ be fairly straightforward, because all of the matching rule infrastructure is already there
matt.fellows
2021-08-04 01:18
could you please raise a feature request at https://github.com/pact-foundation/pact-js/ so we can track?
c.pavan1986
2021-08-04 11:10
Nothing like issues, the database password was changed, but pactflow was still working without any issues. I am assuming there is a pool.of database connections created, and the same connections are used to interact with database. Not like if the application is idle (no activity) the connection will be still open, i was just thinking that the connections would get closed after they are aged..
rejeesh.g
2021-08-04 13:58
Hi Matt, we recently migrated to Pactflow enterprise version "1.11.1". I want know to if there is way to set "Select Team:" to "default" team instead of "none" option as the default option when a user logs in? Additionally "none" option should appear only for Administrator Role.
matt.fellows
2021-08-05 13:15
Hi Rajeesh, The default role and team preference setting is not available via the Pactflow on-premises UI just yet
matt.fellows
2021-08-05 13:15
I believe it can be done directly in the database, if you can drop us a support ticket we can advise you ()
subhashnarla
2021-08-11 07:30
I have a maven project that is working well with provider, consumer and pact folder
bethskurrie
2021-08-11 07:33
Hi @subhashnarla Please see https://docs.pact.io/help/how_to_ask_for_help#4-ask-a-good-question
subhashnarla
2021-08-11 07:35
Context: I have contract testing requirement and i have created the project (placing contact in pact folder) .however I want to integrate to a pactbroker like http://pact.io ....i have the account but unable to integrate my code to it...there are no java/maven examples that i could refer to...Can you point me to any examples (not teh curl commands but ways to publish/verify contract via http://pactflow.io using a springboot project)
bethskurrie
2021-08-11 07:37
I don't use maven, but I went to http://docs.pact.io and I typed in "maven publish"
subhashnarla
2021-08-11 07:44
but it also says i add config in pom followed by You can now execute `mvn pact:publish` to publish the pact files.
subhashnarla
2021-08-11 07:45
can i skip the mvn command and do something in java code that automatically publishes contract
bethskurrie
2021-08-11 07:45
You should only ever publish from CI, so it should nto happen automatically.
bethskurrie
2021-08-11 07:46
Best to ask in the #pact-jvm channel, because it's not Pactflow you're having problems with, it's the Java Pact library
srinagasai.krishnasan
2021-08-13 14:32
Anyone here tried running a server through `gretty` using `startProviderTask`. I am having problems as the gretty server once it starts running, it won't be considered as 100% executed task. I need this gretty task to be called as 100% so I can verify Pact.
Lifecycle should be : gretty starts the war -> pact verification -> gretty stops the war
What's actually happening is : gretty starts the war (server isup and running all good) but the task never executes 100%
kamoljan
2021-08-15 04:06
I am having `PactBroker::Client::Error - Potential duplicate pacticipants:` error, even I removed all integrations from Pactflow console.
kamoljan
2021-08-16 02:18
It will be very useful at least in early stage of my PoC! :slightly_smiling_face:
Beth, thank you!
marcin.baranowski953
2021-08-17 07:26
Hi, I'm trying to run `scripts/publish.sh` from this example project: https://github.com/pactflow/example-provider-restassured
I've provided all needed data for my pact broker and I get output like this (using oas file provided in example):
```==> Uploading OAS to Pactflow
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
</body>
</html>
curl: (6) Could not resolve host: UGFjdGZsb3cgUHJvZHVjdCBBUEkgZGVtbwogIHZlcnNpb246IDEuMC4wCnBhdGhzOgogIC9wcm9k
curl: (6) Could not resolve host: dWN0czoKICAgIHBvc3Q6CiAgICAgIHN1bW1hcnk6IENyZWF0ZSBhIHByb2R1Y3QKICAgICAgZGVz
... lines ommited
curl: (6) Could not resolve host: ZXJzaW9uOgogICAgICAgICAgdHlwZTogc3RyaW5nCiAgICAgICAgcHJpY2U6CiAgICAgICAgICB0
curl: (3) unmatched close brace/bracket in URL position 254:
eXBlOiBudW1iZXIK",
"contractType": "oas",
"contentType": "application/yaml",
"verificationResults": {
"success": false,
"content": "dGVzdGVkIHZpYSBSZXN0QXNzdXJlZAo=",
"contentType": "text/plain",
"verifier": "verifier"
}
}```
What can I do to publish oas and make pactflow verify it against client pact file?
marcin.baranowski953
2021-08-17 08:31
All env vars are set correctly, but thanks to your suggest with `set -x` I've seen something:
``` "content": "b3BlbmFwaTogMy4wLjEKaW5mbzoKICB0aXRsZTogUHJvZHVjdCBBUEkKICBkZXNjcmlwdGlvbjog' UGFjdGZsb3cgUHJvZHVjdCBBUEkgZGVtbwogIHZlcnNpb246IDEuMC4wCnBhdGhzOgogIC9wcm9k dWN0czoKICAgIHBvc3Q6CiAgICAgIHN1bW1hcnk6IENyZWF0ZSBhIHByb2R1Y3QKICAgICAgZGVz Y3JpcHRpb246IENyZWF0ZXMgYSBuZXcgcHJvZHVjdAogICAgICBvcGVyYXRpb25JZDogY3JlYXRl ...```
The problem is with base64 content which has `'` after `b3BlbmFwaTogMy4wLjEKaW5mbzoKICB0aXRsZTogUHJvZHVjdCBBUEkKICBkZXNjcmlwdGlvbjog` and then all the rest of "content" is divided by spaces.
marcin.baranowski953
2021-08-17 09:43
The source of the problem was `base64` command which divided output into separate lines. I added `-w 0` option to `base64` command and it moved on
marcin.baranowski953
2021-08-17 10:12
What do You think about adding this option in https://github.com/pactflow/example-provider-restassured repo?
marcin.baranowski953
2021-08-17 10:55
Why not? :slightly_smiling_face: Right now I'm busy, but as soon as I find some time, I will provide PR :wink:
matt.fellows
2021-08-17 11:08
Ah, that?s a linux flag. I?m guessing you?re not on OSX, or are using a gnu based base64 command (so the script might need updating to be portable)
matt.fellows
2021-08-17 11:09
It?s possible I didn?t properly CI that repo though in a certain haste for a demo
marcin.baranowski953
2021-08-17 12:27
Another question - after running `can-i-deploy` I get a failure. What does error message - `No schema found for response body` mean in this context?
```{
"summary": {
"success": false
},
"crossContractVerificationResults": {
"success": false,
"results": {
"errors": [
{
"code": "response.body.unknown",
"message": "No schema found for response body",
"mockDetails": {
"interactionDescription": "presign request",
"interactionState": "[none]",
"location": "[root].interactions[0].response.body",
"mockFile": "pact",
"value": {
"url": "url"
}
},
"source": "spec-mock-validation",
"specDetails": {
"location": "[root].paths./api/presign/uploadDocument/documentName.get.responses.200",
"pathMethod": "get",
"pathName": "/api/presign/uploadDocument/documentName",
"specFile": "oas",
"value": {
"content": {
"*/*": {
"schema": {
"example": {
"url": "url"
},
"properties": {
"url": {
"type": "string"
}
},
"title": "GetPresignedUrlResultDTO",
"type": "object"
}
}
},
"description": "OK"
}
},
"type": "error"
}
],
"failureReason": "Mock file \"pact\" is not compatible with spec file \"oas\"",
"warnings": []
},
"verificationDate": "2021-08-17T12:17:29.735+00:00",
"verifier": "atlassian-swagger-mock-validator",
"verifierVersion": "10.0.0"
},
... omitted```
In oas yml file I provide `GetPresignedUrlResultDTO` definition
c.pavan1986
2021-08-20 07:08
If uuid is generated field in all the tables, is there possibility that same uuid can occur in 2 different tables? Is there any specific pattern being used to generate it?
matt.fellows
2021-08-20 07:09
Unlikely, but theoretically possible: https://www.wikiwand.com/en/Universally_unique_identifier (see collisions)
matt.fellows
2021-08-20 07:13
> If uuid is generated field in all the tables, is there possibility that same uuid can occur in 2 different tables? Is there any specific pattern being used to generate it?
what are you trying to achieve?
dimundo
2021-08-20 07:15
> the probability of meeting a dinosaur on the street is 50%. either you will meet or not.
c.pavan1986
2021-08-20 07:20
We are trying to use some of the apis exposed (team, role creation as part of 1.11.0 onprem), I noticed uuid is part of almost all the apis (either request/response). So just want to understand how it work.
matt.fellows
2021-08-20 07:26
They are likely to be unique, but mathematically it?s possible they?re not
c.pavan1986
2021-08-20 07:28
If there is situation that 2 uuid match, is pactflow smart to regenerate a new uuid.
Say x user logs in abc1 is the user id generated,
y user logs in abc2 is the user id generated
So on till 10000 users.
If there occurs a situation user x1 (already loggedin user) and y1 (new user login) are being assigned same uuid abc11. Is pactflow intelligent to regenerate a new uuid for new user y1, since it collides with user x1?
dimundo
2021-08-20 07:37
so, is user id - uuid ? if you will spot collision twice - buy a lottery ticket
matt.fellows
2021-08-20 07:40
Pactflow will either generate a new one or fail, but it?s not possible to have a duplicate in the DB.
leonardo.viana
2021-08-20 14:54
Our Jenkins server blocks requests originating from unknown IPs. Therefore, in order to setup PactFlow webhooks to trigger builds in our Jenkins server, we likely need to do some whitelisting. Is there any doc describing the public IPs or CIDR range that we would need to enter to whitelist PactFlow?
kmckeever
2021-08-20 15:50
Hey there! We currently have our GitLab webhooks set up within Pactflow to kick off our provider verification upon creation of new consumer contracts in the consumer pipeline. Does anyone know if there is a way to have the consumer pipeline check for the provider verification GitLab job to complete, and have that be a dependency upon the consumer kicking off its pact-can-i-deploy job?
matt.fellows
2021-08-20 22:42
But, the can I deploy command does have a poll flag that can wait. We use that at Pactflow rather than couple build pipelines together
leonardo.viana
2021-08-20 22:56
https://pactflow.io/faq/#ip-address-whitelist has exactly what I was looking for, thanks!
kmckeever
2021-08-21 11:32
Somehow I missed that flag when first reading the documentation, but that would work perfectly for our purposes. Thanks, Matt!
sams.prashanth
2021-08-22 06:44
Is there any chances of adding the pacts count per pacticipant in pact-cli?
```broker describe-pacticipant --name xyz```
The above cmd brings the name along with the date created. Would be nice if we can somehow get how many pacts written per pacticipant.
btw, is there any tweak to get the pact test count at the moment?
matt.fellows
2021-08-22 08:26
I might drop you an email offline tomorrow to discuss this, I assume it's related to reporting needs?
matt.fellows
2021-08-22 08:48
Feel free to elaborate here and see if others have similar requirements?
sams.prashanth
2021-08-22 08:56
@matt.fellows We had a plan to show all pact related metrics before executing the contract tests in each services.
Like, whether the OpenApi specs are present or not - if so, how many api requests are available? - if present, is there any pacts written? - if so, how many pacts are written.
I find hard to fetch the number of pacts available specific to the provider from pactflow. I tried using the pact-broker-cli's describe-pacticipant cmd to fetch the details but the data that I get from that cmd is very minimal
matt.fellows
2021-08-22 11:02
The API endpoints I shared earlier are better sources for the number of pacts and integration data
sams.prashanth
2021-08-22 12:31
ah okay; @matt.fellows is there any docs for it? I need to hook it with the pactflow base_url, pacticipant, etc.,
sams.prashanth
2021-08-22 12:50
tried this
```curl -X GET https://****.http://pactflow.io/pacticipants/****-service --header 'Authorization: Bearer ****'```
maybe I am pointing to the wrong endpoints; need guidance on this
sams.prashanth
2021-08-22 12:56
when I hit the `/integrations` or `/metrics` endpoints, it is not fetching pact counts specific to the service name (pacticipant)
matt.fellows
2021-08-27 11:27
You would usually terminate TLS connections on a load balancer on your side (e.g. AWS ALB) and communicate via HTTP to the Pactflow instance
matt.fellows
2021-08-27 11:28
if this is not acceptable due to security, you could consider wrapping the Pactflow container in an reverse proxy (e.g. nginx) and using certificates there
sergio.amorim
2021-08-27 14:43
Hi! I need some information on your side in order the setup pactflow as a supplier in my company, can you help, please?
* Commercial Email
sergio.amorim
2021-08-27 15:04
Actually disregard this info, it is going to be send by email. Thank you
kamoljan
2021-08-28 12:31
Everything is green and provider verified, however, `can-i-deploy` is failing in provider:
```Computer says no ¯_(?)_/¯
There is no verified pact between the version of region-api-consumer currently deployed to production (6a6245dc7484b006e8e4400850e81966cc53dd10) and version fdddd64+1630153727 of region-api```
Am I doing something wrong?
matt.fellows
2021-08-29 03:01
It says the provider is verified against `main` but you're asking if the provider `6a624...` Is compatible with the production version of the consumer.
kamoljan
2021-08-29 06:20
`Deletion Integration` feature helped?
And thank you for pointing it out, I will try to play around with tags.
joaoproenca
2021-08-30 10:06
thanks @matt.fellows Please make sure to include me in the reply as well (Sergio CCd me) as I will be helping out in this process while Sergio is on vacations :slightly_smiling_face:
todor.m.kolev
2021-08-30 18:21
Hi, has anybody else had an issue verifying a pact from a Gitlab pipeline on the jvm? Do I need to make pactflow trust Gitlab somehow?
```Cause: au.com.dius.pact.core.pactbroker.InvalidNavigationRequest: Failed to fetch the root HAL document
?
Cause: org.apache.http.conn.HttpHostConnectException: Connect to http://xxx.pactflow.io:443 [http://xxx.pactflow.io/54.153.195.108, http://xxx.pactflow.io/54.79.255.137] failed: Connection timed out (Connection timed out)
```
Other than that I am able to curl pactflow from my pipeline.
todor.m.kolev
2021-08-30 22:50
There's a proxy yep, which has pactflow whitelisted. I am able to upload pacts but not verify them.
todor.m.kolev
2021-08-30 23:13
Just the broker URL and token. I've checked that these are present and correct and that if I copy them from my Gitlab pipeline to my local machine everything works fine. I was thinking that this might be because I have the proxy certificate added to my local java truststore via keytool.
todor.m.kolev
2021-08-31 09:33
I should first try configuring the client to send all requests via the proxy, shouldn't I! https://github.com/pact-foundation/pact-jvm/issues/1146
johnathan.gilday
2021-09-01 20:47
The CI/CD workshop instructs GitHub Actions users to create a Personal Access Token to share with Pactflow so that Pactflow can send authenticated web hooks to GitHub.
Personal Access Tokens (PAT) are discouraged in my organization, because they're tied to an individual. Has anyone found a workaround to using a PAT to authenticate the request from Pactflow to GitHub Actions? Is this something a Pactflow GitHub App could do? Does such a thing exist?
mike.geeves064
2021-09-01 21:24
Probably not the best solution, but in the past for similar cases I've just had a shared "deployment" user and permissioned accordingly, then used a token from them
mike.geeves064
2021-09-01 21:24
Interested to hear if there's a nicer way though :slightly_smiling_face:
matt.fellows
2021-09-02 00:08
Same, I recall that being recommended docs by Github at some point in time
2021-09-02 07:35
Notification from *Pactflow Contract Testing Platform* :ghost:
pavel.azarau
2021-09-02 17:34
Hi folks. We?re using Pactflow as a Broker. To run the contract tests on a Provider side we?re usig Gradle. Upgrading plugin from 4.1.11 to 4.2.10 brought an issue with connecting to Pactflow (to retrieve the pacts).
```java.lang.RuntimeException: Call to fetch pacts from Pact Broker failed with an exception
Caused by: au.com.dius.pact.core.pactbroker.RequestFailedException: Request to path '/' failed with response 'HTTP/1.1 401 Unauthorized'
at au.com.dius.pact.core.pactbroker.HalClient.handleHalResponse(HalClient.kt:310)
at au.com.dius.pact.core.pactbroker.HalClient.access$handleHalResponse(HalClient.kt:142)
at au.com.dius.pact.core.pactbroker.HalClient$getJson$1.invoke(HalClient.kt:293)
at au.com.dius.pact.core.support.KotlinLanguageSupportKt.handleWith(KotlinLanguageSupport.kt:38)
at au.com.dius.pact.core.pactbroker.HalClient.getJson(HalClient.kt:287)
at au.com.dius.pact.core.pactbroker.HalClient.fetch(HalClient.kt:251)
at au.com.dius.pact.core.pactbroker.HalClient.fetch(HalClient.kt:246)```
pavel.azarau
2021-09-02 17:43
the gradle configuration looks like
```pact {
broker {
pactBrokerUrl = 'https://<companydomain>.http://pactflow.io/'
pactBrokerToken = project.hasProperty("brokerToken") ? brokerToken : '<default read only token>'
}
.....```
Is there some kind of incompatibility between the latest Gradle plugin and pactflow? same config works as soon as I downgrade the pluging version back to 4.1.11
pavel.azarau
2021-09-02 17:56
and it seems like it happens when the task is triggered remotely with
```./gradlew --parallel clean composeUp pactVerify -PbrokerToken=<token which works with the previous plugin version> -Dpact.provider.tag=main -PpublishProviderVersion=<version> -Ppact.filter.pacturl=https://<companydomain>.http://pactflow.io/pacts/provider/<provider>/consumer/<cosumer>/pact-version/<pact-versio>/metadata/<some id> -PconsumerTag=<consumer tag>```
matt.fellows
2021-09-03 03:26
I think the github app is the best approach (from this perspective), because it uses OAuth tokens and not PAT. But that means the only way to ever integrate is to create an app, which is painful
todor.m.kolev
2021-09-03 12:16
Hi, pactflow only displays the latest two contract verifications for me (in the Overview tab). Is this a limitation on the free plan or there's another reason why the "LOAD MORE" button is inactive?
bethskurrie
2021-09-05 07:38
@todor.m.kolev try the matrix view if you want to see all the verifications.
gibraltor999
2021-09-07 12:26
Hi, Can someone provide information on 1. On prem hosting vs Pact managed hosting and 2. what do you mean by 5 contracts in the free version?
johnathan.gilday
2021-09-07 21:29
makes sense. My team also arrived at "create a service user and use its PAT"
bethskurrie
2021-09-08 07:20
@gibraltor999 the 5 contract limit means "5 integrations" eg. a->b, b->c, c->d, a->d, d -> e.
bethskurrie
2021-09-08 07:23
In regards to the on-prem application, the technical docs are here: https://docs.pactflow.io/docs/on-premises/
todor.m.kolev
2021-09-08 10:52
Hi, I am trying to to create a provider pacticipant in pactflow so I can then trigger the verification test when the consumer publishes a new pact. So I've executed `broker create-or-update-pacticipant` but then if I try to run `broker record-release` or `broker record-deployment` for a specific version of the provider I get an error - "Version does not exist". Is there a step I am missing?
matt.fellows
2021-09-08 11:01
The provide needs to verify a test to be able to deploy. The verification is what creates a version
matt.fellows
2021-09-08 11:03
The deploy step should be part of your CD pipeline, you wouldn't usually do this in advance. Having a provider (pacticipant/ in the system is enough to create the webhook
todor.m.kolev
2021-09-08 11:10
Hey, thanks! I was trying to avoid having to coordinate the releases of the consumers and the provider. I was thinking that by releasing the provider first and then informing pactflow what version of the provider I have in a specific environment, then each consumer can deploy independently as long as can-i-deploy passes for them.
todor.m.kolev
2021-09-08 11:11
What you're saying is to create the pacts first and then run the verification step in the provider, right? Then record the release of the provider.
todor.m.kolev
2021-09-08 11:21
What version of the provider should then publishing of a new consumer pact trigger? Currently, my idea was that when I new contract gets published that will (via a webhook) will trigger the provider pipeline which checks out the sha that's currently in the environment the consumer is trying to deploy to, who then runs the verification test. Does registering deployments and releases with the broker change anything at all here. What is it useful for, do I even need it or just using tags is sufficient ?
matt.fellows
2021-09-08 11:49
The reason I ask is if supported, you should be able to use branches / environments in your verification
matt.fellows
2021-09-08 11:50
I think this also answers the other part of your question: https://docs.pact.io/pact_nirvana/step_8
matt.fellows
2021-09-08 11:50
(that guide should be helpful, but we've not yet modified it to direct everyone to the new branch/env selectors)
bethskurrie
2021-09-08 22:43
@todor.m.kolev here's a diagram of what the consumer CI/CD pipeline would look like https://docs.pact.io/pact_nirvana/step_4#consumer-pipeline
todor.m.kolev
2021-09-09 07:11
The problem I having with this flow is while trying to onboard both the consumer and provider. Say the first environment both the consumer and provider are trying to deploy to is SIT. When the consumer first runs can-i-deploy that (in my opinion) should run against the provider version that's currently deployed in SIT. Which would mean that the provider should be deployed first but if you run its pipeline first then the can-i-deploy step would fail as there will not be any successful verifications between provider SIT and consumer SIT. I ended up taking the can-i-deploy step out of the provider pipeline as it looks like if either one of the pacticipants breaks the contract the verification will catch that so long as the verification is always run against the current SIT version of each pacticipant.
bethskurrie
2021-09-09 07:12
The consumer requires the provider to be deployed, but the provider does not require the consumer to be deployed.
bethskurrie
2021-09-09 07:14
The consumer may deploy between the provider verification step and the provider deployment, and the pending pacts feature means the provider build does not always fail.
bethskurrie
2021-09-09 07:14
Something is not set up correctly if you have had to disable can-i-deploy.
bethskurrie
2021-09-09 07:15
> Which would mean that the provider should be deployed first but if you run its pipeline first then the can-i-deploy step would fail as there will not be any successful verifications between provider SIT and consumer SIT.
This is incorrect. The provider will not fail if there is no consumer in SIT.
bethskurrie
2021-09-09 07:16
If you're having problems, this workshop should make things clear https://docs.pactflow.io/docs/workshops/ci-cd
todor.m.kolev
2021-09-09 11:20
https://pact-foundation.slack.com/archives/CLS16AVEE/p1631171648047100?thread_ts=1631098988.043200&cid=CLS16AVEE
So if the consumer tries to deploy then it will first publish a new contract and tag it with `master` for example. This will trigger the provider verification step using the sha of the Provider that's currently in SIT. If that passes then when consumer next runs can-i-deploy its pipeline will be allowed to proceed. At the end, after the consumer has been deployed successfully, it will tag its version with `SIT` .
todor.m.kolev
2021-09-09 11:25
https://pact-foundation.slack.com/archives/CLS16AVEE/p1631171572046500?thread_ts=1631098988.043200&cid=CLS16AVEE
When the provider attempts deployment into SIT then it will want to pull the pact that's been tagged by the consumer as SIT. That won't exist unless the consumer pact has been published, then the consumer deployed to SIT and then its pact tagged with `SIT` .
todor.m.kolev
2021-09-09 11:27
https://pact-foundation.slack.com/archives/CLS16AVEE/p1631171680047500?thread_ts=1631098988.043200&cid=CLS16AVEE
```pact-broker can-i-deploy
--pacticipant ProviderName
--version $CI_COMMIT_SHORT_SHA
--to SIT
... ```
todor.m.kolev
2021-09-09 11:30
I didn't find the work shop very useful as it doesn't seem to incorporate the concept of having a specific version deployed in an environment. In my case that's important as that's the SHA I need to git checkout in the Provider pipeline before I run the verification test (triggered when a new pact is published). Equally, when the Provider runs it's deployment pipeline and it's about to deploy into SIT it will want to verify pacts tagged with SIT as this is what the consumers have deployed in their SIT environment currently.
varnit.garg2424
2021-09-12 15:51
Hi, I have a very basic question related to pact broker. I need to publish my contract to the broker (trial version of pact flow)
Trying to use this below command:
```pact-broker publish --consumer-app-version 1.0.0 --broker-base-url <my_url> --broker-token <myToken> <path to json contract file> --tag master```
Is there any pre-requisite to it? Upon trying to run this command, I'm getting this error: `command not found: pact-broker`
mike.geeves064
2021-09-12 16:15
What docs/examples are you following? (there are different ways you can download/run pact commands, depending on usage, what you are trying to do etc)
varnit.garg2424
2021-09-12 16:30
Been following these resources:
https://docs.pact.io/pact_broker/publishing_and_retrieving_pacts
https://github.com/pact-foundation/pact-ruby-cli
Did `docker pull pactfoundation/pact-cli:latest`
and then running this cmd:
`docker run pactfoundation/pact-cli broker publish --consumer-app-version 1.0.0 --broker-base-url <url> --broker-token <token> <path_to_pact.json> --tag <tag>`
Even though I'm passing the right path to my pact json file in local, getting this - `"Specified pact file '<pact_contract>.json' does not exist. This sometimes indicates one of the arguments has been specified with the wrong name and has been incorrectly identified as a file path."`
Am I doing the things correct so far?
varnit.garg2424
2021-09-12 16:31
I'm basically trying to publish the consumer contract json file to the broker.
mike.geeves064
2021-09-12 16:32
If you're using the docker command, you'll need to mount the path containing the JSON, since the docker container won't have access to your filesystem
mike.geeves064
2021-09-12 16:33
In the publish pacts section, the example command includes:
-v ${PWD}:${PWD} \
mike.geeves064
2021-09-12 16:34
Which will mount the current directory (assuming that's where your pact files are stored) so that you can access them at the same path within the docker container
varnit.garg2424
2021-09-12 16:41
Wow wow wow!
Overlooked it somehow.
It's working now. Thanks a lot @mike.geeves064 :slightly_smiling_face:
varnit.garg2424
2021-09-12 17:54
Guys, anyone faced this issue before while trying to publish the contract using pact cli ?
(it is working in case of pactflow but not if I use my own server where broker is deployed)
```Error making request - SocketError Failed to open TCP connection to <server> (getaddrinfo: Name does not resolve) /usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.50.0/lib/pact_broker/client/base_client.rb:109:in `put'```
matt.fellows
2021-09-12 21:44
That tells me the broker you're connecting to is not published to the DNS correctly
matt.fellows
2021-09-12 22:25
:taco: for @mike.geeves064 thanks for answering so many questions and helping out of late!
roy_collings
2021-09-13 10:45
Hi all - I'm wondering if there's an api endpoint (in HAL, or ideally one I can just curl) that shows me the endpoints covered by a providers spec in Pactflow? It'll be very useful for our consumers to know which endpoints they can now write PACT tests for.
bethskurrie
2021-09-13 10:58
@roy_collings are you talking about bi-directional contracts here? Do you mean, which endpoints are in the OAS?
bethskurrie
2021-09-13 11:00
You can get the latest provider contract and inspect that, but you'd need to parse the oas yourself.
matt.fellows
2021-09-13 12:14
Thanks Roy - I also saw your email, but hadn?t completely gotten to #inboxzero today (I was off last Friday), so I owe you one :email:
matt.fellows
2021-09-13 12:18
```? curl -H"Authorization: Bearer ${PACT_BROKER_TOKEN}" https://testdemo.pactflow.io/contracts/provider/pactflow-example-provider-dredd/latest | jq -r .content | base64 -d <aws:pact-prod>
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5823 100 5823 0 0 26348 0 --:--:-- --:--:-- --:--:-- 26348
openapi: 3.0.1
info:
title: Product API
description: Pactflow Product API demo
version: 1.0.0
paths:
/products:
post:
summary: Create a product
description: Creates a new product
operationId: createProduct
requestBody:
description: Create a new Product
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/Product"
examples:
application/json:
value:
id: "1234"
type: "food"
# price: 42
responses:
"200":
description: successful operation
content:
"application/json; charset=utf-8":
schema:
oneOf:
- $ref: "#/components/schemas/Product"
examples:
application/json:
value:
id: "1234"
type: "food"
# price: 42
get:
summary: List all products
description: Returns all products
operationId: getAllProducts
responses:
"200":
description: successful operation
content:
"application/json; charset=utf-8":
schema:
type: "array"
items:
$ref: "#/components/schemas/Product"
examples:
application/json:
value:
- id: "1234"
type: "food"
price: 42
# name: "pizza"
# version: "1.0.0"
# see https://github.com/apiaryio/dredd/issues/1430 for why
"400":
description: Invalid ID supplied
content: {}
/product/{id}:
get:
summary: Find product by ID
description: Returns a single product
operationId: getProductByID
parameters:
- name: id
in: path
description: ID of product to get
schema:
type: string
required: true
example: 10
responses:
"200":
description: successful operation
content:
"application/json; charset=utf-8":
schema:
$ref: "#/components/schemas/Product"
examples:
application/json:
value:
id: "1234"
type: "food"
price: 42
# name: "pizza"
# version: "1.0.0"
# see https://github.com/apiaryio/dredd/issues/1430 for why
"400":
description: Invalid ID supplied
content: {}
"404":
description: Product not found
content: {}
components:
schemas:
Product:
type: object
required:
- id
- name
# - price
additionalProperties: false # WARNING! without this, a consumer can add stuff that doesn't exist! See also https://bitbucket.org/atlassian/swagger-mock-validator/issues/84/test-incorrectly-passes-when-mock-expects
properties:
id:
type: string
type:
type: string
# name:
# type: string
version:
type: string
# price:
# type: number```
roy_collings
2021-09-13 12:25
Aha!!! Thanks Matt - no idea what I was doing wrong, but yup, works for me too!
roy_collings
2021-09-13 12:38
In case it's handy for anyone, I'm giving our consumers this script to list endpoints covered by the provider spec from the command line:
```#!/usr/bin/env bash
BEARER_TOKEN=${BEARER_TOKEN:?"BEARER_TOKEN not set"}
PROVIDER=${PROVIDER:?"PROVIDER not set"}
# Change to your company
PACTFLOW="https://mycompany.pactflow.io"
curl -s -H "Authorization: Bearer $BEARER_TOKEN" $PACTFLOW/contracts/provider/$PROVIDER/latest | \
jq -r .content | \
base64 -D | \
egrep "^ */"```
matt.fellows
2021-09-13 22:37
Would you be interested in looking at some clickable prototypes and answering a few questions about our thinking?
aforeman
2021-09-14 21:00
Hello! Our next contract testing AMA is locked in for the last week of September. This month we have *Bedford West, Head of Test Engineering at M1 Finance*, joining us. See the event info https://www.linkedin.com/events/contracttestingama-rollingoutpa6843394751811715072/.
M1 Finance is a US-based personal financial app providing customers with automated investing, borrowing and banking products. Bedford joined M1 Finance earlier this year to bring rigour around the company?s testing processes. In this short time, he?s devised and begun executing a test automation strategy, of which an important component is contract testing with Pactflow.
Bedford will join as for a structured Q&A about his experience implementing Pactflow at a fast-growing FinTech during a period of rapid growth and expansion.
We are also welcoming questions from the audience. Head over to #pactflow-ama to post any questions for Bedford in advance. And, if the time isn't friendly, no stress - pose your question and we'll have it answered and share the recording with you post event.
Hope to see you there!
akke.luukkonen
2021-09-16 10:35
Hello everyone, just figured I'd drop by and report that the *System Account* *Read/Write* Token regeneration doesn't seem to work (from Settings > Users > System Accounts > Three dots > Regenerate Read Write Token). Regeneration for Read Only tokens works as expected.
akke.luukkonen
2021-09-16 10:37
I'm doing initial setups for our CI/CD and I was lazily including the token directly in the pipeline files before bothering to wrap it up in a vault. Noticed by accident that the regenerated key looked eerily similar and then compared them to be identical.
matt.fellows
2021-09-16 11:16
I just took a look now. It actually seems that regenerating the read/write account is regenerating the read only token
matt.fellows
2021-09-16 11:24
If you need to regenerate it urgently, issue a POST to the url:
```curl -X POST 'https://<account>.http://pactflow.io/settings/tokens/:uuid/regenerate' -d {}```
matt.fellows
2021-09-16 11:25
You can fetch the tokens (and their uuids) from the resource `/settings/tokens`
akke.luukkonen
2021-09-16 11:57
Thanks, but I'm in no rush with this as mentioned I'm doing some initial setups so it isn't running yet in any production environment.
matt.fellows
2021-09-19 23:27
From memory, it was a difference in the way Pact JVM reported successes vs the Ruby based ones
matt.fellows
2021-09-19 23:28
Java wouldn?t send back individual success/fail if the whole pact passed, but the others would
matt.fellows
2021-09-19 23:28
Our UI used the interaction based feedback. But if the whole contract passed, that should also update the interaction :white_check_mark:
przemek.sech296
2021-09-20 04:37
Hey :wave:
I signed up for `Introduction to Pact with Pact JS` tomorrow. Can I confirm the time?
The invitation says
```Topic: Pact Introductory Training - Pact JS (EU)
Time: Sep 21, 2021 05:00 PM Melbourne / 8am GMT / 9am CET```
where I believe `5 PM Melbourne != 8am GMT`
mike.geeves064
2021-09-20 07:27
Ah, GMT stays where it is, we are currently on BST (British Summer Time) which is GMT+1, so that should be:
7am UTC / 8am BST / 9am CET
tjones
2021-09-20 08:14
GMT isn't dead, it's just replaced by UTC (which is not a timezone) for standardising time. So it is only dead from the perspective of "no longer used as a time standard". Since GMT is UTC + 0 , though, it's all semantics
mike.geeves064
2021-09-20 08:16
:smile: Nod. There's a whole long interesting tangent into the history of it
mike.geeves064
2021-09-20 08:17
It's referred to as on GMT and then on BST for summer time, rather than UTC or having a BWT (winter) or something though :thinking_face:
mike.geeves064
2021-09-20 08:18
I have "There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors" on my old work mug :D
matt.fellows
2021-09-20 11:27
I remember this old video going around of a lady teaching the Peters projection. Fascinating but not surprising!
matt.fellows
2021-09-20 11:27
My take away from this thread: the workshop is tomorrow at 5pm AEST -> you can all work around the rest :laughing:
matt.fellows
2021-09-20 11:31
Also based on 977 I guess I?m a Dymaxion. I?ll leave it to you to guess the 3 things there that I do (hint: XML is _not_ one of them)
mike.geeves064
2021-09-20 12:25
:joy: so at a DevOps meetup, a recruiter was talking about their first assignment and the list of skills the client was asking for; Dev in multiple languages, scripting, databases, infrastructure, IaC, automation, workflows etc. As long list. My friend and I were thinking yes ok that's like us. Then they went onto say about how companies need to think about what area they actually want because that person doesn't exist, they're a unicorn. So we had a brief "we're unicorns! :open_mouth:"
I had/have a unicorn in training in still working on.
It's far more meaningful than "senior" imo :grinning:
nada
2021-09-21 19:27
Hi ? my team is using a react hooks (https://redux-toolkit.js.org/rtk-query/usage/queries) to hit the backend. Do you maybe have some example how I can setup consumer tests in this case?
dabfleming
2021-09-22 20:03
Just out of curiosity does anyone have experience with deploying Pactflow on-prem using Okta for SSO? About to be going down this road...
matt.fellows
2021-09-23 03:20
I know we have a bunch of Okta users for our SaaS platform, but not sure about on-prem
aforeman
2021-09-28 05:47
Hear how hyper-growth FinTech, https://www.m1finance.com/ rapidly rolled out contract testing with Pactflow in https://pactflow.io/case-studies/m1-finance/?&utm_source=slack&utm_campaign=m1-finance.
We interviewed @bedfordwest, Head of Test Engineering at M1 to learn about his journey:
"Starting from a greenfield approach where there was limited testing in place to validate assumptions, M1 now has a cohesive strategy comprising more thorough integration, end-to-end, unit tests?plus contract testing. The greatest success, in Bedford?s mind, is giving the engineering and product teams across the company greater confidence in their build and deploy processes."
And, if you have a question for Bedford, join him live at our https://www.linkedin.com/events/contracttestingama-rollingoutpa6843394751811715072/ - Tues 28 Sept @ 3pm PST / Wed 29 Sept @ 8am AEST (if you're in UK/EU, you can access the recording on https://www.youtube.com/channel/UCuN2fL_eCtHruv2SP3WJgNQ)
sams.prashanth
2021-10-05 20:11
Just to confirm again
*In Consumer-Driven,*
1. If both the consumer and provider services are SOAP based, do we support ?
2. What if either one of the services is SOAP based and the other is REST service.. do we support ?
*In Bi-Directional approach,*
3. Provider as a SOAP service, is it possible to validate the provider xml file document against the pact generated from the SOAP based Consumer service ?
matt.fellows
2021-10-06 02:56
At this moment, we only support two modes:
1. Standard Pact mode (i.e. Pact generated by consumer, verified by provider)
2. Bi-directional: consumer publishes Pact, provider publishes OAS
2021-10-06 07:34
Notification from *Pactflow Contract Testing Platform* :ghost:
sams.prashanth
2021-10-06 09:03
Forget to ask the past workshop sessions as a video. Can you share that, Matt.
matt.fellows
2021-10-06 09:42
I sent it to all those that had registered - did you not? I'll DM you offline
sams.prashanth
2021-10-06 09:45
Thank you @matt.fellows. Just now I searched for it; yeah I got it :slightly_smiling_face:
garry.jeromson973
2021-10-06 10:05
Hullo! We?re currently using Pactflow with a free account; we?ve been successfully registering deployments up until today, when `pact-broker record-deployment` started failing with the error `This version of the Pact Broker does not support recording deployments. Please upgrade to version 2.80.0 or later.` . We?re using the `@pact-foundation/pact` (version 9.16.1) Node.js package to run `pact-broker` . Had a look at https://docs.pact.io/pact_broker/recording_deployments_and_releases/ and it?s pretty clear, but it?s still a little confusing as we didn?t change anything in our pipeline. Will Pact Broker free be updated to 2.81 at some point? Cheers.
bethskurrie
2021-10-06 10:07
@garry.jeromson973 the user that is doing the record-deployment needs to have `environment:read:*` permission.
bethskurrie
2021-10-06 10:07
That should be a system user with the CI/CD role, which has been migrated to have that permission.
bethskurrie
2021-10-06 10:09
It was written for the OSS Pact Broker code, which has no concept of permissions. I'll update it to have something more helpful.
garry.jeromson973
2021-10-06 10:10
Thanks for the lightning response! Will have a shufty and regenerate our CI/CD broker token
bethskurrie
2021-10-06 10:11
you don't need a new token - you just need to make sure the user that is recording the deployment has that permission.
bethskurrie
2021-10-06 10:11
The best way to ensure your CI/CD is always working correcltly is to use the CI/CD role
bethskurrie
2021-10-06 10:12
Sorry for the interruption - as we've applied the Pactflow permissions model to the OSS routes, there was unfortunately some backwards incompatible changes we had to make to ensure things were properly secured.
garry.jeromson973
2021-10-06 10:15
All good! Just as a sanity check: this means I shouldn?t use the CI token found under the _API tokens_ menu when I?m logged in, but rather should create a new user only for CI/CD, with the CI/CD role?
bethskurrie
2021-10-06 10:17
Thanks for reminding me - we need to rename "Read/write token (CI)" to "Read/write token" now we have system accounts.
garry.jeromson973
2021-10-06 10:18
Hmm? how can I make a system account? :laughing: (Maybe I didn?t have enough coffee yet but I don?t see it in the web UI)
bethskurrie
2021-10-06 10:20
You said you're on the free tier - @matt.fellows they do have system accounts now don't they?
anagha.sulakhe
2021-10-06 20:31
Hi, I?m seeing inconsistencies in messaging on pactflow vs pact CLI. I?ve followed the steps from https://github.com/pactflow/pact-workshop-dotnet-core-v1.
The pact-broker says - ?yes? to deploy for provider, but Pactflow says pact verification has failed a test. see attached screenshots. What am I missing here? Thanks in advance.
anagha.sulakhe
2021-10-06 21:21
I shared above the output of can-i-deploy command from provider. is there any other output than what i?ve shared?
bethskurrie
2021-10-06 21:27
Oh, sorry, I missed that photo because it's mostly black on a black background for me!
bethskurrie
2021-10-06 21:28
It's saying "No missing dependencies" because there is no version of the consumer in that environment that it needs to support.
bethskurrie
2021-10-06 21:29
To work out if something is safe to deploy, the code looks at all the integrated applications that are already in that environment, and checks if there is a successful verification result between the app version you're about to deploy, and the app version that's already deployed. The consumer is not deployed yet, so there's nothing stopping the provider from being deployed.
bethskurrie
2021-10-06 21:31
It's an unusual situation though, and I'm surprised it comes up in a workshop.
matt.fellows
2021-10-06 23:38
That can-i-deploy command has also been modified from the workshop, it?s using `--to-environment` which probably won?t work, because recording deployments and environments aren?t being used.
matt.fellows
2021-10-06 23:38
I?d use the CI/CD workshop Anagha for learning that in detail: https://docs.pactflow.io/docs/workshops/ci-cd/
przemek.sech296
2021-10-12 23:50
Morning,
I watched the ?Pact Introductory Training - Pact JS (EU)?, it was great session. I assume Java one is equally excellent.
I was wondering if there?s a plan to do a similar one for the dotnet core?
przemek.sech296
2021-10-12 23:51
Btw., I was among ones who can?t read with understanding and got stacked in Google Meet :facepalm: :rolling_on_the_floor_laughing:
matt.fellows
2021-10-13 00:09
oh, so sorry Przemek! It?s one of those options in our calendar that gets added by default (even if you don?t select it), so that is probably what led to the confusion
matt.fellows
2021-10-13 00:09
As for dotnet, we don?t have any immediate plans, but probably wouldn?t want to run one until the new 4.x.x branch is merged, then we would ensure the related workshop (https://github.com/DiUS/pact-workshop-dotnet-core-v3/) is updated too
przemek.sech296
2021-10-13 07:16
Thanks for sharing.
I?m more of video person and enjoyed a lot comments around the code. Good job on JS one. :slightly_smiling_face:
przemek.sech296
2021-10-13 09:36
Nothing comes to my mind :thinking_face:
It was nicely tailored to my (noob) level :slightly_smiling_face:
david.simpson
2021-10-15 09:45
Hi, I'm just doing the setup for the CI/CD workshop and getting an error with can-i-deploy?
david.simpson
2021-10-15 09:46
```========== STAGE: can-i-deploy? ==========
13
14
Environment with name 'production' does not exist
15
make: *** [Makefile:73: can_i_deploy] Error 1
16
Error: Process completed with exit code 2.```
matt.fellows
2021-10-15 09:47
I think it may be a new prerequisite to create the environment in advance
matt.fellows
2021-10-15 10:39
FYI I just updated the docs for this: https://github.com/pactflow/docs.pactflow.io/commit/fa00e17bdff54a06e95d6252b60a53444f68a5d0
arman.najafian
2021-10-15 15:15
Hello, WIP are enabled by default in PactFlow. Now I have a service that has published their pacts but the provider service hasn?t verified them at all. In the PactFlow UI it shows as `Unverified` , does that mean the pact is in a WIP state?
matt.fellows
2021-10-16 04:18
It's a JSONPath like expression that pact uses to describe the location of a problem. $ is the root object
matt.fellows
2021-10-16 08:34
You won't see the WIP status in the UI because it's dynamic based on what configuration you use to verify
matt.fellows
2021-10-16 08:35
If WIP is enabled (or I guess not disabled) then this pact will also be a WIP because it's never been verified
matt.fellows
2021-10-16 08:36
The Pactflow UI just shows whether a pact has been verified (and if it's successfully verified or not) or not (unverified)
matt.fellows
2021-10-17 01:03
Planned maintenance on our database is now underway
See https://status.pactflow.io/ for updates
arman.najafian
2021-10-18 11:16
Thanks Matt. Just seen your AMA about pending and WIP pacts and cleared some things up for me, thanks for having those sessions recorded! Another issue we are having currently is a consumer is publishing a pact and the provider pulls it down and tries to verify it but fails due to it not having set up upstream mocks to deal with that request. So the solution to this would be to run the verification with the `--enable-pending` flag so that if the pact is failing it won?t fail the providers pipeline? We are using develop tags and not feature tags (which is a bad practise?), now if I pass in the `--enable-pending` flag but I also make a breaking change and fail another interaction with the consumer will the pipeline fail or will it pass due to the `--enable-pending` flag?
muralis
2021-10-19 18:44
Hello, We are seeing weird behavior with Pact publish functionality. Lets say there are 3 contract tests executed in Producer side (Spring boot based) - 2 passed and 1 failed. It either publishes as "all pass" or "all fail" to Pactflow. Has anyone else experienced this issue?
uglyog
2021-10-19 21:39
Sounds like you have one contract test with three interactions (i.e. the same consumer/provider). If this is so, than that is the expected behaviour.
dabfleming
2021-10-20 16:28
Not immediately obvious to me so I'll just ask, what's the best place to report a minor UI bug in Pactflow?
matt.fellows
2021-10-21 00:18
If you could please raise https://github.com/pactflow/roadmap/issues/new that would be great
ben.crinion
2021-10-21 11:11
Hi, is it possible to use the environments feature with pactflow? I'm getting an error saying i'm not using the right version of the broker? Is the broker version a customer specific setting?
matt.fellows
2021-10-21 11:50
double check you?re on the latest anyway, it was previously a hidden API but now it?s ?official? older clients may not support it. I recall there being a bug or something though.
ben.crinion
2021-10-21 12:04
I think we were trying to run can-i-deploy. I'll put together full details of what we tried and the response we got. I just wanted to check i was doing something sane first.
ben.crinion
2021-10-21 13:28
That's the result when I run `docker run --env-file .env pactfoundation/pact-cli:latest broker list-environments --verbose`
bethskurrie
2021-10-21 22:59
Or rather, the user that is running the can-i-deploy command needs read:environment.
bethskurrie
2021-10-21 23:00
If this is on CI, you should be using a system account user with the CI/CD role, which has the right permissions.
artur
2021-10-22 16:01
will the `Pactflow Developer Preview Program: gRPC/Protobufs walkthrough` be recorded? I'm not sure if I can make it, but would be very interested
rajnish.maurya
2021-10-25 08:04
Hi,
I created a webhook to trigger our GitLab pipeline, and supplied the following (a GitLab pipeline trigger) as the url:
`https://<our-gitlab-url>/api/v4/projects/<project-id>/ref/<branch>/trigger/pipeline?token=${user.<token-as-a-secret>}&variables[PACT_CONSUMER_TAG]=${pactbroker.consumerVersionTags}`
When I test, the webhook doesn't succeed. Throws following error:
```[2021-10-25T07:49:32Z] INFO: user-agent: Pact Broker v2.89.0
[2021-10-25T07:49:32Z] INFO:
[2021-10-25T07:49:32Z] ERROR: Error executing webhook Net::OpenTimeout - execution expired
[2021-10-25T07:49:32Z] INFO: Webhook execution failed```
We need VPN to connect to our GitLab account. I think that is the cause for failure. But want to confirm if I am not missing anything else.
matt.fellows
2021-10-25 08:13
If that's not an option you could also try the webhookless flow: https://docs.pact.io/pact_nirvana/step_4/#alternative-webhookless-workflow
stefan.smith
2021-10-25 11:31
Hi, I'm using `v0.50.0` of the Pact CLI (via the docker image) to publish pacts to Pact Flow. I'm trying to set the branch name but it isn't appearing in Pact Flow. When I run `publish ... --branch=my-branch --verbose`, I see `PUT /pacticipants/<CONSUMER>/versions/<GIT_SHA>` with body `{"branch":"my-branch"}` but the `201` response returns with a body containing `"_embedded":{"branches":[],"tags":[]}` . Everything else about the pact publish works. Is this caused by the Pact CLI or Pact Flow? I see there's a newer resource (`pb:publish-contracts` at `/contracts/publish`). Does the CLI need to use this resource for branches to be stored?
stefan.smith
2021-10-25 11:42
Note, if I explicitly call `curl -XPUT "https://<SUBDOMAIN>.http://pactflow.io/pacticipants/<CONSUMER>/branches/my-branch/versions/<GIT_SHA>"` , the branch is associated with the consumer version. This implies it's an issue with the Pact CLI not making this call today when the `--branch` arg is supplied
matt.fellows
2021-10-25 12:03
do you mean version 1.50.0 of the `broker` command or the version of the docker image?
In any case, some new things here I need to get on top of. This is probably one for @cgoodison or @bethskurrie tomorrow.
stefan.smith
2021-10-25 12:06
```docker run pactfoundation/pact-cli:latest version```
returns `0.50.0`
stefan.smith
2021-10-25 12:09
```docker image inspect pactfoundation/pact-cli:latest | jq '.[0].RepoDigests'```
returns
```[
"pactfoundation/pact-cli@sha256:219463c0dd64683b325ed5b38fd3367f889e8004c6fbb1b3bd9508677ff97588"
]```
stefan.smith
2021-10-25 14:51
Hi, beyond the brief mention of encryption in https://docs.pactflow.io/docs/user-interface/settings/secrets, are there any documented assurances around how Pactflow handles secrets? For example:
? What is the encryption mechanism?
? Do Pactflow operations staff have the ability to decrypt a tenant's secrets?
? Is HTTP traffic encrypted end-to-end from the browser to the host encrypting the secret?
? What about upon retrieval for use by a webhook?
? How does Pactflow minimise the likelihood of a secret being written to an application log?
? Does Pactflow's operations team make use of any automated tooling to avoid/detect/alert on secrets that leak into application logs?
Thanks!
matt.fellows
2021-10-26 01:31
Hey Stefan, drop us an email to and we can share with you a bunch of details about our security practices/processes
bethskurrie
2021-10-26 03:40
@stefan.smith the branch is just not yet displayed in the UI. It should be out very soon.
bethskurrie
2021-10-26 03:43
Also, do please updated to the latest version of the CLI - the branch as set in the client you are using is not supported.
bethskurrie
2021-10-26 03:48
`sha256:518f3118081e066b3d8e406984714139cca1c10f5c39afbf0f9228feaed9eda0`
bethskurrie
2021-10-26 03:49
Hm. I think I can see a problem. The image with "latest" is different from the image with 0.50.0.14
bethskurrie
2021-10-26 04:53
Ok, @stefan.smith I've fixed the docker image release script so that the `latest` is as it should be.
bethskurrie
2021-10-26 04:54
The branch support is already out in the Pactflow UI - apologies, I missed that release. When you use the most recent version of the CLI, it should all work as expected now. Btw, I do recommend not using the `latest` tag - probably best to pin it to a known version.
j.malyjasiak
2021-10-26 05:51
I posted a https://pact-foundation.slack.com/archives/C01RPBC2KNG/p1635172936003200 regarding Pact Broker -> Pact Flow migration recently but I could?ve picked a better channel for that ? thanks for the suggestion! The question was as follows:
Hello! We are currently considering various migration options from multiple internal pact brokers to a SaaS version of PactFlow. As of today the plan is to run both in parallel and send all the data (tags, pacts) to PactFlow and additionally ask for a possibility to get our databases dumps imported so that we have no gaps in data. Webhooks can be migrated at any point in time from our perspective (mostly terraform) and I?m more concerned about verification results (no idea how to migrate them seamlessly while still using existing pact brokers for verifications). Do you have any experience in putting pact broker behind a proxy to duplicate most of the traffic to PactFlow or backfilling historical data with db dumps? We find it quite crucial due to backward compatibility documented with some of the contracts. Thanks!
j.malyjasiak
2021-10-26 05:59
There is one more topic that I?d appreciate your opinions and experience on. I?ve seen that triggering provider verification pipelines in Git repositories hidden behind firewalls has been brought up a couple times already. Have you find a way to work around it except moving the trigger source from Pact Broker/PactFlow to the consumer?s CI/CD pipeline (pulling provider?s code and executing the verification included)?
matt.fellows
2021-10-26 06:46
Another option would be to build an internal API that can receive any webhook from the broker, and then invoke builds that way. It?s another level of indirection, but I?ve spoken to a few customers who?s tried that previously
matt.fellows
2021-10-26 06:49
Lastly, would something like this help https://pact.canny.io/feature-requests/p/support-a-polling-endpoint-for-a-webhookless-workflow ?
matt.fellows
2021-10-26 06:50
This way, you could trigger the provider build from your consumer, without having to check-out the provider locally etc.
stefan.smith
2021-10-26 06:54
Thanks for the super quick response on this @bethskurrie! Agree that `latest` isn't usually a tag you should track (e.g. when deploying an image to Prod). On the other hand, a repointable tag that indicates the newest stable release of a CLI tool can be really handy for automatically staying up to date (notwithstanding breaking changes, of course). I see other popular images on dockerhub use the `latest` tag for this, although https://vsupalov.com/docker-latest-tag/ as well, I guess a repointing stable tag per major version might be a better idea, eg `stable-v0` and `stable-v1`.
alex.macpherson
2021-10-27 08:44
hello! i posted a question in the #pact-scala-itv channel as our pact verifications have recently started failing with the scala client unable to decode the pact it downloads from pactflow. i've done some debugging and found the point in the library at which the decoding fails, but it's curious to me that decoding would start to fail out of the blue without any sort of dependency changes, so i'm wondering whether there has been a change made to how pacts are sent in JSON format from the broker to the client for verification?
looks like the `_links.curies` or `_links.consumer-versions` arrays are at fault, as i can reproduce the issue when running verification from a local file, and removing either one of these arrays fixes the decoding issue. anyone have any ideas re. this? https://pact-foundation.slack.com/archives/C9W94RTBQ/p1635323693003700
alex.macpherson
2021-10-27 08:46
i understand it's not the quickest to get things fixed in the scala-pact library, but this issue has entirely bricked our pact integration, so if anyone has a suggested workaround we can implement in pactflow/otherwise in the interim, that would be greatly appreciated!
alex.macpherson
2021-10-27 09:01
in fact is the `pb:consumer-versions` array new? i don't see it in the pactflow hal-browser when i request a pact, but it's included in the JSON if the pact is fetched directly from pactflow :thinking_face:
``` "pb:consumer-versions": [
{
"title": "Consumer version",
"name": "<version>",
"href": "https://<customerName>.http://pact.dius.com.au/pacticipants/<serviceName>/versions/<version>"
}
],```
matt.fellows
2021-10-27 09:39
Hi Alex, I don't think we've changed that recently but can take a look. the HAL relations may not appear depending on the Accept header in use. How are you configuring the pact file for downloading?
alex.macpherson
2021-10-27 09:48
i believe all of this is handled within the scala-pact library. we configure scala-pact to use a pact source with the below config
```pactBrokerWithVersionSelectors(
url = pactConf.brokerUrl,
provider = "<provider service name>",
consumerVersionSelectors = pactConf.consumerTags
.split(',')
.map(ConsumerVersionSelector(_, None, Some("<consumer service name>"), Some(true), None, None, None, None))
.toList,
providerVersionTags = List(pactConf.providerTag),
pendingPactSettings = PendingPactSettings.PendingDisabled,
publishResultsEnabled = Some(BrokerPublishData(pactConf.projectVersion, buildUrl)),
pactBrokerAuthorization = Some(PactBrokerAuthorization.BearerToken(pactConf.brokerToken)),
pactBrokerClientTimeout = Some(30.seconds)
)```
alex.macpherson
2021-10-27 09:50
scala pact then sends the following requests:
```Attempting to fetch relation 'pb:provider-pacts-for-verification' from broker
cURL for request: curl -X GET 'https://<company>.http://pact.dius.com.au' -H 'Accept: application/hal+json' -H 'Authorization: <REDACTED>'
cURL for request: curl -X POST 'https://<company>.http://pact.dius.com.au/pacts/provider/<provider service name>/for-verification' -H 'Accept: application/hal+json' -H 'Content-Type: application/json' -H 'Authorization: <REDACTED>' -H 'Content-Length: 228'
Attempting to fetch pact from pact broker at: https://<company>.http://pact.dius.com.au/pacts/provider/<provider service name>/consumer/<consumer service name>/pact-version/35be99c43127acc4f4c45c0470cb0f1960778a18/metadata/c1tdW3RdPW1hc3RlciZzW11bbF09dHJ1ZSZzW11bY3ZdPTE1ODM0
cURL for request: curl -X GET 'https://<company>.http://pact.dius.com.au/pacts/provider/<provider service name>/consumer/<consumer service name>/pact-version/35be99c43127acc4f4c45c0470cb0f1960778a18/metadata/c1tdW3RdPW1hc3RlciZzW11bbF09dHJ1ZSZzW11bY3ZdPTE1ODM0' -H 'Accept: application/json' -H 'Authorization: <REDACTED>'```
alex.macpherson
2021-10-27 09:53
the final request yields the following JSON which scala-pact is now unable to decode
```{
"consumer": {
"name": "<consumer service name>"
},
"provider": {
"name": "<provider service name>"
},
"interactions": [
// loads of interactions
],
"metadata": {
"pactSpecification": {
"version": "2.0.0"
}
},
"createdAt": "2021-10-26T14:59:42+00:00",
"_links": {
// lots of links, but these ones seem relevant
"pb:consumer-version": {
"title": "Consumer version",
"name": "<version>",
"href": "https://<company>.http://pact.dius.com.au/pacticipants/<consumer service name>/versions/<version>"
},
"pb:consumer-versions": [
{
"title": "Consumer version",
"name": "<version>",
"href": "https://<company>.http://pact.dius.com.au/pacticipants/<consumer service name>/versions/<version>"
}
],
"curies": [
{
"name": "pb",
"href": "https://<company>.http://pact.dius.com.au/doc/{rel}?context=pact",
"templated": true
}
]
}
}```
alex.macpherson
2021-10-27 09:54
as i say, removing either of the `curies` or `pb:consumer-versions` arrays from `_links` fixes the issue if i'm running the verification against a local file, but as this is all handled internally by scala-pact, it's non-trivial to remove either of these fields in flight without setting up e.g. a proxy to remove them :confused:
alex.macpherson
2021-10-27 09:56
someone else has raised the issue on the scala-pact github pointing to `pb:consumer-versions` as well, so i'm wondering whether pactflow has only recently started returning this field in the curl request, as this was not failing previously? https://github.com/ITV/scala-pact/issues/231
alex.macpherson
2021-10-27 09:58
of course the ideal solution is to fix in the scala-pact library, but unfortunately it doesn't look like this is a particularly regularly maintained project, so i fear it could be a while before this is fixed - hence my asking whether there's any other known workarounds for this (other than disabling pact tests in our pipelines until fixed/ripping out scala-pact and replacing with pact-jvm)
alex.macpherson
2021-10-27 09:58
i really appreciate any support you're able to provide on this @matt.fellows :slightly_smiling_face:
alex.macpherson
2021-10-27 10:19
have done a bit more digging and looks like the `pb:consumer-versions` field has only recently been added - this appears to be the problem field for the `scala-pact` library.
https://github.com/pact-foundation/pact_broker/commit/3458c9e5
https://pact-foundation.slack.com/archives/C9VPNUJR2/p1633920350325700?thread_ts=1633564474.290200&cid=C9VPNUJR2
matt.fellows
2021-10-27 11:14
That does tell me there is a difference in implementation for how the pacts for verification response is interpreted by scala pact
matt.fellows
2021-10-27 11:36
The only suggested workaround that comes immediately to mind, is to make your own call to the pacts-for-verification endpoint and then feed those into the verifier as direct URLs
alex.macpherson
2021-10-27 11:44
thanks Matt. I've raised a PR with scala-pact to remove the pb:consumer-versions field until someone has more time to fix the type definition across the library (will take more time) but this should unblock anyone currently using scala-test with pactflow
alex.macpherson
2021-10-27 11:45
i'll certainly explore your suggestion though if the PR doesn't get movement soon :crossed_fingers:
matt.fellows
2021-10-27 11:54
FYI this was the master tracking issue for that feature: https://github.com/pact-foundation/pact_broker/issues/307
matt.fellows
2021-10-27 11:55
and sorry about the issue. It certainly raises the question about getting Pact using Pact
matt.fellows
2021-10-27 11:55
(ironically, Pact was not ideally suited to hypermedia APIs, but in the v3/v4 spec we have added some features to make that easier to test)
alex.macpherson
2021-10-27 13:25
appreciate the info - and no worries re. the issue! hopefully it'll be resolved properly soon :slightly_smiling_face:
alex.macpherson
2021-10-27 16:02
just to follow on from the above, this has been fixed in `scala-pact:4.3.0` - thanks @jackbwheatley for getting this resolved so quickly!
bethskurrie
2021-10-27 21:06
> ironically, Pact was not ideally suited to hypermedia APIs
It wouldn't have helped, because the provider can send back extra fields that the consumer has never specified.
bethskurrie
2021-10-27 21:09
I was talking with Trav about this the other day - that clients need to be able to handle and unexpected input gracefully, and that this is one of the problems with statically typed languages as API clients. Classic example is parsing a field to an enum - when a new option is added to the API, the clients all blow up because they can't parse the new enum value. That shouldn't be a breaking change. Clients gotta expect the unexpected.
bethskurrie
2021-10-27 21:11
Great talk by Mike Amundson on this https://slides.yowconference.com/yow2017/Amundsen-TwelvePatternsEvolvableAPIs.pdf?feature=oembed
bethskurrie
2021-10-28 23:14
@stefan.smith yes, you're right. In one of my other docker images I've got a major version tag, so you can keep track of that one, knowing you won't get any breaking changes.
bethskurrie
2021-10-28 23:16
The pact cli is a bit tricky, in that it bundles a bunch of other gems, and every time we update one of the gems, it just bumps the minor version, irrespective of what the bump was to the updated gem. I'll have a think about how I could make that a bit smarter, so we can have a major version tag that we can be sure won't include a breaking change.
kiran.rudrangi
2021-10-29 20:14
Hello, This is my first ping here :slightly_smiling_face: . I am trying to implement Contract test & before doing some hands on with the example repo https://docs.pactflow.io/docs/workshops/introduction/readme. It looks like a BUG on PactFlow Verification, If we use the same names again for Consumer & Providers, it shows status as Success before verification itself. Below are steps to reproduce the issue. Any thoughts here ?
1. *Published contracts from consumer - showed contract state as Unverified.*
2. *Verify contracts on Provider - changed contract state as Success.*
3. *Now delete all the pacts from pack flow.*
4. *Repeat Step 1 - showed contract state as Success directly instead of Unverified.*
bethskurrie
2021-11-01 06:19
@kiran.rudrangi when you delete the pacts, it deletes the pact publications, but not the verification results.
bethskurrie
2021-11-01 06:20
The verification results belong to the pact *content* which does not get deleted when you delete the pact publications.
bethskurrie
2021-11-01 06:20
So, when you publish a pact with the same content as previously deleted one, it just hooks up your new publication to the existing content, and as that content was already verified, it shows you the existing status.
bethskurrie
2021-11-01 06:22
Through the UI, you'll need to choose "Delete Integration" from the "..." menu next to the "consumer <> provider" title on the overview page.
kiran.rudrangi
2021-11-01 13:19
It worked with `Delete Integration` option. Thanks @bethskurrie. :slightly_smiling_face:
kiran.rudrangi
2021-11-01 16:29
@bethskurrie I am done with all 11 steps mentioned at `Pact JS workshop` (https://docs.pactflow.io/docs/workshops/introduction/readme) and looking to take project to `CI/CD using CircleCI`. Do you have any links/examples to guide ?
2021-11-01 23:31
Here are a number of useful hands-on labs that teach all of the key concepts https://docs.pact.io/implementation_guides/workshops
stefan.smith
2021-11-02 13:37
Is there a way to configure https://docs.pact.io/pact_broker/configuration/settings/#main_branch_candidates in Pactflow? It's a config option for self-hosted brokers but it would be useful to support this in Pactflow. I'd like to modify the auto-detection of main branches based on published branches so that it only picks up `main` or `master` but not `develop`
matt.fellows
2021-11-02 21:48
It?s something being rolled out slowly across the ecosystem. I believe we can configure it on our side, could you please drop a request to and we can manage it there? (we won?t lose sight of it then)
michael.branders
2021-11-03 08:37
Hi
I see that the roll Test Maintainer is deprecated and been replaced by the user roll. But i don't see a user roll in the predefined rolls and i do see Test Maintainer there still. What should be the default roll a new developer should get? User is not a roll you can select in the preferences. Do i need to create the roll as defined in the documentation myself?
matt.fellows
2021-11-03 09:49
You might on an account that has the old role as the default, in which case that's fine I think
matt.fellows
2021-11-03 09:51
It's just a label change I think, and you can always change the permissions associated if you need
bethskurrie
2021-11-03 09:54
@stefan.smith you could just set the mainBranch explicitly for each pacticipant?
bethskurrie
2021-11-03 09:55
@michael.branders if you want to reset your roles to the latest default permissions, here are the docs https://docs.pactflow.io/docs/permissions/predefined-roles#resetting-permissions-for-predefined-roles
bethskurrie
2021-11-03 09:55
You can't do it through the UI, but you can do it through the HAL browser.
stefan.smith
2021-11-03 12:12
Thanks. We've decided for now to explicitly set it. Was thinking about how this would need some automation at scale however.
stefan.smith
2021-11-03 13:06
Noticed a UI bug in Pactflow. On the overview page, when the latest version of a pacticipant is associated with multiple branches, the branch names are concatenated together without spaces or commas. Presumably they should be comma-separated?
bethskurrie
2021-11-05 01:57
@stelios.arakliotis we've updated your config. Only main and master will be considered candidate names now.
dawoud.sheraz
2021-11-08 13:41
Hello. Is every secret that gets added in Pactflow accessed through ${user.<secret>}?
https://pactflow.io/blog/secrets/
matt.fellows
2021-11-08 13:43
On the webhooks screen you'll be able to see which secrets you have access to and the variable interpolation you can use there
michael.branders
2021-11-08 15:05
Are there any changes needed for ower firewall to work with pactflow? To use webhooks for example? If there are can you direct me to the information so i can give it to the engineering team so the can add the rules to the firewall. We would be using the cloud version of pactflow.
dabfleming
2021-11-08 17:12
Hey, I was trying to do some testing of the new support for branches and noticed that the latest enterprise image: `http://quay.io/pactflow/enterprise:latest` is `Pactflow Version: 8987606f9 / Pact Broker Version: 2.85.1` so doesn't yet support branches. Would appreciate an updated image so we can start moving to proper branches.
bethskurrie
2021-11-08 20:47
@michael.branders yes - there is a list of IP address ranges you can use in your firewall rules on this page https://pactflow.io/faq/
bethskurrie
2021-11-08 20:48
Scroll down to "Do you provide a list of IP addresses/ranges that we can use to lock down access to our (CI) systems?"
dabfleming
2021-11-08 22:08
:+1: Thanks! Looking forward to refining our process with branches etc. when we can.
kriegster108
2021-11-08 22:50
Question about ``cross contract verification``, it seems super new, I was able to successfully validate a consumer contract with a OAS contract and got it working using PactFlow + Pact CLI container, but it appears the validation result doesnt work? And the UI on pactflow doesnt register any type of association between the Pact Consumer and the OAS Provider I used.. It seems to be doing alot of stuff under the hood to make the usecase work, but the feedback isnt very transparent? Or am I not looking in the right place to see my validation result?
matt.fellows
2021-11-08 22:59
https://docs.pactflow.io/docs/workshops/bi-directional#%EF%B8%8F-project-status
Ah yes, it?s currently in ?developer preview? Hunter - so the UI is currently not representing reality
matt.fellows
2021-11-08 22:59
The UI support is currently in progress, with a target release later this month
kriegster108
2021-11-08 23:12
https://broker/contracts/provider/Bar/version/1.2.1051
I was able to push my OAS contract using the route above ^^
Have a few questions:
Is there like any documentation on this API route? I cant seem to find any, other than what is in the github examples you provide.
{
"content": "OAS Contract base64",
"contractType": "oas",
"contentType": "application/yaml",
"verificationResults": {
"success": *true*,
"content": "????",
"contentType": "text/plain",
"verifier": "verifier"
}
What would I normally fill in for the `verificationResults` obj? I have just `success` = true, but what should go inside that `content` property? I dont have verification results, our swagger generator that we have in dotnet is just automatically generating my OAS for me, which in a way is "testing" the integration
matt.fellows
2021-11-08 23:16
The basic documentation is https://docs.pactflow.io/docs/workshops/bi-directional/contracts/oas#publishing-the-provider-contract--results-to-pactflow. We?ll be adding support via the CLI to do this in the near term
matt.fellows
2021-11-08 23:17
Content can be anything, we?ll eventually be rendering those in the UI verbatim. It can be blank, I believe
kriegster108
2021-11-08 23:56
you gotta have a base64 encoded something for the content portion, just tested it
kriegster108
2021-11-08 23:56
but thats fine, I was just making sure i wasnt supposed to provide something specific
kriegster108
2021-11-09 21:35
Hey quick question, are you guys utilizing the atlassion tool for comparing PACT + OAS under the hood?
kriegster108
2021-11-09 21:35
also, say I wanted to build a pact testing tool so my devs could verify somehow locally that their contracts are still valid before going into CI, is there any ideas in place around that?
bethskurrie
2021-11-09 21:35
> are you guys utilizing the atlassion tool
Yes. We have plans to write our own, but we're starting with this one.
bethskurrie
2021-11-09 21:35
> verify somehow locally that their contracts are still valid before going into CI,
Yes
bethskurrie
2021-11-09 21:39
@kriegster108 regarding the content, I'll see about making that optional, but for now, put in something like "OAS auto generated from code".
kriegster108
2021-11-10 18:37
Question, I am trying to build a CURL request to push my contract up to pactflow
kriegster108
2021-11-10 18:37
I have it working in postman, but when i run it in CI, the API keeps returning a 303
kriegster108
2021-11-10 18:41
`HTTP/1.1 303 See Other`
`Server: Cisco Umbrella`
`Date: Wed, 10 Nov 2021 18:34:16 GMT`
`Content-Type: text/html`
`Transfer-Encoding: chunked`
`Connection: keep-alive`
`Location: http://ihsmarkit.pactflow.io/contracts/provider/Bar/version/1.2.1056`
`X-Frame-Options: SAMEORIGIN`
`X-Content-Type-Options: nosniff`
`X-Xss-Protection: 1; mode=block`
kriegster108
2021-11-10 20:24
i noticed all the cert errors im seeing on the build machine, do not exist on my machine
kriegster108
2021-11-10 20:24
two of my coworkers, when they try to nav to that url, they get SSL errors too
kriegster108
2021-11-11 18:10
OMG, finally convinced my architects that contract testing with you guys is the best direction forward. will probably schedule a meeting in the coming weeks on licensing
mastrobardo
2021-11-21 12:54
Hello guys, im trying to reset my password but i cannot get the verification code. It is probably arriving a bit late, as the help i received on the slack channel to unlock the account did actually helped me to receive a new validation code, but seems already expired. anyone can help out here?
matt.fellows
2021-11-21 13:00
The passcode doesn't expire that quickly, I suspect maybe it's something else.
pollet_22
2021-11-22 08:48
Hello everyone
i am new in pactflow can any one help me with learning materials that can help me to understand pactflow and using it in my company to test contract between two endpoints ?
matt.fellows
2021-11-22 11:42
Note Pactflow is a commercial offering around a set of OSS tools, including Pact
Here are some links that might help:
https://docs.pactflow.io/#tutorials
https://docs.pactflow.io/docs/workshops
http://docs.pact.io
dawoud.sheraz
2021-11-22 16:03
Hi. Some questions/comments on webhook setup step in CI/CD workshop(https://docs.pactflow.io/docs/workshops/ci-cd/set-up-ci/configure-webhook/). I have not been able to get webhook working with github. I receive 404 when the webhook triggers. It might be due to fact that some info is outdated and I wanted to have a confirmation about it:
? When generating token on Github, the steps mention using *public_repo* scope. However, from https://docs.github.com/en/rest/reference/repos#create-a-repository-dispatch-event, to use a repo dispatch event, the token should have complete repo scope.
? A typo in headers. `Accept: Accept: application/vnd.github.everest-preview+json` . The docs from Github suggest using `application/vnd.github.v3+json` with accept.
? Authorization requires token as a prefix instead of Bearer(https://docs.github.com/en/rest/overview/resources-in-the-rest-api#oauth2-token-sent-in-a-header and https://stackoverflow.com/questions/63042765/cannot-trigger-dispatches-from-github-api)
What possible reasons could be there that the Github api call returns 404? Pact docs suggests this might be due to token permissions and I have verified this from Github docs too. But the token generated is per Github docs.
alejandro.pena
2021-11-22 21:52
Hi all! I'm facing a situation I'm not understanding. What can make pactflow to tag an interaction as "Verification Status Unknown". I see the test is being executed in standard output but for some reason is not being recognized in the broker.
matt.fellows
2021-11-22 21:52
it means the verification results from the provider side is not being sent through
bethskurrie
2021-11-22 21:54
@dawoud.sheraz you get a 404 when the authentication credentials are not correct. They do this because it's more secure than returning a 403, because then it would tell you that the endpoint exists. But it's very unhelpful!
bethskurrie
2021-11-22 21:55
There is a bug in some of the pact jvm libraries where the results for the passing tests don't get sent to the verification results endpoint.
alejandro.pena
2021-11-22 21:55
But what can cause that @matt.fellows? The rest of interactions are being sent as usual
alejandro.pena
2021-11-22 21:56
ahhh, ok! That would make sense, because the behavior even changes depending on the number of tests I include
bethskurrie
2021-11-22 21:57
If you are not, can you please upgrade. If you still find the issue, we would love it if you can provide us with an executable reproduction of the issue, because we've been trying to track down the issue for a while, but haven't managed to reproduce it ourselves.
alejandro.pena
2021-11-22 21:59
I'm afraid I cannot update to last version. Is there any workaround to avoid this problem?
bethskurrie
2021-11-22 22:05
Ok, I don't have a webhook, but I do have a curl script here https://github.com/pact-foundation/pact_broker/blob/master/script/trigger-release.sh
bethskurrie
2021-11-22 22:06
I would recommend getting it working with curl first, then creating the webhook from it.
bethskurrie
2021-11-22 22:07
If you can give us the exact code to reproduce it, we can try and get it fixed.
dawoud.sheraz
2021-11-25 03:32
Yes, you were right. It was the token issue, rather issue with user account whose token was used. Certain write permissions were missing.
stefan.smith
2021-12-01 09:57
I think I'm hitting a bug in the Pactflow UI, specifically on the "matrix" page.
*Steps to reproduce*
1. Visit Matrix page
2. Click the end-of-row "Verified X days ago" link for verification result *X*. I am taken to the detailed verification page for verification result *X*.
3. Click the browser Back button. I see the Matrix page.
4. Click the end-of-row "Verified X days ago" link for verification result *Y*.
*Expected behaviour*
I am taken to the detailed verification page for verification result *Y*.
*Actual behaviour*
I am taken to the detailed verification page for verification result *X*. I have to repeat steps 3 and 4 to actually arrive at the detailed verification page for verification result *Y*.
*More info*
If you keep clicking links on the Matrix page and clicking Back, it becomes clear that the UI takes you to the _*previously-requested*_ verification result. Seems like there some sort of state update issue.
matt.fellows
2021-12-01 10:00
Hey! Funny, I was just thinknig before you posted this if your query was answered about webhooks (it was). Thank you for both your careful questions and detailed reports - if only all of our bug reports came in this clearly!
joelgrimberg
2021-12-01 11:31
The http://pactflow.io/features page mentions that for Contract Types you can have Pact + openAPI specification when you use pactflow . I have a Pactflow free account set up, but I cannot find any info about openAPI when I am logged in. How is openAPI implemented in Pactflow?
joelgrimberg
2021-12-01 11:33
question #2: the http://pactflow.io/features mentions about testing mode, that you can use consumer or provider driven testing when you use pactflow. does Provider-driven mean that when I push another version of my API (provider), it triggers that consumer tests? or .. ?
matt.fellows
2021-12-01 11:54
For posterity, the docs for the feature are here: https://docs.pactflow.io/docs/workshops/bi-directional
matt.fellows
2021-12-01 11:54
as sent in the email, just heed the warning that there is currently no UI support. We?re hoping to drop that this side of xmas
matt.fellows
2021-12-02 11:53
Hi Stefan, I believe a fix went out for this today. If you get a moment to please confirm if it has addressed your issue, that?d be ace
stefan.smith
2021-12-02 12:40
*Feature request: sort Matrix table by verification time but without grouping by consumer*
Today, it seems that the table is sorted by verification result but then displays results grouped by consumer version. For a long-running feature branch, this means that whenever a new version is published, all the historic results for that branch get displayed at the top of the table. This pushes other branches' results off the end of the table and sometimes means we cannot see any rows for verification against the Prod version of our Provider (more likely when the Provider hasn't deployed recently). We'd really like to see results sorted by verification date-time (descending), without the grouping by consumer.
stefan.smith
2021-12-02 12:45
Separately, may have discovered a bug on the Matrix page but not sure. I _think_ we're seeing a behaviour where verification results against our Prod environment (which are listed in the HAL API) are not appearing in the Matrix UI. We see results for the most recent consumer feature branch against the Provider's Develop, Staging and Prod versions but earlier consumer feature branches only display results for the Provider's Develop and Staging versions. This is even though the consumer's Pact digest hasn't changed for multiple weeks. As mentioned, all results are visible in the HAL API's matrix endpoint but they don't get listed on the UI.
bethskurrie
2021-12-03 00:50
Hi @stefan.smith. Can you let me know the version numbers of the data you're expecting to see, and I'll dig into it for you.
bethskurrie
2021-12-03 00:51
They may be dropping off the bottom of the results due to the way the data is being sorted. We're well aware of the shortcomings of the Pactflow version of the matrix page, and we have a card in our "up next" column to add support for filtering and sorting.
bethskurrie
2021-12-03 00:52
What you can do in the mean time is switch to the OSS page, which supports the filtering and ordering. Click the "View old UI" button at the top of the page, and then click on the little grid icon that is in the row of the integration you want to inspect.
stefan.smith
2021-12-03 12:58
Thanks. We'll have a look at using the old UI for now. Looking forward to the UI enhancements!
imikhtyuk
2021-12-06 05:53
Hello,
I'm currently working on POC for bi-directional contracts testing, but facing an issue with tests verification..
I've published consumer pact into pactflow (in standard way, same as pactbroker), and also provider openapi spec (based on this https://docs.pactflow.io/docs/workshops/bi-directional/contracts/oas/#publishing-the-provider-contract--results-to-pactflow example), however in pactflow UI I can't see any provider contract details.
I can see provider versions uploaded using 'pacticipants/parties/versions' API and pact-broker cli though.
can-i-deploy returns something like 'There is no verified pact between version 0.0.1-620341 of products and the latest version of parties (0.0.1-620352)'
I'm using pactflow image c595d861bf18 and cli v1.88.77
Can you please suggest what can be wrong and how can I troubleshoot it?
bethskurrie
2021-12-06 05:54
Hi @imikhtyuk there is no UI for bi-directional contracts at the moment. We're working on it as I type!
imikhtyuk
2021-12-06 05:56
Thanks Beth. What about cli? Am I right that verification should happen when I execute 'can-i-deploy'? At the moment it's not working for me (I've provided output in 1st message). Should I use some special version of cli?
And yes, I'm using on-prem pactflow
imikhtyuk
2021-12-06 05:58
Output from can-i-deploy:
Computer says no ¯_(?)_/¯
CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
---------|--------------|----------|-----------|----------|--------
products | 0.0.1-620341 | parties | ??? | ??? |
aforeman
2021-12-08 22:26
Hi everyone!
A note that the inaugural Pactflow Contract Testing Report is now available for free download,
And, join @matt.fellows @bethskurrie and I this time tomorrow morning as we discuss the report and the different ways you can execute contract testing in our final AMA for 2021 >> https://www.linkedin.com/events/contracttestingama-comparingpac6869498736658264064 hope to see you there!
bethskurrie
2021-12-08 22:27
@dabfleming I can't believe it's been this long, but I just remembered I had to tell you that the latest onprem is out with branches and environments support.
tjones
2021-12-09 03:04
Pactflow question: How do I get a ?deploy on Fridays? t-shirt, as modelled by @abubics at the LAST conference?
matt.fellows
2021-12-09 03:41
You break into the not-so-well-hidden tee-shirt box at Level 3, 31 Queen st :stuck_out_tongue:
bethskurrie
2021-12-09 03:43
It was just raised in our fortnightly team meeting that someone has already broken into the t-shirt box :laughing: Outrage!
aforeman
2021-12-09 22:00
:wave: We are live! Join @matt.fellows @bethskurrie and I now over on YouTube https://www.youtube.com/watch?v=5LcW_8lgySs for the next 30 mins.
If you have a question about contract testing with Pact - using the self-hosted Pact Broker or Pactflow, drop it in the thread here or as a comment on the video. :party_parrot:
aforeman
2021-12-10 03:00
*Launching Pactflow's refreshed brand and new mascot!* :pactflow-platypus-slack:
Because it?s ok to deploy on Fridays with Pactflow, we?re setting a new trend from today?launches and announcements will occur on Fridays!
So, for the first Friday launch, the team and I are pleased to officially present our *brand refresh* and a *new member of our team*?plus the chance for you to score Pactflow swag! Read on to find out how?
*Our new look*
Today we officially launch our refreshed branding. We?re sure you?ve noticed some gradual changes of late. Read why we?ve undertaken the refresh and what it means in this blog post by the Product Designer who led the project - Carl Thompson. https://pactflow.io/blog/evolving-the-pactflow-brand
*Help name our new team member and you could score Pactflow swag!*
Meet our new team member?our platypus mascot! Representative of our Australian roots, going forward, you'll see more of this new face around the traps at Pactflow. But, before then, we need your help to select a name! Poppy? Percy? Maybe Parker? :pactflow-platypus-slack:
Select your preference from the shortlist https://dius.typeform.com/to/aQdxO0Lt and provide your email if you want to go into the draw to score brand new Pactflow branded swag! :party_parrot:
We?ll announce the winners of the swag packs, along with the new name for our mascot, next Fri 17 Dec!
przemek.sech296
2021-12-11 02:21
I strongly believe the mascot should be called Plato :pactflow-platypus-slack: :slightly_smiling_face:
przemek.sech296
2021-12-11 02:31
Btw, not emu, not koala, but platypus should the symbol of Australia.
> ??Now look,? said Ridcully. ?I?m a man who knows his ducks, and what you?ve got there is laughable.??
> It?s been said, cruelly, that a platypus is what a duck would look like if it was designed by a committee.
? ?The Last Continent? Terry Pratchett
diva.pant1
2021-12-15 14:31
Hi everyone, I am trying to learn about contract testing and Pact specifically. I went through some articles on the pactflow website. If we use bi-directional flow, than we are using the pactflow right, which is different from Pact?
diva.pant1
2021-12-15 14:34
And when we are using Pact we essentially creating the mocks, right? But in schema testing using OpenAPI spec for example, it just generates the schema. Can we consider it as mock as well?
matt.fellows
2021-12-15 22:27
OAS can be used to generate mocks, there are plenty of OSS tools that can do that if you need?
j3rry.wan9
2021-12-15 23:31
Hello, as a Pactflow customer, I'd like to know if Pactflow is affected by the Log4j vulnerability? https://www.lunasec.io/docs/blog/log4j-zero-day/
tjones
2021-12-15 23:55
By example has several advantages over by-spec - a major problem with using specs for testing is that it is hard to check if an implementation is spec compliant without testing all possible responses
tjones
2021-12-15 23:59
The spec tends to be too general - for example, optional fields are hard to get right. If there are two payload types switched on a `type` field:
```{
type: "hasPropA",
a: { ... some object}
}```
and
```{
type: "hasPropB",
b: { ... some other object}
}```
is often expressed as:
```{
type: "hasPropB" | '"hasPropA",
a: (OPTIONAL) { ... some object}
b: (OPTIONAL) { ... some other object}
}```
which means spec compliant (but wrong) messages like:
```{
type: "hasPropB",
}```
would pass spec based testing
tjones
2021-12-16 00:00
the spec is the grammar. You can use grammar to validate a response, but you can't use the grammar to be sure that all responses have meaning
2021-12-16 01:15
Notification from *Pactflow Contract Testing Platform* :ghost:
diva.pant1
2021-12-16 15:43
When we say Pactflow provides scalable and faster solution over Pact, in theory how does it do that specifically? Making it *faster and scalable*
mike.geeves064
2021-12-16 15:50
Probably not directly what you'd consider addressing scalable, but on the non-tech-functional side I believe they have much more advanced permissions management to make it easier for scaling up teams to provide different access for different roles etc. So easier to follow better security practices and so on with minimal admin overhead as your needs grow :slightly_smiling_face: I'm not sure about what goes on "under the hood" though, it would be interesting to find out more though :slightly_smiling_face:
diva.pant1
2021-12-16 17:21
Yes I am also confused about what it means exactly :thinking_face: @bethskurrie @matt.fellows
matt.fellows
2021-12-16 20:37
Bidirectional is certainly a faster way to do contract testing, but it's also partly that with a hosted solution you can get going with pact right away - no need to setup your own pact broker which can take time and resources away from other activities - like writing pact tests!
matt.fellows
2021-12-16 20:38
With respect to scale, again it's more than we have features like users, teams and RBAC that larger companies need to do Pact at scale.
matt.fellows
2021-12-16 20:38
You can't have everybody in the org with permission to delete contracts - you can imagine the accidents!
matt.fellows
2021-12-16 20:39
Fun story: I accidentally did this on an internal acct (instead of a test one) so the team used RBAC to restrict which apps I could delete so I wouldn't accidental do it in a demo :rolling_on_the_floor_laughing:
mike.geeves064
2021-12-16 20:41
I was looking at e.g. "Designed for scale
Built for large teams, with the choice of a fully managed service or self-hosted deployment." - and taking it to mean around the admin side which is a headache I like to offload where possible :slightly_smiling_face:
I don't think it mentions anything about HA, load scaling etc though :thinking_face:
songqing.gu
2021-12-16 22:58
Hello everyone, just want to confirm one thing, provider contract, looks to me, only applies to Rest API, not messaging API? or how messaging API works for Bi-contract testing?
bethskurrie
2021-12-16 23:00
Correct - for now, we only support OAS+HTTP pact verifications. We will be adding support for more contract types as time goes on.
songqing.gu
2021-12-16 23:42
In case of an application with both consumer and provider roles, when Can-i-deploy end point gets called, what will happen in Broker's logic?
matt.fellows
2021-12-17 03:31
It will ensure it?s safe to deploy into that environment and says something like ?if I deploy here, will I break any of my consumers?? and also ?if I deploy there, do all of my providers satisfy my needs?
aforeman
2021-12-17 05:30
Last Friday we https://pact-foundation.slack.com/archives/CLS16AVEE/p1639105200174800 and gave you the chance to help select our new platypus mascot's name.
_*Today I am thrilled to announce the name of our new platypus mascot..*_
_*PAXTON!*_ :pactflow-platypus-slack:
Paxton the Pactflow platypus is representative of our Aussie roots. From the outset, we've nurtured a positive culture and spirit centered around our vision to ?create a world that doesn?t worry about deployments?.
This bubbly platypus is a reflection of all of those good things wrapped up. Just as it is a positive symbol for customers to identify with, it's a symbol for the Pactflow community to embrace and feel good about. Read more about why we chose a platypus https://pactflow.io/blog/evolving-the-pactflow-brand/.
Thank you to everyone who took the time to vote. The winners of the Pactflow swag packs have been emailed. If you missed out, stay tuned for more ways to score swag in 2022.
diva.pant1
2021-12-18 17:40
hi I am bit confused after buying pactflow plan, how to get started and what are the next steps.
Also https://pactflow.io/pricing/ what does it mean to have 25 users or 10 users. Are these users the developers who will write the code, or anyone who wants to access the pact files??
mike.geeves064
2021-12-18 21:42
When you say get started, do you mean with writing Pacts at all or Pact Flow specifically? Have you looked at https://docs.pactflow.io/ ? "Getting Started with Pactflow" for example :slightly_smiling_face:
mike.geeves064
2021-12-18 21:43
The 10 or 25 users depends on the tier you have signed up for your "Team"
diva.pant1
2021-12-19 00:35
About the Pactflow specifically. Also Pactflow is compatible with Ruby right?
mike.geeves064
2021-12-19 07:44
I'm not sure what you are trying to ask about PactFlow. You use it as a broker for verifying Pacts against, the same as the open source broker, just with additional functionality available
diva.pant1
2021-12-20 01:08
I have few pacts written between consumer and provider, and published them on pact-broker, now I want to start using pactflow, how do I migrate those pacts to pactflow?
mike.geeves064
2021-12-20 01:11
If you've only just started looking at Pact so only have a couple of test examples that's almost certainly going to be easier to just run the consumer/provider tests again, but pointing to your PactFlow account instead of your local broker...
diva.pant1
2021-12-20 02:16
https://pactflow.io/try-for-free/?utm_source=homepage&utm_content=header
In the free version of pactflow, when you say 5 contracts, does it imply to contracts between different consumers and different providers, or 5 contracts with same consumer and different provider?
diva.pant1
2021-12-20 02:18
If we start with free plan of the pactflow can we later migrate to paid version, with our contracts that we created in the free version?
diva.pant1
2021-12-20 02:26
If we have created 5 pacts in our free plan, and if we change our pact, or update the provider, the pact will remain the same. I mean it will be in the 5 pacts limit, right ?
uglyog
2021-12-20 02:27
As long as the provider name does not change. But if needed, you can delete the old one
diva.pant1
2021-12-20 02:31
Ok but in general there is no difference in the features of free plan against the others right, except for the contract limitation, and gmail/ github support?
bethskurrie
2021-12-20 02:35
You'll keep all your current data @diva.pant1 as you upgrade. Yes, the limits on the contracts will be removed.
matt.fellows
2021-12-20 02:41
So it could be either scenario. We don?t limit who the contract is with, just the total number of contracts (essentially integration points)
matt.fellows
2021-12-20 02:42
In terms of users, anybody that is publishing contracts or verifying them would be expected to have their own unique access to Pactflow (user/password or federated) and would count toward your limit
diva.pant1
2021-12-20 17:20
What can be the disadvantage, of using pact-only test over the Pactflow's bi-directional flows. Over the past couple of days I have understood that bottom line is, the pact-only test are reliable, and hence gives us confidence in our tests. But what about the time, cost of running tests etc. I want to understand every other possible way, pact-only approach can be disadvantageous over bi-directional or OAS/spec
matt.fellows
2021-12-20 20:48
Let?s answer here https://pact-foundation.slack.com/archives/C5F4KFKR8/p1640019896375300
aforeman
2021-12-22 02:56
*To our valued customers and community?thank you!*
While 2021 has certainly had some challenging moments, as a toast to our joyous new mascot?Paxton?we want to celebrate the good stuff with you. Here's some highlights:
> Introduction of monthly AMAs https://youtube.com/playlist?list=PLwy9Bnco-IpdgdZce-rXeR7EF2_6T8z7S
> Beginning the conversation about provider-driven / bi-directional contracts https://pactflow.io/blog/bi-directional-contracts/
> Welcoming new team members?Candy, Shuying, Shane & Harsh
> You heard how important OSS is to us https://pactflow.io/blog/why-build-an-open-source-company/
> The release of the inaugural Pactflow Contract Testing Report https://pactflow.io/pactflow-contract-testing-report-2021/
> A bunch of new features to improve collaboration https://pactflow.io/blog/tag/feature/
> A shoutout in the 2021 ThoughtWorks? Tech Radar report https://pactflow.io/blog/pactflow-thoughtworks-tech-radar-2021/
> AND our brand refresh https://pactflow.io/blog/evolving-the-pactflow-brand/
As the team head off for our summer holiday break here in Melbourne :flag-au: from Fri 24 Dec, we wish you a safe and joyful holiday period and look forward to helping you worry less about deployments in 2022 :slightly_smiling_face:
lixiaoyan68
2021-12-23 08:54
Hi team, does pactflow support the solution Grpc + NET ? Where can I find any documents?
matt.fellows
2021-12-23 08:56
Hi Lisa, I replied to your email regarding this - did you not receive it?
bruno855
2022-01-05 06:07
I have renamed *ConsumerCharacter* to *ConsumerCharaters*, and it still shows up in PackFlow, and it seems to me that there is no way to destroy it.
Have you any idea to remove it?
bruno855
2022-01-05 06:09
how to manage unnecessary integrations, I don't see how to destroy them (the button is grayed out)
bruno855
2022-01-05 08:07
Actualy, I'm alone as administrator (I have a free subscription), so one team.
matt.fellows
2022-01-05 08:23
There is a delete integration drop down if you select that integration, that should do it
aubilla
2022-01-05 14:46
Hi :wave: Good 2022 y?all
I?ve got a question regarding API Tokens in Pactflow
Should I create a single Read/Write(CI) token for my whole organization or should I leave each team to manage their own API tokens?
It would much easier for me, as an administrator, to provide an environment variable with the token that everyone can use. Are there any limitations to this approach that I?m not aware of?
bruno855
2022-01-05 16:54
Thank you Matt, but where is the drop down ? I noticed the old ui is correct, there no zombie integration.
bruno855
2022-01-05 17:56
to clarify, i have two integrations I had previously deleted which are still present in the new gui, but which don't exist in the old gui (correct).
bruno855
2022-01-05 18:00
In other words, if we delete an integration in the new gui, an entry remains present in the list of integrations, which is not the case with the old gui.
uglyog
2022-01-05 22:02
Unless you need to control access for individual teams (like disable a particular team's token), I would just use one shared token linked to a service account.
matt.fellows
2022-01-05 22:20
Ron beat me too it, this was identified late last year and is already there. The issue is you can?t delete an integration via the UI if there are no pacts
matt.fellows
2022-01-05 22:22
You can issue an API `DELETE /https://<account>.http://pactflow.io/integrations/provider/<provider>/consumer/<consumer>`
juan.cruz
2022-01-06 03:30
Hi all! :wave:
Does bi-directional testing support anything similar to pending states?
Im trying to add bi-directional testing to my pipeline, I want consumer's pull requests to be checked against the latest provider version, but I don't want those checks to fail if they're hitting endpoints which hasn't been published yet in our provider.
Also, where do we bind consumers with providers in a bi-directional workflow? Can a consumer have multiple providers? Would this require having a pact for each provider?
Thank you for your work btw, Pact is amazing :muscle:!
matt.fellows
2022-01-06 13:17
Hi Juan!
I don?t believe we currently support pending for bi-directional. But I think what you?re asking is a bit different to what pending is - pending is a _provider_ feature, not a consumer one. The consumer?s build should always fail if it?s not compatible with its provider
> Also, where do we bind consumers with providers in a bi-directional workflow? Can a consumer have multiple providers? Would this require having a pact for each provider?
Yes, exactly. The provider can actually publish its contract in advance, but it doesn?t advertise its consumers. When a consumer publishes its contract, an integration is established. Each consumer must have its own pact with a provider, yes, because otherwise we can?t track (down to the field level) what each consumer needs of a provider at any point in time
juan.cruz
2022-01-06 13:38
> The consumer?s build should always fail if it?s not compatible with its provider
In our case, we're maintaining several consumers and providers. When a new feature is requested, it's very likely to be implemented in a consumer before it's provider pact is published (we define the api beforehand, but publishing a pact requires for our integration tests to be written, which takes some days). We want to prevent our teams blocking each other, having consumer pull requests to be blocked until providers have finished publishing their pacts would prevent us from implementing can-i-deploy.
Our consumer team usually creates one pull request for each endpoint, a new feature consisting of 4-5 different pull requests. cross-validation would prevent us from having these partial merges, as it would require for all the api to be implemented at once.
Another use case, when a feature is ready in our consumer, but still under development in our backend, we release it as feature-flagged to our lower level environments (or even prod in some scenarios) to show QA the UI and check if there's any undesired side-effect, while the provider is being completed (i.e, maybe we only managed to implement a POST, GET and summary endpoint, but DELETE an PUT are still pending development)
So, should I not use cross-validation on consumers at all?
matt.fellows
2022-01-07 11:56
> having consumer pull requests to be blocked until providers have finished publishing their pacts would prevent us from implementing can-i-deploy.
that choice is entirely yours. With bi-directional, you actually use `can-i-deploy` to determine compatibility (unlike with regular pact in which case you need to see a valid verification result from a provider for a given pact). But in either case, all we can tell you is what we know about compatibility.
Usually, you would leave a PR to be red until such time that the provider implements the feature, at which point the PR should be green and ready to merge. See this workshop for the recommended flow: https://docs.pactflow.io/docs/workshops/ci-cd/
> . We want to prevent our teams blocking each other,
of course! But also you can?t escape reality, if the consumer is not compatible with a provider yet, then we?ll tell you about it. If you know better, then you?re free to override the results of `can-i-deploy` of course.
> Another use case, when a feature is ready in our consumer, but still under development in our backend, we release it as feature-flagged to our lower level environments (or even prod in some scenarios) to show QA the UI and check if there?s any undesired side-effect, while the provider is being completed (i.e, maybe we only managed to implement a POST, GET and summary endpoint, but DELETE an PUT are still pending development)
see also https://docs.pact.io/pact_broker/tags/#using-tags-with-feature-toggles-instead-of-feature-branches and https://docs.pact.io/pact_broker/tags/#for-feature-toggles-in-the-consumer
juan.cruz
2022-01-07 17:58
> if the consumer is not compatible with a provider yet, then we?ll tell you about it
I'm looking for a middle point where it ignores missing endpoints, but complains if an existing endpoint isn't compliant. I'll try to extend that functionality by downloading the verification results
> see also https://docs.pact.io/pact_broker/tags/#using-tags-with-feature-toggles-instead-of-feature-branches
Sweet, that's exactly what I was needing!
Thank you Matt, much appreciated!
huhuang
2022-01-08 05:41
Hi, I?m trying to install pactflow following the Docker compose example in this Doc: https://docs.pactflow.io/docs/on-premises/docker-compose-example/
huhuang
2022-01-08 05:42
There is error after running the docker compose command.
Any help will be appreciated.
Below is the console output:
```[+] Running 3/3
? Container pact_simplesaml_1 Created 0.1s
? Container pact_postgres_1 Created 0.1s
? Container pact_pactflow_1 Created 0.1s
Attaching to pact_pactflow_1, pact_postgres_1, pact_simplesaml_1
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
simplesaml_1 | [Sat Jan 08 05:08:14.247412 2022] [ssl:warn] [pid 1] AH01906: localhost:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
simplesaml_1 | [Sat Jan 08 05:08:14.247495 2022] [ssl:warn] [pid 1] AH01909: localhost:443:0 server certificate does NOT include an ID which matches the server name
postgres_1 | 2022-01-08 05:08:14.247 UTC [1] LOG: starting PostgreSQL 14.1 (Debian 14.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
postgres_1 | 2022-01-08 05:08:14.248 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2022-01-08 05:08:14.248 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2022-01-08 05:08:14.250 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2022-01-08 05:08:14.253 UTC [29] LOG: database system was shut down at 2022-01-08 05:05:46 UTC
postgres_1 | 2022-01-08 05:08:14.257 UTC [1] LOG: database system is ready to accept connections
simplesaml_1 | [Sat Jan 08 05:08:14.270925 2022] [ssl:warn] [pid 1] AH01906: localhost:443:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
simplesaml_1 | [Sat Jan 08 05:08:14.270956 2022] [ssl:warn] [pid 1] AH01909: localhost:443:0 server certificate does NOT include an ID which matches the server name
simplesaml_1 | [Sat Jan 08 05:08:14.275610 2022] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.10 (Debian) PHP/7.1.13 OpenSSL/1.0.1t configured -- resuming normal operations
simplesaml_1 | [Sat Jan 08 05:08:14.275644 2022] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
pactflow_1 | 2022/01/08 05:08:14 Waiting for:
pactflow_1 | 2022/01/08 05:08:14 Connected to
pactflow_1 | /home/pact_broker_fork/lib/pact_broker/version.rb:2: [BUG] Segmentation fault at 0x00007f792bbf4000
pactflow_1 | ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux-musl]
pactflow_1 |
pactflow_1 | -- Control frame information -----------------------------------------------
pactflow_1 | c:0052 p:---- s:0316 e:000315 CFUNC :RGLoader_load
pactflow_1 | c:0051 p:0193 s:0311 e:000310 TOP /home/pact_broker_fork/lib/pact_broker/version.rb:2 [FINISH]
pactflow_1 | c:0050 p:---- s:0304 e:000303 CFUNC :require
pactflow_1 | c:0049 p:0195 s:0299 e:000298 METHOD /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83
pactflow_1 | c:0048 p:0037 s:0282 E:002238 EVAL /home/pact_broker_fork/pact_broker.gemspec:4 [FINISH]
pactflow_1 | c:0047 p:---- s:0278 e:000277 CFUNC :eval
pactflow_1 | c:0046 p:0022 s:0271 e:000270 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:626
pactflow_1 | c:0045 p:0008 s:0263 e:000262 BLOCK /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:563 [FINISH]
pactflow_1 | c:0044 p:---- s:0260 e:000259 CFUNC :chdir
pactflow_1 | c:0043 p:0018 s:0255 e:000254 BLOCK /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/shared_helpers.rb:52 [FINISH]
pactflow_1 | c:0042 p:---- s:0252 e:000251 CFUNC :synchronize
pactflow_1 | c:0041 p:0016 s:0248 e:000247 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/shared_helpers.rb:51
pactflow_1 | c:0040 p:0061 s:0242 e:000241 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:562
pactflow_1 | c:0039 p:0060 s:0233 e:000230 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:548
pactflow_1 | c:0038 p:0013 s:0224 e:000223 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/path.rb:156
pactflow_1 | c:0037 p:0006 s:0218 e:000217 BLOCK /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/path.rb:171 [FINISH]
pactflow_1 | c:0036 p:---- s:0213 e:000212 CFUNC :each
pactflow_1 | c:0035 p:0077 s:0209 e:000208 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/path.rb:170
pactflow_1 | c:0034 p:0017 s:0203 e:000202 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/path.rb:105
pactflow_1 | c:0033 p:0018 s:0198 e:000197 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/source/path.rb:113
pactflow_1 | c:0032 p:0031 s:0194 e:000192 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:600
pactflow_1 | c:0031 p:0033 s:0186 e:000185 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:585
pactflow_1 | c:0030 p:0006 s:0180 e:000179 BLOCK /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:629 [FINISH]
pactflow_1 | c:0029 p:---- s:0176 e:000175 CFUNC :any?
pactflow_1 | c:0028 p:0008 s:0172 e:000171 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:628
pactflow_1 | c:0027 p:0506 s:0168 e:000167 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:119 [FINISH]
pactflow_1 | c:0026 p:---- s:0156 e:000155 CFUNC :new
pactflow_1 | c:0025 p:0030 s:0145 e:000144 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/dsl.rb:234
pactflow_1 | c:0024 p:0020 s:0139 e:000138 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/dsl.rb:13
pactflow_1 | c:0023 p:0082 s:0131 e:000130 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/definition.rb:34
pactflow_1 | c:0022 p:0048 s:0124 e:000123 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:195
pactflow_1 | c:0021 p:0019 s:0119 e:000118 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:143
pactflow_1 | c:0020 p:0012 s:0114 e:000113 BLOCK /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/setup.rb:20
pactflow_1 | c:0019 p:0012 s:0111 e:000110 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/ui/shell.rb:136
pactflow_1 | c:0018 p:0009 s:0105 e:000104 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/ui/shell.rb:88
pactflow_1 | c:0017 p:0090 s:0100 e:000099 TOP /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/setup.rb:20 [FINISH]
pactflow_1 | c:0016 p:---- s:0096 e:000095 CFUNC :require_relative
pactflow_1 | c:0015 p:0079 s:0091 e:000090 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:61
pactflow_1 | c:0014 p:0071 s:0083 e:000082 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:28
pactflow_1 | c:0013 p:0024 s:0078 e:000077 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli.rb:476
pactflow_1 | c:0012 p:0054 s:0073 e:000072 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27
pactflow_1 | c:0011 p:0040 s:0065 e:000064 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127
pactflow_1 | c:0010 p:0239 s:0058 e:000057 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399
pactflow_1 | c:0009 p:0008 s:0045 e:000044 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli.rb:30
pactflow_1 | c:0008 p:0066 s:0040 e:000039 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476
pactflow_1 | c:0007 p:0008 s:0033 e:000032 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/cli.rb:24
pactflow_1 | c:0006 p:0109 s:0028 e:000027 BLOCK /usr/local/bundle/gems/bundler-2.1.4/exe/bundle:46
pactflow_1 | c:0005 p:0002 s:0022 e:000021 METHOD /usr/local/bundle/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123
pactflow_1 | c:0004 p:0111 s:0017 E:001ae8 TOP /usr/local/bundle/gems/bundler-2.1.4/exe/bundle:34 [FINISH]
pactflow_1 | c:0003 p:---- s:0013 e:000012 CFUNC :load
pactflow_1 | c:0002 p:0112 s:0008 E:001f90 EVAL /usr/local/bundle/bin/bundle:23 [FINISH]
pactflow_1 | c:0001 p:0000 s:0003 E:002550 (none) [FINISH]
pactflow_1 |
pactflow_1 | -- Ruby level backtrace information ----------------------------------------```
huhuang
2022-01-08 05:43
The end of the console output is :
```pactflow_1 | 7f792f771000-7f792f772000 r--p 003af000 fe:01 568323 /usr/local/lib/libruby.so.2.7.3
pactflow_1 | 7f792f772000-7f792f77b000 r--p 003af000 fe:01 568323 /usr/local/lib/libruby.so.2.7.3
pactflow_1 | 7f792f77b000-7f792f77c000 rw-p 003b8000 fe:01 568323 /usr/local/lib/libruby.so.2.7.3
pactflow_1 | 7f792f77c000-7f792f795000 rw-p 00000000 00:00 0
pactflow_1 | 7f792f795000-7f792f7aa000 r--p 00000000 fe:01 2927981 /lib/ld-musl-x86_64.so.1
pactflow_1 | 7f792f7aa000-7f792f7f2000 r-xp 00015000 fe:01 2927981 /lib/ld-musl-x86_64.so.1
pactflow_1 | 7f792f7f2000-7f792f828000 r--p 0005d000 fe:01 2927981 /lib/ld-musl-x86_64.so.1
pactflow_1 | 7f792f828000-7f792f829000 r--p 00092000 fe:01 2927981 /lib/ld-musl-x86_64.so.1
pactflow_1 | 7f792f829000-7f792f82a000 rw-p 00093000 fe:01 2927981 /lib/ld-musl-x86_64.so.1
pactflow_1 | 7f792f82a000-7f792f82d000 rw-p 00000000 00:00 0
pactflow_1 | 7ffc9419f000-7ffc9499e000 rw-p 00000000 00:00 0 [stack]
pactflow_1 | 7ffc949b2000-7ffc949b5000 r--p 00000000 00:00 0 [vvar]
pactflow_1 | 7ffc949b5000-7ffc949b7000 r-xp 00000000 00:00 0 [vdso]
pactflow_1 | ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
pactflow_1 |
pactflow_1 |
pactflow_1 | Aborted
pactflow_1 | 2022/01/08 05:08:14 Command exited with error: exit status 134
pact_pactflow_1 exited with code 134```
matt.fellows
2022-01-08 06:13
I've seen that before (I think) when the license is mounted as a directory and not a file
huhuang
2022-01-08 06:25
Hi Matt, thanks again for helping me when you?re in vacation. I posted here but didn?t expect any response till next week. Sorry again for bothering you in your vacation.
And yes you?re right it?s about the license file. I just double checked it and found the path I mounted has a typo :sweat_smile:
Now the localhost:8080 is up and running . Thanks a lot!
matt.fellows
2022-01-08 06:46
Don't feel sorry, it's exactly why this channel / workspace exists. I just happen to be out and about and opened it up :sunglasses:
matt.fellows
2022-01-08 22:38
I believe that?s already set, but I?ll follow up tomorrow when I?m back at work
lixiaoyan68
2022-01-09 12:36
@matt.fellows Hi Matt, I read the document about plugins. Several questions, and could you kindly answer it?
1. Do we have an plugin example for gRPC under the new plugin framework?
2. How do the new plugins work with existed pact-mock-service, pact-verifier? In the consumer side, the requests to provider go to plugin at first (eg, gRPC/protobuf -> REST/json), and then go to pact-mock-service?
mch
2022-01-09 17:45
Hi, are there any plans to make main branch visible and settable from the web interface?
matt.fellows
2022-01-09 23:42
Hi Lisa!
1. The example gRPC project is in the plugins repo. We don?t officially support gRPC yet, but we do support protobufs. see also the thread on that in #protobufs
2. The plugins framework is completely separate to the pact-mock-service (Ruby) tool. The new https://docs.pact.io/implementation_guides/cli/#native-binary-new verifier supports plugins.
Do note that the plugins framework is still quite new, and at last check it was still in an early beta stage.
cc: @uglyog as I?m at least 2-3 weeks behind where things are at.
diva.pant1
2022-01-10 14:08
Which template should I use to generate a project in Pactflow. Not sure which template to choose, need some guidance
huhuang
2022-01-13 03:29
Hi, I met one issue while setting up the pactflow in our private cloud. The saml login is keeping saying ?redirected you too many times?
huhuang
2022-01-13 03:35
Any clue what could be the root cause?
So far I?ve tried below things:
1. Set up pactflow using the docker compose in my local Mac and it works!
2. Set up pactflow in private cloud (Kubernetes based). I met the above issue.
3. In the private cloud, set the pactflow Env PACTFLOW_SAML_AUTH_ENABLED to false and remove all the SAML Env Vars. It leads to the pactflow login page without any login method provided. Nothing I can do from that page.
4. Use the PACTFLOW_BASIC_AUTH_ENABLED. But when I tried to get the api token, the page has issue like below:
matt.fellows
2022-01-13 04:46
yes, the basic auth is really just to test that the basic API access works. It?s not designed for other use (API tokens and several other key Pactflow functionality is associated with users, and when basic auth is enabled there are no users)
matt.fellows
2022-01-13 04:46
if (1) works but not (2), that tells me you have a configuration issue (which you probably already know!)
matt.fellows
2022-01-13 04:47
I?d be looking first at your log files, they might tell you what?s going on.
Usually when I hear Kubernetes, I think of proxies and other things.
matt.fellows
2022-01-13 04:48
Because the SAML redirect happens via a browser, you will need to look at the network tab to see what the SAML response looks like.
matt.fellows
2022-01-13 04:49
If you can share your Pactflow configuration details with us in a support ticket () and share the logs for a complete start to finish failed SAML login, that should help us start to help you
huhuang
2022-01-13 05:21
Thanks a lot @matt.fellows
Is there a ticket system I should go to file a ticket? or Just email the details to ?
cyrus.devnomad
2022-01-14 13:51
Hi
I have configured a webhook in Pactflow which triggers a pipeline in Azure DevOps. This webhook was working properly for weeks. Yesterday my Azure token expired and since yesterday I am unable to make that same webhook working again.
I have created a new token in azure the same way I did before, converted the token to Base64 and entered it in the Pactflow webhook form.
After that when I click on [Test] button of the webhook form, I get the following results:
```authorization: [REDACTED]
...
HTTP/1.1 302 Found
...
Webhook execution failed```
I also get an empty page with the title "Object moved" returned.
I've sent the same query with the same parameters from Postman from an internal and an external network and in both cases I get the expected correct response.
Any ideas?
cyrus.devnomad
2022-01-14 16:55
Aha, I found the cause of the problem. Here is how I got into this problem and what was its resolution for others who might experience the same problem:
I tried to configure the webhook in Pactflow for the pact verification pipeline in Azure DevOps according to the following page:
https://docs.pact.io/pact_broker/webhooks/template_library/
I originally (before my token expired) was able to make the webhook work with following configuration:
Team: None
Description: Pact changed webhook for ApiClient and ProductService
Consumer: ALL
Provider: ProductService
Events: [Contract published with changed content or tags] selected
URL: https://dev.azure.com/mycompany/myorganisation/_apis/build/builds?api-version=6.0
Header:
Content-Type: application/json
Accept: application/json
Authorization: Basic _TokenConvertedToBase64_
Body:
{
"definition": {
"id": 63
},
"sourceBranch": "refs/heads/master",
"parameters": "{\"pact_url\": \"${pactbroker.pactUrl}\"}"
}
Basic auth Username: _something, must not be empty_
Basic auth Password_: TokenFromAzure_
Enabled: checked
But yesterday the webhook stopped working and I found out that my Azure token had expired.
So I created a new Azure token and came back to enter it in the Pactflow webhook. In the meantime
I had forgotten that the fields [Basic auth Username] and [Basic auth Password] were necessary and believed
to make the webhook work again by entering the new Azure token converted to Base64 in the [Authorization] field. This led then to the error I mentioned above.
After some debugging I figured out that the Authorization field was not required at all and that the fields
that really matter were [Basic auth Username] and [Basic auth Password]. So then I just deleted the Authorization field from the header, entered something for example "Username" into username field and the token from Azure into the password field. After that the webhook started to work again.
matt.fellows
2022-01-15 02:45
by completing the basic auth username/password fields, that would essentially replace the `Authorization` header. I suspect because you just base64 encoded the token (not the base64 encoded `username:token` ) that the header overrode your other settings, and it was incorrect
matt.fellows
2022-01-15 02:46
TL;DR - basic auth is usually of the format `Authorization: Basic <base64(username:password)>` but you were probably doing `Authorization: Basic <base64(password)>` type thing
matt.fellows
2022-01-15 02:47
the form probably shouldn?t allow both username/password to be set _and_ the Authorization header.
cyrus.devnomad
2022-01-17 06:45
Thank you for your clarifications, now I understand better what I was doing wrong.
lambat.ishanya
2022-01-18 05:15
HI Team Just want to know how we can achieve spec validation for pact to be made via pact flow is their any provision to verify schema in pact flow?
james_fraser
2022-01-27 14:44
Hey folks :wave: got a quick question about the Pactflow API - say I have a provider with two consumers
I can use /pacts/provider/<provider>/consumer/<consumer>/latest/ to get the latest interactions for each consumer, but is there an endpoint to get the interactions for both consumers at the same time?
I'm setting up a pipeline and just wondering if I can automatically test against any newly added consumers or whether I need to manually add a new step for each new consumer
lio.lunesu
2022-01-27 17:36
~In our CI I?m seeing 401 Unauthorized from http://pactflow.io. Compared the token and it?s correct. Any others seeing this?~
```[2022-01-27T17:33:18.481Z] > Task :server:pactPublish FAILED
[2022-01-27T17:33:18.481Z] Could not fetch the root HAL document
[2022-01-27T17:33:18.481Z] au.com.dius.pact.core.pactbroker.RequestFailedException: Request to path '/' failed with response 'HTTP/1.1 401 Unauthorized'```
~This job with same token worked fine yesterday.~
matt.fellows
2022-01-27 23:27
Hi James, we?ll connect via the support ticket. I think you?re currently doing it pact-by-pact, but really you?ll want to dynamically configure the pacts that get verified
james_fraser
2022-01-28 11:35
Hi Matt, yup support clarified exactly what I needed - I didn't realise that was an option so thanks very much for your help :smile:
matt.fellows
2022-01-28 11:37
I don?t think we have a Python tutorial, and it seems the Python docs aren?t very clear, but you might find running through a workshop in another language might help with concepts
2022-01-28 11:37
Here are a number of useful hands-on labs that teach all of the key concepts https://docs.pact.io/implementation_guides/workshops
james_fraser
2022-01-28 11:39
Yeah I kinda pieced it together from a few samples so I've possibly just missed the intended usage somewhere - I'll take a look at those workshops and see if there's anything else I should be trying :thumbsup: cheers
j3rry.wan9
2022-01-31 01:06
Hi team, is there an easy way to figure out what the expiration date is for my current Business 250 Annual Plan?
bethskurrie
2022-01-31 01:07
Your administrator or finance person can log into ChargeBee, where this information should be available.
bethskurrie
2022-01-31 01:08
Usually the person with the ChargeBee account was the person who signed up, unless you have deliberately changed it.
gueorgui
2022-01-31 11:13
When uploading pact verification results to PactFlow (from a ruby provider test ran on CircleCI), I?m getting a `503 Service Temporarily Unavailable` error, is anyone else experiencing the same?
matt.fellows
2022-01-31 12:26
Can you please send a request to with your account details and as much information as possible?
bogdanm.rusu
2022-01-31 15:03
Hi team, we have tried to deploy Pactflow on-prem inside an EKS cluster with helm-chart but we receive the following message in the container log "Could not locate Gemfile or .bundle/ directory ". Do you know what this might be about?
matt.fellows
2022-02-01 01:42
We saw the support ticket and will respond there. It's harder to say without other info (eg the helm chat)
gueorgui
2022-02-01 06:38
Hi @matt.fellows, sorry for not getting back to you earlier. It?s still happening, and it turns out it doesn?t just happen on CI, but locally as well.
Basically, the interactions get verified, and then when the pact broker tries to upload results to pactflow, it seems like it?s getting back an HTML error page for a 503 error instead of the expected JSON:
```INFO: Creating xxxxx version 10b6c0ea88d3a3f354341dc4fac6d268b3b6478d
with branch "gt/xxxxx
"
/Users/gueorgui/.rbenv/versions/2.6.8/lib/ruby/gems/2.6.0/gems/json-2.6.1/lib/json/c (JSON::ParserError)se': 859: unexpected token at '<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
</body>
</html>
'
from /Users/gueorgui/.rbenv/versions/2.6.8/lib/ruby/gems/2.6.0/gems/json-2.6.1/lib/json/common.rb:216:in `parse'
from /Users/gueorgui/.rbenv/versions/2.6.8/lib/ruby/gems/2.6.0/gems/pact-1.61.0/lib/pact/hal/http_client.rb:85:in `body'```
matt.fellows
2022-02-01 06:40
If you can enable debug/verbose logs on that publish we'll find out for sure
gueorgui
2022-02-01 06:58
Good call, let me see. Doesn?t seem like a corporate proxy though, as it happens on both my machine (on a random private wifi in Taiwan) and on Circle CI (in what I assume is an AWS datacentre)
ruud.von.faber
2022-02-09 09:18
Hi all, I'm currently in the process of evaluating http://Pact.io/Pactflow and I looking for the resource specs/installation requirements, CPU, RAM, Storage, ...? Can anyone point me in the right direction? Thanks in advance.
sudhanshu.testing
2022-02-11 05:17
Hello All, I started evaluating PACT for our Contract Test and preferring JAVA as language. I just wanted to understand if We really need to write the Provider script in Developer?s repo (Service Repo). OR we can have a mechanism to implement it on Contract Test Repo itself ?
abubics
2022-02-11 05:21
The intent with Pact (as contract by example, and inline verification) is to prove that things work by exercising real implementations.
You write unit-style tests on the consumer side (to make sure it works as you expect), and verify those realistic recorded interactions against the real provider code (to make sure the provider also works as you expect).
You can do that via a separate repo (just as you can program whatever you want, given infinite time & resources), but you lose significant reasoning power.
To put it simply: if you're not testing your actual provider code, how do you know it's working?
sudhanshu.testing
2022-02-11 07:18
makes sense Boris. So ideally it would be recommended to put Consumer tests in Consumer Service repo and Provider tests in Provider repo to make it more significant. And Validation would happen through PACT broker.
matt.fellows
2022-02-11 11:07
the pact broker doesn?t do validation (Pact does that), it?s more of the collaboration part where you share the contracts aronud
charles.kuo
2022-02-11 18:33
@matt.fellows
~I am new with Pact and Pactflow~
~Just watched youtube video for Demo: Pact + Cypress integration http://youtube.com/watch?v=jTuuYMFJBBQ~
~I had Cypress API test suite against deployed API environment and was wondering if I can convert it to Pact + Cypress integration pact test?~
~Is there a repo for that demo?~
Link is in article - thanks
varnika.singh
2022-02-15 09:13
Hi, I have just started working on pactflow. is it possible to enable single sign on for pactflow?
matt.fellows
2022-02-15 09:26
All accounts have GitHub/Google which can be configured via the UI, business plans have SAML2.0.
matt.fellows
2022-02-15 09:27
diva.pant1
2022-02-15 20:15
can the free Pactflow account be accessed by different users in the company
athaper
2022-02-17 10:43
Hii , just had a question
If the pact contents didn?t change and the consumer updated its code and ran tests , then will the `can-i-deploy` module let us deploy ? and will the provider still need to run tests again and verify the pact from their side before consumer can deploy ?
matt.fellows
2022-02-17 10:46
No, if the pact contents haven?t changed between consumer versions, the contract is ?pre-verified? and `can-i-deploy` should continue to pass
fushinoryuu
2022-02-17 17:38
Hello everyone, I am running into an issue that I hope someone can help me with.
My company uses Octopus Deploy but they don't have docker installed in the worker agents, so I've had to create a powershell script instead of `can-i-deploy`.
Locally, when I run the script, everything works and I get a response from pactflow. But when the requests runs on Octopus, it comes back with a 404. I am printing out the url that is being hit, and it works fine on Postman.
Has anyone ran into this issue? Or is there logs on Pactflow that I can look at?
fushinoryuu
2022-02-17 19:35
```param(
[string]$pactBroker = "https://ess.pactflow.io",
[Parameter(Mandatory = $true)]
[string]$pacticipant,
[Parameter(Mandatory = $true)]
[string]$toEnv,
[Parameter(Mandatory = $true)]
[string]$pactBrokerToken,
[string]$dataDogEventsUrl = "https://api.datadoghq.com/api/v1/events",
[Parameter(Mandatory = $true)]
[string]$dataDogToken
)
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$PSVersionTable
$firstLine = Get-Content "..\..\ReleaseNotes.md" -First 1
$branchName = $firstLine -Replace "Branch: ", ""
$canIDeployUrl = "$pactBroker/pacticipants/:pacticipant/latest-version/:tag/can-i-deploy/to/:toEnv"
$pactFlowHeaders = @{ "Content-Type" = "application/json"; "Accept" = "application/hal+json"; 'Authorization' = 'Bearer ' + $pactBrokerToken; }
$encodedPacticipant = [Uri]::EscapeDataString($pacticipant)
$encodedTag = [Uri]::EscapeDataString($branchName)
$encodedToEnv = [Uri]::EscapeDataString($toEnv)
$formattedCanIDeployUrl = (($canIDeployUrl -Replace ":pacticipant", $encodedPacticipant) -Replace ":tag", $encodedTag) -Replace ":toEnv", $encodedToEnv
Write-Host "`nChecking if '$pacticipant' with tag '$branchName' can be deployed to '$toEnv':"
Write-Host "`t$formattedCanIDeployUrl`n"
$pactFlowResponse = Invoke-RestMethod $formattedCanIDeployUrl -Method "GET" -Headers $pactFlowHeaders
Write-Host (ConvertTo-Json $pactFlowResponse -Depth 10)
if (($null -eq $pactFlowResponse.summary.deployable) -or ($true -eq $pactFlowResponse.summary.deployable)) {
exit 0
}
else {
$dataDogEvent = @{}
$dataDogEvent.alert_type = "error"
$dataDogEvent.title = "$pacticipant Failed to Deploy to $toEnv"
$dataDogEvent.text = $pactFlowResponse.summary.reason
$dataDogEvent.tags = [System.Collections.ArrayList]@()
$dataDogEvent.tags.Add("documents_service")
$dataDogEvent.tags.Add("environment:$toEnv")
$dataDogHeaders = @{ "Content-Type" = "application/json"; "Accept" = "application/json"; 'DD-API-KEY' = $dataDogToken; }
Invoke-RestMethod $dataDogEventsUrl -Method "POST" -Headers $dataDogHeaders -Body (ConvertTo-Json $dataDogEvent)
exit 1
}```
fushinoryuu
2022-02-17 19:47
These are the logs from Octopus:
```Name Value
---- -----
PSVersion 5.1.14409.1018
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.14409.1018
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
Checking if 'Documents Service' with tag 'task/SRVC-233-pactflow-deploying' can be deployed to 'devsb':
https://ess.pactflow.io/pacticipants/Documents%20Service/latest-version/task%2FSRVC-233-pactflow-deploying/can-i-deploy/to/devsb
InvalidOperation: The remote server returned an error: (404) Not Found.
At C:\Octopus\Work\20220215231229-1676071-4608\Infrastructure\Scripts\CanIDeploy-Octopus.ps1:33 char:21
+ ... wResponse = Invoke-RestMethod $formattedCanIDeployUrl -Method "GET" - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
at <ScriptBlock>, C:\Octopus\Work\20220215231229-1676071-4608\Infrastructure\Scripts\CanIDeploy-Octopus.ps1: line 33
at <ScriptBlock>, <No file>: line 1
at <ScriptBlock>, C:\Octopus\Work\20220215231229-1676071-4608\Infrastructure\Scripts\Octopus.FunctionAppenderContext.ps1: line 185
at <ScriptBlock>, C:\Octopus\Work\20220215231229-1676071-4608\Infrastructure\Scripts\Bootstrap.Octopus.FunctionAppenderContext.ps1: line 3705
at <ScriptBlock>, <No file>: line 1
at <ScriptBlock>, <No file>: line 1
The remote script failed with exit code 1
The action Can I Deploy on OctopusServer failed ```
nouri.tawfik
2022-02-17 20:42
Sorry I used another solution.
My script download the pact-cli from GitHub and use it as cli like docker ....
matt.fellows
2022-02-18 00:21
Thanks Tawfik - yes, we recommend you just use our CLI tools directly in this case. You don?t have to use docker
matt.fellows
2022-02-18 05:51
:zap:*New feature coming soon:* *see the value of contract testing faster* :zap:
We heard that you wanted to get more of your team involved with contract testing and were seeking ways to see the value and ROI sooner?well, that?s about to become a reality! :pactflow-platypus-slack:
In two weeks time, *Pactflow launches Bi-Directional Contract Testing* which will enable new modes of contract capture and compliance, a broader set of use cases, and a simplified developer experience.
With this new Pactflow-exclusive feature, turn your existing tools into a powerful contract testing solution accelerating adoption and rapidly improving time-to-value and ROI.
Read more about what Bi-Directional Contract Testing means and register to be one of the first to hear when this game-changing feature is live https://pactflow.io/bi-directional-contract-testing?&utm_source=slack&utm_medium=social&utm_campaign=bdct-pre-launch&utm_content=organic_cta&utm_term=18-Feb.
ringo
2022-02-24 07:39
Hello, I have a pricing question. If I signup for a Business subscription for 25 users now. If I have to increase the amount of users mid-year, how is the billing handled? Do I have to go to 50 users immediately, or can I pick an amount of users in between like 35?
https://pactflow.io/pricing/
matt.fellows
2022-02-24 11:08
Hi Ringo! We currently only support tiered based pricing, but are investigating the possibility of per-user based
nathan.deamer
2022-02-25 12:05
Hi All,
Trying to get the pact ?Can I Deploy? badges into my README. Do I have to use ?tags? for it to work?
(I am using branches but not tags when I publish the pacts: `pact-broker publish build/pacts --consumer-app-version=$(git rev-parse --short HEAD) --branch=$(git rev-parse --abbrev-ref HEAD)`
bethskurrie
2022-02-28 02:43
@nathan.deamer I was actually adding the branch/environment version of this endpoint the other day.
aforeman
2022-02-28 23:59
_*Save the date*_ :point_right: _*to find out how a US bank has reduced its reliance on long and expensive manual end-to-end tests!*_
Join @matt.fellows from the Pactflow team and @francisco.moreno from our consulting partner, https://sngular.com/ to hear how we worked together to support a US bank to transform their application testing strategy for an application modernization initiative, reducing their reliance on long and expensive manual end-to-end tests with contract testing.
This is a live global event, please register https://bit.ly/Pactflow-SNGULAR to access the link to join. If the time is not friendly, we can share the recording with you. Hope to see you there! :raised_hands: :pactflow-platypus-slack:
matt.fellows
2022-03-01 09:38
:wave: Introducing Pactflow?s new Developer Advocate - @yousafn !
Our community ? which incorporates the Pact contract testing community and our commercial product Pactflow ? is the most important thing to us (https://pactflow.io/blog/why-build-an-open-source-company) and so it was natural, as we grow, that we create a role focussed outward on that community to engage, lean into and nurture it. We were after someone with passion, somebody with humility but also a problem solver, and somebody who knew not just Pact the technology, but how to make it work in organisations and the complex environments in which it exists.
This sounds like a :unicorn_face: , but it?s all true and we are so lucky to have him join the team.
You may have seen him around?here on Slack, through testing MeetUps, the Ministry of Testing, his past employer and current Pactflow consulting partner and InfinityWorks?he?s been an invaluable contributor to our community, as a maintainer of projects like `jest-pact` and `msw-pact` and through contributions to various parts of our ecosystem including the frameworks themselves and our documentation.
As our our ?community shepherd?, Yousaf is a voice for the customer ? you ? and will be focussed on reducing barriers to adoption, fostering a healthy contributor community and building more value in our community.
For a great read on Yousaf joining the team and how we got here, I?d encourage you to read this delightful https://blog.you54f.com/2022/02/18/standing-on-the-shoulder-of-giants/, which should serve to inspire anybody on the possibilities that emerge if you keep your mind open.
Please join us in welcoming Yousaf to the Pactflow team :pactflow-platypus-slack:
yousafn
2022-03-01 11:48
Hey, thanks for the warm welcome Matt.
I am so excited to be here, and to be able to dedicate my full time to Pact and it's incredible community.
If you have any questions, thoughts, wishes or just want to vent or chat, feel free to reach out, my door is always open.
james.troughton
2022-03-02 11:58
Hello everyone :wave: I have just signed up for a demo pactflow account. I am quite keen to see how the automatic stub generation works. I am trying to access the stub that is generated for the example integration `A pact between Example App and Example API` that has been given. I am making a request to
```curl --location --request GET 'https://{myDomain}.http://pactflow.io/pacts/provider/Example API/consumer/Example App/latest/stub/alligators/Mary' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer {token}'```
But I am getting a 400 Bad Request back, what am I doing wrong?
yousafn
2022-03-02 12:43
Hi James, thanks for signing up,
I don't mind having a look with you. Did you copy the stub url from an option in the Pactflow ui?
yousafn
2022-03-02 12:46
Your curl snippet looks ok, although I'd normally encode the spaces in the url, and not sure if you need the `/alligators/mary`
I've just tried it against mine to retrieve a pact and it worked fine
```curl --location --request GET 'https://you54f.pactflow.io/pacts/provider/example%20provider/consumer/example%20consumer/version/LOCAL_DEV' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer xyz'```
Will test it with the `/stub/` endpoint and let you know
james.troughton
2022-03-02 13:13
Hey @yousafn I built the URL from this blog post https://pactflow.io/blog/hosted-stubs/
yousafn
2022-03-02 13:16
Great thanks for that, I've just got it working, using the detail from the latest docs https://docs.pactflow.io/docs/stubs
```curl -v --location --request GET 'https://you54f.pactflow.io/pacts/provider/example%20provider/consumer/example%20consumer/latest/stub/dogs' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer xyz'```
Let me go through the post above and run through it :slightly_smiling_face:
james.troughton
2022-03-02 13:22
yeah sure
```{
"consumer": {
"name": "Example App"
},
"provider": {
"name": "Example API"
},
"interactions": [{
"_id": "ef69ceef4d7fb82af014da950a3d9028a905c4de",
"description": "a request for an alligator",
"providerState": "there is an alligator named Mary",
"request": {
"method": "get",
"path": "/alligators/Mary",
"headers": {
"Accept": "application/json"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json;charset=utf-8"
},
"body": {
"name": "Mary"
},
"matchingRules": {
"$.body.name": {
"match": "type"
}
}
}
}, {
"_id": "4b3c23c364f420e1d1296d56a47695de0428d0af",
"description": "a request for an alligator",
"providerState": "there is not an alligator named Mary",
"request": {
"method": "get",
"path": "/alligators/Mary",
"headers": {
"Accept": "application/json"
}
},
"response": {
"status": 404,
"headers": {
}
}
}, {
"_id": "e57e7ac251a8bd078fcb81cad1e577cbafebcef5",
"description": "a request for an alligator",
"providerState": "an error occurs retrieving an alligator",
"request": {
"method": "get",
"path": "/alligators/Mary",
"headers": {
"Accept": "application/json"
}
},
"response": {
"status": 500,
"headers": {
"Content-Type": "application/json;charset=utf-8"
},
"body": {
"error": "Argh!!!"
}
}
}],
"metadata": {
"pactSpecification": {
"version": "2.0.0"
}
},
"createdAt": "2022-03-01T21:09:21+00:00"
}```
agustina.bosso
2022-03-03 10:44
Hello eveyone!! I am trying to login to Pactflow and I cannot change my password because email is not sent. Any ideas?
agustina.bosso
2022-03-03 10:46
Here is the URL for your shiny new toy (you can log in with your Github credentials) --> I tryed with my github credentials but is not working
bethskurrie
2022-03-04 02:16
@james.troughton if you're still stuck on this, send us an email to and we can have a look at the logs.
matt.fellows
2022-03-04 04:17
:rocket: *[New Feature] Bi-Directional Contract Testing is LIVE!* :rocket:
To help more teams get value from contract testing faster, we?re excited to announce the launch of this new, game-changing feature!
Bi-Directional Contract Testing enables new modes of contract capture and compliance, a broader set of use cases, and a simplified developer experience.
:zap:? Turn your favourite tools into a powerful contract-testing solution
:zap:? Everyone, from developers to testers can get involved
:zap:? See the value and return on investment of contract testing sooner
Find out more and get started http://ow.ly/xykg50I9O97
https://youtu.be/8_abWl1N32Q
diva.pant1
2022-03-08 10:53
How do we ensure which team is responsible for fixing tests?
By putting consumer version and tags is the way to ensure accountability to a particular team responsible for a particular consumer or provider test?
Otherwise it will be a chaos in the organization. Let me know how to manage that through Pactflow
matt.fellows
2022-03-08 21:36
What do you mean @diva.pant1? Pending pacts is the way to prevent consumers from putting bad expectations in the contract, so it doesn?t fail the provider build. In those instances, the consumer will not be able to release (as `can-i-deploy` will fail), so they are responsible for fixing.
In the case a provider makes a backwards incompatible change, it?s their responsibility.
bethskurrie
2022-03-08 22:13
The CI/CD workshop would be a good way to understand our recommended workflow https://docs.pactflow.io/docs/workshops/ci-cd
andrevdrodrigues
2022-03-09 16:04
Hi @bethskurrie, is possible to inject behavior in these hosted stubs?
i.g. in same cases is I need that some stubs have different behaviors depending on the input parameters or the data exchange between services or even to keep state between endpoints.
I know that pactflow doesn't necessarily have that scope. However, I saw this post about hosted stubs and was thinking about this possibility.
william.pritchard
2022-03-11 11:58
Is it possible to have a Consumer C1 which has a relationship with a Provider P1 but P1 is also a consumer and C1 is its provider. e.g. in `can-i-deploy` it would look something like this.
```CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS?
--------------------------|-----------|----------------------------|-----------|---------
C1. | 1.0.0. | P1. | 2.0.0 | true
P1. | 2.0.0 | C1. | 1.0.0. | true ```
I?ve looked through some documentation but couldn?t find any reference to it.
matt.fellows
2022-03-11 12:18
I believe you can do it, yes. You will need to ensure you only ever make changes in one direction of the relationship at a time, otherwise can-i-deploy will very correctly stop you from deploying. If you have changes in both directions that each depend on the other, you will be in the situation where you must deploy both applications at the same time, at which stage you would probably be better off if both services were part of a single application. Bi-directional dependencies tend to cause a lot of issues in both testing and deployments!
matt.fellows
2022-03-11 12:19
The main use cases I?ve seen for this is when there is a ?callback? API situation, where P1 notifies C1 that it has ?done the thing?. What?s the scenario you have?
alexstout009
2022-03-14 14:35
Hey we are looking into pactflow for our contract testing. We have a workflow where if a consumer uploads a new contract pactflow will trigger a web hook to our other components workflow to verify the pacts however it still means that the first run of the workflow fails and the 2nd run passes given the other component has run the check to validate the consumer contract. Is there a way using the can-i-deploy to wait for the contracts to be verified without having to have a case where it always fails the 1st time until the provider has verified?
matt.fellows
2022-03-14 23:49
I think you want these properties:
``` [--retry-while-unknown=TIMES] # The number of times to retry while there is an unknown verification result (ie. the provider verification is likely still running)
# Default: 0
[--retry-interval=SECONDS] # The time between retries in seconds. Use in conjuction with --retry-while-unknown
# Default: 10```
yousafn
2022-03-21 13:13
Hey hey,
Just a small thing to shout about.
On Friday we added a search component to the Pactflow docs page
https://docs.pactflow.io/
So hopefully finding information should be a little bit easier, feedback welcome. The indexing isn't 100% so some of the results aren't the most useful, but it's a good start
james.troughton
2022-03-22 12:37
Hey, does anyone have a good example of bi directional contract testing? One thing I am struggling to understand is the tests that are run against the provider contract?
yousafn
2022-03-22 14:26
Hi we have example repos on the website, you can see the link for use cases for oas specs in the table
https://docs.pactflow.io/docs/examples#providers
From the provider side, the provider would provide their own functional testing against their oas spec and service, the results of which are uploaded to Pactflow along with the oas spec.
The validation in Pactflow will take place against the uploaded oas spec and a consumers pact file when they call can-i-deploy to ensure that the consumers pacts are valid against the oas spec against whichever environment the consumer is targeting.
james.troughton
2022-03-22 14:31
Ahh nice one thanks @yousafn so for this example
https://github.com/pactflow/example-provider-restassured/blob/master/src/test/java/com/example/springboot/ProductsAPITest.java
A mock is created from the `oas/swagger.yml` file and then those `@Tests` run against it?
james.troughton
2022-03-22 15:01
Does the created consumer contract in bi directional testing still get turned into a stub automatically with pact flow?
aforeman
2022-03-22 22:32
*How does a bank reduce its reliance on long and expensive manual end-to-end tests?* :thinking_face:
Join @matt.fellows @yousafnand I in two days week to hear how we worked with @francisco.moreno and his team at https://www.sngular.com/ (a Pactflow consulting partner) to support the bank to transform their application testing strategy for an application modernization initiative to reduce their reliance on long and expensive manual end-to-end tests with contract testing.
_*The end result: the bank reduced their manual testing costs by 70%!*_ :tada:
This is a live global event, please register https://www.eventbrite.es/e/reducing-the-cost-of-testing-in-app-modernization-a-banking-case-study-tickets-272559361387?utm_campaign=pact+flow+event+link to access the link to join or, if the time is not friendly, we can share the recording with you.
Hope to see you there!
jonathan.fortunati
2022-03-24 23:42
Hi folks! I am trying to learn more about Pactflow, and am going through the CI/CD workshop in Pactflow university, but am getting stuck at the section of "how to break everything." .... I can't break everything! I have my consumer set up so that it expects an extra field that my provider does not provide, and I have run my consumer tests and published my pact to Pactflow. This triggers the provider workflow verify-changed-pact, via webhook which fails as expected. However, when I run the provider build workflow manually as described in the "Run the provider build" section, the build passes just fine, which I wasn't expecting based on the next page, which has me switch the enablePending flag to true. Am I misunderstanding when things should fail? Thanks!
bethskurrie
2022-03-24 23:46
what are the selectors configured in the provider tests @jonathan.fortunati?
aforeman
2022-03-24 23:52
:pactflow-platypus-slack: *Missed today's event? The recording is now available for you to watch on https://www.youtube.com/watch?v=LyVSmRtwFkc* :eyes:
Thank you to the team at Sngular? @francisco.moreno @alejandro.pena and Juan?and my colleagues @matt.fellows and @yousafn for sharing their knowledge and experience on the topic of contract testing and how it fits with app modernisation projects on banking and financial services
*Some of the topics we covered:*
? What is contract testing, what is Pactflow and introduction to the new feature?https://pactflow.io/bi-directional-contract-testing/
? The value of contract testing
? Case study - how Fran, Alex and the team at Sngular worked with the US bank on their app modernisation project
? Case study - setting up for success - three steps to ensure scale including how to engage stakeholders and get the team onboard
? Case study - preparing and implementing a group of practice framework
? Case study - comparison of costs for E2E tests vs contract testing
? + Q&A
Got another question? Pop it in the chat :raised_hands:
jonathan.fortunati
2022-03-24 23:54
For selectors I've been cycling between `_consumerVersionSelectors_: [{ _tag_: 'master', _latest_: true }, { _deployed_: true } ], _// the new way of specifying which pacts to verify_`
and
_`//consumerVersionSelectors: [{ tag: 'master', latest: true } ]`_
bethskurrie
2022-03-24 23:55
they should both work, because you've got the master pact in there both times.
bethskurrie
2022-03-24 23:55
when you run the verification, what is the output? you should see the URL of the pact being verified.
bethskurrie
2022-03-24 23:56
when you follow that link, is it the pact that has the new expectation in it?
jonathan.fortunati
2022-03-25 00:06
I did notice the version used for verification corresponded to the SHA from my commit associated with the consumer that was not expecting an extra field
bethskurrie
2022-03-25 00:07
double check that the pact in the broker with the latest master tag is the one you think it is
bethskurrie
2022-03-25 01:58
@jonathan.fortunati if you dm me your tenant, and I'll log in and have a look for you.
brendan.j.donegan
2022-03-28 08:24
Hi, just looking to get some more info about the proposed domain switch for Pact flow users with a custom domain - having to switch over everything simultaneously seems.. inconvenient? Is there something I'm missing or anything that can be done to ease the transition?
matt.fellows
2022-03-28 08:29
You can start switching now. The domain is just an alias so you can use them at the same time
brendan.j.donegan
2022-03-28 08:30
Ah good, that makes more sense. My pre coffee brain read it as being a hard cutover :joy:
brendan.j.donegan
2022-03-28 08:34
Just looking again at the email, it certainly reads like on April 4th we will have to have everything updated?
brendan.j.donegan
2022-03-28 08:34
"On Monday 4th April at 01:00 UTC we will be decommissioning the legacy domain name http://pact.dius.com.au. We don't expect any outage for the Pactflow application.
ACTION REQUIRED
If you currently use a domain name ending in http://pact.dius.com.au when accessing Pactflow (via the UI, API or Pact client tooling), you need to update the hostname you use to connect to our services to the http://pactflow.io domain immediately to avoid losing access to your service.
EXAMPLE
If your Pactflow hostname is currently http://acme.pact.dius.com.au, you will simply need to change it to to http://acme.pactflow.io. You can do this immediately without impact to your service - there is no need to wait until the cutoff date."
matt.fellows
2022-03-28 10:46
Yes, by Monday the fourth you must have cut over. We sent out a notice 30 days ago and this is a follow up
matt.fellows
2022-03-28 10:47
I saw your email tho, we can look to defer if this is going to cause issues for you
matt.fellows
2022-03-28 11:04
Be great for you to confirm if you received it - if you didn?t, perhaps others didn?t either
matt.fellows
2022-03-28 11:06
The subject was ?ACTION REQUIRED: decommissioning legacy domain http://pact.dius.com.au?
matt.fellows
2022-03-28 11:11
It?s probably already gone from spam if it was there. I can see you were on the distribution list (I?ll DM you the emails I have on file that were on that list)
nathan.deamer
2022-03-29 14:03
Automating removal of orphaned pacts:
Given A had a pact with B.
When A no longer needs to communicate with B
And the consumer pact has been removed from code / no json generated.
Then I need to go to pactflow and delete the integration so the provider is no longer verifying a pact it doesn?t need to anymore.
Is there anyway to automate this?
fushinoryuu
2022-03-29 20:19
Hi everyone, I have a question about cleaning up old pacts.
My team has been using Pactflow for over a month, but I'm already seeing a lot of old pacts and I'm wondering how we are suppose to clean these up?
Thanks for your help!
bethskurrie
2022-03-29 20:52
@nathan.deamer I've been working on a fix for this. You shouldn't need to delete it, just stop publishing.
bethskurrie
2022-03-29 20:55
The issue was, it was returning "the pact for the latest version with a pact that has tag X", which meant there was always a pact being returned. What it needed to do was return "find the latest version that has tag X and return the pact if it exists".
bethskurrie
2022-03-29 20:56
I think I've updated the logic for branch selectors, but not sure I've done it for tags yet.
bethskurrie
2022-03-29 21:03
What you should just need to do is create an pacticipant version resource that doesn't have the pact with that provider.
bethskurrie
2022-03-29 21:04
If the consumer has pacts with other providers still, then just publish as normal. If the consumer has no more providers then you'll need to create a pacticipant version resource that has no pacts, so the broker knows that there is no pact for that consumer any more.
fushinoryuu
2022-03-30 15:50
Thanks Brendan,
I was hoping there would be a better solution than manually cleaning it up.
bethskurrie
2022-03-31 21:34
These are the oss docs for the feature, but we can enable it for you https://docs.pact.io/pact_broker/administration/maintenance
bethskurrie
2022-03-31 21:35
It always keeps the latest from each branch however, so it might not "clean up" the UI, however.
fushinoryuu
2022-03-31 21:36
O, that would be great. Let me look over the docs and I will let you know. Thanks!
bethskurrie
2022-03-31 21:57
We have monitoring that detects when your instance is getting too much data, and the queries are slowing down, at which point, we contact our customers to enable the clean up.
bethskurrie
2022-03-31 21:57
You're way off though! It usually takes a few years of data before you notice any issues.
bethskurrie
2022-04-03 22:18
@nathan.deamer i've just dug into the code to work out what it's currently doing
bethskurrie
2022-04-03 22:19
``` # The latest pact publication for each tag
# This uses the old logic of "the latest pact for a version that has a tag" (which always returns a pact)
# rather than "the pact for the latest version with a tag"
# Need to see about updating this.
def latest_by_consumer_tag```
bethskurrie
2022-04-03 22:19
```# Updated logic - the pacts for the latest version of each main branch,
# not the latest pact that belongs to the main branch.
def latest_for_main_branches```
bethskurrie
2022-04-04 00:21
```# The pacts for the latest versions with the specified tag (new logic)
# NOT the latest pact that belongs to a version with the specified tag.
def for_latest_consumer_versions_with_tag(tag_name)```
bethskurrie
2022-04-04 00:33
I've created a script with the http data builder to show how this is expected to work.
bethskurrie
2022-04-04 00:34
The instructions for running it locally on docker are here: https://github.com/pact-foundation/pact_broker/blob/master/ISSUES.md
bethskurrie
2022-04-04 00:35
The scenario just needs to be updated to use `script/data/decommissioning-an-integration.rb` instead of `script/data/reproduce-issue.rb`
bethskurrie
2022-04-04 00:38
(note - I was running this against the head version of the code that has my latest branch fix applied - it will behave differently against the released version).
nathan.deamer
2022-04-05 14:27
I have a question about `record-release`
Would you usually only use this for things like mobile applications where you need to support multiple consumer versions?
And not for microservices which might be changing every day? (Otherwise you?ll have a lot of `record-support-ended` for previous versions)
nathan.deamer
2022-04-05 14:31
*Assuming microservices are on a rolling deployment / no long lived multiple instances
justin
2022-04-05 15:15
I think you use record-deployment for microservices where there's only one 'active' version. I don't think there's any reason you couldn't use releases, but you'd have to manage taking old versions offline, while deployments only allow one active version automatically.
nathan.deamer
2022-04-05 15:25
Yeah, that was my thought. I don?t really want to have to call `record-support-ended` on each release - having to work out the previous version numbers etc
bethskurrie
2022-04-05 20:58
@nathan.deamer record deployment is for single instancea. Record release is for mobile apps and libraries.
bethskurrie
2022-04-05 20:59
alejandro.velez
2022-04-06 15:09
hello! our company is assessing pact/pactflow and so far it?s been working out nicely! we?ve setup a system account, CI/CD in buildkite, and our first contract :slightly_smiling_face: is there any limit we should be aware of in terms of webhooks calls or verification counts? I don?t think we will exceed the 5 contracts anytime soon but want to be cognizant of your resources. thanks in advance!
alphonse.bendt
2022-04-08 11:50
hi folks, how can i determine the tags for a pact using the pactflow REST API/HAL Browser?
The tags do not seem to be part of the response of a specific version `/pacts/provider/{provider}/consumer/{consumer}/version/{version}`
Also i didn't see any other relevant resource to discover the tags. Am i missing something?
best regards,
yousafn
2022-04-08 12:44
Hey everybody :wave:
Get more of your team involved and maintain traction to ensure a successful rollout of your contract testing initiative using our tooling integration, made possible with Bi-Directional Contract Testing.
This week we hosted two Pactflow customer webinars providing demos using Cypress for consumer contracts, OpenAPI documentation for provider contracts and Postman. The recordings are now available on YouTube if you?d like to view / share with your team:
Using OpenAPI Documentation in Contract Testing with Pactflow https://youtu.be/a9K43CHSRM0
Using Cypress and Postman for Contract Testing with Pactflow https://youtu.be/tl1PtesLJVI
Plus some more details. Check the thread :thread: :point_down:
yousafn
2022-04-08 12:45
Plus I?ve pulled together links and resources to help you get started:
The back story: why we created Bi-Directional Contract Testing https://pactflow.io/blog/introducing-bi-directional-contract-testing/
:eyes: Check out the Bi-Directional Contract Testing https://docs.pactflow.io/docs/workshops/bi-directional-contract-testing
? In-browser Katacoda tutorial - interactive and very easy to start with :partying_face:
? Mix & match Bi-Directional Consumer & Providers
:new: Read the Pact Cypress adapter https://pactflow.io/blog/use-cypress-in-contract-testing/
:writing_hand: View the examples used in the demo:
- Consumer Cypress https://docs.pactflow.io/docs/examples/bi-directional/consumer/cypress/
- Provider Postman https://docs.pactflow.io/docs/examples/bi-directional/provider/postman/
:checkered_flag: Tick off the https://docs.pactflow.io/docs/workshops/bi-directional/pre_requisites#prerequisites for getting started
? https://pactflow.io/try-for-free/ , ask for an invite to or https://docs.pactflow.io/docs/user-interface/settings/users/#invite-users to your Pactflow account
? Create or log in to your GitHub
alan.hanafy
2022-04-11 19:05
hi, sorry i dont have an answer for you, but i was wondering how you are using the REST API. is there some documentation on it? what client are you using? how are you authenticating?
cameron.allan
2022-04-12 01:45
Hi @yousafn thanks a lot for that demo. In the Cypress & Postman video you speak about unused fields in the fixtures that would unnecessarily prevent the provider from making changes. Would it make sense to remove these fields from the fixture even though that wouldn't represent the true (whole) response? Or is this a compromise of using this method?
Timestamp:
https://youtu.be/tl1PtesLJVI?t=1826
alphonse.bendt
2022-04-12 10:33
There is some documentation in the HAL Browser thats provided by pactflow.
i am using curl as a client, authentication is done via a bearer token that is setup inside the pactflow Ui
lio.lunesu
2022-04-12 22:30
Hi! I?ve changed the casing of a pacticipant (from `Abc` to `abc`) and the new UI seems confused showing both, but *empty*. The old UI only shows the new casing `abc` with the contracts. I had manually deleted the old `Abc` from the old UI before publishing the contracts.
lio.lunesu
2022-04-12 22:38
Browser console shows ERR_BLOCKED_BY_CLIENT. Turns out the new API is being blocked by uBlock Origin!
lio.lunesu
2022-04-12 22:38
The new UI still shows both variants of the pacticipant `Abc` and `abc`. The old UI only shows `abc`
yousafn
2022-04-13 15:41
Hey @cameron.allan , sorry I missed this in my notifications. No worries, my pleasure. Hope to more in the future and maybe some specific videos where I go over the details of each of our tool integrations.
Re: fixtures on the consumer side. I suppose it depends on where you are in your journey.
Say you just a testing team, maintain a set of mocks, which your consumers use, but represent your providers.
You may have a single mock service that provides the full set of provider responses, so it isn?t coupled to a consumers specific needs.
Using the full fat mock for validation against the provider would ensure that your mocks don?t stay out of sync.
If your consumer adds a new field, they would want some tests against the fixtures, in which it would fail, giving them feedback pre deployment.
If the case mentioned where a consumer fixture has a field unused by the consumer, the provider would remove it and publish their OAS spec. It would fail against the consumer fixture?s generated Pact file. You would then want a conversation to update your fixtures, and re-run your consumer side tests, make sure they are good, generate a Pact, verify and you are good to deploy.
If you can generate the Pact files from the consumer side code directly, I am always in favour of that, and fortunately you can use the existing Pact framework on your consumer side, along our new Bi-Directional cross-contract validation.
yousafn
2022-04-13 15:41
If there is anything you?d like to see specifically covered in future demos, feel free to drop any of us a line!
yousafn
2022-04-13 22:23
No worries man, if you have any specifics about your particular domain be sure to ask on here as there is loads in the community who want to help :slightly_smiling_face:
lucka
2022-04-14 17:13
Hi,
while working on our cicd pipeline to reach platinum pact nirvana :blush:, I am sometimes deleting existing pacts to simulate a new flow between consumer and provider. When deleting all pacts, the matrix is cleared as well. But as soon as I publish the same pact again, all matrix entries reappear. Is this expected or should the matrix entries be deleted as well? Can I delete the matrix entries myself?
matt.fellows
2022-04-14 22:07
Deleting the pacts doesn't remove this data, but deleting the integration will
drakulavich
2022-04-15 12:14
Hi everyone!
Recently I've created pacticipant on pactflow side without name. Can't delete it using api now :slightly_smiling_face:
Any ideas?
``` "name": "",
"displayName": "",
"createdAt": "2022-04-04T07:54:22+00:00",
"_embedded": {
"labels": []
},
"_links": {
"self": {
"href": "https://caspian.pactflow.io/pacticipants/"
},
"pb:versions": {
"href": "https://caspian.pactflow.io/pacticipants//versions"
}```
matt.fellows
2022-04-16 01:11
Strange! Sounds like a weird bug - do you know how you managed to create it?
matt.fellows
2022-04-16 01:11
I?ll raise a card to look next week (note it?s a long weekend here in Melbourne)
matt.fellows
2022-04-16 01:12
I don?t think the pacticipant API would accept this, I?m wondering if perhaps a contract with an empty provider did
drakulavich
2022-04-16 13:12
Thank you for the reply, Matt!
I don't remember how I managed to create it. I think I was playing with `publish.sh` script to send OAS and somehow one of environment variables was not properly initialized.
matt.fellows
2022-04-17 04:06
Ah that makes sense, thanks for the pointer. We can remove the empty pacticipant for you next week when we're back
pavlo.sprogis
2022-04-19 10:31
hi guys, I?m trying to implement bi-directional testing PoC with pactflow and I have noticed the following problem: in my `swagger.json` I have the field with `"format" : "S2S-Token"` (this is our custom token) and pactflow fails with during verification:
```Validator Error unknown format "S2S-Token" is used in schema at path "#/properties/value"```
even though I have required headers and payload in request, looks like pactflow does not like this custom format field, is there a way to fix this?
matt.fellows
2022-04-19 10:51
Can you share your OAS? IF not here, could you please send a request to so we can take a look?
yousafn
2022-04-19 10:55
Hey @pavlo.sprogis, does your `format` property have an associated primitive `type`?
https://github.com/OAI/OpenAPI-Specification/blob/main/versions/2.0.md#data-types
yousafn
2022-04-19 10:57
datatypes for oas3
https://swagger.io/docs/specification/data-models/data-types/
pavlo.sprogis
2022-04-19 11:39
> Can you share your OAS?
I will ask the team and management, since we have plenty of regulations, soc compliance, etc. That?s why I removed information about all endpoints and left only 1 for testing
pavlo.sprogis
2022-04-19 11:39
> Hey @pavlo.sprogis, does your `format` property have an associated primitive `type`?
yes, it is string:
pavlo.sprogis
2022-04-19 11:40
{
?name? : ?name here?,
?in? : ?header?,
?description? : ?authentication key?,
?required? : true,
?type? : ?string?,
?format? : ?S2S-Token?
```}```
pavlo.sprogis
2022-04-19 12:07
yes, I do not see any errors, only warnings: ?siblings values alongside $refs are ignored. ?
yousafn
2022-04-19 12:13
I've managed to recreate with `openapi: 3.0.3` will upload an example on a PR to view and test
yousafn
2022-04-19 13:03
Hi @drakulavich, I managed this today, as I also didn't have my PACTICIPANT env var set (as was running the script outside of a Makefile).
1. Select integration from integrations list
2. select view contract
3. Click on the 3 dots and click delete
4. Click delete on the confirmation
5. Refresh the page and it should be removed from the list.
Not tried via the API, but hope that may help :thumbsup:
yousafn
2022-04-19 13:05
ideally we would have server side validation that stops that happening in the first place, and the planned CLI tool, could provide that feedback sooner to the user, pre upload
drakulavich
2022-04-19 13:49
Thank you for the update. As far as I remember, in my case I even couldn't open the page for that participant. I ended up cleaning all participants except empty one using `curl -X DELETE` to https://XYZ.pactflow.io/pacticipants/pacticipant_name/versions
drakulavich
2022-04-19 13:50
Integration panel doesn't have it right now. It's only visible from API calls.
nathan.deamer
2022-04-19 15:13
Hi all,
Looking at the `contract_requiring_verification_published` webhook https://docs.pact.io/pact_broker/webhooks#the-contract-requiring-verification-published-event.
Is there a reason this doesn?t fire when a new contract is published? (e.g. no previous contract between services)
craig.schwarzwald
2022-04-19 22:39
@matt.fellows is there any chance PactFlow may take "ownership" of the "Atlassian owned" Open Source swagger-mock-validator project? I had opened this defect nearly 15 months ago when our company started running into it, but to date it doesn't seem to have been worked on. There's another issue in particular that had a lot of comments that I believe should ideally result in an enhancement idea for this compare tool as well: https://bitbucket.org/atlassian/swagger-mock-validator/issues/84/test-incorrectly-passes-when-mock-expects where we should auto add "additionalProperties=false" to every object in the api-spec just before the compare happens similar to how it already strips the "required" flag off of every property in the spec because consumers may not want to utilize every property.
craig.schwarzwald
2022-04-19 22:42
additionalProperties=false should be added to any/every object that doesn't already have an additionalProperties value specified. It makes sense for OAS to default that value to true, but for this compare tool specifically, it makes sense for that value to default to false.
matt.fellows
2022-04-19 22:50
Good question! I?ll bring back to the team (a bunch on leave this week). And aim to get back to you next ewek
yousafn
2022-04-19 22:59
I'm with you and Tim on this one for sure. I had that same issue before and it seemed like such a gap, from the validation from the Pact side of the fence
matt.fellows
2022-04-20 00:32
We are likely to fork that project soon Craig as we use it in our BDCT feature, and need to support v3 and v4 pacts properly
matt.fellows
2022-04-20 00:34
I think there are issues with that tool that can be overcome without a significant rewrite. For example, polymorphic payloads aren?t supported (e.g. `anyOf` , `oneOf` etc.)
matt.fellows
2022-04-20 00:36
I spiked an https://github.com/mefellows/pact-oas-validator about a year ago using modern OAS parser, and was able to overcome this issue fairly easily. There are no doubt other issues we?ll need to overcome, but I think because Atlassian built it a while ago retrofitting that behaviour is both a) difficult to do and b) potentially going to impact their usage of the tool.
matt.fellows
2022-04-20 00:37
> additionalProperties=false should be added to any/every object that doesn?t already have an additionalProperties value specified. It makes sense for OAS to default that value to true, but for this compare tool specifically, it makes sense for that value to default to false.
In the context of our BDCT feature, I?m not convinced that tool should do that or if Pactflow should pre-validate it. I could definitely see it having a flag however that controls the behaviour
craig.schwarzwald
2022-04-20 02:35
Yeah, I think that's probably ideal, to have an option to run the swagger-compare with additionalProperties defaulted to false, so users can have the choice at execution time.
Forking the repo and taking control of that fork also sounds like the best solution both for PactFlow and the community. As pact continues to makes changes in the future, especially now with so many people implementing BDCT, you're going to want to make sure that any breaking changes pact implements with new major versions still have a way to get compared successfully to OAS.
matt.fellows
2022-04-20 02:38
BDCT is already starting to resonate with our customers and prospects, so we?ll definitely need to deal with it soon anyway
matt.fellows
2022-04-20 02:39
We also ?drink our own champagne? and do our best to feel the pains of our customers. The lack of some validation like `additionalProperties` and others will need adressing. As we move beyond OAS, the formula will also mature
craig.schwarzwald
2022-04-20 02:49
As mentioned in the issue ticket I posted, setting `additional properties=false` will do a strict compare, but specifying that in the actual OAS may not be what you want all the time (and teams may forget or not know to). It sounds like you have a good direction on this.
It's exciting to see two of the biggest issues I've been having with pact-swagger compares for over a year get some traction toward resolutions.
matt.fellows
2022-04-20 02:51
For transparency, I think we could address the `additionalProperties` more quickly than the other. But we definitely need to solve both
yousafn
2022-04-20 15:55
Hey, had a go at fixing this and we have a fork now, although it is on GH, the original repo is on BB, but we can still track changes.
https://github.com/pactflow/swagger-mock-validator
https://www.npmjs.com/package/@pactflow/swagger-mock-validator
Next steps are to pull into the Pactflow application, and add support for v3/v4 Pacts
Will look at getting the additionalProp change in, as a configuration option both in the tool, and in Pactflow land.
madhulika.mitra
2022-04-21 09:27
Hi Team . We are trying to integrate Wiremock and generate pact using WireMockPactGenerator. We are using Junit5, which uses wireMockRuntimeInfo instead of Wiremock server. Any idea on how we could integrate a listener here? https://bitbucket.org/atlassian/wiremock-pact-generator/src/master/ #Atlassian
yousafn
2022-04-21 09:33
Hi @madhulika.mitra,
Thanks for joining up!
Just adding @argos83 into the discussion, as he is the current maintainer, and we discussed over in your GitHub issue, that were unable to raise an issue against the above repository for discussion.
For a little more context, the question was initially raised on pact-jvm's GH https://github.com/pact-foundation/pact-jvm/issues/1535
matt.fellows
2022-04-22 01:19
and done :point_right: https://www.npmjs.com/package/@pactflow/swagger-mock-validator/v/11.0.0
matt.fellows
2022-04-22 01:20
:clap: Yousaf - brought this forward for us. We?ll aim to get this into the next BDC release
madhulika.mitra
2022-04-22 11:28
Hi @argos83 - Could you let me know for any means here, to implement pact generator in combination with Wiremock runtime info?
jordan.nazemi
2022-04-25 16:13
I've been seeing some confusing behavior from the Pactflow dashboard while trying to put together a demo on bi-directional contract usage. See the attached screenshot.
I have three provider branches set up, master and dev are in-line with each other and based on the same commit. The third branch, "test," is meant to mimic a potential feature branch. So far I have not published any consumer contracts. This page looks exactly how I'd imagine it would with two differing contracts being presented across three branches.
jordan.nazemi
2022-04-25 16:14
Now, I'm going to run my pipeline for the dev and master branches of the consumer (who like the provider are based on the same commit). What I *EXPECT* to see is that there are two separate entries one for dev + master provider to dev + master consumer, and one for test provider to dev + master consumer. *HOWEVER*, instead, I see a single entry for the test provider to the dev + master consumer and the dev + master provider contract has just... disappeared. Even weirder, the matrix page just shows a non-entry for the provider columns. What's going on here?
yousafn
2022-04-25 16:32
Hey Jordan,
The matrix page doesn't support BDC providers at the moment, but this is on our radar to address asap
yousafn
2022-04-25 16:33
I think the overview is only showing the latest, and not letting you see the full list, I'll get back to you on that, I know there is a list that the team are working on to get the UI experience slick, so the feedback is really appreciated
jordan.nazemi
2022-04-25 17:11
Ahhh gotcha. can-i-deploy seems to be picking up all the contracts properly so I think you're on the money with what's going on! It's good to know this is on the radar.
jordan.nazemi
2022-04-25 17:12
What is the best method to identify why a contract between two services is invalid when it's not displayed via the dashboard?
yousafn
2022-04-25 17:13
I believe the can i deploy output should have a link to the correct failing contract pairing
yousafn
2022-04-25 17:15
You can always drop issues onto the Pactflow public roadmap issues board https://github.com/pactflow/roadmap/issues
we track the project here https://github.com/pactflow/roadmap/projects/1 so you can generally see what we are working on
andrew.fraser
2022-04-26 07:23
Hello :wave: we're looking at using bi-directional contracts for events in javascript and python using pactflow. I didn't completely understand the rough timescales on the https://github.com/pactflow/roadmap/projects/1, where does it sit?
matt.fellows
2022-04-26 08:38
BDCT is supported on all languages (consumer). Provider is language agnostic as you BYO provider verification tool
andrew.fraser
2022-04-26 12:21
I'm referring to this section of the pact guide https://youtu.be/IetyhDr48RI?t=485
matt.fellows
2022-04-26 12:47
Right. You can do that with python now, but not with bidirectional as that only supports a provider with an OAS
wng
2022-05-03 15:55
hi, we're trying to implement BDCT and are a bit confused how to publish the provider contract with the appropriate tags, branch and env. I followed this doc but the contract uploaded doesn't have any branch info and not sure how to pass it in. https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow.
yousafn
2022-05-03 16:35
Hi Wayne,
Thanks for bringing this to my attention, (and apologies for the confusion) it was on my list of items to address as I had previously queried it. Hope the following is helpful.
So I believe the current process is as follows,
1. https://github.com/pact-foundation/pact_broker/blob/master/lib/pact_broker/doc/views/index/pacticipant-branch-version.markdown (API Reference) / https://github.com/pactflow/example-bi-directional-provider-restassured/blob/d562158cd0920eb57e5ba7007e65db4a9f08cbe9/Makefile#L26 / https://github.com/pactflow/example-bi-directional-provider-restassured/blob/master/scripts/create_branch_version.sh
2. https://github.com/pact-foundation/pact_broker/blob/master/lib/pact_broker/doc/views/index/publish-contracts.markdown (API Reference) / https://github.com/pactflow/example-bi-directional-provider-restassured/blob/d562158cd0920eb57e5ba7007e65db4a9f08cbe9/Makefile#L32 / https://github.com/pactflow/example-bi-directional-provider-restassured/blob/master/scripts/publish.sh
> branch version should be created before publishing provider contracts
3. Use https://docs.pact.io/pact_broker/client_cli/readme#can-i-deploy `pact-broker can-i-deploy` where environments are first used / https://github.com/pactflow/example-bi-directional-provider-postman/blob/984f635a2317faea9137d9aa52a17f77324e5568/Makefile#L74
4. Use https://docs.pact.io/pact_broker/client_cli/readme#record-deployment `pact-broker record-deployment` where environments can be recorded post deployment / https://github.com/pactflow/example-bi-directional-provider-postman/blob/984f635a2317faea9137d9aa52a17f77324e5568/Makefile#L82
Notes:-
1. The current advice here https://docs.pactflow.io/docs/bi-directional-contract-testing/provider#step-5-deploy-your-application regarding tagging, is out-of-date and will be removed.
2. Both 1 and 2 in the steps above, will be available via the https://docs.pact.io/pact_broker/client_cli/readme soon. Roadmap item tracking CLI publishing tool is https://github.com/pactflow/roadmap/issues/68
will get the team to cast their eyes over cc: @matt.fellows / @bethskurrie
matt.fellows
2022-05-04 04:24
That sounds right to me Yousaf. Apologies for the confusion Wayne, we?re in a migration between tags / branches and occasionally the advice is inconsistent
lei.shi
2022-05-04 10:34
Hi team. just trying the free version packflow. the page link to how to publish contract is not accessible. https://docs.pactflow.io/docs/bi-directional-contract-testing/publishing/
yousafn
2022-05-04 13:49
Hey @lei.shi thanks for the report! I will get this addressed
If the url has a trailing forward-slash, the redirect isn't working.
The are getting the trailing forward-slash added to the URL when deeplinking to the page, or directly navigating to the url.
1. https://docs.pactflow.io/docs/bi-directional-contract-testing/publishing/
2. Click on https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow.
3. takes us to https://docs.pactflow.io/docs/bi-directional-contract-testing/publishing/contracts/oas#publishing-the-provider-contract--results-to-pactflow
This URL is shown, when we select `publishing contracts` from the side bar.
1. https://docs.pactflow.io/docs/bi-directional-contract-testing/publishing
2. https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow.
3. takes us to https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow
The difference between the 2. The first link is being generated as relative, if the `trailing forward-slash`e is present, so it is trying to find it under `/publishing...` where it doesn't exist
yousafn
2022-05-04 14:54
Hey @lei.shi this is now resolved!
whether you land on
1. https://docs.pactflow.io/docs/bi-directional-contract-testing/publishing
2. https://docs.pactflow.io/docs/bi-directional-contract-testing/publishing/
I will be updating the page later with the content from this post
https://pact-foundation.slack.com/archives/CLS16AVEE/p1651595753620349?thread_ts=1651593302.639899&cid=CLS16AVEE
matt.fellows
2022-05-05 05:06
So sorry Lei, I read ?just trying the free version packflow? as you were trying to signup to Pactflow via that link. :facepalm: sorry, shouldn?t multi-task - thanks for saving my bacon Yousaf!
yousafn
2022-05-05 10:07
Just to let you know @wng that the following sections are now updated.
1. https://docs.pactflow.io/docs/bi-directional-contract-testing/provider#step-4-publish-your-provider-contract-and-verification-results
2. https://docs.pactflow.io/docs/bi-directional-contract-testing/provider#step-5-deploy-your-application
3. https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow
Hope this helps a little!
yousafn
2022-05-05 10:11
:wave: Hello everybody,
Have you tried to publish a Pact or Provider contract to the Pactflow platform and been unsuccessful?
Got an account but not tried it out yet?
Drop me a reply in a :thread: and we'll see if we can get you sorted!
lredpath
2022-05-05 11:24
I?ve just arrived here and was going to ask the same question - thank you for updating the documentation. Our team spent some time with Pact and Pactflow last week as part of a hack week project and I think the ?create branch version? step was missing at the time. I?ve updated our publish script and its now working. :thumbsup:
lredpath
2022-05-05 11:25
:wave: hi - I?m Luke, I?m a mobile developer at http://Community.com - we?re currently evaluating the use of Pact (and specifically Pactflow) to enable us to do bi-directional contract testing of our mobile and web apps against our backend OpenAPI specs. We had a company hack week last week where we spent some time on this so I may have some questions.
lredpath
2022-05-05 11:26
My first question - when I look at the matrix tab for an integration, I see a list of consumer versionsbut I don?t see any provider details, for example:
lredpath
2022-05-05 11:27
Should I be seeing verification results for each provider version checked here?
lredpath
2022-05-05 11:28
My second question is more of a feature request - we?re currently evaluating how we could make the OpenAPI specs available in SwaggerUI format to act as a human-readable API repository. I see that Pactflow can already render SwaggerUI for each provider contract - it would be great if it could serve up an index page that shows each provider and has a link to the latest version of its OpenAPI spec.
lredpath
2022-05-05 11:29
I realise this overlaps a bit with SwaggerHub and I know we could probably make this work with SwaggerHub by automating the publishing of our docs using the SwaggerHub API but it would be nice if we didn?t need a separate tool.
yousafn
2022-05-05 11:48
Wonderful, thanks for the feedback @lredpath. Look forward to hearing how you get on, and the reception from the team after the hack week now it's working.
yousafn
2022-05-05 11:50
Hey buddy,
So I just updated the docs this morning re: the matrix tab. Let me know what you think
https://docs.pactflow.io/docs/user-interface/bi-directional#matrix-view-tab
TL;DR - We are working on a BDCT specific view, currently this matrix is still tailored towards the traditional CDCT, and thusly just shows the consumer generated Pacts.
yousafn
2022-05-05 11:53
Thanks for the wonderful feedback, so you may, or may not know that https://pactflow.io/blog/pactflow-joins-the-smartbear-family/, who hold SwaggerHub and Swagger in their stable of products.
This might be of particular interest
> From a practice perspective, you will still get same love from Pactflow?but with additional supporters and resources behind us from our new parent company. In the coming months we will be integrating more closely with the ?siblings? in our new family?for example https://swagger.io/ and more?so keep an eye out for these perks! We will keep you updated as things progress.
So we will take your feedback on board, and be sure you let you know. We have a developer preview platform, I will try and find some details, you will be able to register, and we can invite you for early access to new features, and you can directly help shape the roadmap!
yousafn
2022-05-05 11:55
RE: https://github.com/pactflow/roadmap#developer-preview-programs
RE: https://github.com/pactflow/roadmap/discussions/67
RE: https://github.com/pactflow/roadmap/projects/1
You can submit a feature request via an issue :muscle:
lredpath
2022-05-05 12:02
The reaction was positive, we still have lots of details to flesh out, mainly around how we produce OpenAPI specs for our existing APIs. We?re trying to document an plan for this now. We?re already starting to use PactSwift for our mobile client because even as a standalone mocking tool its providing our API client implementation with much better test coverage.
yousafn
2022-05-05 12:10
> mainly around how we produce OpenAPI specs for our existing APIs
Yes, this is always a challenge, especially for legacy systems, that can be cumbersome to integrate with.
I do have a couple of shortcuts, creating a postman collection to test an existing service, and generating an openAPI spec off the back of it.
? https://github.com/joolfe/postman-to-openapi
? https://github.com/kevinswiber/postman2openapi
We use one of the above in https://github.com/pactflow/example-bi-directional-provider-postman
I've been on the hunt recently for OSS tools around open api, and have a little cli tool that contains a selection of all the awesome things I've found, I thought it was a good way to bring my GitHub stars list to life, and for me to learn more about CLI tooling. I need to release it at some point :slightly_smiling_face:
lredpath
2022-05-05 12:33
We do use Postman already but I think we?re hoping to migrate away because it needs to be manually maintained.
lredpath
2022-05-05 12:34
Most of our backend services are written in Elixir, we?ve already explored `open_api_spex` but are investigating the possibility of using our existing backend test suite to generate the open api docs.
jaswanth.ooty
2022-05-05 23:15
:wave: Hello Yousaf!
I'm about to wrap up my business case proposal that I'm putting forward in my company.
I've been trying to build POCs for most of the scenarios we use.
The two pieces I'm trying to get my head around are
1. Setting up an example for a BFF -> AWS API gateway. Although @matt.fellows pointed me to this one which is really helpful. https://github.com/pact-foundation/docs.pact.io/pull/148/files
2. The other pending scenario is for a kafka consumer in ruby (via Avro schemas). I'm about to dig into this one https://github.com/pact-foundation/pact-message-ruby
I am sure that there is a lot of support from pactflow for different scenarios, but I'm just looking for docs/samples that can push me through.
Appreciate the help. Please point me to any other docs if you think it'll help :pray: ta!
yousafn
2022-05-05 23:54
Hey man, still super new to the world of avro, so will defer that one for now, but will see what I can rummage.
On point 1, yes that has been on my mind too. I have situations I have encountered with api gateway acting as both an auth and transformational layer, for accessing on prem services. The gateway services were written in Nodejs and some of the on prem services would co ordinate multiple downstream calls before returning back to the client, all in under 29 seconds ( gateway window is 30 )
We would short circuit in certain downstream services to provide the client with a we are processing this request, and the client would then poll, until their request was satisfied or errored.
Client side, we had various lambdas, one was the main BFF which served the interactions between a website served in s3 built with react.
All the clients needs awsv4 signed requests.
I can think of a few other past client scenarios, I need to sketch them out.
Pretty late here now but I'll see what I can cook up tomorrow / next week. Feel free to fire me over some kind of sample architecture diagram here or via pm if you want, and I could try and work an example of something that replicates it
matt.fellows
2022-05-06 02:14
> The other pending scenario is for a kafka consumer in ruby (via Avro schemas). I?m about to dig into this one https://github.com/pact-foundation/pact-message-ruby
Hi Jas, we don?t currently support Avro. Whilst I know people have requested/discussed it, it?s not highly voted for (https://pact.canny.io/feature-requests/p/support-apache-avro)
It?s probably possible to do it though, but you could look to use a strategy like https://medium.com/@ivangsa/consumer-driven-contract-testing-for-grpc-pact-io-d60155d21c4c to transform the payload to/from Avro into JSON so that Pact can store it in the contract.
jaswanth.ooty
2022-05-06 04:10
Yup that's what I thought too :pray: I saw that there was a level of abstraction there. Happy to contribute to the thought there and add a layer on top for avro specifically when I get to work on it :thumbsup: ta!
matias.cardenas
2022-05-09 07:21
Hi! we are also new to Pactflow here, and we are trying to work in our first POCs using it, in this case, for the consumer side with *cypress*. However, following the existing sample project I am coming with the issue that if I `intercept` the endpoint and then use `cy.visit` in the test I get the following error:
```The `content-type` of the response we received from your web server was:
> `application/json`
This was considered a failure because responses must have `content-type: 'text/html'````
I can set at the endpoint the header `{ 'Content-Type': 'text/html' }` and the ignore it, in that case it doesn't fail and publish the contract, nevertheless it will fail when checking against the provider as the response is sent as string.
Is there a way to workaround this or I am missing something here?
matt.fellows
2022-05-09 08:07
I?m guess, that if a `201` created returns a JSON body (and that?s your expectation), the header should be of the correct content type
matias.cardenas
2022-05-09 08:15
This is the test, it's a pretty simple one:
```const orderResponse = require('../fixtures/created_order.json')
describe('Orders creation', () => {
beforeEach( () => {
cy.intercept(
{
method: 'POST',
url: '/orders/'
},
{
statusCode: 201,
body: orderResponse,
headers: { 'Content-Type': 'text/html' },
},
).as('postOrder')
cy.setupPact('pactflow-example-bi-directional-consumer-cypress', 'pactflow-order-api-postman-provider')
cy.setupPactHeaderBlocklist(['Content-Type'])
})
it.only('Displays created order', () => {
cy.visit({method: 'POST', url: '/orders/'})
})
after(() => {
cy.usePactWait(['postOrder'])
})
})```
matias.cardenas
2022-05-09 08:16
> I?m guess, that if a `201` created returns a JSON body (and that?s your expectation), the header should be of the correct content type
Yes indeed, and that's the issue I am actually having, cypress complains when I want to use `cy.visit` and to the intercepted endpoint if it is returning `application/json` (which is what by default it does when I don't set any `content-type` on it :disappointed:
matt.fellows
2022-05-09 08:17
I?m confused,
```headers: { 'Content-Type': 'text/html' },```
You want the response type to be HTML?
matt.fellows
2022-05-09 08:17
Shouldn?t this be:
```headers: { 'Content-Type': 'application/json' },```
matias.cardenas
2022-05-09 08:17
> You want the response type to be HTML?
no no, it's the only way I manage to cypress not to complain . If I remove that test won't pass
matias.cardenas
2022-05-09 08:19
Everything I shared is the consumer tests errors:
```The `content-type` of the response we received from your web server was:
> `application/json`
This was considered a failure because responses must have `content-type: 'text/html'````
matt.fellows
2022-05-09 08:22
Does https://github.com/pactflow/example-bi-directional-consumer-cypress/ work for you?
matt.fellows
2022-05-09 08:25
Oh, wait:
```cy.visit({method: 'POST', url: '/orders/'})```
Why are you ?visiting? the API?
matt.fellows
2022-05-09 08:25
You should be visiting a web page and interacting with it, to get it to `POST` to `/orders`
matt.fellows
2022-05-09 08:26
Or, using other APIs that Cypress have availaible for API testing (if you do this though, I?d argue you?re usinsg the wrong tool for the job and could probably just use Pact with whatever unit testing tool you use)
matias.cardenas
2022-05-09 08:28
hmmm I see, regarding this:
> Why are you ?visiting? the API?
I was doing it because I saw it was being used in the example you shared: https://github.com/pactflow/example-bi-directional-consumer-cypress/
matias.cardenas
2022-05-09 08:29
regarding other cypress defined methods indeed I checked `cy.request` but apparently that one ignores `intercept` endpoints, it would only work if I add other stubs perhaps with mountebank, but not straightforward like this
matt.fellows
2022-05-09 08:34
Yeah the example visits a web page, and the web page makes the API call, which is intercepted.
kannan.jai
2022-05-09 08:35
Hi, I have a provider verification test build which always passes even if there is a failure in the pact verification. Am I missing something? I am using maven command to run the tests.
Please help. Thanks.
matt.fellows
2022-05-09 08:35
For a POST, you should write a test that checks the web page completes a form (or whatever it does) which then issues a POST call to an API that will be intercepted
matias.cardenas
2022-05-09 08:38
> Yeah the example visits a web page, and the web page makes the API call, which is intercepted.
oh I see, that is why! I wasn't able to stop the difference haha
matias.cardenas
2022-05-09 08:38
indeed it's not necessary for us to validate the UI in these contract tests, I might move to check the Mountebank example then
kannan.jai
2022-05-09 08:54
Sorry Matt if I sound stupid but I am new to Pact, what do you mean by pending enabled?
matt.fellows
2022-05-09 09:05
Ignore that for now then. I'd just like to see the log output from the verification task. I'm assuming there is a way to get this
gustavs.slutins
2022-05-09 11:58
Hello. Im having trouble getting my pact verified by the Provider using pactflow. The configuration is junit5+spring jvm. I can get the consumer pact uploaded to the pactflow fine, but when i run the verification test on the Provider i get Connection refused. Im using the token for @PactBrokerAuth. Is there any troubleshooting tips anyone can recommend?
matt.fellows
2022-05-09 11:59
Connection refused indicates a request is being sent to a server that is probably not running. Is your provider running when you run the tests?
Perhaps if you can share your provider code we can take a look
gustavs.slutins
2022-05-09 12:28
@matt.fellows Thanks for quick response. I shared the repo on github. The provider is customer. https://github.com/Gustavs920/pactflowtest. I used the read/write personal token as token, which is not committed here.
kannan.jai
2022-05-09 13:52
Hi Matt, I tried this (both true and false for enablePendingPacts) but no luck. Most probably I am doing something wrong or at the wrong place. Could you please help?
gustavs.slutins
2022-05-09 14:02
Nevermind. Figured it out with the test project from pactflow. Thanks anyway
matt.fellows
2022-05-09 21:16
that flag looks correct to me. Are you able to share any debug level logs to see what requests are sent to Pactlfow? In the request to discover which pacts should be verified, we should see it disabled
gustavs.slutins
2022-05-10 06:56
I moved the authentication and host to the application.yml and it started working.
danielflieger
2022-05-10 08:07
It?s may a silly question but what exactly does it mean to have 5 contracts in the Starter Free packet? Interactions or pact files?
I would understand that I can write 5 pact files for 5 endpoints. One endpoint = one pact file which can have more then 5 interactions, right?
Or is a contract = one interaction in a pact file? :)
matt.fellows
2022-05-10 08:15
> I would understand that I can write 5 pact files for 5 endpoints. One endpoint = one pact file which can have more then 5 interactions, right?
exactly!
fushinoryuu
2022-05-10 20:27
Hello, I have a question/issue that I just noticed today.
Whenever out consumer app publishes a new pact, the webhook fires in Pactflow and kicks off a build to verify the pact in the provider. But I notice that its firing multiple times. This is causing our contract tests to take up more build agents than necessary, and I can see this turn into an issue in the future once we start to adopt pactflow more widely. Has anyone else seen this?
matt.fellows
2022-05-10 22:39
I'm assuming you don't have multiple webhooks configured? If you could share the details of the duplicates we can look into it.
kannan.jai
2022-05-12 08:25
Apologies matt, I got carried away with other works. Will try to get the logs over to you today. Thanks for your assitance.
akke.luukkonen
2022-05-12 10:07
Might be related to this as well i.e. intentionally fires multiple times:
https://pact-foundation.slack.com/archives/C9VPNUJR2/p1650638136806229
william.pritchard
2022-05-12 11:12
Hi, I?ve switched my webhooks to run only on `contract_requiring_verification_published` to help reduce the amount of builds we run. When I?m sending across the trigger pipeline my payload body contains CONSUMER_TAGS, PACT_CONSUMER and PACT_URL. Now I?ve just been reading through the documentation around `contract_requiring_verification_published` and I?ve noticed that you?ve mentioned I need to pass in `${pactbroker.providerVersionNumber}` and `${pactbroker.providerVersionBranch}`. I was wondering what would be possible issues I would face by not including these values in my webhook payload? It seems to be working currently
william.pritchard
2022-05-12 11:15
This is the documentation I was reading - https://docs.pact.io/pact_broker/webhooks#using-webhooks-with-the-contract_requiring_verification_published-event
pennellbeth
2022-05-12 16:39
Hello there!
This question is more of a conceptual one, as I'm trying to get my head around the best implementation of pact for my use case.
The project I'm working on is looking to implement a microservice design, but most of the services will be reliant on 3rd party APIs and integrations including Salesforce, HooYu and Equifax.
Its one of the reasons I've been so excited about the release of bidirectional contract testing. Originally I thought it'd be a case of curling an endpoint to get a public facing version of an Open API spec... until I actually _saw_ the inside of the APIs (SF especially).
Am I right in thinking that the base concept for the provider side spec is just "get an OAS in any way you can" and push it to Pactflow, for which Postman-to-oas seems the most sensible call as this is a tool we already use. Do you then need to centre the postman collection on all endpoints and functionality? Or just the bits the consumer cares about (e.g. were not going to use all of Salesforce functionality for our application)
matt.fellows
2022-05-13 03:26
> Its one of the reasons I?ve been so excited about the release of bidirectional contract testing. Originally I thought it?d be a case of curling an endpoint to get a public facing version of an Open API spec... until I actually _saw_ the inside of the APIs (SF especially).
:laughing:
matt.fellows
2022-05-13 03:29
> Am I right in thinking that the base concept for the provider side spec is just ?get an OAS in any way you can? and push it to Pactflow,
Well, sort of. That?s all that Pactflow needs. But the key thing, is that the OAS is a valid representation of the provider (it can be a subset which is safe, but can?t contain things the provider _can?t_ do, because it?s used to determine safety for consumers)
> for which Postman-to-oas seems the most sensible call as this is a tool we already use. Do you then need to centre the postman collection on all endpoints and functionality? Or just the bits the consumer cares about (e.g. were not going to use all of Salesforce functionality for our application)
Technically, only the bits the consumer needs. The upside of mapping out the entire provider experience, is that if a new consumer comes along, or new requirements for the existing consumer mean it would now consume more of the provider?s interface, you don?t need to wait for a provider to update the tests and the corresponding OAS.
If it were me though, hearing your problems, I?d probably just map the APIs you know you use as a starting point and going from there. The SF surface area is quite big, so it might be wasteful to go too far beyond that.
yousafn
2022-05-13 10:07
Would you be interested in beta testing the new `publish-provider-contract` method in the Pact CLI for Bi-Directional Contract Testing.
It will simplify the existing direct API calls suggested here https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/#publishing-the-provider-contract--results-to-pactflow
If so, just drop me a reaction to this :mailbox: or a reply in a :thread:
yousafn
2022-05-13 12:55
Please see this https://github.com/pactflow/example-bi-directional-provider-postman/pull/13 for documentation use of the beta CLI. Feel free to drop any feedback there, or here.
pennellbeth
2022-05-13 16:09
Yeah, I've managed to obtain an OAS spec for it today, but that's presented me with a new problem in that the spec file is 15.8MB.
So there's no chance that's going to fit into a header b64 encoded. Is there another way to upload provider specs when they exceed header size from the PUT request to the pact broker / pactflow?
akke.luukkonen
2022-05-16 08:43
> I was wondering what would be possible issues I would face by not including these values in my webhook payload?
Testing the contracts with the wrong version of the provider. Are you currently running all tests against the HEAD of `master` or similar?
akke.luukkonen
2022-05-16 08:45
I hope I'll have time soon to experiment & help with this, but I haven't yet had time to even go through the BDC docs since the past <n> sprints have been a bit too busy :see_no_evil:. I'm at least very interested in the concept in general from what I've briefly read.
yousafn
2022-05-17 07:44
Hey @pavlo.sprogis, this change went out with the Pactflow platform yesterday, so you should able to retest, I haven't checked it since deploying (I've just got up) but tested with your test case prior to release, so :crossed_fingers: should be good
matt.fellows
2022-05-17 07:47
FYI Pactflow now automatically sets `additionalProperties` to false during verification (the package above also now has an additional flag to conditionally set this if required)
yousafn
2022-05-17 08:00
Release notes here for SMV https://github.com/pactflow/swagger-mock-validator/releases/tag/11.0.0
tati.shep
2022-05-17 10:53
Hi Pactflow team, wanted to ask if there was a release of some new version of Pactflow yesterday or today? We are on Pactflow Version `a48afdec7` currnently, and bi-directional contract verification started failing suddenly without any obvious reason on our side (or I haven?t found it yet). So, I wanted to double-check if this is on Pactflow side. Thanks!
kannan.jai
2022-05-17 10:54
Hello team, I am seeing this error while running provider verification tests - though this does not affect the test run, I was just wondering if I should be concerned? Please advise. Thanks.
kannan.jai
2022-05-17 10:55
```14:49:21.847 [ERROR] o.a.w.l.Aj - au/com/dius/pact/core/pactbroker/PactBrokerClient$fetchPactsUsingNewEndpoint$1$1$1$notices$1
org.aspectj.weaver.BCException: Whilst processing type 'Lau/com/dius/pact/core/pactbroker/PactBrokerClient$fetchPactsUsingNewEndpoint$1$1$1$notices$1;' - cannot cast the outer type to a reference type. Signature=Lau/com/dius/pact/core/pactbroker/PactBrokerClient$fetchPactsUsingNewEndpoint$1$1$1; toString()=au.com.dius.pact.core.pactbroker.PactBrokerClient$fetchPactsUsingNewEndpoint$1$1$1 class=PactBrokerClient$fetchPactsUsingNewEndpoint$1$1$1
when processing type mungers
when weaving
at org.aspectj.weaver.AbstractReferenceTypeDelegate.getFormalTypeParametersFromOuterClass(AbstractReferenceTypeDelegate.java:130)
at org.aspectj.weaver.bcel.BcelObjectType.ensureGenericSignatureUnpacked(BcelObjectType.java:770)
at org.aspectj.weaver.bcel.BcelObjectType.getSuperclass(BcelObjectType.java:230)
at org.aspectj.weaver.ReferenceType.getSuperclass(ReferenceType.java:993)
at org.aspectj.weaver.bcel.BcelWeaver.weaveParentsFor(BcelWeaver.java:1309)
at org.aspectj.weaver.bcel.BcelWeaver.weave(BcelWeaver.java:1122)
at org.aspectj.weaver.tools.WeavingAdaptor.getWovenBytes(WeavingAdaptor.java:549)
at org.aspectj.weaver.tools.WeavingAdaptor.weaveClass(WeavingAdaptor.java:385)
at org.aspectj.weaver.loadtime.Aj.preProcess(Aj.java:115)
at org.aspectj.weaver.loadtime.ClassPreProcessorAgentAdapter.transform(ClassPreProcessorAgentAdapter.java:51)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at au.com.dius.pact.core.pactbroker.PactBrokerClient$fetchPactsUsingNewEndpoint$1$$special$$inlined$map$lambda$1.invoke(PactBrokerClient.kt:255)
at au.com.dius.pact.core.pactbroker.PactBrokerClient$fetchPactsUsingNewEndpoint$1$$special$$inlined$map$lambda$1.invoke(PactBrokerClient.kt:138)
at au.com.dius.pact.core.support.json.JsonValueKt.map(JsonValue.kt:263)
at au.com.dius.pact.core.pactbroker.PactBrokerClient$fetchPactsUsingNewEndpoint$1.invoke(PactBrokerClient.kt:249)
at au.com.dius.pact.core.pactbroker.PactBrokerClient$fetchPactsUsingNewEndpoint$1.invoke(PactBrokerClient.kt:138)
at au.com.dius.pact.core.support.KotlinLanguageSupportKt.handleWith(KotlinLanguageSupport.kt:38)
at au.com.dius.pact.core.pactbroker.PactBrokerClient.fetchPactsUsingNewEndpoint(PactBrokerClient.kt:247)
at au.com.dius.pact.core.pactbroker.PactBrokerClient.fetchConsumersWithSelectors(PactBrokerClient.kt:214)
at au.com.dius.pact.provider.junitsupport.loader.PactBrokerLoader.loadPactsForProvider(PactBrokerLoader.kt:209)
at au.com.dius.pact.provider.junitsupport.loader.PactBrokerLoader.load(PactBrokerLoader.kt:111)
at au.com.dius.pact.provider.junit5.PactVerificationInvocationContextProvider$resolvePactSources$$inlined$flatMap$lambda$1.invoke(PactJUnit5VerificationProvider.kt:77)
at au.com.dius.pact.provider.junit5.PactVerificationInvocationContextProvider$resolvePactSources$$inlined$flatMap$lambda$1.invoke(PactJUnit5VerificationProvider.kt:34)
at au.com.dius.pact.core.support.KotlinLanguageSupportKt.handleWith(KotlinLanguageSupport.kt:38)
at au.com.dius.pact.provider.junit5.PactVerificationInvocationContextProvider.resolvePactSources(PactJUnit5VerificationProvider.kt:77)
at au.com.dius.pact.provider.junit5.PactVerificationInvocationContextProvider.provideTestTemplateInvocationContexts(PactJUnit5VerificationProvider.kt:38)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$0(TestTemplateTestDescriptor.java:103)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:107)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:42)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150)
at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)```
matt.fellows
2022-05-17 11:17
hmm that looks like an AOP thing (it?s been years since I?ve written Java, but that looks a bit like it)
matt.fellows
2022-05-17 11:18
Yes, we did make a minor change to enforce `additionalProperties` set to `false`
matt.fellows
2022-05-17 11:18
If you were having passing tests before, it?s likely you were getting false positives
matt.fellows
2022-05-17 11:19
tati.shep
2022-05-17 11:36
> What error are you getting?
`Response body is incompatible with the response body schema in the spec file: should NOT have additional properties` looks like it?s related to the changes.
tati.shep
2022-05-17 11:36
> If you were having passing tests before, it?s likely you were getting false positives
interesting, thank you!
matt.fellows
2022-05-17 11:43
basically, if you had it set to `true` (or omitted) it meant that the consumer could ask for any fields in the response, and it would always pass
matt.fellows
2022-05-17 11:44
if you can share the request body (consumer) and provider OAS (or just the schema for the request that?s failing) we can tell you. If you?re unable to share here and can?t get to the bottom of it, please raise a ticket at and we can look into it privately for you
yousafn
2022-05-17 12:02
Hey @tati.shep, for a bit of detail on the change, and background thread for reading, please checkout the release notes
https://github.com/pactflow/swagger-mock-validator/releases/tag/11.0.0
This point is relevant in your case, with the previous behaviour.
> Test incorrectly passes when mock expects a field that is not in the spec
https://bitbucket.org/atlassian/swagger-mock-validator/issues/84/test-incorrectly-passes-when-mock-expects
Will look to the get the documentation on the Pactflow docs site updated asap, and I could probably add some additional error messaging back to the user, if it's relevant. (Would a link to the issue help save you searching for help?)
tati.shep
2022-05-17 15:09
You are right that our contract tests wrongly passed before. I didn?t work much with OAS and relied on the written unit tests and mocks they used to write contract tests. Apparently, mocks were outdated, but we didn?t notice that as we used the default configuration. Thanks a lot for your help and for enforcing that setting! :smile:
tati.shep
2022-05-19 09:14
hey @matt.fellows @yousafn, me again. I think I encountered a bug in swagger-mock-validator with `allOf` notation in OAS. I prepared an example with contract and OAS files. I actually took that example from http://swagger.io. I?d appreciate if you have a look and share if something is wrong with the files.
Run `npx @pactflow/swagger-mock-validator ./swagger.yaml ./contract.json`
tati.shep
2022-05-19 09:17
These are the errors I see when I run that command:
``` {
code: 'response.body.incompatible',
message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - name',
mockDetails: {
interactionDescription: '46745cc6-f980-4d57-95b8-3c85539f7812',
interactionState: '[none]',
location: '[root].interactions[0].response.body[0]',
mockFile: './contract.json',
value: { name: 'Meow', petType: 'Cat' }
},
source: 'spec-mock-validation',
specDetails: {
location: '[root].paths./cats.get.responses.200.content.application/json.schema.items.allOf.0.additionalProperties',
pathMethod: 'get',
pathName: '/cats',
specFile: './swagger.yaml',
value: undefined
},
type: 'error'
},
{
code: 'response.body.incompatible',
message: 'Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - petType',
mockDetails: {
interactionDescription: '46745cc6-f980-4d57-95b8-3c85539f7812',
interactionState: '[none]',
location: '[root].interactions[0].response.body[0]',
mockFile: './contract.json',
value: { name: 'Meow', petType: 'Cat' }
},
source: 'spec-mock-validation',
specDetails: {
location: '[root].paths./cats.get.responses.200.content.application/json.schema.items.allOf.1.additionalProperties',
pathMethod: 'get',
pathName: '/cats',
specFile: './swagger.yaml',
value: undefined
},
type: 'error'
}```
yousafn
2022-05-19 09:31
Hey, thanks for the examples, I assume that is running `@pactflow/swagger-mock-validator` v11 with the default flags?
I think this falls in line with the comments made by Ben Sayers here
https://bitbucket.org/atlassian/swagger-mock-validator/issues/84/test-incorrectly-passes-when-mock-expects#comment-59384182
You'll note at the bottom someone advises a tool which will resolve the anyOf schemas which may make the tool perform in the way you are expecting, I've yet to try it out.
Just thinking on these examples, it might be good to add these in CI, showing the output with the 4 diff combinations of flags,
Looking at the spec, the only required field is `petType` but the Pact expects both?
tati.shep
2022-05-19 09:37
yes, I?ve installed `@pactflow/swagger-mock-validator` v11 and used with the default flags.
> Looking at the spec, the only required field is `petType` but the Pact expects both?
there are not `required` fields in my example. it?s just an object which consists from 2 other objects using `allOf` . Pact complains that they are both present. I assume that Pact expects them not to be present at all.
yousafn
2022-05-19 09:40
`petType` is shown as required on line 28 of the swagger, name is not on `38`
tati.shep
2022-05-19 09:42
yeah, you are right, sorry about it. Anyway, `swagger-mock-validator` complains about both fields
yousafn
2022-05-19 09:42
so I assume the `allOf` is saying you can have a response with
? Pet with its required field `PetType` and no name (as its optional)
? Pet with its required field `PetType` and name (as its optional)
Your Pact file is expecting both to be present
yousafn
2022-05-19 09:46
I will spend a bit of time on this tomorrow, as have a fair few meetings today, so really, REALLY appreciate you providing all the examples.
yousafn
2022-05-19 09:47
> yeah, you are right, sorry about it. Anyway, `swagger-mock-validator` complains about both fields
No problem. Just want to clarify so we are on the same page :slightly_smiling_face:
tati.shep
2022-05-19 09:53
JIC this is from http://swagger.io : `allOf` takes an array of object definitions that are validated _independently_ but together compose a single object. So, Pact file is expecting both to be present and I expect it to be the correct behaviour. https://swagger.io/docs/specification/data-models/oneof-anyof-allof-not/
yousafn
2022-05-19 10:01
Just about to head into another meeting but if you set both fields to required in the oas spec what is the validation outcome?
tati.shep
2022-05-19 10:07
the outcome is the same. it?s satisfied only when both of the fields are missing, like `"body": []` .
yousafn
2022-05-19 19:31
:question: time
A topic we love to hate, but we have to use.
*3rd Party API?s*
It is listed in our BDCT use cases https://docs.pactflow.io/docs/bi-directional-contract-testing/#use-cases
For Consumer-driven Contract testing we?ve said the following and the reasons is expanded on https://docs.pact.io/getting_started/what_is_pact_good_for#why-pact-may-not-be-the-best-tool-for-testing-public-apis
> Pact is not ideally suited to 3rd party APIs because 3rd parties are unlikely to validate your Pacts
For Bi-Directional Contract testing we said it can help
> By pulling in the third party?s API specification (OAS) regularly, you can continually ensure your consumers are compatible
But we are light on examples!
What common public API integrations would you like to see an example of?
If you have an OpenAPI spec of a public API, feel free to share.
If you want to get involved and showcase to a load of :eyes: ?s and demo your own example, let your devo :avocado: set the stage for you.
Even if you don?t have an answer to the above, but want to see some :meat_on_bone: on the :bone: , then just drop us some kind of emoji.
Drop us a reply in a :thread:
matt.fellows
2022-05-20 00:42
I think the classic example is:
1. 3rd party publishes an OAS describing their API
2. On a regular schedule (or ideally, triggered by a notification from the provider that the OAS has changed) a provider side job uploads the provider contract to Pactflow.
a. In this case, you would have to trust the provider tests their own API, so no actual verification work would need to be done
3. On the consumer side, it?s the same as other BDC examples (can use Pact or generate the examples from other tools/mocks)
rajnavakoti
2022-05-20 12:48
We recently started using pactflow tool for contract testing. I see in pactflow we can create different teams. But while publishing pacts to pactflow how to set the 'team name' or use team uuid ? we are using pact CLI to publish pacts.
we are using the below pact CLI cmd to publish pacts. I see there is no supported arguments for in 'publish' command to set team-uuid. We use --team-uuid when using 'create-webhook' command, looking for something similar for 'publish' command
```docker run --rm -v ${PWD}:${PWD}
pactfoundation/pact-cli publish ${{ pact_folder_path }}
--consumer-app-version ${{ git_commit_version }}
--tag ${{ git_branch }}
--broker-base-url ${{ pact_broker_url }}
--broker-token ${{ pact_broker_token }}```
matt.fellows
2022-05-21 02:04
You need to assign the consumer application to the team first, either through the UI or via APIs
harris
2022-05-23 03:17
HI there, quick questions regarding BDCT graphql, I know supporting graphql schema is on a roadmap, but is there any way I can use quick BDCT POC for graphql? What is the best strategies for the provider side? convert graphql schema to openapi doc? , btw our provider is REST api but wrapped in graphql mesh, and consumer is graphql. thanks :slightly_smiling_face:
matt.fellows
2022-05-23 03:19
I haven?t come across any ways to make BDCT and GraphQL work with OAS at this stage. If you can do that, that?s definitely plausible. I?d be surprised though, because the queries would be pretty darn hard to transpose into OAS JSON Schemas
harris
2022-05-23 03:40
cool thanks for that, the reason I want to try BDCT is I can use current MSW, but yeap I will stick to CDCT for now :slightly_smiling_face: thanks
matt.fellows
2022-05-23 03:40
when we get to GraphQL we?ll definitely announce it here in the #graphql chan
yousafn
2022-05-23 10:27
:wave:
I assume you?ve seen our MSW adapter https://docs.pactflow.io/docs/bi-directional-contract-testing/tools/msw
I?ve not got support in for graphQL yet.
I mentioned
> Not currently tested, please feel free to help contribute or let us know if you want to see it!
I?ve not used gQL for some years now on a project.
On gQL -> OpenAPI https://github.com/schwer/graphql-to-openapi
On the OpenAPI -> gQL, I have seen https://github.com/IBM/openapi-to-graphql
Wonder if they might be worth having a play around with?
Open to contributions to the pact-msw-adapter :raised_hands::skin-tone-2:
matt.fellows
2022-05-23 10:35
Yeah, so https://schwer.github.io/graphql-to-openapi/ is useless as far as a contract test is concerned. Because, it turns a structured query into a `string`, so now we have zero confidence that a given consumer query is compatible with a given provider (the other way around is much easier, as IBM have discovered)
matt.fellows
2022-05-23 10:36
i.e. in their example:
```query scalarQuery (
$input: String!,
$otherInput: String!
) {
arrayOfStrings(input: $input, otherInput: $otherInput)
}```
Is type as a `string` in the OAS. So now any non-empty query will pass a contract test, because it will satisfy `string`
rajnavakoti
2022-05-23 11:55
@matt.fellows where can I find the api documentation for this ? I tried to find it on https://docs.pact.io/pact_broker/advanced_topics/api_docs page but didn't find the right endpoint to do this ? can you please provide the link for API documentation (to assign customer app to a team) thanks in advance
matt.fellows
2022-05-23 12:47
We currently don?t have published docs for those Pactflow APIs, but we could probably point you at the ones needed. The simplest path is to navigate to the UI (see https://docs.pactflow.io/docs/user-interface/settings/teams) or you could also use the https://github.com/pactflow/terraform-provider-pact/
matt.fellows
2022-05-23 12:48
You could probably look at the network requests from the users page to see how they update the team membership, and then use that
bethskurrie
2022-05-24 00:28
@nathan.deamer that's a good question. I have a vague memory of it being because there is no main branch configured for the provider, so it can't tell what branch to trigger the build for.
bethskurrie
2022-05-24 08:20
Yes - just double checked the code. The webhook expects to fill in a branch name and a version number. If there has been no version published by the provider, and it doesn't know what the main branch is of the provider, it can't fill in either of those things.
bethskurrie
2022-05-24 08:21
We could potentially use `main` and `HEAD` as stand in values in that situation.
rajnavakoti
2022-05-24 15:24
Hey @matt.fellows one more followup question - we are using *docker run --rm pactfoundation/pact-cli:latest broker create-webhook* to create webhooks in *pactflow*.
we are using `--team-uuid` tag in the command to assign webhook to a particular team. When we use both --consumer and --provider arguments in the command the webhook is not getting assigned to the team. But if are not passing both, the webhook is assigned to the team. Is this intentional function ? or bug ?
`--consumer ${consumer}`
`--team-uuid ${team_uuid}`
This works
`--provider ${provider}`
`--team-uuid ${team_uuid}`
This works
`--consumer ${consumer}`
`--provider ${provider}`
`--team-uuid ${team_uuid}`
This is not working. webhook is created by not assigned to team
Sorry, I know this can be done through pactflow UI, but our team wants to use less UI and do everything as part of pipeline
agarwal.akash333
2022-05-26 01:37
Hi! It would be really cool to be able to quickly glance at the number of interactions for each version of a Pact, and maybe even number of failing interactions.
matt.fellows
2022-05-26 01:38
We?re looking to move to a more ?application centric? view in the UI, which will provide a home for each application and help clarify these sorts of things.
Although in this case you want it at the integration. level, right?
agarwal.akash333
2022-05-26 01:45
Hey Matt! A simple stat of number of interactions (total, and perhaps failing too) in the Overview tab, ?View Pact? page and perhaps the Matrix tab as well would do the trick. I am basically trying to quickly identify if the number of interactions changed between commits. It?s small but can be really helpful when you?re reviewing other people?s code/working together on a branch :slightly_smiling_face:
matt.fellows
2022-05-26 01:59
Thanks. It should be doable, but will need to pop it on the backlog. We?re in the process of moving our roadmap and feature request tool to another system (Product board). It should appear there in due course, but I think this is likely to be wrapped up in more of a UI redesign
agarwal.akash333
2022-05-26 02:05
No worries and that?s great, I?m glad it?s in the works regardless :slightly_smiling_face:
matt.fellows
2022-05-26 02:15
For clarity (I don?t want to mislead), reporting and insights is an area we want to improve. This specific feature would still need consideration etc.
matt.fellows
2022-05-26 02:16
You can (currently) follow the Pactflow roadmap here: https://github.com/pactflow/roadmap/projects/1
james.weng
2022-05-26 03:40
Is there an example of CI/CD using jvm? Would need some help how to pass tags/environments/consumer app version/provider version from gradle when doing provider /consumer test.
matt.fellows
2022-05-26 04:56
All of our examples can be swapped in for the workshops, to an extent.
But something like this is how I?ve done it in the past:
https://github.com/pactflow/example-provider-springboot/blob/master/build.gradle#L37-L44
harris
2022-05-27 00:07
hello just follow up question, for provider side, I can still use Pact? if converting grpahql to OpenAPI format hard to do that, I am thinking to stick provider side as Pact CDCT style. so workflow will be
`consumer` generate Pact using MSW adator, `provider` write provider side like CDCT and compare consumer generated pact.
the reason I want to do that is reusability from unit testing + contract in consumer side. so people easy to use
do you think it is good idea or not much getting benefit?
yousafn
2022-05-27 00:29
unfortunately you aren?t going to be in for a pleasant time using the msw adapter + CDCT provider verification due to no ability to use provider states or matchers in the request/response interactions.
yousafn
2022-05-27 00:30
if using regular CDCT provider verification, I would advise using `jest-pact` / `mocha-pact` dependant on your framework, or use `pact-js` directly, which is test framework agnostic, and you can use with whichever framework you are using.
yousafn
2022-05-27 00:32
It is just a wrapper around `pact-js` with convenience options for jest. so you should be able to follow this post https://pactflow.io/blog/contract-testing-a-graphql-api/
yousafn
2022-05-27 00:32
and this example from the pact-js repo https://github.com/pact-foundation/pact-js/tree/master/examples/graphql
harris
2022-05-27 00:33
```provider.addInteraction({
state: "Server is healthy",
uponReceiving: 'A request for API health',
willRespondWith: {
status: 200,
body: {
status: Matchers.like('up'),
},
},
withRequest: {
method: 'POST',
path: '/graphql',
},
})```
yousafn
2022-05-27 00:34
I?m about to hit the sack now mate, have a look at the pact-js example and it should be relatively straight-forward, see how you get on and if you get stuck, pop up a repo and ill have a gander tomo
harris
2022-05-27 00:36
:rolling_on_the_floor_laughing: haha yes those example are so useful cheers for good work :slightly_smiling_face:
james.weng
2022-05-27 05:32
Can i clarify the interactions for a consumer/provider CI/CD process. In the first PR for consumer would it always fail because the provider is not verified?
The other question i have is about the purpose of webhooks for pactflow. How should i use them in the process?
In my head im thinking
First consumer PR run
1. Run the consumer pact tests
2. Publish to pact
3. Run can i deploy? (will fail because provider is not verified)
but conversely if i try to deploy provider first without consumer run it will also fail because
1. Run the provider pact tests (nothing in broker cause consumer is not run and step will fail)
So should it be ?
1. Run consumer pipeline (fail because provider not verified)
2. Run provider pipeline (pass)
3. Rerun consumer again and make it pass
? Process seems quite lengthy and easy to "block" each other
Please advice
tjones
2022-05-27 05:48
Are you deploying as soon as you merge the PR? If yes, then your consumer PR run is right. However, you can improve it by running CI on push (but not PR) like this:
1. Run consumer pact tests
2. Publish to pact broker
This means the contract might already be verified at the time of the PR
> but conversely if i try to deploy provider first without consumer run it will also fail because
I think this depends. Personally, I think the consumer needs the provider, but the provider doesn't need the consumer. So, I would expect `can-i-deploy` for a provider with no verified consumers to pass.
tjones
2022-05-27 05:51
A subtle but important point is that the verification is done on consumer _contracts_, not consumer _versions_. So, if the consumer version changes, but the contract hasn't changed from a previously verified contract, pact knows that the new version's contract is already verified.
james.weng
2022-05-27 05:51
understand so i guess i will only run "can i deploy" after merge and during pr/pushes , i will run step 1&2 (Run and Publish). Can you explain to me more about webhooks and purpose in pactflow. I still dont really get it
tjones
2022-05-27 05:51
> "can i deploy"
I would run can-I-deploy right before you deploy, whenever that is
tjones
2022-05-27 05:53
The pact broker supports a few webhooks: https://docs.pact.io/pact_broker/webhooks
Which you can use to tie together the steps where you mention "rerun"
tjones
2022-05-27 05:53
So you can say:
`contract_requiring_verification_published` -> run provider verification
tjones
2022-05-27 05:54
I don't have a lot of experience with the webhooks - with small teams doing active development, it's ok to do this stuff manually or just wait for the next commit to re-trigger the workflow
tjones
2022-05-27 05:55
Something that is useful is to put `can-i-deploy` in a job all on its own, so that you can tell the difference between something broken in CI and a "hold off on this one for a bit" deployment
tjones
2022-05-27 05:56
depending on whether or not it's a good fit for your CI, you can also use `--retry-while-unknown=TIMES` to have `can-i-deploy` keep waiting on the verification status
james.weng
2022-05-27 05:59
Thanks. If you put the can i deploy to a separate job then if the whole process is continous delivery then would it then auto go to deployment even if that breaks?
tjones
2022-05-27 05:59
Ideally no. Ideally a successful can-i-deploy is a prerequisite for a deployment
james.weng
2022-05-27 06:00
yeap. so just wanted to clarify what u meant by having a separate job. do u mean triggering it as a separate job but it is still blocking the deployment if it fails?
tjones
2022-05-27 06:01
(also after a deployment, you need to tell the broker that you have deployed with `record-deployment`)
tjones
2022-05-27 06:02
Ah! Good question - I meant just for reporting reasons. There's a difference between an email (or whatever) that says `CI job can-i-deploy failed` and `CI job deploy failed`
tjones
2022-05-27 06:03
you don't _need_ them to be separate jobs. I just like to reduce the layers of indirection in CI pipelines - if you put `can-i-deploy` as the first line in a job called `deploy` then sometimes you will get failures you're expecting
tjones
2022-05-27 06:03
I like to have most pipelines be expecting to succeed unless something is wrong
tjones
2022-05-27 06:04
alternatively, and if your CI supports it, you might want to have a different status (skipped or blocked or whatever) when your `can-i-deploy` fails.
james.weng
2022-05-27 06:08
so a consumer would have a unique pacticipant and a provider will also have one?
tjones
2022-05-27 06:08
pacticipant is just a name for "consumer or provider". There have been some regrets about this naming choice
tjones
2022-05-27 06:10
For example:
```pact-broker can-i-deploy /
--pacticipant="$CONSUMER_NAME" /
--version="$CONSUMER_VERSION" /
--to-environment=production```
tjones
2022-05-27 06:11
hmm, I'm not sure. I think the current recommended practice is to use environment instead of tags. I'd have a look in the docs
james.weng
2022-05-27 06:15
what are your thoughts of spring cloud contract versus PACT and are there any other alternative contract testing tool that is worth considering
tjones
2022-05-27 06:17
I think someone did a comparison recently between SCC and Pact on youtube somewhere. I don't know SCC too well, but Pact has the substantial advantage that it supports several languages out of the box - so if you have a Spring consumer and a node provider, you can still use it
tjones
2022-05-27 06:17
I think Pact is the best there is, but then I used to be a pact maintainer, so I have to say that.
james.weng
2022-05-27 06:19
Yeap PACT has quite a lot of advantages compared to the rest and interestingly there are not many other competitors trying to provide contract testing solutions. In your opinion do you see any drawback / unsuitable scenarios in using contract testing
tjones
2022-05-27 06:21
Sometimes people do "schema" or "spec" validation in place of contract testing (and worse, sometimes they call this contract testing) - I think this is dangerous, because you're not confirming whether your implementations actually understand each other, just that they produce messages that conform to the same grammar. A good example of why this is a problem is Chomsky's "Colourless green ideas sleep furiously" - it's grammatically correct english, but not understood to mean anything.
See @matt.fellows?s excellent blog post on this topic here:
https://pactflow.io/blog/schemas-are-not-contracts/
tjones
2022-05-27 06:23
> In your opinion do you see any drawback / unsuitable scenarios in using contract testing
Currently (as you're experiencing) the effort required to understand it is high.
Specifically consumer driven contract testing is not a good fit if you don't know who your consumers are (eg, for a public API).
I think it's absolutely essential if you have any kind of complex architecture (eg, more than 2 services)
tjones
2022-05-27 06:26
There are graphql examples in the docs somewhere. I'm not too familiar with graph-ql because I personally don't like it. To me, it feels more like RPC, and it makes it really easy to give the consumer more information than it should have about the data model.
Sometimes graphql people will say "we don't need contract testing, because the contract is just graphql". I don't think this is true, because you can still have breaking changes - queries that are not valid for previous versions.
For messages, Pact doesn't actually send the messages during testing or verification (whereas for HTTP it does actually send the requests/responses) - so any of the message examples will work for kafka
bethskurrie
2022-05-31 06:26
@nathan.deamer the more I think about this, the less sure I am that this makes sense.
bethskurrie
2022-05-31 06:30
If there are no provider versions at all, that means the provider has not even got the provider-changed verifications running yet.
bethskurrie
2022-05-31 06:31
if they haven?t got the pacts verifying in their own build, it would seem strange that they would start with the pact changed workflow first.
jarekrzdbk
2022-05-31 14:47
Hello, does pactflow provide mechanism for backups?
sorry if it is somewhere, I couldnt information on this.
jarekrzdbk
2022-05-31 14:48
also I submitted form Pactflow Contract Testing Report - 2021. but did not get any response yet, should I wait for response, or is it possible to get this report any other way?
bethskurrie
2022-05-31 22:20
@jarekrzdbk are you talking about the saas product or the onprem product? For the saas product, we use AWS?s RDS backup feature, which gives us 5 minute snapshots. We also dump each database to a secure external location every day.
For the on-prem product, it will be up to the ops team to do their own backups of the database instance.
matt.fellows
2022-05-31 22:28
If you need more email and we can get an NDA setup to share our policy info
matt.fellows
2022-05-31 22:28
Hmm I'll follow this up. I believe it should be automated. Did you check spam?
jonatan.jaworski
2022-06-01 02:38
Hi team - Is there any place where we can check the broker version in Pactflow? So people checking pages https://docs.pact.io/pact_broker/webhooks#the-contract-content-changed-event can confirm which features are available at the moment.
matt.fellows
2022-06-01 04:51
If you head to the API (HAL Browser), or hit an API, you?ll see the version in the headers:
matt.fellows
2022-06-01 04:53
Good point though, we should possibly have this information more accessible
j3rry.wan9
2022-06-01 05:52
Hi team, whom should I contact about renewing my company's Pactflow subscription?
j3rry.wan9
2022-06-01 06:07
@matt.fellows Could you please help? I raised support request 701 for this.
matt.fellows
2022-06-01 09:25
:wave: hello there!
We are currently looking at some use cases to combine the value of SwaggerHub and Pactflow.
If one or more of the below sounds like you, I?d love to connect with you:
1. You follow a design first workflow to build your APIs
2. Are a Pactflow BDCT user
3. Use SwaggerHub
Just give this post a :thumbsup: and I?ll DM you for more.
jarekrzdbk
2022-06-01 09:48
Thank you @matt.fellows
I will mention it to my team, for now I am gathering info about different options on prem and saas etc.
I created simple POCs for contract tests using Pact and Pactflow, presented it to my team and now documenting all I was able to find
jarekrzdbk
2022-06-01 09:50
Yes, I checked spam,
maybe it did not work because I used my personal email... I will try with company email.
Thank you for quick responses.
matt.fellows
2022-06-01 10:25
Hi Jay, have you tried updating to the latest version of the libraries? If not, we?d appreciate a https://github.com/pact-foundation/pact-jvm/issues with the setup that?s failing so we can address it. There was definitely an issue with an earlier release
kannan.jai
2022-06-01 10:27
Hi Matt, thanks for getting back. No, I have not tried with the latest version. Will give it a try. Thanks.
mike.key
2022-06-01 21:05
Not sure if this is the correct channel for this question....I am a pactflow user and want to demonstrate stub server with pactflow....however I have been pushing my teams to use pact spec v3 or v4 and I notice this doc says only spec versions 1.* and 2 are supported. Is there a compelling reason I should/shouldn't want to use the latest spec versions vs. the benefits of stub server?
matt.fellows
2022-06-01 23:29
It depends on whether or not you?re using the v3/v4 features (e.g. generators). If you?re using them, then the stub server unfortunately currently won?t support the contracts
mike.key
2022-06-01 23:45
Thanks Matt?perhaps I've misunderstood the need/desire to stick with the latest specs?is there any doc on which and/or why I should or shouldn't use a specific spec?
aforeman
2022-06-02 05:53
Hey @jarekrzdbk - if you have any more issues, please send me a DM with the email address(es) you have used and I'll take a look :slightly_smiling_face:
jonatan.jaworski
2022-06-03 01:37
Hi team! I found that the https://docs.pactflow.io/docs/permissions/predefined-roles#cicd is missing the `webhook:manage:team` permission. It would be handy to have it added so that teams can configure their webhooks as code and deploy them using system accounts. Thoughts?
jonatan.jaworski
2022-06-03 02:20
I don't know how to do it, really. I was about to create my custom role with that permission added in the meantime. I guess I could do a PUT to `/admin/roles/{ci/cd role guid}` , but I don't know what the API contract looks like.
slin
2022-06-08 05:32
Hi there, Shuying from the Pactflow Team here :wave:.
I?ve been working on a new Pactflow user journey and couple interfaces to improve overall user experience, and I?d love some volunteers to complete a 45-min online user test. I can be flexible to work with your time schedule.
Please emote a :thumbsup: or reply under this thread, I?ll DM you about more detail. Cheers! :pactflow-platypus-slack:
anja.gruss
2022-06-09 11:13
hello, any pointers on how to analyze a pactflow broker message for the provider contract
"source sequence is illegal/malformed utf-8" with a error reference code
anja.gruss
2022-06-09 11:42
running the cat file command and decoding it on my machine seems to create utf-8 files, but on macos I am running it locally without the -w0 flag
yousafn
2022-06-09 12:07
Hi @anja.gruss
You can use our CLI tool now which will base64 the file for you - tracking it here
https://github.com/pactflow/roadmap/issues/68
which you can use this like
https://github.com/pactflow/example-bi-directional-provider-postman/pull/13
I will be updating the docs this week.
yousafn
2022-06-09 12:08
Just tested this out now :thumbsup: so should work for you :crossed_fingers:
``` docker run --rm -v /${PWD}:/${PWD} \
-w ${PWD} \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest pactflow publish-provider-contract \
oas/swagger.yml \
--provider "pactflow-example-bi-directional-provider-postman" \
--provider-app-version 152b2ac \
--branch test-pactflow-command \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results newman/newman-run-report-2022-06-09-12-20-39-465-0.json \
--verification-results-content-type text/plain\
--verifier postman```
yousafn
2022-06-09 12:08
can you provide any output from your CI logs, or where you are getting the error
anja.gruss
2022-06-09 12:09
I will try it with the CLI thingy first, and then get back to you, thanks!
matt.fellows
2022-06-09 12:38
If you can share the code here (or send to with that code) we can see what happened on this side of the fence and (hopefully) pinpoint the issue
matt.fellows
2022-06-09 12:39
I think we did see some issues the other day, where it looked like some fields (such as the provider name) were not populated correctly and instead had some sequence of `%` and spaces etc. in it.
anja.gruss
2022-06-09 13:31
@yousafn I just realized that the pact image that is used at the client uses the ruby gem install
```RUN gem install pact pact_broker-client pact-mock_service pact-message```
and then copies some custom scripts (can-i-deploy, pact-publish...)
how would we add the publish-provider-contract in this scenario?
yousafn
2022-06-09 13:59
pact_broker-client latest will have a `pactflow` binary, the command is listed on the readme and website
yousafn
2022-06-09 14:01
```pactflow publish-provider-contract \
oas/swagger.yml \
--provider "pactflow-example-bi-directional-provider-postman" \
--provider-app-version 3a0994c \
--branch test-pactflow-command \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results newman/newman-run-report-2022-06-09-14-00-55-366-0.json \
--verification-results-content-type text/plain\
--verifier postman```
anja.gruss
2022-06-10 10:16
hm, so currently I fail with an error for the verification-results path, and cannot find a good hint about this variable
abhishek.lamba
2022-06-10 10:18
Hi Team, I am doing my first bidirectional contract test. But at contract verification fails with following error
abhishek.lamba
2022-06-10 10:18
Request Header Is Incompatible
Value is incompatible with the parameter defined in the spec file: should have required property 'value'
Mismatched Pact Path:
[root].interactions[0]
Mismatched Provider Path:
[root].paths./**.post.parameters[0]
abhishek.lamba
2022-06-10 10:55
but there is no such property 'value' in either provider side or consumer side\
matt.fellows
2022-06-10 11:07
Regarding your previous error (thanks for sharing the error code), I?m seeing `source sequence is illegal/malformed utf-8` in the logs.
matt.fellows
2022-06-10 11:10
> hm, so currently I fail with an error for the verification-results path, and cannot find a good hint about this variable
this should be a path to a file, containing verification results for the provider test (the test that checks an OAS is compliant with the actual provider).
anja.gruss
2022-06-10 11:10
is there any documentation on this? when and how is that file created? and by who?
matt.fellows
2022-06-10 11:11
We are talking about step 3 and 4 here: https://docs.pactflow.io/docs/bi-directional-contract-testing/provider#step-3-verifying-the-provider-contract-testing-your-api
matt.fellows
2022-06-10 11:12
The publishing docs are here (courtesy of Yousaf): https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing
This is the bit that I think you?re asking about
anja.gruss
2022-06-10 11:16
so this is basically the result of https://docs.pactflow.io/docs/bi-directional-contract-testing/provider step 2 in this document?
matt.fellows
2022-06-10 11:17
Yes! Choose tool, test your provider, share the provider test output+OAS to Pactflow
matt.fellows
2022-06-10 11:17
How are you testing the provider? Or, separately, are you generating the OAS (e.g. from spring/code)?
matt.fellows
2022-06-10 11:22
Perfect, so for now, we just need a dummy file (e.g. an empty text file that says ?generated by springfox?). I think it?s a mandatory property.
yousafn
2022-06-10 13:40
Thanks buddy, here are the results you can see locally. Just for context, what are you seeing in your Pactflow broker, on each of the screens for your provider and consumer contract? Just wondering if there is enough detail as per the local output to help you diagnose. Will have a look through the errors now
`npx @pactflow/swagger-mock-validator Matching.yaml CA.json`
```? npx @pactflow/swagger-mock-validator Matching.yaml CA.json
Mock file "CA.json" is not compatible with spec file "Matching.yaml"
2 error(s)
request.authorization.missing: 1
request.header.incompatible: 1
0 warning(s)
{
warnings: [],
errors: [
{
code: 'request.authorization.missing',
message: 'Request Authorization header is missing but is required by the spec file',
mockDetails: {
interactionDescription: 'A valid post response',
interactionState: 'We have resposne from Ontology',
location: '[root].interactions[0]',
mockFile: 'CA.json',
value: {
description: 'A valid post response',
providerState: 'We have resposne from Ontology',
request: [Object],
response: [Object]
}
},
source: 'spec-mock-validation',
specDetails: {
location: '[root].security[0].apiKeyHeader',
pathMethod: 'post',
pathName: '/api/v1/ontology/normalizations',
specFile: 'Matching.yaml',
value: []
},
type: 'error'
},
{
code: 'request.header.incompatible',
message: "Value is incompatible with the parameter defined in the spec file: should have required property 'value'",
mockDetails: {
interactionDescription: 'A valid post response',
interactionState: 'We have resposne from Ontology',
location: '[root].interactions[0]',
mockFile: 'CA.json',
value: {
description: 'A valid post response',
providerState: 'We have resposne from Ontology',
request: [Object],
response: [Object]
}
},
source: 'spec-mock-validation',
specDetails: {
location: '[root].paths./api/v1/ontology/normalizations.post.parameters[0]',
pathMethod: 'post',
pathName: '/api/v1/ontology/normalizations',
specFile: 'Matching.yaml',
value: {
name: 'X-Tenant-ID',
in: 'header',
description: 'Internal Tenant ID',
required: true,
schema: [Object]
}
},
type: 'error'
}
]
}
Error: Mock file "CA.json" is not compatible with spec file "Matching.yaml"```
abhishek.lamba
2022-06-10 13:43
```{
code: 'request.header.incompatible',
message: "Value is incompatible with the parameter defined in the spec file: should have required property 'value'",
mockDetails: {
interactionDescription: 'A valid post response',
interactionState: 'We have resposne from Ontology',
location: '[root].interactions[0]',
mockFile: 'CA.json',
value: {
description: 'A valid post response',
providerState: 'We have resposne from Ontology',
request: [Object],
response: [Object]
}```
yousafn
2022-06-10 13:45
``` {
code: 'request.header.incompatible',
message: "Value is incompatible with the parameter defined in the spec file: should have required property 'value'",
mockDetails: {
interactionDescription: 'A valid post response',
interactionState: 'We have resposne from Ontology',
location: '[root].interactions[0]',
mockFile: 'CA.json',
value: {
description: 'A valid post response',
providerState: 'We have resposne from Ontology',
request: [Object],
response: [Object]
}
},
source: 'spec-mock-validation',
specDetails: {
location: '[root].paths./api/v1/ontology/normalizations.post.parameters[0]',
pathMethod: 'post',
pathName: '/api/v1/ontology/normalizations',
specFile: 'Matching.yaml',
value: {
name: 'X-Tenant-ID',
in: 'header',
description: 'Internal Tenant ID',
required: true,
schema: [Object]
}
},
type: 'error'
}```
`X-Tenant-ID` is required but not in your Pact file
yousafn
2022-06-10 13:49
The first error
``` {
code: 'request.authorization.missing',
message: 'Request Authorization header is missing but is required by the spec file',
mockDetails: {
interactionDescription: 'A valid post response',
interactionState: 'We have resposne from Ontology',
location: '[root].interactions[0]',
mockFile: 'CA.json',
value: {
description: 'A valid post response',
providerState: 'We have resposne from Ontology',
request: [Object],
response: [Object]
}
},
source: 'spec-mock-validation',
specDetails: {
location: '[root].security[0].apiKeyHeader',
pathMethod: 'post',
pathName: '/api/v1/ontology/normalizations',
specFile: 'Matching.yaml',
value: []
},
type: 'error'
},```
there are two `apiKeyHeader`'s defined in the spec
``` securitySchemes:
apiKeyHeader:
type: apiKey
name: Ocp-Apim-Subscription-Key
in: header
apiKeyQuery:
type: apiKey
name: subscription-key
in: query```
it is an error because `Ocp-Apim-Subscription-Key` is a defined header but is not present in the pact file
yousafn
2022-06-10 13:53
:raised_hands: Thanks for sharing the spec and pact file! Will look to improve our docs over time for failure diagnosing so it can be easy to help you all get on and fix things when they fail
jonatan.jaworski
2022-06-14 22:06
Hi team! The documentation for the admin endpoints is missing in the HAL browser. Does anyone know where to find that, or if that's something that will be available at some point? I'd like to do simple things like listing all the active system accounts (the UI doesn't allow filtering at the moment) and similar operations. Thanks!
bethskurrie
2022-06-15 00:42
@jonatan.jaworski we have it on our roadmap to get API docs published, but at the moment, we don?t have anything publicly available unfortunately.
bethskurrie
2022-06-15 00:45
The endpoint you want is `/admin/system-accounts`, however, there is no filtering supported on that endpoint at the moment, so you?d need to filter the response yourself.
jonatan.jaworski
2022-06-16 01:41
Hi! This might be a silly idea, but considering that the Open Api spec allows including "examples" (At least through Swagger), have you considered allowing using these for provider stubs? Could be useful for bi-directional testing.
matt.fellows
2022-06-16 02:32
Not a silly idea! Something we are certainly looking into.
We do want to be able to provide stubs for Provider contracts like we do for consumer contracts
matt.fellows
2022-06-16 02:32
SmartBear has some technologies in this space we?ll look to integrate to over time (e.g. their virtual server and mocking capabilities that SwaggerHub has).
druiz
2022-06-16 16:18
Hey there! I'm getting the following error:
`There is no verified pact between version 1.0.0 of demo-app and a version of GameService currently deployed or released to production (no version is currently recorded as deployed/released in this environment)` when running can-i-deploy. How do I configure/set the environment for my consumer + provider contracts? I couldn't find it in the Pact Broker Client CLI commands.
prasadsolanki
2022-06-16 16:19
Hi, I have just started using Pact. I created a free starter account here -https://pactflow.io/try-for-free/?utm_source=homepage&utm_content=hero . Successfully received email with credentials and url to Pactflow account. I could also log into it and see default example set up . But now I am trying to connect to the same account and it gives me error , the url is not reachable
prasadsolanki
2022-06-16 16:20
I am sure the url is correct as I am opening it through my email itself
yousafn
2022-06-16 16:50
Hi @jonatan.jaworski,
This is indeed something we are looking into and would love to garner feedback from users, new, seasoned and all in-between.
We have a developer preview program https://github.com/pactflow/roadmap#developer-preview-programs
which you can register for. This feels like a perfect candidate to get some real world feedback about how people would want to use it, what it might look like and go from there.
We've always wanted to be open and transparent and listening to the needs of the community and our customers, and what better way that our customers being able to directly shape our roadmap themselves.
To anyone following this thread, if you any ideas, big, small, something someone else is doing that you think the Pact or Pactflow team should look at, or integrations with other SmartBear products you may use or simply just know of, please don't hesitate to reach out
rholshausen
2022-06-16 23:23
It might be a proxy server on your network doing that. Maybe try from another machine or on another network.
matt.fellows
2022-06-17 01:54
You need to tell Pactflow when you have deployed your applications. You do so calling the `pact-broker record-deployment` command after you have deployed your application.
See https://docs.pact.io/pact_broker/recording_deployments_and_releases.
I?d also recommend going through the CI/CD workshop :point_down: for the detail here (howtolearn)
2022-06-17 01:55
Here are a number of useful hands-on labs that teach all of the key concepts: https://docs.pactflow.io/docs/workshops and https://docs.pact.io/implementation_guides/workshops
wng
2022-06-17 19:49
For BDCT, we're having some trouble figuring out this error.
```Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - id
Mismatched Pact Path:
[root].interactions[0].response.body.paymentMethods[0]
Mismatched Provider Path:
[root].paths./userprofile/v1/paymentmethods.get.responses.200.content.application/json.schema.properties.paymentMethods.items.additionalProperties```
In our provider contract, if we follow the provider path, items can be `anyOf`, as seen below
``` responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
paymentMethods:
type: array
items:
anyOf:
- $ref: '#/components/schemas/creditCard'
- $ref: '#/components/schemas/bankAccount'
schemas:
creditCard:
type: object
properties:
id:
type: integer
userId:
type: integer
externalId:
type: string
type:
type: string
lastFour:
type: string
isPrimary:
type: boolean
issuer:
type: string
verificationStatus:
type: string
expiration:
type: object
properties:
year:
type: integer
month:
type: integer```
We're wondering why the contract comparison can't find the schema definition? `paymentMethods[0]` is matching the credit card schema currently in our consumer contract
As an aside, should BDCT questions be in #general or #pactflow (since it's currently just a pactflow feature)?
yousafn
2022-06-17 21:19
Hey, some pointers here in the docs
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#allof-support-and-other-logical-keywords
Have you got the ref snippet for bankaccount?
If it differs, your provider could only ever provide one response, and your consumer could require both, or one.
BDC questions are best asked in here but appreciate it can be confusing, and some OSS users prefer not to see Pactflow messaging in general.
I never minded personally prior to joining the team, but can appreciate and understand.
Do you think a bi directional slack channel would be useful, especially given it is a big feature, and new so there is lots of overlap and interesting conversation that would be nice to group
wng
2022-06-17 21:29
Thanks for the response @yousafn. Yes, the bank account object is different
``` bankAccount:
type: object
properties:
id:
type: integer
userId:
type: integer
externalId:
type: string
type:
type: string
lastFour:
type: string
isPrimary:
type: boolean
issuer:
type: string
verificationStatus:
type: string```
Since Pact does not support those key words then, I assume we just don't want to contract test those use cases then? Or is there another route we should go?
I think a separate BDCT channel would be super helpful since it is a big feature and the documentation is still limited
wng
2022-06-17 21:30
Full context: the end point is /getAllPaymentMethods and it can return a credit card, bank account object or both, or multiples of any of the above
yousafn
2022-06-17 22:38
Are you able to use CDCT with the traditional pact verifier flow on this endpoint or are you looking at BDCT only? With Pact CDCT, you can control the behaviour of the your provider, with provider states which would give you finer grained control of the provider response.
You can run the bi directional validator locally, where you can control the flags to perform different validation behaviours. Pactflow is opinionated about its choice of flags, to prevent breaking changes that were not caught by the default behaviour of the original Atlassian tool.
https://www.npmjs.com/package/@pactflow/swagger-mock-validator
Not sure on the best advice at the moment but will noodle with the team and get back to you.
yousafn
2022-06-17 22:42
@wng you raised a canny issue earlier about SwaggerHub/Pactflow integrations.
The above may be of interest, and the pactflow roadmap would probably be the best place for that kind of ticket as it would cover Pactflow and SmartBear commercial offerings, not directly related to the OSS roadmap board ( canny )
Apologies as I know it can be a bit confusing
matt.fellows
2022-06-17 22:44
We really should be able to support the `anyOf` and `oneOf` operators. Unlike the `allOf` which I think is more problematic (albeit we could modify the behaviour to suit the standard use case), I think the validation behaviour is clearer
matt.fellows
2022-06-17 22:46
As Yousaf said, let?s bring this back to the team for review and come back to you (reminder set)
ben.a.hubbard
2022-06-20 11:36
Might be a stupid question but I've had a look around the docs and couldn't find anything. What are the options around configuring multiple brokers for consumers? (an existing pact broker and a newer pactflow instance) Or is there a way to transition to pactflow while keeping the history of verifications etc?
yousafn
2022-06-20 11:46
No this is an excellent question. I have posed it to the team. I am unsure if there is a way at the moment (although have tabled a couple of possible options)
Also you have got me wondering if there are cases of users with multiple OSS brokers, and how they manage them (if they do exist)
matt.fellows
2022-06-20 11:47
I believe we can take a dump of your current broker, and use that as a basis for your Pactflow account - i.e. start your account from the OSS and presto, you have Pactflow with all of your data.
matt.fellows
2022-06-20 11:48
then you would need to point your builds from the old OSS broker to Pactflow (it would require an outage as far as your existing setup is concerned, so could do it out of band potentially to reduce disruption).
matt.fellows
2022-06-20 11:49
Alternatively, the idea of a ?backup? where you shadow all (or, many) requests to pactflow also is possible, but we don?t have a solution for it atm
matt.fellows
2022-06-20 11:49
Depending on what data you need, it might be a few API calls to fetch and pump in the things you need
anja.gruss
2022-06-20 12:18
is it just me or is the verifier paramter in the publish-provider-contract command not really doing a lot?
yousafn
2022-06-20 12:26
Hi,
It's just a free text state to state which verifier tool is being used (which is up to the user to provide)
What behaviour are you expecting
anja.gruss
2022-06-20 12:30
not much, I just was not sure if it's necessary at all :slightly_smiling_face:
yousafn
2022-06-20 12:34
So the rationale behind it, although it is free-text, is if we see recurring tools being used, we may be able to provide first class support for the results and render them nicely in the UI, if it was something users had an appetite for.
ben.a.hubbard
2022-06-20 12:54
Thanks for the replies! I think the multiple brokers for a consumer would be preferable right now as we just want to test out pactflow (bi-directional etc) before fully committing. Can you let me know @yousafn if you hear anything?
ben.a.hubbard
2022-06-20 12:54
Although knowing how to switch over would still be useful! Is there anything I can read up about on that @matt.fellows?
matt.fellows
2022-06-20 23:15
And also over time, display that context within the Pactflow UI (so you can see your tech stack, tools used for testing, integration technologies etc.)
drakulavich
2022-06-23 12:14
Hello!
Trying to migrate from gitlab ci to github actions. I used https://github.com/pactflow/example-bi-directional-provider-dredd as a reference (Thank you Pactflow team for that). Sending the provider contract works on local machine but is failing on github with strange error:
```NoMethodError - undefined method `request_uri' for #<URI::Generic ?>```
drakulavich
2022-06-23 12:16
`PACT_BROKER_BASE_URL` and `PACT_BROKER_TOKEN` are specified the same way as in the example repo. Don't understand what is failing exactly:
```PACTICIPANT=ghapi \
""docker run --rm -v /home/runner/work/marketplace-api/marketplace-api:/app -w "/app" -e PACT_BROKER_BASE_URL -e PACT_BROKER_TOKEN pactfoundation/pact-cli"" pactflow publish-provider-contract \
/app/src/test/resources/openapi/swagger.json \
--provider ghapi \
--provider-app-version fefdb4369a5ef60abdfc19ac6edf82aa8d7d6794 \
--branch ci-publish_reports \
--content-type application/json \
--verification-exit-code=0 \
--verification-results /app/target/failsafe-reports/failsafe-summary.xml \
--verification-results-content-type text/xml \
--verifier junit5```
matt.fellows
2022-06-23 22:30
try adding the `--verbose` flag, that should print out detail about the arguments it received and what requests it?s firing
bethskurrie
2022-06-24 03:22
```$ irb
irb(main):001:0> URI("foo").request_uri
NoMethodError (undefined method `request_uri' for #<URI::Generic foo>)
irb(main):002:0> URI("https://foo").request_uri
=> "/"```
bethskurrie
2022-06-24 03:23
`<URI::Generic ?>` that suggest to me that the PACT_BROKER_BASE_URL env var is empty..
bethskurrie
2022-06-24 03:24
You can see in my example it included the value of the URL in the error `<URI::Generic foo>`
bethskurrie
2022-06-24 03:38
The simplest way to do it is duplicate the publish/verify commands in your build and run them in parallel until everything has been pushed out to prod, then turn off the old one.
matt.fellows
2022-06-24 03:52
If it were empty, i?d expect the CLI to tell me I?m missing an important argument - does it not do that?
drakulavich
2022-06-24 05:20
Thank you for the suggestions. The output with `--verbose` flag:
```NoMethodError - undefined method `request_uri' for #<URI::Generic ?>
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/hal/http_client.rb:49:in `create_request'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/hal/http_client.rb:25:in `get'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/hal/link.rb:41:in `get'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/hal/link.rb:45:in `get!'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/hal_client_methods.rb:24:in `index_resource'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/versions/create.rb:48:in `branch_versions_supported?'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/versions/create.rb:11:in `do_call'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/base_command.rb:25:in `call'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/base_command.rb:13:in `call'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pactflow/client/provider_contracts/publish.rb:37:in `create_branch_version_and_tags'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pactflow/client/provider_contracts/publish.rb:24:in `do_call'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/base_command.rb:25:in `call'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/base_command.rb:13:in `call'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pactflow/client/cli/provider_contract_commands.rb:36:in `publish_provider_contract'
/usr/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
/usr/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
/usr/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
/usr/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/lib/pact_broker/client/cli/custom_thor.rb:23:in `start'
/usr/lib/ruby/gems/2.7.0/gems/pact_broker-client-1.64.0/bin/pactflow:4:in `<top (required)>'
/usr/bin/pactflow:25:in `load'
/usr/bin/pactflow:25:in `<main>'```
drakulavich
2022-06-24 05:23
The env vars in github workflow yaml:
``` PACT_BROKER_BASE_URL: https://mycompany.pactflow.io
PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}```
bethskurrie
2022-06-24 05:29
```$ PACT_BROKER_BASE_URL="" be bin/pact-broker publish example/scripts/pact.json --consumer-app-version 1
PactBroker::Client::Error - Please specify the pact_broker_base_url```
bethskurrie
2022-06-24 05:33
```export PACT_BROKER_BASE_URL=""
bundle exec bin/pactflow publish-provider-contract \
script/oas.yml \
--provider Foo \
--provider-app-version 1013b5650d61214e19f10558f97fb5a3bb082d44 \
--branch main \
--tag dev \
--specification oas \
--content-type application/yml \
--verification-exit-code 0 \
--verification-results script/verification-results.txt \
--verification-results-content-type text/plain \
--verification-results-format text \
--verifier my-custom-tool \
--verifier-version "1.0" \
--verbose```
bethskurrie
2022-06-24 05:33
```NoMethodError - undefined method `request_uri' for #<URI::Generic ?>
/pact_broker/client/hal/http_client.rb:49:in `create_request'
/pact_broker/client/hal/http_client.rb:25:in `get'
/pact_broker/client/hal/link.rb:41:in `get'
/pact_broker/client/hal/link.rb:45:in `get!'
/pact_broker/client/hal_client_methods.rb:24:in `index_resource'
/pact_broker/client/versions/create.rb:48:in `branch_versions_supported?'
/pact_broker/client/versions/create.rb:11:in `do_call'
/pact_broker/client/base_command.rb:25:in `call'
/pact_broker/client/base_command.rb:13:in `call'
/pactflow/client/provider_contracts/publish.rb:37:in `create_branch_version_and_tags'
/pactflow/client/provider_contracts/publish.rb:24:in `do_call'
/pact_broker/client/base_command.rb:25:in `call'```
bethskurrie
2022-06-24 05:36
If I do `unset PACT_BROKER_BASE_URL` then I get an error
`No value provided for required options '--broker-base-url'`
matt.fellows
2022-06-24 05:51
It?s very easy in YAML to be out by an indentation and for the values to look like they?re there, but actually aren?t present at all. GH should print the environment output (possibly redacted) at the top of the run.
bethskurrie
2022-06-24 06:21
I?ve updated the validation for the command.
```export PACT_BROKER_BASE_URL=""
bundle exec bin/pactflow publish-provider-contract \
script/oas.yml \
--provider Foo \
--provider-app-version 1013b5650d61214e19f10558f97fb5a3bb082d44 \
--branch main \
--tag dev \
--specification oas \
--content-type application/yml \
--verification-exit-code 0 \
--verification-results script/verification-results.txt \
--verification-results-content-type text/plain \
--verification-results-format text \
--verifier my-custom-tool \
--verifier-version "1.0" \
--verbose
No value provided for required option --broker-base-url or environment variable PACT_BROKER_BASE_URL ```
drakulavich
2022-06-24 06:26
Thank you so much!
I found the root cause after adding explicit `echo`s. The problem was in my make file. I have:
```cp -n .env.example .env || true```
So, on github runner it creates an `.env` that looks like:
```PACT_BROKER_BASE_URL=
PACT_BROKER_TOKEN=```
drakulavich
2022-06-24 06:38
BTW, all examples on github have `.env` target in the pact related actions: https://github.com/pactflow/example-bi-directional-provider-dredd/blob/master/Makefile#L34
For me it doesn't expose variables from the existing `.env` file on macos. So I ended up adding the snippet on the top of Makefile:
```# Export all variable to sub-make if .env exists
ifneq (,$(wildcard ./.env))
include .env
export
endif```
Am I doing it wrong way?
matt.fellows
2022-06-27 22:28
It's been added to our backlog for review. Officially we'd like to be able to support it
drakulavich
2022-06-29 11:29
Hi everyone!
All examples on github have `.env` target in the pact related actions: https://github.com/pactflow/example-bi-directional-provider-dredd/blob/master/Makefile#L34
For me it doesn't expose variables from the existing `.env` file on macos. So I ended up adding the snippet on the top of Makefile:
```# Export all variable to sub-make if .env exists
ifneq (,$(wildcard ./.env))
include .env
export
endif```
Am I doing it wrong way?
matt.fellows
2022-06-29 11:41
I think the `.env` was copied over from the canonical `example-consumer` example, but the use of the `dotenv` package hasn?t gone with it (for several reasons).
francisco.moreno
2022-07-03 17:30
Hi, all. I post here this article that we wrote in our company. Hope helps!
chriswilliamsef
2022-07-04 14:50
We?re running into the same thing now - a large swagger def. of 16k lines with 150 uses of `allOf` in, is there any update or sense of timescale for this being reviewed? Thanks :slightly_smiling_face:
matt.fellows
2022-07-04 23:24
It?s currently on the prioritised backlog if issues to solve around the new launch. I can?t give timeframes at this stage.
matt.fellows
2022-07-04 23:24
We are considering a few approaches, one of them is to look at pre-parsing through something like https://www.npmjs.com/package/json-schema-merge-allof
matt.fellows
2022-07-04 23:25
So you could potentially look at a similar solution - merge the `allOf` and published the tweaked OAS
chriswilliamsef
2022-07-05 10:50
Thanks for your replies, we?re going to look at the `json-schema-merge-allof` solution I think, and accept that the OAS file we push to the broker isn?t _quite_ the original
matt.fellows
2022-07-05 10:54
One problem I can foresee is that debugging problems could be challenging, because you won?t be comparing the uploaded OAS with the (mutated) one that was validated. But my assumption is that most errors should still look the same
chriswilliamsef
2022-07-05 11:00
yea, depends how agressive / opinionated the lib is in how it generates it?s output - gotta try it to see i guess!
ben.a.hubbard
2022-07-05 12:20
The Matrix tab on pactflow UI isn't displaying accurate verification statuses for me. "Can I deploy" and cross-contract-verification-result pages in the UI do give the correct result however. It's happening fairly frequently so wasn't just a one off. I've tried the refresh button but it doesn't change. Should they not just be displaying the same status?
chriswilliamsef
2022-07-05 16:32
We had better luck combining https://www.npmjs.com/package/json-schema-resolve-allof and https://github.com/APIDevTools/json-schema-ref-parser to turn our Openapi spec into something that now passes :heart:
chriswilliamsef
2022-07-05 16:33
only downside is with no `$ref` the spec file jumped from a 506K yaml, to a 695K JSON, to a 2.8MB de-$ref?ed / de-allOf?ed monster
bethskurrie
2022-07-05 23:06
@ben.a.hubbard can you send an email to with the details of the issue please? If you include screenshots and the URLs of the problem contracts and matrix page, we should be able to debug the issue.
chriswilliamsef
2022-07-06 08:11
I don?t think you can remove the `allOf`s without also removing the `$ref`s
ganesh.panchagnula-no
2022-07-06 19:02
Hello, I have just signed up for pactflow and created an account. Now I would like to publish the pact file generated by the consumer to my pactflow account. I have the authentication token with me. My question is how to publish the pact file to http://pactflow.io on Windows/.Net core. I have seen this example, however this is not working. any ideas on this?
`$Headers = @{`
Authorization = "Basic $([System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("dXfltyFMgNOFZAxr8io9wJ37iUpY42M:O5AIZWxelWbLvqMd8PkAVycBJh2Psyg1")))"
}
$res = Invoke-WebRequest -Uri "https://test.pactflow.io/pacts/provider/Animal%20Profile%20Service/consumer/Matching%20Service/version/1.0.0" -Headers $Headers -Method Put -InFile .\matching_service-animal_profile_service.json -ContentType "application/json"
matt.fellows
2022-07-07 03:52
The advice is to use https://docs.pact.io/implementation_guides/cli.
I don?t believe any official docs still point to a manual powershell publish.
ganesh.panchagnula-no
2022-07-07 09:50
thanks Matt. The links on the above page pointed me to Pact ruby standalone executable at https://github.com/pact-foundation/pact-ruby-standalone/releases. I did download pact-1.89.02-rc1-win32 zip, unzipped into local folder. Tried to run pactflow.bat from bin folder command prompt with this:
```pactflow publish-provider-contract C:\pact-workshop-dotnet-core-v3\pacts\ApiClient-ProductService.json --broker-token=myAuthenticationtoken --broker-base-url=https://mydomain.pactflow.io --provider ProductService --provider-app-version 1.0 --branch main --content-type application/json --verification-exit-code=0```
ganesh.panchagnula-no
2022-07-07 09:56
I did also installed the certificate and set the environment variable to the path as explain here https://docs.pact.io/pact_broker/advanced_topics/using-tls#extracting-a-copy-of-the-certificate-from-a-running-pact-broker, however I am still getting the same error
matt.fellows
2022-07-07 10:00
hmm you shouldn?t need to install custom certificates - it should work out of the box
matt.fellows
2022-07-07 10:00
Can you please share verbose logs? One guess would be an intermediate proxy presenting an invalid certificate
ganesh.panchagnula-no
2022-07-07 10:05
Please find the logs:
```opening connection to http://mydomain.pactflow.io:443...
opened
starting SSL for http://mydomain.pactflow.io:443...
Conn close because of connect error SSL_connect returned=1 errno=0 state=error: certificate verify failed
OpenSSL::SSL::SSLError - SSL_connect returned=1 errno=0 state=error: certificate verify failed
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/ruby/lib/ruby/2.4.0/net/protocol.rb:44:in `connect_nonblock'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/ruby/lib/ruby/2.4.0/net/protocol.rb:44:in `ssl_socket_connect'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/ruby/lib/ruby/2.4.0/net/http.rb:948:in `connect'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/ruby/lib/ruby/2.4.0/net/http.rb:887:in `do_start'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/ruby/lib/ruby/2.4.0/net/http.rb:876:in `start'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/hal/http_client.rb:79:in `block in perform_request'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/hal/http_client.rb:93:in `until_truthy_or_max_times'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/hal/http_client.rb:64:in `perform_request'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/hal/http_client.rb:25:in `get'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/hal/link.rb:41:in `get'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/hal/link.rb:45:in `get!'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/hal_client_methods.rb:24:in `index_resource'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/versions/create.rb:48:in `branch_versions_supported?'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/versions/create.rb:11:in `do_call'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/base_command.rb:25:in `call'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/base_command.rb:13:in `call'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pactflow/client/provider_contracts/publish.rb:37:in `create_branch_version_and_tags'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pactflow/client/provider_contracts/publish.rb:24:in `do_call'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/base_command.rb:25:in `call'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/base_command.rb:13:in `call'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pactflow/client/cli/provider_contract_commands.rb:36:in `publish_provider_contract'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/vendor/ruby/2.4.0/gems/pact_broker-client-1.65.0/lib/pact_broker/client/cli/custom_thor.rb:23:in `start'
C:/Users/testuser/Downloads/pact-1.89.02-rc1-win32/pact/lib/app/pactflow.rb:28:in `<main>'```
ganesh.panchagnula-no
2022-07-07 11:05
@matt.fellows you might be correct. I was trying to connect through my company laptop and getting that connection error. I did try on my personal laptop, and it was successful. So something on my company laptop is stopping the successful connection. I have two questions if you could help
1. Any idea what's stopping me to make a successful SSL connection on my company laptop. Ultimately this is the environment we would be running our tests
2. And in Pactflow, I see this . There is no information about consumer. When I checked "pactflow publish-provider-contract", it has no options to provide information about consumer. Is there another command to publish the contract generated by the consumer to pactflow?
matt.fellows
2022-07-07 11:59
that?s great news. Now to isolate your corporate environment issue:
1. There are likely two paths to success. 1 set or unset the `http_proxy` and `https_proxy` environment variables. If you don?t know what these are, find somebody in your organisation that does. They?ll be able to advise what to do with them. If not this, you?ll need to get a copy of the intermediate proxy SSL certificate and get that into your trust store (the docs above you linked are the right ones). This is unlikely the problem though, so start with the env vars
2. You don?t publish the consumer with your provider contract - each of your consumers publish contracts, and when they do, there will be an integration in the left nav for each consumer that says `ProductService` is a provider
ganesh.panchagnula-no
2022-07-07 13:46
thanks Matt . I will check with IT support team on proxy environment variables.
And just be clear on point 2), you said each of my consumer should publish the contract. How do I do this through pactflow?
matt.fellows
2022-07-07 13:47
ganesh.panchagnula-no
2022-07-07 13:57
Ok, some progress made. As described here I did use _pact-broker publish_ command to publish the pact file generated by the consumer
ganesh.panchagnula-no
2022-07-07 13:59
And it did update status on Pactflow . I think now I need to perform the verification by the provider and upload the results to the pactflow
alexstout009
2022-07-07 16:21
Hi, we are trying to setup a new provider for pact flow but when running the pact.verify() we get this error
```Error retrieving https://cinch.pactflow.io/pacts/provider/marketplace-gateway-component/for-verification status=404 (Pact::Hal::ErrorResponseReturned)```
harris
2022-07-07 23:39
question regarding using pactflow(hosted broker), is there any disadvantage to use both BDCT and CDCT ? and store pacts in pactflow?
if team, use REST services( both provider and consumer) AWS lambda + API gateway=REST and they want to use BDCT, and the other services (graphql) uses CDCT, is there any issue to mix both contract way? no right? because each pacts are mapping separately and only concern their contracted services each other. so people can select whatever they want to use?
madhulika.mitra
2022-07-08 14:57
Hi - Our OAS has a lot of allOfs , oneOfs and we are getting false failures while doing BDCT. Is there a quick setting on the pact side to make "additionalProperties":true - rather than adding it on multiple times on schema? Requesting help here.
bethskurrie
2022-07-12 20:46
@ben.a.hubbard sorry this has been so slow to fix. The pactflow dev team is in Boston at the moment (we flew from Australia) , and we're in strategy meetings all day. I identified where the issue is happening, I just haven't worked out the fix for it yet.
ben.a.hubbard
2022-07-13 07:42
No worries! Thanks for the update :slightly_smiling_face: Enjoy your time in Boston
noor.hashem
2022-07-14 17:50
Hi all, I am verifying a pact on pactflow and it shows that it is verified in terminal but once I go to pactflow it still shows unverified. Is there a command I need to run to get it to ber verified in pactlfow. I am simply just running yarn test for now.
matt.fellows
2022-07-14 20:34
You'll need to publish the verification results. There is a property you need to enable in CI only (usually) to do that
matt.fellows
2022-07-14 20:40
Interesting question. If we set it to true, you are likely to get false positives - which are worse
tjones
2022-07-15 05:13
Also, it's good practice to get your token from an environment variable or encrypted secret instead of pasting in the file directly
yousafn
2022-07-15 09:05
Hey @madhulika.mitra sorry for the late reply, we have all been away on an offsite for the week.
The current functionality provided in swagger-mock-validator, although configurable, we have been opinionated in pactflow, with the defaults chosen and these are not currently configurable by our end users because it can lead to false positives.
As mentioned, if you can share your oas either here or directly with myself or Matt, that would be useful to see how we can help
rxiao
2022-07-19 18:04
Hi team, I already have a OAS and wanna publish it to pactflow. I?m wondering why it will give me this error?
```./pact/bin/pactflow publish-provider-contract ./openapi.yaml --provider=CC-Go-Template-Service-Provider -a v0.01 --broker-base-url=https://mydns.pactflow.io -k $PACT_TOKEN --branch main --contentType=application/yaml --verification-exit-code=0```
`ERROR: "pactflow publish_provider_contract" was called with arguments ["./openapi.yaml", "--contentType", "application/yaml"]`
`Usage: "pactflow publish-provider-contract CONTRACT_FILE ... --provider=PROVIDER -a, --provider-app-version=PROVIDER_APP_VERSION -b, --broker-base-url=BROKER_BASE_URL"`
PS: I?ve read the https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow page and its repo, I?m wondering if it?s necessary to have the collection first and run the newman in CI then publish?
yousafn
2022-07-19 18:06
You are missing some properties. :+1::skin-tone-4:
--verification-results REPORT_PATH \
--verification-results-content-type REPORT_CONTENT_TYPE \
--verifier VERIFIER
yousafn
2022-07-19 18:09
You can provide any result file, even a plain text file if you do not have a codebase yet to verify
rxiao
2022-07-19 18:10
```./pact/bin/pactflow publish-provider-contract ./openapi.yaml --provider=CC-Go-Template-Service-Provider -a v0.01 --broker-base-url=https://mydns.pactflow.io -k $PACT_TOKEN --branch main --contentType=application/yaml --verification-exit-code=0 --verification-results ./result.txt --verification-results-content-type text/plain --verifier postman```
`ERROR: "pactflow publish_provider_contract" was called with arguments ["./openapi.yaml", "--contentType", "application/yaml"]`
`Usage: "pactflow publish-provider-contract CONTRACT_FILE ... --provider=PROVIDER -a, --provider-app-version=PROVIDER_APP_VERSION -b, --broker-base-url=BROKER_BASE_URL"`
rxiao
2022-07-19 18:11
still same error:thinking_face:, and the consumer pact already published in pactflow.
rxiao
2022-07-19 18:29
I think it?s -a
```Options:
--provider=PROVIDER
# The provider name
-a, --provider-app-version=PROVIDER_APP_VERSION
-b, --broker-base-url=BROKER_BASE_URL```
Even I change to full-name, still same error:
```./pact/bin/pactflow publish-provider-contract ./openapi.yaml --provider=CC-Go-Template-Service-Provider --provider-app-version=v0.01 --broker-base-url=https://confluent.pactflow.io -k $PACT_TOKEN --branch main --contentType=application/yaml --verification-exit-code=0 --verification-results ./result.txt --verification-results-content-type text/plain --verifier postman```
matt.fellows
2022-07-19 18:30
Ah sorry. It might have been the formatting of the output. Sorry I'm on mobile
rxiao
2022-07-19 22:17
find out the problem, seems like the result must be json file:
`--verification-results ./result.json`
Thanks for your concern Yousaf, Matt!
rxiao
2022-07-20 03:03
hmmmm, weird then, cause that?s the only change I made in the cmd line:thinking_face:
noor.hashem
2022-07-20 14:30
Hi all, is there full compatibility for ci/cd integration with both pact and pactflow? What would be the main differences for ci/cd when using pactflow vs when using just pact and having you own broker.
matt.fellows
2022-07-20 22:28
Yes there is. You can perform the same automation workflows with both tools.
The main difference is that the OSS broker doesn't have any real authentication, user, secret management etc.
The main differences are listed here: http://pactflow.io/features
yousafn
2022-07-20 23:44
The self verification results by the provider signified a failure, this would normally be the error code from your providers test run.
We are looking for evidence as part of the test run, that the provider can implement the specification. The onus is on the provider testing team in BDCT, where the Pact Verifier would do the heavy lifting for us here, in CDCT.
As such the provider self verification are free text so we currently can provide the user granular detail, but as we see common tools being used, we may be able to build parsers to extract suitable information a particular tools output. ( junit format might be something nice and standardised )
Ultimately it would be nice to get more insight so we could say providers tests covered x% of the provider spec, so you know what you can safely consume
yousafn
2022-07-20 23:45
Clicking on the view log button will show the user the corresponding report uploaded by the provider team, which was associated with the failure exit code ( or a flag to say no success )
tjones
2022-07-20 23:52
It's not clear to me whether this message is saying that the provider code has the wrong version of something pact related, or whether it means the contract was violated. It might be better to say something like "Your ${contractType} contract is violated by the provider"
lewis.prescott079
2022-07-21 06:16
The message isn't very clear when you are verifying openapi spec. It's not really the codebase that's involved
yousafn
2022-07-21 14:45
in this instance you are self-verifying a provider codebase against the provider's specification (currently only OAS is supported) to ensure that implementation of the provider matches that documented in the specification.
the provider contract we are speaking of consists of the spec + verification results, so it possibly be more correct to say,
"Your provider specification is not compatible with your provider codebase" and the user to be guided to view the results and determine what action to take.
I would agree that the messaging could be improved and more directive as to the next steps, the current docs around the provider publication are here
https://docs.pactflow.io/docs/bi-directional-contract-testing/provider#step-4-publish-your-provider-contract-and-verification-results
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#interpreting-verification-result-failures
https://docs.pactflow.io/docs/user-interface/bi-directional#provider-contract-tab
Thanks for the feedback Tim/Lewis
yousafn
2022-07-21 14:49
There is a some detail on the cross compatibility failures between the consumer + provider for BDCT but not enough on the provider contract (spec plus evidence) failures.
I think a genuine quick win improvement is getting the UI screenshots up there on the docs, along with highlights to describe the valid/invalid views and next steps (along with provided next steps in the error message too) - danke
tjones
2022-07-22 00:06
Specifically the word codebase is confusing, I think. To me that reads like ?the problem is the version of something that your code is based on, not your code?
tjones
2022-07-22 00:07
Also ?not compatible? is ambiguous for the same reason- could you not run it, or did it fail?
tjones
2022-07-22 00:18
I'd prefer something like "Your ${contractType} contract is violated by the provider"
tjones
2022-07-22 04:38
Other options:
? Your ${contractType} contract is not correctly implemented by the provider
? Your ${contractType} contract is broken by the provider
? Your ${contractType} contract provider verification failed
wilco.van.esch
2022-07-22 14:27
Hi all! We are looking at BDCT using Pactflow. I'd love a sanity check on some of my thinking on the practical workflow.
Say a developer on a Provider team makes a deliberate breaking change to an endpoint and updates the OAS accordingly. The API tests would have to be updated at the same time to ensure the contract which will be generated from the OAS matches the change being made to the endpoint. To allow a test specialist to update the API tests, would you then have the developer and test specialist make their changes on the same branch (whether it be async or in a pair programming session)? How are others doing this?
Also, say the Consumer is ready with an update to match the change on the Provider side. The Provider isn't able to merge their change yet, because they're breaking a Consumer. Because of this, the Consumer also isn't able to merge their change, because the Provider doesn't meet their new expectations yet. Is this resolved by the Provider and Consumer using identically named branches in their respective repos so that the contracts can be tagged and verified against their working branches? Or is this where the WIP Pacts feature would tell the Consumer that they get a fail for the main branch but they may get a pass from the WIP pacts which includes the new Provider one and which will unblock that Provider? :confused-dog-2741:
ben.a.hubbard
2022-07-22 15:47
For the merging a provider's breaking change question I've just used https://docs.pact.io/faq#how-can-i-make-a-breaking-change-to-a-provider part of the original pact FAQ before. It's not BDCT specific but I feel it doesn't really matter and still applies that any changes should be backwards compatible with consumers.
I'm still learning BDCT myself so there may be another way!
tjones
2022-07-24 14:47
I don't know much about how BDCT are supposed to work, but I agree with Ben.
In OAS documentation, there's often definitions of breaking changes like "removing an endpoint" or "renaming a field". This is overly wide - it's not a breaking change to remove an endpoint that no one is using.
In Pact, the definition of breaking change is any change to an API which _actually_ causes a failure in a consumer or provider.
Pact isn't supposed to enable you to make this kind of breaking changes, it's supposed to help you _not_ make breaking changes, so that you can safely deploy with no downtime.
Even without pact, how would you deploy a completely breaking API change safely? Unless you can atomically deploy the consumer and the provider, you'd still need a consumer that can talk to the old and new provider.
matt.fellows
2022-07-24 23:16
What everyone said.
This https://docs.pact.io/faq#how-can-i-make-a-breaking-change-to-a-provider is our recommended practice for breaking changes, but is pretty much the above.
j3rry.wan9
2022-07-25 08:24
Hi team, is it possible to create/disable/delete users using the REST API endpoint `admin/users` in Pactflow? If yes, where can I find the documentation? I'm trying to build some kind of user onboarding automation, and the built-in HAL Browser isn't very useful IMHO.
hazem
2022-07-25 23:40
question about https://pactflow.io/blog/hosted-stubs/, this seems like a really useful feature as described in the blog post, eg starting your react app with the consumer pact stubs.
however, how would one handle the case where a consumer (let's say react web app), has many providers? it looks like the api support in pactflow supports a stub endpont of 1:1 , provider:consumer. when really i want All Providers for a given consumer.
yousafn
2022-07-26 09:33
hey hey,
you would get a hosted api stub for each consumer/provider pair, and provide them to your apps configuration, in the same way that you could configure your app to point at several _real_ provideres
alan.still
2022-07-26 12:26
Hi Team :wave: . I?m having some issues with a new Webhook that does not seem to be Authorising with the GitHub url i am trying to trigger. Testing the trigger always returns a 404 from GitHub.
We have other webhooks (all still working) that use the same auth token (stored in Secrets) and I can also trigger the GitHub repo using a simple curl request from my Terminal.
I have also regenerated our GitHub Auth Token, just to see if that could help.
Happy to provide links to our webhooks on pactflow, if you think it will help. Thanks
wng
2022-07-26 14:50
Resurrecting this thread for a different use case.
We are using an api gateway and writing contract tests in the gateway, but adding a new consumer contract there breaks the pipeline if the provider oas isn't set up yet and we have no way to set the consumer to be a WIP pact. Overriding isn't a great option every time temporarily and once the consumer contract is published, it breaks all branches of our CI/CD (unless we set it up wrong) since it pulls all pacts from the pacticipant list even in adjacent branches (i.e. wip pact is on PACT-123, but renovate-123 sees pacts from it's branch and on PACT-123). Thoughts? @matt.fellows
wng
2022-07-26 14:58
Guess our alternative is to write consumers as `wip-api-gateway` and then transition it to `api-gateway` as the pacticipant once it's ready
matt.fellows
2022-07-26 22:53
We?ll definitely need to see the webhook configuration to understand why it?s failing.
matt.fellows
2022-07-26 22:55
There should be webhook logs in the UI that show the outbound request and response, is there anything there that indicates what is going on?
matt.fellows
2022-07-27 02:13
Sorry I thought I had responded to this. Thanks for your response Yousaf.
matt.fellows
2022-07-27 02:14
> when really i want All Providers for a given consumer.
Is that how you configure your app now? A single provider endpoint even if you have separate providers?
matt.fellows
2022-07-27 02:14
Usually, I would have my API client endpoint configurable, so then I can configure the location of each provider independently. In this case, it should work nicely with different stub URLs
matt.fellows
2022-07-27 02:15
Hi Jerry, we currently don?t have the API documented as of now, but it?s something we have on the backlog for the next quarter to deal with.
matt.fellows
2022-07-27 02:16
You can CRUD users, with the exception that you _can?t_ currently pre-create users for SSO providers (e.g. Google, SAML)
matt.fellows
2022-07-27 02:17
I?m not sure if this will be helpful, but our terraform provider calls the APIs you will need.
matt.fellows
2022-07-27 02:17
Here are the pact tests for that: https://github.com/pactflow/terraform-provider-pact/blob/master/client/client_pact_test.go#L539-L675
cyrus.devnomad
2022-07-27 06:45
Hi. I'm trying to run the bi-directional example with Postman:
https://docs.pactflow.io/docs/examples/bi-directional/provider/postman/
I'm trying to go through that example on a Windows WSL2 system.
I get as far as trying to execute the command "make test_and_publish" which fails with the following error:
```docker run --rm -v //home/someuser/ContractTesting/Bi-Directional/cypress_postman/ -w /home/someuser/ContractTesting/Bi-Directional/cypress_postman/example-bi-directional-provider-postman -e PACT_BROKER_BASE_URL -e PACT_BROKER_TOKEN pactfoundation/pact-cli:latest pactflow publish-provider-contract \
oas/swagger.yml \
--provider "pactflow-example-bi-directional-provider-postman" \
--provider-app-version 1b8982-master+1b8982 \
--branch master \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results newman/newman-run-report-2022-07-27-06-17-15-353-0.json \
--verification-results-content-type text/plain\
--verifier postman
/pact/entrypoint.sh: exec: line 15: pactflow: not found
make[1]: *** [Makefile:57: publish_provider_contract] Error 127
make[1]: Leaving directory '/home/someuser/ContractTesting/Bi-Directional/cypress_postman/example-bi-directional-provider-postman'
make: *** [Makefile:49: test_and_publish] Error 2```
Any hints what could be the cause of the problem?
matt.fellows
2022-07-27 07:18
What happens when you just run `docker run --rm pactfoundation/pact-cli:latest pactflow publish-provider-contract`
cyrus.devnomad
2022-07-27 07:39
Then I get just the following error:
/pact/entrypoint.sh: exec: line 15: pactflow: not found
cyrus.devnomad
2022-07-27 07:53
Ok, after pulling the latest image that error disappeared, I now get this:
No value provided for required options '--provider', '--provider-app-version', '--broker-base-url'
So does this mean that the `make test_and_publish` command in the postman example was not pulling the latest docker image?
cyrus.devnomad
2022-07-27 07:56
Hmmm, in fact I never pulled any docker image myself, so that command must have pulled the first docker image by itself, is that so?
matt.fellows
2022-07-27 07:57
either that, or you?ve previously run other pact examples that use it, and just happened to have a slightly older version
matt.fellows
2022-07-27 07:57
or, in fact, the same example a month or two ago probably didn?t use that command
cyrus.devnomad
2022-07-27 07:59
Aha, thats very likely, because I've been indeed running pact examples for some time now in that same environment. So that explains everything. Thanks a lot.
alan.still
2022-07-27 08:48
I can?t see anything too obvious in the logs. Here is a snippet of a failing webhook
`DEBUG: Webhook context {"base_url":"https://moo-print.pactflow.io","event_name":"test"}`
`INFO: HTTP/1.1 POST https://api.github.com/repos/moo-print/manufacture-psp-adapter/dispatches`
`INFO: accept: application/vnd.github.v3+json`
`INFO: user-agent: Pact Broker v2.101.0`
`INFO: content-type: application/json`
`INFO: authorization: [REDACTED]`
`[2022-07-26T11:40:06Z] INFO: host: http://api.github.com`
`[2022-07-26T11:40:06Z] INFO: content-length: 352`
`[2022-07-26T11:40:06Z] INFO: {"event_type":"pact_provider_webhook","client_payload":{"pact_url":"https://moo-print.pactflow.io/pacts/provider/mfg-psp/consumer/mfg-work-order-notification/pact-version/9b6a9a249b42f255bca7a678185531badfedcfe3/metadata/Y3ZuPTNjM2ViMTY1YWMxMzg1OWU1OWNlZDk1Mjk4YTgxODE2Y2U0ZGNmZGUmY3Z0W109cGFjdC10ZXN0Jnc9dHJ1ZQ","consumer_version_branch":"pact-test"}}`
`INFO: HTTP/1.1 404 Not Found`
`INFO: server: http://GitHub.com`
`INFO: date: Tue, 26 Jul 2022 11:40:06 GMT`
`INFO: content-type: application/json; charset=utf-8`
`INFO: content-length: 125`
`INFO: x-github-media-type: github.v3; format=json`
`INFO: {"message":"Not Found","reference/repos#create-a-repository-dispatch-event"}`
`INFO: Webhook execution failed`
alan.still
2022-07-27 08:57
and this snippet, for another (working) webhook, calls out to the same GitHub repo,
alan.still
2022-07-27 08:58
`DEBUG: Webhook context {"base_url":"https://moo-print.pactflow.io","event_name":"test"}`
`INFO: HTTP/1.1 POST https://api.github.com/repos/moo-print/manufacture-psp-adapter/dispatches`
`INFO: accept: application/vnd.github.v3+json`
`INFO: user-agent: Pact Broker v2.101.0`
`INFO: content-type: application/json`
`INFO: authorization: [REDACTED]`
`INFO: host: http://api.github.com`
`INFO: content-length: 344`
`INFO: {"event_type":"pact_provider_webhook","client_payload":{"pact_url":"https://moo-print.pactflow.io/pacts/provider/mfg-psp/consumer/mfg-order-event-log/pact-version/d61886c80617f08cce3a8db75915587782bfa54e/metadata/Y3ZuPWY1ZDBiYjhkNGM2ODIyNjVmNzczYmVlYzcxMDk2NDdjZjk2OGVmOWYmY3Z0W109cGFjdC10ZXN0Jnc9dHJ1ZQ","consumer_version_branch":"pact-test"}}`
`[2022-07-26T11:40:49Z] INFO: HTTP/1.1 204 No Content`
`INFO: server: http://GitHub.com`
`INFO: date: Tue, 26 Jul 2022 11:40:49 GMT`
`INFO: x-oauth-scopes: repo, workflow`
`INFO: x-accepted-oauth-scopes:`
`INFO: x-github-media-type: github.v3; format=json`
alan.still
2022-07-27 09:02
We use the same auth token to access GitHub, which is stored in our Pactflow Secrets
matt.fellows
2022-07-27 09:57
Are you editing via the UI? One guess is the 404 is because of token permissions. Sometimes browsers auto complete the basic auth section
matt.fellows
2022-07-27 09:58
I know a repository dispatch can 404 if the token is incorrect, so that would be my guess. Also check the webhook team is also associated with the secret and is able to use it
alan.still
2022-07-27 12:24
Yes, I have made a few edits through the UI, and yes it looks like the Basic Auth section had been auto-completed! Thank you for that, I don?t think I would have ever searched that!
matt.fellows
2022-07-27 12:39
You're welcome. We need to find a better way here, sorry for that confusion!
yousafn
2022-07-29 12:12
Happy feature release Friday! :rocket:
Check out our latest UI enhancement - Can I Deploy.
Can I Deploy is the promised land for contract testers! The much-loved feature in Pactflow is now available in our UI, improving visibility into which applications can be deployed and why.
Can I Deploy in the UI provides a rich query interface for the Matrix to ensure you can safely deploy your application, providing additional context not readily available via the CLI.
Read how it will help you and your team deploy with more confidence and watch @slin?s demo https://pactflow.io/blog/can-i-deploy/
Or log into your account and check it out by clicking the ?New? button in your account toolbar.
Have a fantastic Friday and weekend all :the_horns::skin-tone-4:
hazem
2022-08-04 15:31
Question about debugging a provider verification in the *past*.
We've been evaluating pact and pactflow for the past month (noob here, :wave::skin-tone-4:), and yesterday we inadvertently made a breaking change in a provider service that went undetected since the provider verification oddly passed. So my question, is there a way to replicate a verification for a provider commit based on the same moment in time the test ran?
I'm at a loss to how the provider verification passed. In order to quickly fix, we fixed the consumer to meet the new specifications of the provider api.
Next, I wanted to debug more and understand what happened. So then I essentially reverted the initial breaking change on a provider branch to try and replicate breaking the api again and this time it failed as I would have expected.
hazem
2022-08-04 15:37
``` _, err := pact.VerifyProvider(t, types.VerifyRequest{
ProviderBaseURL: fmt.Sprintf("http://127.0.0.1:%d", port),
PactLogDir: brokerConfig.LogsDir(),
ConsumerVersionSelectors: []types.ConsumerVersionSelector{
types.ConsumerVersionSelector{
MainBranch: true,
},
types.ConsumerVersionSelector{
MatchingBranch: true,
},
types.ConsumerVersionSelector{
DeployedOrReleased: true,
},
},
BrokerURL: brokerConfig.BrokerURL,
BrokerToken: brokerConfig.BrokerAccessToken,
EnablePending: true,
IncludeWIPPactsSince: &includeWipPactsSince,
PublishVerificationResults: allowPublishVerificationResults(),
ProviderBranch: gitInfo.Branch,
ProviderTags: []string{gitInfo.Branch},
ProviderVersion: gitInfo.Version,
StateHandlers: stateHandlers,
})```
bethskurrie
2022-08-05 01:00
There should be output in the test logs that tells you exactly which versions of the consumer are being verified.
bethskurrie
2022-08-05 01:09
```DEBUG: The pact at https://test.pactflow.io/pacts/provider/example-provider/consumer/example-consumer/pact-version/011de3a974a121cbf474b931e4de355393b52220 is being verified because the pact content belongs to the consumer versions matching the following criteria:
* latest version from branch 'master' (acda70158a6d338e677a99a19d3fbc9c8371ae90)
* consumer version(s) currently deployed to dev (acda70158a6d338e677a99a19d3fbc9c8371ae90) and prod (acda70158a6d338e677a99a19d3fbc9c8371ae90)
* latest version from the main branch 'master' (acda70158a6d338e677a99a19d3fbc9c8371ae90)
DEBUG: This pact has previously been successfully verified by example-provider. If this verification fails, it will fail the build. Read more at https://docs.pact.io/go/pending```
hazem
2022-08-08 18:57
Thanks for the follow-up @bethskurrie. I did a bit more digging into the CI logs and I'm not able to find the message example from above. However, in the case where I revert the provider code and rerun, then I do see example message from above when it fails as expected. We're using pact-go module for running our verify tests so not sure if that somehow doesn't emit the ruby logs in non-failure cases.
Interestingly, we are running can-i-deploy on our PR branches per another slack thread :joy: . And this does show our Provider PR Commit has a verification against the Consumer main branch commit. So presumably this the suspect verification I'm trying to debug.
Ultimately the breakage was on a matcher in body response verification. The field is an enum and I was expecting a regex of a list of values that actually changed.
I also looked into overriding the consumer version in ConsumerVersionSelector, but looks like setting the version is ignored now.
lmendonca
2022-08-08 22:35
Hi team :wave: ,
I have been tasked by my company to analyze and Introduce Pact contract testing to our workflows. We have some Questions in regards to Bi-directional Contract testing for Spring Boot Microservices. We have already set up a Consumer driven POC using below tools and Java JUnit tests.
? Pact broker used with docker - https://github.com/pact-foundation/pact-broker-docker
? Pact CLI tool - https://github.com/pact-foundation/pact_broker-client - (Download standalone source code as zip from https://github.com/pact-foundation/pact_broker-client/releases/tag/v1.65.0 )
1. Considering that I didn't face any issues exercising the paid features mentioned here https://pactflow.io/pricing/ with an exception for SAMl integration, in the free versions above, why would I need the paid version?
2. How can I ensure that none of our developers are able to game the system somehow as we see some room for it? My particular question is that if Provider decides to write the pacts when consumer is not ready, How do I ensure that `can-i-deploy` is accurately telling me that both consumer and provider are in sync if I never ran the consumer test yet?
3. Continuation from point 2, Do I have to run both consumer and provider tests if consumer pact was developed after the provider test in order to release the consumer?(I think so but just wanted to confirm if there is a different approach or not)
I may have more questions, which I can bring up as they brew. Could you please let me know if this is the right forum to bring these questions up, or if I should bring them up elsewhere.
matt.fellows
2022-08-08 23:28
> Considering that I didn?t face any issues exercising the paid features mentioned here https://pactflow.io/pricing/ with an exception for SAMl integration, in the free versions above, why would I need the paid version?
If you want to use the bi-directional feature, this is only available in Pactflow. Is this your intent? We have features other than SAML in Pactflow, which you can see https://pactflow.io/features/. You can also see our public https://github.com/pactflow/roadmap/projects/1 to see if what we?re working on is going to be useful to you.
matt.fellows
2022-08-08 23:38
> How can I ensure that none of our developers are able to game the system somehow as we see some room for it? My particular question is that if Provider decides to write the pacts when consumer is not ready, How do I ensure that `can-i-deploy` is accurately telling me that both consumer and provider are in sync if I never ran the consumer test yet?
What do you mean by ?provider writes the pacts?? Do you mean publishing their provider contract (an OAS) that is invalid?
bethskurrie
2022-08-08 23:56
Yeah, you can?t set a consumer version in the selector. But you can provide the URL of a specific pact to be verified (as is done for webhook triggered verifications).
matt.fellows
2022-08-09 01:09
> Continuation from point 2, Do I have to run both consumer and provider tests if consumer pact was developed after the provider test in order to release the consumer?(I think so but just wanted to confirm if there is a different approach or not)
In the cases of BDCT no (because the OAS is not changing, getting the provider to re-publish the same OAS doesn?t give us any new information). With the Pact model, yes, because the provider must verify each contract individually
matt.fellows
2022-08-09 01:09
> Could you please let me know if this is the right forum to bring these questions up, or if I should bring them up elsewhere
You?re in the right place :slightly_smiling_face:
hazem
2022-08-09 03:42
I just came across a workflow issue with a new pact for a provider that has yet to be merged by the consumer. Details in thread...
hazem
2022-08-09 03:45
From the tutorials and examples I've done the following:
? As a Consumer, created a pact on a feature branch.
? Provider implemented the pact on a feature branch and merged to main.
At this point it seems like this is a WIP Pending pact as it still remains on the consumer feature branch. In addition, the provider has no other verified pacts, eg first pact scenario.
? Now from a new branch on the provider I get the following error:
```Failures:
1) Failed to load pact - Could not load pacts from the pact broker 'https://REDACTED.pactflow.io' - NotFound("No pacts were found for this provider")```
Example Verify Request:
``` _, err := pact.VerifyProvider(t, types.VerifyRequest{
ProviderBaseURL: fmt.Sprintf("http://127.0.0.1:%d", port),
PactLogDir: brokerConfig.LogsDir(),
ConsumerVersionSelectors: []types.ConsumerVersionSelector{
types.ConsumerVersionSelector{
MainBranch: true,
},
types.ConsumerVersionSelector{
MatchingBranch: true,
},
types.ConsumerVersionSelector{
DeployedOrReleased: true,
},
},
BrokerURL: brokerConfig.BrokerURL,
BrokerToken: brokerConfig.BrokerAccessToken,
EnablePending: true,
IncludeWIPPactsSince: &includeWipPactsSince,
PublishVerificationResults: allowPublishVerificationResults(),
ProviderBranch: gitInfo.Branch,
ProviderTags: []string{gitInfo.Branch},
ProviderVersion: gitInfo.Version,
StateHandlers: stateHandlers,
})```
occurs with: `http://github.com/pact-foundation/pact-go/v2 v2.0.0-beta.14.0.20220628130155-b610d4b9149d`
hazem
2022-08-09 03:55
And executing from the main branch of the provider does not cause this error for the WIP
matt.fellows
2022-08-09 08:05
Thanks Hazeem.
I believe because we use the `t.Logf` function here, they are only printed on test failure *or* if you pass the `-v` flag to the tests.
I could force this to be printed to stdout, but is not a very idiomatic. I?ll have a think about the best way to tackle this
hazem
2022-08-09 16:02
Also discovered https://github.com/pact-foundation/pact-go/blob/b610d4b9149d649783944049595ecdd27e864e1c/provider/verify_plugin_request.go#L87 option does not seem to get used as it does in 1.7.0
lmendonca
2022-08-10 06:09
Hi @matt.fellows, I apologize for the delayed response. Yes I was looking for bi-directional feature.
lmendonca
2022-08-10 06:11
For this https://pact-foundation.slack.com/archives/CLS16AVEE/p1660001885006759?thread_ts=1659998145.922109&cid=CLS16AVEE, I figured out the answer I was looking for. Thank you!
lmendonca
2022-08-10 06:24
https://pact-foundation.slack.com/archives/CLS16AVEE/p1660007379263709?thread_ts=1659998145.922109&cid=CLS16AVEE - If I only validate the consumer pact to OAS in BDCT, how would verification results be published to pact broker and how does `can-i-deploy` fit into this? This page isnt very clear about that. I conceptually understand the diagram but practically, I am making assumptions that I am not 100 percent sure of.
lmendonca
2022-08-10 06:26
Consumer based contract test is quite straight forward, but If I wanted to have a Provider based contract test where provider code is developed first, this is the workflow I see happening without use of Swagger hub;
1. Provider creates the contract if provider is ready first or is in earlier development.(This is done first time or only when consumer is not ready or needs a change as a result of a broken contract)
2. Provider publishes the contract to broker.(This is done first time or only when consumer is not ready or needs a change as a result of a broken contract)
3. Provider validates the contract against its own open api spec json file generated during Integration tests(If executed before contract tests) or contract test generates it itself as a before test hook. - I almost have a working example for this with `swagger-mock-validater`.
4. Provider runs provider tests and publishes passing results to broker. - At this point Provider doesnt care about an actual consumer because there is none.
5. Provider publishes the open api spec json to artifactory.
6. Provider runs `can-i-deploy` pact CLI command to verify if provider is ready for release against the pact-broker.
7. Provider can release if step 1 to 6 are passing.
8. Consumer development begins and consumer writes a consumer contract.
9. Consumer retrieves the published provider open-api spec json from artifactory to validate pact against it using same tool as step 3.
10. Consumer publishes the contract to broker.
11. Consumer triggers the provider tests by cloning the provider repo with latest version specified in `gitops-version` and actually runs the provider tests to publish the verification results to pact broker.(May not be needed since a comparison with OAS is sufficient - Confirming with Pact team)
12. Consumer runs `can-i-deploy` to verify if consumer is ready for release. (May not be needed since a comparison with OAS is sufficient - Confirming with Pact team)
13. Consumer can release if steps 8-12 passed.
14. A month later, provider changes its code, breaking the contract but does not know it yet.
15. Provider repeats steps 3-6 to verify that its not breaking the contract with the consumer(This time it does care about consumer because a released or an active version of the consumer exists).
16. Provider fails on the repeated step 4 as the tests will fail because code was modified in step 14.
17. Teams coordinate the failure and decide if the fix needs to be on provider or consumer side.
18. If provider makes the fix, then contract is reverted to original content and steps 3-6 are repeated. If consumer has to fix, Then steps 1-12 are repeated and both provider and consumer are released together to ensure accurate interface in production.
matt.fellows
2022-08-10 06:50
> Consumer triggers the provider tests by cloning the provider repo with latest version specified in `gitops-version` and actually runs the provider tests to publish the verification results to pact broker.(May not be needed since a comparison with OAS is sufficient - Confirming with Pact team)
Exactly. The OAS is the only information we need to ensure the consumer contract is compatible with the Provider contract - this is why you don?t need to trigger a provider build when a new consumer contract is published. Triggering a provider build is just going to result in the same OAS being published.
matt.fellows
2022-08-10 06:51
> 1. Consumer runs `can-i-deploy` to verify if consumer is ready for release. (May not be needed since a comparison with OAS is sufficient - Confirming with Pact team)
both provider and consumer should always run that command prior to deployment
matt.fellows
2022-08-10 06:52
> Teams coordinate the failure and decide if the fix needs to be on provider or consumer side.
Yes, that too. Ideally, the provider uses an expand and contract pattern, and never breaks a consumer. But of course we live in the real world, and sometimes that?s not possible
thomas.koppensteiner
2022-08-10 06:57
Hello, I'm also interested in pactflow supporting OpenAPI v3.1.
A new version of the swagger-parser has been released and it supports v3.1. Do you use this parser or another one?
https://github.com/swagger-api/swagger-parser/releases/tag/v2.1.0
matt.fellows
2022-08-10 07:23
Sorry I missed this. Yes, albeit we don?t yet have timelines published for completing in
matt.fellows
2022-08-10 07:28
No we don?t use that package, sorry (we don?t use Java).
We don?t have timelines published just yet, but it?s definitely something we would like to support in the near future
ben.a.hubbard
2022-08-10 13:32
Hey, I've got the following errors when executing provider oas publish in Jenkins using the linux CLI:
/pact/lib/ruby/bin.real/ruby: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /pact/lib/ruby/bin.real/ruby)
/pact/lib/ruby/bin.real/ruby: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by /pact/lib/ruby/bin.real/ruby)
I didn't experience these at all when running from local following https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#pact-standalone-cli. Are there any pre-requisites for running that I might've missed?
matt.fellows
2022-08-10 13:41
The issue looks clear to me though - the version of glibc is out of date (2.17 was https://sourceware.org/glibc/wiki/Glibc%20Timeline in 2012)
lmendonca
2022-08-10 15:43
@matt.fellows I needed some help with BDCT with pactflow. I wanted to check with you how contracts appear instead of pacts
lmendonca
2022-08-10 15:44
Would you or someone from the team be able to meet with me for 15 to 30 mins?
matt.fellows
2022-08-10 22:00
I'm not sure I follow the question, what are you asking? How provider contracts are displayed?
hazem
2022-08-10 23:24
Thanks for the tip/reminder about debugging with PactURLs.
In case others search for regex matching with enums, my mysterious pact verification was the result of a bad regex:
1. Consumer defined matching a json response field for expected enums of: `(OBJECT|STRING|INTEGER|FLOAT|BOOLEAN|TIME)`
2. Provider verified the pact.
3. Next provider broke the api by changing the enums and added a suffix of `_RESULT` and this unintentionally passed the _not so great_ enum matcher, and allowed verification of the pact for the broken api.
Outcome: The consumer's match regex should have been: `^(OBJECT|STRING|INTEGER|FLOAT|BOOLEAN|TIME)$`
bethskurrie
2022-08-10 23:29
@hazem how would you feel about adding that into the docs? https://docs.pact.io/getting_started/matching/gotchas
matt.fellows
2022-08-11 01:32
Ah good spot I don't think that option can be wired into the new core. I'll double check
danielflieger
2022-08-11 12:18
Hi there! I have a question regarding structuring contracts within pactflow.
Am I correct that on the left side the user can see the highest level of the communication between consumer & provider?
All endpoints & interactions for these participants are grouped there. This means that when the user opens the detail page of the contract which may have a long list of endpoints to cover in the pact he would see a long list with interactions and endpoints.
So, i was wondering if there is a way to structure all interactions per endpoint and not present the full list of interactions for all endpoints between consumer and provider. Thanks!
matt.fellows
2022-08-11 12:21
Interesting idea. We are looking at some redesign at the moment, and will see how this migth be able to inform that design.
cc: @slin
matt.fellows
2022-08-11 12:22
Knowing the endpoint isn?t always straightforward, but might be doable. There are possibly other ways to group as well
danielflieger
2022-08-11 12:29
Is there a way to group it on the left side for every endpoint? So having a pact for each endpoint which be somehow grouped for one consumer & provider pair via the teams feature. Or what would be other ways on your eyes? Thanks Matt. :)
matt.fellows
2022-08-11 12:42
you can group integrations by team, but I think you want it for the _interaction_ level
matt.fellows
2022-08-11 12:43
What is an ?endpoint? to you? It has very different meanings to different people!
matt.fellows
2022-08-11 12:45
`/orders/:id` - we?d probably all agree that?s orders.
`/orders/:id/items` - is that orders or items?
`/orders/:id/items/:id/actions` - is that orders or items or something else?
matt.fellows
2022-08-11 12:46
It gets trickier too, because you may not easily be able to detect the dynamic parts of the request without a number of samples (and even then, it?s risky). Probably numbers are placeholders, but not necessarily. Harder for strings
danielflieger
2022-08-11 12:49
I see. That?s more complex than I thought. In your example I would expect `product` to be the highest level which groups all interactions in it.
ben.a.hubbard
2022-08-11 13:08
I don't have access to make those changes so I've just passed this on. Not sure how long it will take them though :upside_down_face: I've never had to update GLIBC myself but they are a fair bit out of date, do you know if later versions are backwards compatible?
matt.fellows
2022-08-11 13:19
Yes, glibc versions all function calls. So a version compiled on glibc 1.0 will still work on 100.0, just not the other way around (when you link against glibc, it ?helpfully? uses the latest symbols, and there really isn?t a way to not do that, so you can?t then use it on older glibc environments - i.e. the very problem we have now!)
ben.a.hubbard
2022-08-11 13:37
Makes sense thanks! I had just read https://superuser.com/questions/537683/how-to-fix-lib-x86-64-linux-gnu-libc-so-6-version-glibc-2-14-not-found which mentioned "upgrading it can have far-reaching implications" so I wanted to double check, but looking back they're probably just saying that as the version was still "experimental"
lmendonca
2022-08-11 13:55
I mean to ask if there is any practical documentation on bi-directional flow. Like api calls i need to make to upload open api spec and provider contract to pactflow
lmendonca
2022-08-11 13:57
Actually, I will spend some time today to get it working. If I stumble upon any outstanding questions, I will reach out. Thanks Matt!
hazem
2022-08-11 15:15
Also curious about the expectations of this workflow for provider branches where the first provider pact is merged to main on provider branch but still a WIP on consumer's. Shouldnt the provider branches detect these wip/pending pacts still, rather than failing with no pact error?
tylerkron
2022-08-11 15:22
Hello! We are seeing an oddity within Pactflow and hoping to get some help. We have a pact between two services. The pact exists in Pactflow (attached picture). However when running provider verification, we are getting a `NoPactsFound` exception (JVM Library). Digging into the code, the library is using the following endpoint `/pacts/provider/<provider>/for-verification` . This endpoint has no pacts on it. Any ideas on why this might be?
yousafn
2022-08-11 15:50
Hey, is that consumer branch the only pact that has been published?
Note it is not the main branch, nor deployed to an environment, nor has tags.
On the provider side for verification, what consumer version selectors are you using to retrieve the pacts for verification?
https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors
There are two verifications of pacts on the provider side, which provider verification task are you referring too?
https://docs.pact.io/pact_nirvana/step_4#verifying-pacts
> 1. When the provider code changes
> ? The verification task will run as part of the normal CI/CD pipeline for the provider
> ? The verification task will fetch and verify all the relevant pacts from all consumers from the Pact Broker to ensure no regressions have occurred.
> 2. When the pact changes
> ? We recommend that a separate pipeline is set up to verify just the changed pact (more details on this later in the guide).
You would traditionally have a webhook, setup on new contracts, or contracts being changed that would trigger a provider verification with a pactUrl directly
https://docs.pact.io/pact_broker/webhooks#the-contract-requiring-verification-published-event
tylerkron
2022-08-11 16:26
For this specific pact, this is the first time publishing and trying to verify it. We have other examples that work. The difference between this one and the working pact is:
? this is a Pact v4 spec contract (the working one is v2)
? this is a message based contract (the working one is HTTP based)
lmendonca
2022-08-11 16:47
@matt.fellows I got bidirectional flow started for two of our services in pactflow. We are a large organization with lots of micro-services so we will likely be headed towards the enterprise or paid version. I got the OAS uploaded from provider side and also see it in correct format in pactflow ui. But when I upload the consumer pact, I see this error.
lmendonca
2022-08-11 16:48
It's only telling me about a validation failure. I don't see any obvious issues in the open api spec on the pactflow ui
hazem
2022-08-11 21:12
Yeah, I'd be curious as to your ConsumerVersionSelectors too. I'm not certain, but it feels like a WIP pact since it is still on a consumer branch.
A couple things come to mind from my experience:
? ConsumerVersionSelector: `MatchingBranch: true` with provider branch `PE-354`
? VerifyRequest config
? `IncludeWIPPactsSince: [date]`
? `EnablePending: true`
I dont use tags and opt for branches instead and seeing as how you dont tag, that might fix it. :fingerscrossed:
mike.key
2022-08-11 22:21
There are no version selectors yet, we typically don't add version selectors until the contract is published in all of our environments. Pending pacts are set to true on the provider verification
saleh.elnagar
2022-08-15 08:24
Hi there, I have a question.
What should we do when we have both applications are consumer and provider at the same time?
matt.fellows
2022-08-15 08:29
What?s the actual problem you?re trying to solve? It?s totally normal for that situation to happen
saleh.elnagar
2022-08-15 08:32
Let's say which one should we deploy first?
Asking about the best practices in this case?
matt.fellows
2022-08-15 09:10
Ideally you don't have to worry about sequencing changes. Why is this a perceived problem?
Ideally, you make a change to an application and deploy that, and not worry about dependencies (Pactflow will tell you if you're going to break one)
matt.fellows
2022-08-15 09:12
But obviously, if a consumer needs a new API then the provider needs to go first (or you feature toggle the consumer)
tpaktopsp
2022-08-15 18:49
Hello everyone. Might be missing something simple, is it possible to override authentication on stubs to make them publicly available ? We are using `Bearer token Authentication` for our own application and it is conflicting with stubs `authentication header` .
Or maybe at least I can specify custom header to use for pact-flow authentication instead of using common `Authentication` header.
matt.fellows
2022-08-16 04:05
Thanks Maxim. It?s something for us to think about. I believe initially we had no authentication on it but received feedback that we should enable the default auth due to security concerns.
I have some ideas, would be keen for your perspective
matt.fellows
2022-08-16 04:05
1. Use a separate auth header for Pactflow (e.g. `Pactflow-Authentication`)
matt.fellows
2022-08-16 04:06
2. Disable auth altogether (or have the option to do so as http://e.ga system preference)
matt.fellows
2022-08-16 04:07
3. Pass through the Pactflow bearer token to the stub also (two birds with one stone).
matt.fellows
2022-08-16 04:08
4. Allow users to specify their own custom authentication token (e.g. continue to allow your Bearer token, but the token is user defined and checked by Pactflow). I believe this is how SwaggerHub auto-mock works
tpaktopsp
2022-08-16 05:26
Not sure I understand option 3.
Option 4 wouldn?t work for us as we have different tokens we use for testing different user roles. For example, if user tries to perform admin action they will get 403
tpaktopsp
2022-08-16 05:27
Option 1 is probably the most straightforward - we would just add extra header when running in test env. Though, `pact-js` might need to be updated to exclude this header somehow?
tjones
2022-08-16 05:29
Option 3 is that the pactflow token (which you probably use in your CI already) would be valid as an auth token for the stubs
tpaktopsp
2022-08-16 05:31
Ah right that is what we pass anyway. Wonder how would our test pick it up and pass to stub ? We are using hosted stubs as replacement for our backend so our code doesn?t know its pact stubs just different URL
tjones
2022-08-16 05:31
I'm not certain about this, but if you don't mention a header in your test expectation, I believe the test still passes (this is potentially a problem if the presence of the header would cause your provider to fail, as verification will only use the headers you verified).
tjones
2022-08-16 05:32
You would mock your auth provider layer so that it just returns the pactflow token. Depending on your design, this might not work for you, and is probably the driver behind option 4
tpaktopsp
2022-08-16 07:35
Right, so if no changes to pact-js required as it will skip its verification option 1 would be the easiest and will require minimal changes on our side too
dpal
2022-08-16 09:17
Hi there, I have an issue importing "import { Matchers, PactV3 } from '@greet_bot/pact" this library for my event consumer tests. I'm having node version node -v v18.7.0. Please help. Thanks
alexey.shchukin
2022-08-16 09:21
Hi @matt.fellows,
thanks for your reply.
The problem Saleh is talking about is the following:
We have two applications, and both of them are a consumer and a provider at the same time. There is a circular dependency.
```(Consumer) A -> B (Provider)
(Provider) A <- B (Consumer)```
Thus, we can?t deploy app A (consumer), because it require app B (provider) in place and vice versa.
Am I understanding it correctly, the only way to resolve this is feature toggles?
Thanks in advance.
matt.fellows
2022-08-16 09:32
Well you just need to get one of the apps deployed initially to break the cycle, and then you just need to not get into a situation where they both have changes that depend on one another (because Pact aside, this is an awkward situation to be in!)
dpal
2022-08-16 09:40
`npm notice Run npm install -g npm@8.17.0 to update!`
`npm notice`
`npm ERR! code 1`
`npm ERR! path /<home_dir>/node_modules/@pact-foundation/pact-core`
`npm ERR! command failed`
`npm ERR! command sh /var/folders/3l/mcrvlw1x2bddv5zlx1fncpdr0000gq/T/install-627208a4.sh`
`npm ERR! gyp info it worked if it ends with ok`
`npm ERR! gyp info using node-gyp@9.0.0`
`npm ERR! gyp info using node@18.7.0 | darwin | arm64`
`npm ERR! gyp info find Python using Python version 3.9.13 found at "/opt/homebrew/opt/python@3.9/bin/python3.9"`
`npm ERR! gyp http GET https://nodejs.org/download/release/v18.7.0/node-v18.7.0-headers.tar.gz`
`npm ERR! gyp WARN install got an error, rolling back install`
`npm ERR! gyp ERR! configure error`
`npm ERR! gyp ERR! stack FetchError: request to https://nodejs.org/download/release/v18.7.0/node-v18.7.0-headers.tar.gz failed, reason: unable to get local issuer certificate`
`npm ERR! gyp ERR! stack at ClientRequest.<anonymous> (/opt/homebrew/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:130:14)`
`npm ERR! gyp ERR! stack at ClientRequest.emit (node:events:513:28)`
matt.fellows
2022-08-16 09:54
The issue looks to be that node can't download something from an official source, so my guess is you're behind a corporate proxy that is blocking it. Best to go and talk to somebody in your org that knows about networking
matt.fellows
2022-08-16 09:55
Usually you shouldn't have to do anything, aside from ensure you have basic dev build tools on it, which unless you're running a custom lean docker environment you should have
matt.fellows
2022-08-17 02:19
Thanks for the feedback! The `403` use case is definitely a curveball for consideration.
tjones
2022-08-17 02:20
Ah right! If you consider the auth token part of the contract (a reasonable choice, but I usually don't), then it's certainly more complex
matt.fellows
2022-08-17 02:22
It?s not usually a problem, because provider states can be used as the signal for behaviour of the mock (and even then, it?s usually not required because the mock needn?t be very intelligent).
But when using a pact file as a general stub server where it only receives HTTP interactions from clients (who couldn?t reasonably be expected to ?know? about states etc.), it?s hard to disambiguate certain requests
matt.fellows
2022-08-17 02:23
I don?t think we (Pactflow) want to go down a path where we start building intelligent mocks - that seems a slipperly slope into service virtualisation which is off the garden path for us (and SmartBear already has decent SV tools).
tjones
2022-08-17 02:24
It seems like intelligent mocks would be against the general philosophy of Pact
matt.fellows
2022-08-17 02:27
I?ll bring this back to the team. I think there are a few ways we could improve it without ?devolving? into intelligent mocking, but also keeping the security of data considered
tjones
2022-08-17 02:30
> We are using hosted stubs as replacement for our backend
I'm curious about this use case - if it's for small tests, I would probably just mock the API client (and then use pact to assert that the mock's expectations are part of the contract). If it's for larger tests, how would you manage the provider state?
matt.fellows
2022-08-17 02:48
The use case I most often see, is using them as ?dumb? stubs for UI tests (e.g. Cypress), or as a dockerised backend in a ?n + 1" style integration test (I wouldn?t do that personally, but have seen it more than once)
alexey.shchukin
2022-08-17 13:09
Hi @matt.fellows,
Thank you for your reply.
Unfortunately, no, it?s not theoretical. We are facing the dilemma exactly like this. So, it seems to me that every time we have such a conflict, we have to skip `can-i-deploy` step.
1st scenario is initial deployment, when none of the apps (A or B) is deployed in an environment. In this case we have to skip `can-i-deploy` step for the app, which is deployed first.
2nd scenario, when both of the apps are depends on newer versions of each other, for example:
? A version 1.0 is deployed on dev env.
? B version 1.0 is deployed on dev env.
? A version 2.0 depends on B version 2.0.
? B version 2.0 depends on A version 2.0.
So, in this case we?ll have to skip `can-i-deploy` for one of the apps (A or B) as well, which is deployed first.
Am I missing something?
matt.fellows
2022-08-18 01:19
So assuming you weren't using Pact, what would you do?
Building software that has circular dependencies like this where each change requires both sides to update simultaneously is not ideal - now you have no choice but to deploy them together
matt.fellows
2022-08-18 01:20
You might then argue the boundaries are wrong and they in fact shouldn't be two separate systems
matt.fellows
2022-08-18 01:22
If you want to use can I deploy, you'll need to make changes in such a way that you can deploy one independently of the other i.e. change only one thing at a time
matt.fellows
2022-08-18 01:23
What's the specific use case that's causing the need to change both at once?
lucka
2022-08-18 09:44
Hi, is there an option to view/download the raw JSON of a PACT within the PACT details page?
lucka
2022-08-18 09:48
I figured that I can view the raw JSON when clicking the "API" button. I find that a bit non-intuitive and would appreciate a "View raw json" button on the pact-details page. Pleaseee :-)
matt.fellows
2022-08-18 09:51
There's a drop down to get the URL for the pact for E.g. curl . May I ask why you want/need it?
lucka
2022-08-18 10:01
When I open that URL in another browser tab i get presented the PACT details page again.
Our provider asked for his provider name he is supposed to use and I was not sure where to find it so I looked for the particular JSON property "provider": "xyz"
lucka
2022-08-18 10:03
Its good that a click on "API" deeplinks to the correct API path already, will document it like that in our best practices manual
hazem
2022-08-18 15:27
I did this the other day using Matt's suggestion above and then piping it to jq to remove the hal stuff `jq 'del(._links)' > pacts.json` . My particular case was debugging a pact that unexpectedly passed verification so in order to recreate the build the CI ran. I then changed the verify to look for pacts on my file system.
matt.fellows
2022-08-18 22:09
> When I open that URL in another browser tab i get presented the PACT details page again.
the api uses media types to determine the content to return. If you open in a browser, the accept is `application/html` so it turns the react page.
matt.fellows
2022-08-18 22:09
> Our provider asked for his provider name he is supposed to use and I was not sure where to find it so I looked for the particular JSON property ?provider?: ?xyz?
you don?t need the JSON for that - it?s on the web page
bethskurrie
2022-08-19 00:24
The ?pacts for verification? endpoint requires a post with a body, where the body contains the consumer version selectors.
bethskurrie
2022-08-19 00:25
and then you need to either select the branch with the selectors on the provider side, or turn on wip pacts.
bethskurrie
2022-08-19 00:26
Doing a GET to the endpoint will not provide you with what you want. You can see the actual request the provider is making if you turn on http debug logging.
tpaktopsp
2022-08-19 09:09
> If it?s for larger tests, how would you manage the provider state?
Its something we stumbled upon when we were trying to use `pact mock` in place of real server. Amount of state permutations would grow too much.
Good point though that even if we use stubs we will end up in the same situation. Hm?.
lucka
2022-08-19 09:54
I would still like a button, because a colleague new to PACT is not able to grasp from where he can see the provider state that he is supposed to support only by UI
j3rry.wan9
2022-08-22 23:16
Hi team, my team is trying to do the migration from tags to branches by leveraging the feature called "Automatic branch creation from first tag": https://docs.pact.io/pact_broker/branches#automatic-branch-creation-from-first-tag. However, I'm not sure how I can set the `use_first_tag_as_branch` with http://Pactflow.io as the https://docs.pact.io/pact_broker/configuration/settings#use_first_tag_as_branch page is all about standalone Pact Broker. Could someone please shed some light?
matt.fellows
2022-08-23 00:53
If I understand your _need,_ you need to be able to see the provider states that you need to add to your provider tests, and you believe getting access to the raw pact file will help. Perhaps making them more accessible in the UI would be more helpful?
lucka
2022-08-23 12:07
Right. However, since this approach seems to be a quick-win rather than a solution that follows your UX strategy (to have a self-explaining UI), what about this: The UI could make it crystal clear what a provider has to do when a new PACT is published. E.g. by extending the UI here:
hazem
2022-08-23 22:29
I just finished reviewing https://www.youtube.com/watch?v=tl1PtesLJVI and found Bi-Directional Contract Testing quite interesting!
One question (maybe two dimensional) came to mind. Within a new micro-service architecture, which approach would you recommend, traditional pact or bi-directional pacts? An extra dimension to this is an internal consumer service with an external provider service.
I would imagine the pact approach is quite useful for all of our internal services where we are building new apis. However, one of the bullet points on the slide _Why and when would I use this?_ mentions _Legacy systems that can't be unit tested._ I automatically related this in our product's context as external systems that can't be unit tested.
So what approach would you recommend for interactions of a consumer service with an external provider service from outside my organization? I could see leveraging a postman collections as a way to document the provider interactions and treating this external system as a _legacy provider_
yousafn
2022-08-23 22:51
Hey up Hazem, is your external system, that from another company. Do they provide a sandbox environment or would you be leveraging these against the live services.
I would love to live in a world where public facing or private api?s but available to many consumers are coupled with a great developer experience. A portal you can go to for integrations, test accounts, a sandbox environment whereby you can be alerted to impending changes.
Well documented services with something like OpenAPI, or in the case of event based services AsyncAPI.
Would love to hear more about your use case.
I can see the generation of an OpenAPI spec off the back of a tested Postman collection being useful, as shown in one of our demos, as that subset of the OpenAPI that you have tested, is a rich guarantee, rather than taking the full OpenAPI spec and utilising the existing OAS testing tools, which all have, to some degree limitations, and may not give you complete picture of confidence. Especially when coupled with a system that can only test from the outside in.
matt.fellows
2022-08-24 02:11
> However, one of the bullet points on the slide _Why and when would I use this?_ mentions _Legacy systems that can?t be unit tested._ I automatically related this in our product?s context as external systems that can?t be unit tested.
yep, it?s probably a separate bullet point but you can definitely try this
matt.fellows
2022-08-24 02:12
The key thing is that you have _some_ confidence that the API is correct. So that could be:
1. Your own validation (e.g. a postman collection of tests converted into an OAS)
2. an OAS shared by the provider
If you can reliably push the most up-to-date and (ideally) tested provider to Pactflow, you then get some additional assurances.
matt.fellows
2022-08-24 04:12
If there are multiple states, they are comma separated. But your point is definitely valid. I have been thinking it would be _great_ to have code snippets in that vacant space for provider teams to copy and paste. But that is hard to do across the pact languages, and versions within each language.
matt.fellows
2022-08-24 04:14
we are also looking at an application home page, which could potentially amalgamate all of these into a single place that would be more convenient than having to look at each integration separately
hazem
2022-08-24 15:05
Yeah, that last part their of most up-to-date seems ideal. Thanks for the input, :bow::skin-tone-4:
shen-yu
2022-08-24 23:35
Hi Pactflow team @matt.fellows @yousafn, I also have question about bi-direction. The example it looks like it is setting up intercept and then stub the response, and then it happens when UI react with the network request.
Is it possible to generate pact file with Cypress by making the request without UI interaction? Our API integration(e2e) test is using Cypress to making network request, and then verify the response. Cypress won?t be able to intercept those since it?s not the request coming from UI.
matt.fellows
2022-08-24 23:46
As long as you?re using the `cy.intercept` I believe so. Have you tried it?
@slin is that right?
matt.fellows
2022-08-24 23:47
> Our API integration(e2e) test is using Cypress to making network request
may I understand why you use Cypress for this? It _seems_ like a bad idea to me, but I?d love to know why you chose Cypress for this task
shen-yu
2022-08-24 23:48
cy.intercept setup the rule to intercept the incoming request from browser (application network layer), but if we directly use cy.request to send out network request, then the request is coming from physical layer, and cypress won?t be able to intercept it.
shen-yu
2022-08-24 23:49
Thanks @matt.fellows, the reason we are using cypress to run our API e2e was because we are trying to use the same tool for all our testing. UI e2e, API e2e, etc.
https://docs.cypress.io/guides/guides/network-requests#Testing-Strategies
`Cypress helps you test the entire lifecycle of HTTP requests within your application.`
matt.fellows
2022-08-24 23:51
> , the reason we are using cypress to run our API e2e was because we are trying to use the same tool for all our testing. UI e2e, API e2e, etc.
:scream:
matt.fellows
2022-08-24 23:54
That doesn?t seem right to me. But to answer your question, I don?t know for sure. Reading https://docs.cypress.io/api/commands/intercept#Comparison-to-cy-route seems to suggest you should be able to use it, but I?ve not tested it myself
matt.fellows
2022-08-24 23:55
OK I see that `cy.request` bypasses this: https://docs.cypress.io/api/commands/request#cy-intercept-cy-server-and-cy-route
matt.fellows
2022-08-24 23:56
the point of contract testing is to serialise a contract that is derived from consumer usage - i.e. the actual needs of the consumer
matt.fellows
2022-08-24 23:56
by using `cy.request` directly, you?re bypassing the consumer code altogether so you have no confidence that your consumer actually needs or does make those API calls.
matt.fellows
2022-08-24 23:58
The testing your doing is a functional end-to-end test of the provider. That?s a separate class of testing to what contract testing offers.
shen-yu
2022-08-25 00:02
ah that?s good point, so I guess the bi-direction with cypress plugin probably won?t work with our micro services, but we can probably adopt it with our front end UI test.
matt.fellows
2022-08-25 00:03
but if you have microservice to microservice interaction, you can still use bi-directional
shen-yu
2022-08-25 00:04
yea, our micro services been running integrate test with Cypress, and it worked well. But now we are trying to adopt contact testing, and saw the cypress plugin
tpaktopsp
2022-08-25 06:06
What would be your recommendation beside - use bi-directional contracts? We are using GQL which is not supported yet by bi-directional contracts.
tjones
2022-08-25 06:59
technically the first recommendation is "don't do this" :wink:
Although, I think that page might be a bit misleading. It doesn't mean "don't use pact if you have a UI", it means "don't drive pact through your UI" - as in, don't click the buttons to send the requests
tjones
2022-08-25 07:00
it's better to separate the API client into code that doesn't know about your UI, and then test that with Pact
tpaktopsp
2022-08-25 07:03
and UI tests use real code which tries to send Auth header. Which is matched by ?like? in the contract anyway but pactflow interjects before that
tpaktopsp
2022-08-25 07:04
I tried to move to pact-stub-server but that doesn?t support ?v3? it seems :confused:
tpaktopsp
2022-08-25 07:05
The problem with auth header is in the UI testing on trying to use Pact stub directly from pactflow
tjones
2022-08-25 07:06
and use the responses in the stub as the responses I test against in the pact test
tjones
2022-08-25 07:07
I consider the handler covered by pact, and everything else covered by the UI / unit tests.
That might not work for you, depending on what you want to achieve
tpaktopsp
2022-08-25 07:09
With detox it might be a bit tricky as we would need to replace the whole handlers layers. There is no single entry point to handlers API its many small files
tjones
2022-08-25 07:14
I talked a little bit about how to connect your pact expectations and mock objects here, around the 40 minute mark
https://www.youtube.com/watch?v=wkld_wRsTDE
tpaktopsp
2022-08-25 07:20
Hm?. I wonder if it would work with cypress/detox way of testing - as they are testing black box without replacing any files. The only thing we usually do in those tests is change URLs of external services to some mock implementations.
matt.fellows
2022-08-25 07:21
Sorry I'm just on the go but I wanted to drop this in: https://github.com/pact-foundation/pact-stub-server
yousafn
2022-08-26 13:11
:arrow_up: *Level up your testing game with Full Stack Testing Academy*
Are you hunting for Pactflow training materials for you and your team? Our friend @pietrucha.bartosz ? the founder of Fullstack Testing Academy ? is launching a new Pactflow contract testing module. The course starts on October 6th, 2022.
I can vouch for the content, which covers all you need to know about full-stack web application testing and automation.
From unit, integration, E2E tests, and automation to CI/CD and contract testing, everything is in one place.
Tickets for the 10-week course are ?499 each.
They include lifetime access to on-demand video lessons, as well as ten weeks of support on Slack and via Zoom.
@pietrucha.bartosz is generously providing all Pactflow customers with a ?50 discount. Use the code `PACT_FLOW_50` to secure your spot!
Sign up <https://pactflow.us4.list-manage.com/track/click?u=d975bc32fd84ad3580ed4777e&id=1581d47252&e=f3a6913421
|here>.
aforeman
2022-08-31 05:28
_*Webinar invite: API Contract Testing for a Design-First World*_
Join @matt.fellows and me on Thurs 1 Sept 2022 at 10am EDT / 3pm BST as we discuss:
? The challenge of scaling testing in complex API structures and microservices architectures:tornado:
? Why relying on traditional testing techniques will no longer cut it :no_entry_sign:
? How contract testing, in an API design-first context, helps calm the chaos to help teams deploy with confidence :relieved:
? The important role of Pactflow and https://swagger.io/tools/swaggerhub/ in the modern API-led enterprise :handshake:
Register https://smartbear.com/resources/webinars/api-contract-testing-for-a-design-first-world/?utm_medium=social&utm_source=linkedin&utm_campaign=API-Contract%20Testing-for-a-Design-First-World&utm_content=webinar. If you can?t make it, we?ll email you the recording!
noor.hashem
2022-08-31 16:48
Hi all, when utilizing bidirectional contract testing do we need to verify the provider contract? For example if we have openapi documentation that is well kept, do we need the additional step of using postman for verification? Wouldn't postman redundantly recreate the openapi spec for us?? I see in the video here: https://www.youtube.com/watch?v=a9K43CHSRM0 we use postman to actually generate the openapi spec. But for our team we have openapi spec already.
matt.fellows
2022-08-31 22:06
It's not about recreating the spec, it's about ensuring your provider is compliant with the spec. However you do that we don't mind
matt.fellows
2022-08-31 22:07
In that example we use a postman collection (the tests) and convert to a spec, so we know whatever is generated is valid)
matt.fellows
2022-08-31 22:09
For what it's worth I don't think postman is a good tool for this, but Portman might be ok
sumanta.roy
2022-09-01 01:34
Hi Team,
I have been trying out the Bidirectional contract testing feature and sort of hit a blocker with the below error.
I do have the path defined in the provider spec in the location it's complaining. Any leads on how to go about debugging this would be of great help. Thanks
sumanta.roy
2022-09-01 03:03
Please find below the spec. I have renamed/ updated some fields. Thanks
`{`
`"swagger": "2.0",`
`"x-authapikey": true,`
`"info": {`
`"description": "get points",`
`"version": "1.0",`
`"title": "app apis",`
`"license": {`
`"name": "test"`
`}`
`},`
`"basePath": "/tester/test/v1",`
`"schemes": [`
`"https"`
`],`
`"paths": {`
`"/data/accounts/points": {`
`"get": {`
`"tags": [`
`"test"`
`],`
`"summary": "Get points balance",`
`"description": "Get points balance",`
`"operationId": "getPoints",`
`"x-servers": [`
`{`
`"url": "",`
`"description": "test"`
`}`
`],`
`"produces": [`
`"application/json"`
`],`
`"parameters": [`
`{`
`"in": "header",`
`"name": "token",`
`"description": "User Token",`
`"type": "string",`
`"required": true`
`},`
`{`
`"in": "header",`
`"name": "sub-key",`
`"description": "subscription key",`
`"type": "string",`
`"required": true`
`}`
`],`
`"responses": {`
`"200": {`
`"description": "successful operation",`
`"schema": {`
`"type": "object",`
`"properties": {`
`"Customer": {`
`"type": "object",`
`"properties": {`
`"FirstName": {`
`"type": "string"`
`},`
`"PointsBal": {`
`"type": "integer"`
`}`
`}`
`}`
`}`
`}`
`}`
`}`
`}`
`}`
`}`
`}`
sumanta.roy
2022-09-01 03:13
Here is the consumer contract. Thanks
`{`
`"consumer": {`
`"name": "MyPointsConsumer"`
`},`
`"interactions": [`
`{`
`"description": "a request for checking my points balance",`
`"providerState": "I want to check my points balance",`
`"request": {`
`"headers": {`
`"sub-key": "somekey",`
`"token": "sometoken"`
`},`
`"method": "GET",`
`"path": "/data/accounts/points"`
`},`
`"response": {`
`"body": [`
`{`
`"Customer": {`
`"FirstName": "SUMANTA",`
`"PointsBalance": 99354,`
`}`
`}`
`],`
`"headers": {`
`"Content-Type": "application/json"`
`},`
`"status": 200`
`}`
`}`
`],`
`"metadata": {`
`"pact-js": {`
`"version": "10.1.2"`
`},`
`"pactRust": {`
`"ffi": "0.3.9",`
`"models": "0.4.5"`
`},`
`"pactSpecification": {`
`"version": "2.0.0"`
`}`
`},`
`"provider": {`
`"name": "MyPointsProvider"`
`}`
`}`
matt.fellows
2022-09-01 04:04
OK I can reproduce. Upgrading to OAS3 seems to address this, is that an option for you?
sumanta.roy
2022-09-01 04:07
Thanks Matt. We have most of our specs available in Swagger 2.0 at the moment. Could you please help with how I can raise a support case ? Thanks
matt.fellows
2022-09-01 04:09
No probs.
Please send a ticket to and attach the info in this thread (the OAS, the Pact file and the screenshot/problem). Also make a note that I have requested you to raise the issue, so it will be routed to the dev team. Appreciate you doing this, and sorry for the issues.
I can?t see why it would be causing issues so need to dive deeper.
bethskurrie
2022-09-01 07:17
Also. Find the result in the matrix UI. Click on the link in the status tab. Then click on the ?API? button in the top right.
fabio.rodrigues
2022-09-01 13:27
Hi all :wave:
I'm doing a POC of using Bi-Directional Contract Tests (BCT) with Pactflow
The strategy i used started by using a already existing contract that we already enforce (written as Consumer Driven Contract Test with pacnet nugget). This means that:
- The Consumer contract is already being generated (with PactNet - pactflow)
- it already has checks and validations (some with matchers)
- The pipeline already publishes the contract
- The Provider also verifies the contract, and as the time of writing the validations are passing
Then grabbing the generated swagger file from the provider and checking if the contract is validated (and if not why? ). I only had to do a manually edit to the pact:
- to avoid impacting pipelines and other checks, i changed manually the name of the consumer and provider in the consumer pact by suffixing the pacticipants with '-bi'
In high terms the POCwas:
- Publish in pactflow the consumer contract
- generate the swagger of the provide service
- Publish in Pactflow the generated swagger ( no need at this time to check swagger against the service,since that swagger is generated by the service)
- Profit??
But instead of profit i found some obstacles:
- Looks like pactflow cannot handle well some url parameters: In our case in swagger we have a lot of address defined like this:
'''/v{version}/somecontrolleraction/{someOtherParamert}''' and in our consumer contract we have the address like /v1/somecontrolleraction/{someOtherParamert}. Both version and someOtherParamert are parameters defined in the request. For a concrete example we have the following entry in the swagger (abbreviated for brevity):
```"paths": {
"/v{version}/environments/{environmentKey}/deployed-applications": {
"get": {
"tags": [
"DeployedApplications"
],
"summary": "",
"description": "",
"operationId": "",
"parameters": [
{
"name": "environmentKey",
"in": "path",
"description": "Environment unique identifier",
"required": true,
"schema": {
"type": "string"
}
},
//other parameters removed for brevity
{
"name": "version",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],```
But Pactflow doesn't like that much:
fabio.rodrigues
2022-09-01 13:27
for continuing the POC i had to edit the swagger file, and change from v{version} to v1 in all address, and remove the version parameter block
fabio.rodrigues
2022-09-01 13:28
after that, i got other types of errors, but this types in relation to fields in matchers
fabio.rodrigues
2022-09-01 13:28
```Response body is incompatible with the response body schema in the spec file: should be equal to one of the allowed values
Mismatched at:
[root].interactions[1].response.body.applicationType```
fabio.rodrigues
2022-09-01 13:29
revisionDateTime, etc...
so, is there any way pactflow can handle tha v{version} pattern and what about matchers??
noor.hashem
2022-09-01 13:33
Okay I see what you mean. Now that I have a yaml openapi file that is checked how can I publish it to pactflow directly? I see many different ways to do it but i'm looking for just a simple way to directly publish a yaml openapi file
krystof.sykora
2022-09-01 13:56
Hello folks,
I?ve started working on a Bi-Directional Contract Testing POC and stumbled on a contract violation during the first few basic attempt. After some searching, I?m still not quite sure what the problem might be and so decided to try and ask here. The verification fails in the consumer contract with the reasoning, that no schema seems to have been found for the response body. Never the less from my limited understanding of Pactflow, it seemed to me it should have been provided as part of the API specification published from the provider.
Could you help me identify what it is I?m doing wrong exactly?
Thanks!
noor.hashem
2022-09-01 15:47
Hi all :wave:,
Is there a tool to publish an openapi yaml to pactflow? Or do we need to create a script to do this? I see in this example https://github.com/pactflow/example-bi-directional-provider-postman/blob/master/test/publish.js that there is a js script to do this but i'm wondering if there is some kind of pact tool already created to simply take an openapi file an publish it to pactflow?
matt.fellows
2022-09-02 05:33
Yes, there is a CLI for it: https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow
The example pre-dates the CLI. We?ll get it updated soon
matt.fellows
2022-09-02 11:26
The matchers aren't considered in this mode, but I believe we should support the param. It might be because the param is only a partial path segment instead of the full segment
matt.fellows
2022-09-02 11:28
That would be helpful indeed. If you can't share here, please send to so we can get the team to review
thomas.koppensteiner
2022-09-02 12:36
Hi :wave:,
similar to @fabio.rodrigues and @sumanta.roy I also get the `request contains unknown path or method` error within my pactflow setup.
Could this be related to path params?
thomas.koppensteiner
2022-09-02 13:05
Found it - the path parameter was defined in the wrong place within the OAS file.
matt.fellows
2022-09-02 23:04
You?ll need to specify the content type in the response to be explicitly `application/json`
matt.fellows
2022-09-02 23:04
i.e.
``` "content": {
"application/json": { <---------
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Project"
}
}
}
}```
matt.fellows
2022-09-02 23:05
The header `access-control-allow-origin` is also not present in the OAS. If you update the content type and add the new header, it should pass.
lewiscowles
2022-09-04 17:55
What is the point of having http://test.pactflow.io if examples used in language specific repo's are left broken?
matt.fellows
2022-09-04 23:33
That?s like asking why does BBC broadcast TV when there are broken televisions
matt.fellows
2022-09-05 00:27
I think you?re maybe talking about: https://github.com/pact-foundation/pact-php/pull/268
If so, the issue sounds like the build is relying on a pacticipant `Animal Profile Service` that doesn?t exist. This isn?t a broken URL in Pactflow, it?s a bad test that assumes an application should be there when it can be deleted by anyone (it?s a public broker!).
We have some examples that use Pactflow in other projects, but we always setup the consumer, publish the consumer contract and then verify the provider in the build to avoid this.
We also run them as a separate stage of the pipeline (they are essentially a form of integration test).
woojos
2022-09-07 07:11
Hi, i have an inquiry about clean up process. In docs, there is this feature explained but only for self hosted version https://docs.pact.io/pact_broker/administration/maintenance, and what about cloud version, pactflow?
bethskurrie
2022-09-07 07:11
We keep an eye on the database query times, and enable it when it is deemed necessary, in consultation with the customer.
pabcagi
2022-09-07 08:14
Hi all, one question, why exists two swagger-mock validator one in bitbucket and another in github? which should I use?
https://github.com/pactflow/swagger-mock-validator
https://bitbucket.org/atlassian/swagger-mock-validator/src/master/
dpal
2022-09-07 09:25
Hi, I'm working on message pact. Can you provide me some examples of message pact in pact flow. Thanks.
dpal
2022-09-07 09:26
https://test.pactflow.io/pacts/provider/pactflow-example-provider-js-sns/consumer/pactflow-example-consumer-js-sns/latest sample link is asking for credentials. Cant login using my pactflow credentials. Please help.
matt.fellows
2022-09-07 11:39
Oh, the account above is a test broker and not your account. Your password won?t work there
matt.fellows
2022-09-07 11:39
Where did you get that link from? Everywhere we use it should be accompanied by the user/pass e.g. https://docs.pactflow.io/docs/examples/js/provider/
yousafn
2022-09-07 11:53
they are not maintained by the same teams. The original was created by the Atlassian team, and Pactflow hold a modified fork in GitHub with additional options, and opinionated defaults.
yousafn
2022-09-07 11:56
Spot on Matt
Consumer - https://docs.pactflow.io/docs/examples/aws/sns/consumer/
Provider - https://docs.pactflow.io/docs/examples/aws/sns/provider/
noor.hashem
2022-09-07 13:20
Hi all, I have used a JS script to publish my pact using an openapi yaml file. But when I click on the arrow to show the contents of the contract it simply just shows a loading spinner forever. Has anyone encountered this?
noor.hashem
2022-09-07 13:41
I think it might be due to the fact that in my yaml file I have `$refs`. Because when I used the file from the POC you and Yousef did it worked fine. But as you can see from the image that i've attached we use refs
yousafn
2022-09-07 13:58
Two points.
1. Uploading via a JS file alone, won?t be sufficient as you need to create a branch associated with that version for the provider, this is done when you use the CLI tools for publishing provider contracts.
2. The only time I haven?t seem the OAS render is due to incorrect content types.
Could you provide the file and the script you are using to publish?
yousafn
2022-09-07 14:00
The example isn?t using that file any more, but it hadn?t been removed, have done so now :thumbsup:
matt.fellows
2022-09-07 23:28
Yep, see the docs. You must upload a fully resolved OAS. There are several tools that do this
dpal
2022-09-08 09:12
Thanks for the response. I have got the above link from https://docs.pactflow.io/docs/examples/js/provider/. "The latest version of the Example Pactflow Consumer/Example Pactflow Provider pact is published https://test.pactflow.io/overview/provider/pactflow-example-provider/consumer/pactflow-example-consumer." The here opens the page and asks for credentials. So I have asked for the credentials to see the latest examples.
dpal
2022-09-08 09:14
@matt.fellows @yousafn I do have the yml document of the event data model. Is there a way to convert that to a json object for pact? As we upload the swagger documents in open API format to the pact flow.
nathan.deamer
2022-09-08 10:44
:wave:. Continuing to play around with webhooks
Is there a way I can inject my own variables when I publish a new consumer contract?
https://docs.pact.io/pact_broker/advanced_topics/api_docs/webhooks#dynamic-variable-substitution
My usecase is that instead of using the `build_url`
```"build_url": "https://circleci.com/gh/nathandeamer/pact-mobile-app/125/workflows/3338e9f6-6123-45bc-a9cf-0ba4df3c3148"```
I also need to know which job id I am currently executing within that workflow.
I was wondering if I could put something in the consumer json file, which could then be used by the pactflow webhook.
yousafn
2022-09-08 10:46
> It is using a public tenant on Pactflow, which you can access https://test.pactflow.io/ using the credentials `dXfltyFMgNOFZAxr8io9wJ37iUpY42M`/`O5AIZWxelWbLvqMd8PkAVycBJh2Psyg1`. The latest version of the Example Pactflow Consumer/Example Pactflow Provider pact is published https://test.pactflow.io/overview/provider/pactflow-example-provider/consumer/pactflow-example-consumer.
Details in the readme for the public credentials
yousafn
2022-09-08 10:48
> I do have the yml document of the event data model. Is there a way to convert that to a json object for pact? As we upload the swagger documents in open API format to the pact flow.
This is an SNS based example which wouldn?t use OpenAPI. What have you got documented in the OpenAPI?
Are you trying to test a message based consumer, with Bi-Directional Contract testing?
matt.fellows
2022-09-08 10:58
We don't support that yet sorry. It's something we're aiming for in Q1 2023 (current target)
dpal
2022-09-08 10:59
so I need to follow your examples for the provider for the async api for provider?
nathan.deamer
2022-09-08 10:59
Just seen `${pactbroker.consumerLabels}` so wondering if I could make use of them
yousafn
2022-09-08 11:01
Please see https://docs.pactflow.io/docs/examples
See Kafka for a Java message based demo
noor.hashem
2022-09-08 18:08
Hi all, im using bidirectional contract testing with openapi for the provider and the regular consumer contract. But am getting the attached error for `"unknown format "integer" is used in schema at path "#/properties/value"` . I'm not sure why this error is occurring as believe my expected response is correct and matches the provider.
matt.fellows
2022-09-08 22:37
It looks like it can't parse the OAS fully. Are you able to please share?
noor.hashem
2022-09-09 13:31
Hmm okay I think I may have figured this one out just now. I tried another endpoint and it ended up working I think it was due to the OAS one of the types may have been incorrect.
jordan.nazemi
2022-09-12 16:49
Hi there, sorry if this is a silly question but is there a list of Cypress version compatibility with the pact-cypress-adapter? Just realized it doesn't work on Cypress <8.2 and was curious to see if it was compatible with the newer Cypress 10 releases. DIdn't see anything in the release notes on Github
yousafn
2022-09-12 17:05
Hi, it should work with 9
It's being tested with 9.5.0 in the repo
https://github.com/pactflow/pact-cypress-adapter/blob/main/example/todo-example/package.json
And 9.5.0 in our e2e demo
https://github.com/pactflow/example-bi-directional-consumer-cypress/blob/28669a49080d16cfb62291c1a10d55b431d61474/package.json#L49
I've not tested it with 10 yet, I know there are some package changes like the removal of cypress.json and I need to do update some of my own cypress plugins to support ( possibly :thinking_face: )
What issue are you seeing above 8.2 and in 10?
yousafn
2022-09-12 17:06
Oh doesn't work with less than 8, yeah that makes sense, I believe cypress was version 8 when the adapter was built
yousafn
2022-09-12 17:10
Fair few http logging improvements in 8.2.0
https://github.com/cypress-io/cypress/releases/tag/v8.2.0 so it's plausible it won't work prior but would need to dig into the code. Just on mobile atm
jordan.nazemi
2022-09-12 17:51
From what I've tested the only part that seem to throw errors is that the adapter tries to pull the test name using
```const testCaseTitle = Cypress.currentTest.title```
on line 41 of https://github.com/pactflow/pact-cypress-adapter/blob/955d2150c9d11423e1e91ece6581636bbcc1e8e3/src/index.ts. From what I've gathered https://docs.cypress.io/api/cypress-api/currenttest currentTest was only added in 8.2. I've actually tested locally filling that field manually with a string and everything else seems to work properly.
jordan.nazemi
2022-09-12 17:53
I wish it could allow for a blank string if it pulls None from that value, but I understand not wanting to support backwards compatibility that far. As for 10 I've not tried yet but as you said I've seen they've significantly changed the configs for Cypress so I imagine as-is it may need some messing about to get working.
dpal
2022-09-13 10:34
Hello, the import for the below is not happening for me.
```import au.com.dius.pact.provider.junit5.AmpqTestTarget```
dpal
2022-09-13 10:35
```<dependency>
<groupId>au.com.dius.pact.provider</groupId>
<artifactId>junit5</artifactId>
<version>4.3.14</version>```
yousafn
2022-09-13 11:00
You can raise a PR to support it, if you have a need to use versions lower than 8.2, otherwise even adding to the readme to state the correct versions supported from would be a great start :+1::skin-tone-4:
noor.hashem
2022-09-13 13:15
Hey! Quick question. I have manually created two contracts one for consumer the other for a provider. The consumer contract was created through a normal js file while the provider contract was created through openapi. I am now in the process of trying to create github workflows/actions and wanted a bit of clarity on this CI/CD process. From my understanding what I need to be able to do in the workflows for say the provider side is:
1. Run can-i-deploy
2. Then if it passes publish the openapi yaml file
What about versioning? Could someone highlight a more detailed step by step of what needs to be done in the workflows?
yousafn
2022-09-13 13:29
When you publish the provider contact you supply a version number. Can I deploy if there are no consumers will allows return yes to deploy your provider. Once deployed you record the deployment to an environment with the pact CLI.
If you try and run can I deploy, prior to publishing the provider contract you will be told that the pacticipant cannot be found
stefan.tertan
2022-09-13 15:50
Hi all, I tried implementing an MVP based on `pact-workshop-js`. I got the consumer, provider & Pactflow broker working together & CI setup (using Drone CI).
Most documentation mentions tags for tagging pacts & specifying which pacts to verify, however, it is suggested to use environments & releases/deployments instead of tags for newer versions.
Our usual git flow is to create feature branches, raise a PR & merge to `main`. From main, we can promote (deploy) to staging or production.
I'm quite confused as to what the pipelines should look like and which pact versions to verify where.
I have configured a webhook for the `contract_requiring_verification_published` event to trigger a verification pipeline for the provider.
But there is also a `provider_verification_published` event; should that trigger a consumer pipeline via a Pactflow webhook too?
Also, although the webhook to trigger the provider build should only trigger on contract changes, it seems to trigger every time.
Could someone perhaps take the time to do a Zoom call with me and go over my existing setup?
jordan.nazemi
2022-09-13 16:10
@yousafn Hi Yousaf, I've made a PR https://github.com/pactflow/pact-cypress-adapter/pull/17 for you to review whenever you have a chance. It just locks the currentTest usage behind and if statement to allow for older Cypress versions to use usePactWait and getPactGet.
matt.fellows
2022-09-13 22:36
> But there is also a `provider_verification_published` event; should that trigger a consumer pipeline via a Pactflow webhook too?
no you shouldn?t need that, that sounds like a recipe for build loops that could go on forever :laughing: (although the ?contract requiring verification? should filter out duplicate contracts)
matt.fellows
2022-09-13 22:36
You?re right, we are in the final stages of finishing the work fro branches/releases, and the docs / workshops are last to update
nathan.deamer
2022-09-14 13:30
I?m having some problems with pactflow and adding secrets:
```An error has occurred. The details have been logged with the reference mFZfPkJyZl
Error reference: mFZfPkJyZl```
noor.hashem
2022-09-14 18:38
Hmm I see. I see here: https://github.com/pactflow/example-bi-directional-provider-postman/tree/master/.github/workflows
You have 4 workflows. Are they all necessary? For the provider I simply want to run can I deploy, then if it passes id like to deploy with a new version of the contract published to pactflow.
yousafn
2022-09-14 18:43
This one
https://github.com/pactflow/example-bi-directional-provider-postman/blob/master/.github/workflows/build.yml
Tests app, publishes contract and records deployment of the application if on master/main
bethskurrie
2022-09-14 23:16
@stefan.tertan have you done the CI/CD workshop? https://docs.pactflow.io/docs/workshops/ci-cd/
bethskurrie
2022-09-14 23:17
This walks you through all the webhook set up, though as Matt says, there is a slight update to the webhook configuration for the new ?https://docs.pact.io/blog/2021/10/11/contract-requiring-verification-published-webhook-event? webhook.
bethskurrie
2022-09-14 23:20
Your workflow will match the CI/CD workshop example perfectly. I think it will answer your questions.
matt.fellows
2022-09-14 23:21
Hi Nathan, we?ve seen a few of these recently and they only seem to be happening on your account. We have started the process to find out why, including opening a ticket with AWS.
cc: @bethskurrie
bethskurrie
2022-09-14 23:42
I?ve seen the error logs popping up over the last couple of days. I?ll have a look at it today.
nathan.deamer
2022-09-15 08:54
Thanks. Support seems to have resolved it for us :slightly_smiling_face:
stefan.tertan
2022-09-15 10:03
hi Beth, the workshop is a bit outdated , even the `feat/10.x.x-upgrade` branch still uses tags:
```//consumerVersionTag: ['master', 'prod'], //the old way of specifying which pacts to verify
consumerVersionSelectors: [{ tag: 'master', latest: true }, { deployed: true } ], // the new way of specifying which pacts to verify```
I'm doing the below, but am not sure if it's correct:
```consumerVersionSelectors: [
{deployed: true, environment: 'staging-uk'},
{deployed: true, environment: 'production-uk'},
],```
stefan.tertan
2022-09-15 10:03
I've only done the Pact Workshop at https://github.com/pact-foundation/pact-workshop-js/tree/feat/upgrade-to-10.x.x
stefan.tertan
2022-09-15 10:04
I'm gonna go over the Pactflow Workshop on that link you provided.... hopefully it will help clear the issues that I'm facing :+1:
matt.fellows
2022-09-15 10:05
The JS workshop really takes you up to the point of where a broker would come in, where the CI/CD workshop would kick in
matt.fellows
2022-09-15 10:05
It's more about the mechanics of contract testing, not the CI/CD integration
matt.fellows
2022-09-15 10:06
We'll be updating the docs and workshops soon to reflect this latest thinking (branches/environments)
bethskurrie
2022-09-15 10:53
This is the recommended config for the consumer version selectors https://docs.pact.io/provider/recommended_configuration
nathan.deamer
2022-09-15 11:18
Thank you! Out of interest, what was the cause? / Was special about our account?
jordan.nazemi
2022-09-15 14:58
Hi @yousafn absolutely no rush but just so I know how to plan our QA teams path going forward. Does this seem to be a change you would approve or should we aim to await the Cypress 10 release?
2022-09-15 15:04
This message was deleted.
yousafn
2022-09-15 17:26
Hi buddy, I'm away this week at a conference but will hope to have a look on Tuesday ( Monday is a bank holiday )
I think I would be okay supporting the change, but just need some time to test. Will try and get Cypress 10 on the radar soon, feel free to raise an issue to track.
Are you able to publish a scoped version and use that as an interim?
jordan.nazemi
2022-09-15 18:10
Sounds great, thank you! We're not under a huge crunch to get this implemented, my manager just wanted to know if were should start considering alternative approaches if this is unlikely to be our solution. Have a great time at the conference!
noor.hashem
2022-09-15 19:41
Not so much, it was an error on my end. Just that I had removed the id from the example but it was still in the scheme for the provider. I needed to remove the id from the examples as well as from the schema and then it failed as expected.
matt.fellows
2022-09-15 19:43
That indicates to me there is an issue with the way you're testing your provider. If you can remove a required field but that still pass that would be a concern. Unless of course you're still building up to that
noor.hashem
2022-09-16 14:58
Hi all, it seems that I have gotten the above error once again. This time i've reviewed and cannot see why pactflow is labelling the consumer and provider as compatible when they should not be. From the images attached you can see that the consumer expects and `id` but the provider does not provide one. Why is pact able to label this as compatible?
noor.hashem
2022-09-16 15:16
Also, as you can see from this image, it says, "Compared at: a day ago" but I have released several versions of the provider just a few minutes ago.
noor.hashem
2022-09-19 16:57
Hi all, has anyone ran into some problem where the consumer and provider should not be compatible but pactflow is showing that they are? I have the above ^^ problem I posted last week that is still continuing and i've searched up quit a bit and cannot find much online.
abubics
2022-09-20 00:43
I can't tell from these screenshots what the matchers in the contract are, or what is actually being returned. Probably need to look at the consumer-side test, and the logs of the provider verification.
abubics
2022-09-20 00:44
(I'm not really familiar with the Pactflow UI, and the other screenshots look like Swagger? They all look like example payloads, rather than anything Pact would assert against)
noor.hashem
2022-09-20 00:46
I was under the impression as per the docs on pactflow that it should confirm that the consumer is a subset of the provider. But in this case even without seeing any more info isn't it breaking that rule, we can see that the consumer says it needs an id but the provider is not giving it an id.
abubics
2022-09-20 00:47
Oh, you're doing bi-directional contract testing? I don't know much about that either.
noor.hashem
2022-09-20 00:47
Haha yes i'm doing bidirectional. Okay no worries thanks for looking into though.
cgoodison
2022-09-20 02:09
Hi Noor, are you able to see the more recent verification results in the matrix page for this integration? Wondering if this could be an issue with the overview page.
noor.hashem
2022-09-20 03:13
Yep I see them there and they are also failing. Could this be an issue with content type? In my open api we have content set as `application/vnd.api+json` but in the example https://github.com/pactflow/example-bi-directional-provider-postman/blob/master/oas/swagger.yml they have it set as `application/json` . I tried changing it and I can see that now it seems to work but the issue with this is that all of our openapi files are using `application/vnd.api+json` . There are hundreds of places where the `application/vnd.api+json` is used and would be very difficult to replace it. Is there any other way to allow pactflow to be able to read this content format?
cgoodison
2022-09-20 03:52
so in both your pact file and oas you are setting the content type as `application/vnd.api+json`? I believe in theory that should work with this content type, so this seems to point to an issue with comparison code. I can't think of another work around but will look into it more
cgoodison
2022-09-20 04:15
Are you able to share the actual error message as it appears in the pactflow UI? I created this example using `application/vnd.api+json` , where the pact file is expecting the response to have that type, and the OAS dictates that it can produce that type. I'm not sure if this is the same use case that you are working on?
pact
```{
"consumer": {
"name": "default-consumer-name"
},
"interactions": [
{
"description": "interaction description",
"request": {
"method": "post",
"path": "/does/exist"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/vnd.api+json"
}
}
}
],
"metadata": {
"pactSpecification": {
"version": "1.0.0"
}
},
"provider": {
"name": "default-provider-name"
}
}```
OAS
```{
"info": {
"title": "default-title",
"version": "1.0.0"
},
"paths": {
"/does/exist": {
"post": {
"responses": {
"200": {
"description": "default-response"
}
},
"produces": [
"application/vnd.api+json"
]
}
}
},
"swagger": "2.0"
}```
cgoodison
2022-09-20 04:26
Just to clarify, the overview page is showing an outdated validation, but on the matrix one you can see the most up to date validation result (which is failing as expected?)
cgoodison
2022-09-20 04:27
only asking to check that there are 2 seperate issues you are experiencing so I can troubleshoot both
drakulavich
2022-09-20 08:50
I was impressed by the great workshops and materials by the Pactflow team and decided to create some kind of sandbox to try bi-directional contract tests with RealWorld project: https://dev.to/drakulavich/operation-pact-or-how-i-learned-to-stop-worrying-and-love-contract-testing-4nhh
Would love to see any feedback. Cheers!
yousafn
2022-09-20 11:39
Oooh lovely, I've been meaning to do something with GoThinksters real world app for a while
drakulavich
2022-09-20 12:00
Cool! I like that project because you can show nontrivial problems with it. It's a kind of an app in `production`
yousafn
2022-09-20 12:09
I was most impressed by the interoperability between different language implementations and that they could be tested to that effect via the machine readable spec (OAS), polyglotters dream
noor.hashem
2022-09-20 14:39
Thank you @yousafn and @cgoodison for the investigation on this! Much appreciated.
noor.hashem
2022-09-20 14:52
Hi all, what is the best way to upload more than one pact in github workflows? So far in my consumer github workflows I have a few simple commands just to publish a single pact and run can-i-deploy. But What if I am in a monorepo and there may be several different files where there are many pacts in each file and they may all have different providers and names? How can I scale this to a more realistic CI flow? I saw that some https://github.com/pactflow/example-bi-directional-consumer-cypress/blob/main/scripts/trigger-cross-test.sh were using bash scripts but was not entirely sure what for?
yousafn
2022-09-20 15:20
Think we?ve figured out the issue with a repro documented here
https://github.com/pactflow/swagger-mock-validator/issues/12
yousafn
2022-09-20 15:22
the pact-broker publish command will publish a single pact, or any pacts in a directory. You are pointing it to the `./pacts` directory in your screenshot.
The `trigger-cross-test` is a workflow designed for testing our examples via GitHub actions against different operating systems, and methods of upload ( standalone cli, ruby cli and dockerised cli )
yousafn
2022-09-20 15:25
The pact publish command will perform duplicate detection, so if you create a pact files for the same consumer/provider pair, with different interactions, they will all be uploaded under the version/branch/tag associated with the publish command
noor.hashem
2022-09-20 17:33
But what about the can-i-deploy command? It needs the _`--pacticipant`_ right? How would I be able to run that command and have it check multiple pacts with different providers and consumers??
jordan.nazemi
2022-09-20 19:26
Hey @yousafn thanks for making the change and merging! Would you be able to publish the newest tag when you have the chance so we can start importing the changes? Appreciate it!
omarsaddaoui
2022-09-20 21:54
Thank you for this document. bu in have a question about this call https://gitlab.sngular.com/sngulartech/bi-direccional_contract_testing/-/blob/main/scripts/publish-provider.sh#L28 where you found it. i can't find it anywhere
bethskurrie
2022-09-20 23:08
```pact-broker can-i-deploy --pacticipant monorepo-app-1 --version 1 --pacticipant monorepo-app-2 --version 1 ... --to-environment test```
matt.fellows
2022-09-21 00:12
Awesome, thanks Anton!
We were literally talking about doing something like this the other day :clap:
Can?t wait to have a play around (likely not until next week I?m afraid)
anhtvuong
2022-09-21 05:52
I just clone master on example-provider-java-kafka.git and run ./gradlew clean test which fails with errors below. Any advice? Thanks.
`Starting process 'Gradle Test Executor 5'. Working directory: /ws/vuongan/pacts/example-provider-java-kafka Command: /ws/vuongan/jdk-17.0.4/bin/java -Dorg.gradle.internal.worker.tmpdir=/ws/vuongan/pacts/example-provider-java-kafka/build/tmp/test/work -Dorg.gradle.native=false @/ws/vuongan/dev/.gradle/.tmp/gradle-worker-classpath2637501272871811105txt -Xmx512m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea http://worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 5'`
Successfully started process 'Gradle Test Executor 5'
ProductsKafkaProducerTest > testTemplate(Pact, Interaction, PactVerificationContext) > io.pactflow.example.kafka.ProductsKafkaProducerTest.initializationError FAILED
org.junit.platform.commons.PreconditionViolationException: You must register at least one TestTemplateInvocationContextProvider that supports @TestTemplate method [void io.pactflow.example.kafka.ProductsKafkaProducerTest.testTemplate(http://au.com.dius.pact.core.model.Pact,http://au.com.dius.pact.core.model.Interaction,http://au.com.dius.pact.provider.junit5.PactVerificationContext)]
at app//org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:296)
anhtvuong
2022-09-21 06:08
I successfully run example-consumer-java-kafka with jdk 17 and build.gradle:
plugins {
id 'org.springframework.boot' version '2.7.3'
id "http://au.com.dius.pact" version "4.2.0"
id "io.freefair.lombok" version "6.5.1"
}
apply plugin: 'java'
apply plugin: 'io.spring.dependency-management'
group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '17'
then try jdk 17 on provider. I need to use jdk 17. Any suggestion that I can try with 17?
uglyog
2022-09-21 06:11
:man-shrugging: looks like that example is not working. Try upgrade the Pact version to the latest
anhtvuong
2022-09-21 06:32
Tried with 4.3.14 and same failure:
`> Task :clean`
`> Task :processResources`
`> Task :processTestResources`
`> Task :generateEffectiveLombokConfig`
`> Task :compileJava`
`> Task :classes`
`> Task :generateTestEffectiveLombokConfig`
`> Task :compileTestJava`
`> Task :testClasses`
`> Task :test FAILED`
`ProductsKafkaProducerTest > testTemplate(Pact, Interaction, PactVerificationContext) > io.pactflow.example.kafka.ProductsKafkaProducerTest.initializationError FAILED`
`org.junit.platform.commons.PreconditionViolationException at Preconditions.java:296`
`1 test completed, 1 failed`
gayatree.eee
2022-09-21 06:53
Not able to connect to postgres DB after upgrading the pactflow version to 1.20 from 1.17. Please help
alan.zhu
2022-09-21 10:32
Hey, team
A quick question: how to move the existing default pacts into the team?
jwang
2022-09-21 19:47
@thomas.koppensteiner Can you clarify a bit on what the, "wrong place" is? I am seeing the same error in my setup.
yousafn
2022-09-22 11:49
Hey v1.3.0 is now out :rockon: Will get some time to look at cypress 10 at some point, thanks for the support on this
noor.hashem
2022-09-22 13:55
Hi all, I can see that in some example we are using webhooks to check if the pact has changed etc. But when is this needed? The bidirectional flow I currently built is:
*Provider:*
? Publish openapi to pactflow
? Run can-i-deploy
*Consumer:*
? run the tests `npm run test:pact` to test and create the pacts from the test
? publish the pacts to pactflow
? run can-i-deploy
This flow is simple but seems to work great at least for bidirectional and for one provider and one consumer, is this alright or is there more complexity that I am missing here? Note that I currently only have one provider and one consumer and only one pact for each. Also note that for now we are skipping the "verifying the provider contract" step as we are still looking into that.
jordan.nazemi
2022-09-22 16:53
Thank you! Everything is working great and this should be all we need to get online with Pact. Appreciate it.
matt.fellows
2022-09-23 03:01
We don't really recommend tags, and I can see the action doesn't support it: https://github.com/pactflow/actions/blob/main/publish-provider-contract/publishOAS.sh
What do you need it for? It should be easy to add if you're up for a PR though
noor.hashem
2022-09-26 13:06
Oh I see, I don't mind not using tags I was simply just following along with the docs and saw that they were used. But we don't need them for any particular reason.
dpal
2022-09-27 14:29
Hi @yousafn, I followed your java messaging provider example, now I want to run only the provider test. How to do that? I'm using maven and java. I want to run only the provider test to generate the provider pact json. Please help. Thanks
yousafn
2022-09-27 14:43
https://github.com/pactflow/example-provider-java-kafka run `make test` or `./gradlew clean test` - these verify pact contracts generated by the consumer application.
The provider does not produce a pact file as an output of it's test phase, it generates a verification result, for the pacts its verified as returned by the providers consumer version selectors
dpal
2022-09-27 15:02
okay. I'm a bit confused here. Please correct me. For http services we have Provider OpenAPi documentation which we upload to Pactflow and against which the consumer pact gets validated. I am in an impression that for messaging this will be the same. The only missing part was the openapi documentation of provider message generator. As you have already mentioned that you do not support async openapi so my expectation was the PactVerifyProvider is the place which generate the provider pact.
yousafn
2022-09-27 16:22
> For http services we have Provider OpenAPi documentation which we upload to Pactflow and against which the consumer pact gets validated.
For Bi-Directional contract testing.
Traditional Consumer-Driven contract testing will use a Pact Provider verification build, where it will pull down Pact files from a broker, which have been generated by the consumer codebases
dpal
2022-09-28 08:39
@yousafn so for messaging its only consumer driven contract testing? In your Java example the Pact gets created its from the consumer test?
yousafn
2022-09-28 08:57
Yes correct buddy.
We are aiming for AsyncAPI support in Bi-directional contract testing in q1 2023, but currently it only supports OpenAPI specification.
Consumer driven supports both HTTP and message pact.
matias.cardenas
2022-09-29 13:26
Hi, quick question, has any timeline been set or do you see guys supporting this somewhere soon?
Just asking as we?re trying to move towards adopting OpenAPI 3.1 for our schemas, but that?d also depend on our possibilities of keep using pactflow as our contract testing solution. Thanks in advance!
https://pact-foundation.slack.com/archives/CLS16AVEE/p1660114630946419
yousafn
2022-09-29 14:06
Hey @matias.cardenas,
It is on our product roadmap for late 2022 / early 2023 across both Pactflow & SwaggerHub, as we also would like to both see and support adoption of OAS 3.1
I?ve just reflected a ticket on our open-source roadmap, and you can register your interest there if you wish. Will fill out with further detail shortly :+1:
https://github.com/pactflow/roadmap/issues/81
noor.hashem
2022-09-29 15:38
Hi , quick questions here, if we are using consumer driven contract testing, how can we publish the provider in github workflows, can we still use:
`uses: pactflow/actions/publish-provider-contract@v0.0.2`
Or is there some other way? As in the github workflows in this https://github.com/pact-foundation/pact-workshop-js I could not find any place where the provider contract was being published except in the test itself where it shows the code in the image i've attached.
yousafn
2022-09-29 16:12
Hey,
in CDCT, you don?t publish a provider contract,
verification results are published by a provider build or trigger from a changed pact
see here in the CI/CD workflow https://docs.pact.io/pact_nirvana/step_4#verifying-pacts
Example workflow here for the new changed https://docs.pact.io/pact_broker/webhooks#using-webhooks-with-the-contract_requiring_verification_published-event
https://github.com/pactflow/example-provider/pull/29
matt.fellows
2022-09-30 01:49
BDCT - there are two contracts (consumer and provider contract)
CDCT - just the consumer contract
matt.fellows
2022-09-30 01:51
Hi!
It?s on our backlog as Yousaf said, we?re aiming for this in Q1 2023. Definitely keep abreast of https://github.com/pactflow/roadmap/projects/1 for major features/updates and plans :slightly_smiling_face:
fabio.rodrigues
2022-09-30 08:45
more in depth:
In BDCT, consumer publish a contract, it's expectations on how to use the API, and the Provider publish it's service contract (for example the OpenAPI spec of the service) plus (highly recommended) the API test verification results (that is if the service itself conforms to the spec). Pactflow then will see if both contracts are compatible (like if the consumer enum that it uses in the request is one of the enum valid values)
In CDCT, consumer publish again the contract with it's expectation. The difference now comes in the Provider side. The provider will download a version of the contract, have a way to setup some provider state ( for example if the test requires it, add some data to the database) and run the request against the live service, and then it checks if the given response matches the consumer expectation (did we get a 204 instead of a 200, did all expected headers are passed)
dpal
2022-09-30 08:52
Hi @yousafn, I have one question. When a provider is associated with multiple consumers, when a new version of the provider published to Pactflow, will this update all the consumer checks? As per my understanding it tracks based on the provider name. Please correct me if I'm wrong. Thanks
matt.fellows
2022-09-30 12:00
See http://docs.pact.io/selectors for what pacts you are verifying and what compatibility checks you need
matt.fellows
2022-09-30 12:01
It doesn't update checks, because each check is associated with specific versions of the provider. The new provider version will have a check (verification result) for each pact it verifies
noor.hashem
2022-09-30 13:02
I see, this makes sense but then what if on the consumer side we change the contract, now in the github actions how do we run that new contract against the provider automatically? I see that we can use webhooks but how would webhooks allows us to run the new contract against the provider in the consumer CI
yousafn
2022-09-30 13:11
you would use webhooks which trigger when a contract content changes on the consumer side, to trigger a build on the provider side.
see https://docs.pact.io/pact_nirvana/step_6
If you cannot do that, but have access to the provider codebase, to be able to check it out in your SCM, you can use a webhookless workflow
https://github.com/pactflow/example-consumer-webhookless
> This workflow ensures changed pacts are verified immediately by checking out the provider codebase in the consumer's pipeline, and running the verification as part of the consumer's own tests.
aurelia.rochat
2022-09-30 13:59
I am currently facing an issue which seems to be the same.
I have ?* */ *? as content type both in the OAS specification and in the generated pact, and I stil have that ?*No schema found for response body? error.
Do you think it is the same issue ?
aurelia.rochat
2022-10-02 18:13
No, not yet, as I?d have to change either the endpoints implementations or how the OAS is generated, but I?ll give it a try.
cgoodison
2022-10-02 23:38
hmm not sure what could cause this, are you able to run 'can-i-deploy' locally? The console output might show a more informative error
eddie
2022-10-02 23:47
So digging a bit deeper, the logs show this:
There's some useful info in the (full) logs:
```[Step 4/8] Compiled provider version (optional): Some(1.0.0.771-855b70d)
[21:44:08][Step 4/8] Starting verification...
[21:44:08][Step 4/8] Pact verification failed
[21:44:08][Step 4/8]
[21:44:08][Step 4/8] Verifier Output
[21:44:08][Step 4/8] ---------------
[21:44:08][Step 4/8] The pact at https://xero.pactflow.io/pacts/provider/ManualUpload.BFF/consumer/ManualUpload.SPA/pact-version/bf45cbec498cbe559fd77fccd3807d9273e99344 is being verified because the pact content belongs to the consumer version matching the following criterion:
[21:44:08][Step 4/8] * consumer version(s) currently deployed to test (78047465772cce9c1e37edf0d09181761236670c)
[21:44:08][Step 4/8]
[21:44:08][Step 4/8] Verifying a pact between ManualUpload.SPA and ManualUpload.BFF```
In particular:
> *The pact at [https://xero.pactflow.io/pacts/provider/ManualUpload.BFF/consumer/ManualUpload.SPA/pact-version/bf45cbec498cbe559fd77fccd3807d9273e99344..] is being verified because the pact content belongs to the consumer version matching the following criterion: consumer version(s) currently deployed to test (78047465772cce9c1e37edf0d09181761236670c*
We don't think that SHA (`78047465772cce9c1e37edf0d09181761236670c`) is deployed to any environment, definitely not `test`.
cgoodison
2022-10-02 23:51
ah could be erroneous record deployment somewhere. Could try 'undeployment': https://docs.pactflow.io/docs/bi-directional-contract-testing/deploying/#recording-undeployments
eddie
2022-10-02 23:51
I _think_ we've figured this out. `test` was an old environment that we're no-longer supposed to be using. I think we will need to clear the record of that deployment, probably with https://docs.pact.io/pact_broker/recording_deployments_and_releases#recording-undeployments
aurelia.rochat
2022-10-03 07:35
@matt.fellows you were totally right, specifying application/json in the oas does the trick.
aurelia.rochat
2022-10-03 07:37
Hello, I have a question regardingn the UI and results display.
Although the contracts are compatible, in the ?overview? tab it still shows as ?unknown.?
Any idea why ?
cgoodison
2022-10-03 07:44
Hi Aurelia Have you uploaded a verification for your provider? You can check in the provider contract tab
cgoodison
2022-10-03 08:00
This section shows that it's unverified, you'll need to upload a verification result that indicates your oas contract is implemented by the API. https://docs.pactflow.io/docs/bi-directional-contract-testing/provider/
aurelia.rochat
2022-10-03 09:08
ok, indeed that step is currently missing, thanks for the prompt answer :slightly_smiling_face:
aurelia.rochat
2022-10-03 11:10
I struggle finding concrete example of this verification.
In my case, the OAS is generated automatically, thus I?d like to skip the verification and always publish a successful result for the verification step.
I found an example here : https://www.ontestautomation.com/an-introduction-to-contract-testing-part-6-bi-directional-contracts/, but I don?t know what structure should the verficationResults.content field have.
Can someone help me on this ?
matt.fellows
2022-10-03 12:04
Just put something in there to indicate how you came to the conclusion it was safe
matt.fellows
2022-10-03 12:04
it?s just an arbitrary field - I?d suggest ?automatically generated by <tool>?
noor.hashem
2022-10-03 14:36
This is great @yousafn thank you very much. Is there any preference between using the webhooks approach verses the non webhooks approach of checking out the provider branch and running the tests there?
noor.hashem
2022-10-03 17:03
When I wanted my provider contract to be verified just for a PoC stage I simply made a json file that was basically empty with only `{}` and pointed to it as my verification file.
yousafn
2022-10-04 19:38
Webhooks, specifically https://docs.pact.io/pact_broker/webhooks#using-webhooks-with-the-contract_requiring_verification_published-event
Are the recommendation, along with the WIP pacts and pending pacts features.
omarsaddaoui
2022-10-05 14:55
Hi All, quick question Does anyone of you have an explanation for this? I'm supposed to have the UI swagger. but I have No API definition provided.
yousafn
2022-10-05 14:59
Hey,
How did you upload your contract, and what content type did you provide?
Are you able to share your contract? I assume your OAS doesn't reference any external files, and is dereferenced?
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/#document-limitations
yousafn
2022-10-05 15:22
Thanks for that :+1: How did you upload it?
Just tested now
```pactflow publish-provider-contract \
oas.yaml \
--provider test \
--provider-app-version 1.0.1-some-sha \
--branch test \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results oas.yaml \
--verification-results-content-type application/yaml \
--verifier test```
omarsaddaoui
2022-10-05 15:30
like this.
I don't see what's wrong with it :expressionless:
pactflow publish-provider-contract ./tests/Pact/Consumer/Icp/provider/swagger/oas.yaml \
--broker-token \
--broker-base-url \
--provider ${PARTICIPANT} \
--provider-app-version ${VERSION} \
--branch ${BRANCH} \
--content-type CONTENT_TYPE \\\
--verification-exit-code=0 \\\\N
--verification-results ./tests/Pact/Consumer/Icp/provider/collections/ContratStockreport.postman_collection.json \\
--verification-resultats-content-type REPORT_CONTENT_TYPE \
--verifier postman
yousafn
2022-10-05 15:33
`application/yaml` for your OAS and `application/json` for the collection, although you should be passing in the output of your test run, I assume with newman. as `text/plain`
this param is also spelt incorrectly `--verification-resultats-content-type` it should be `--verification-results-content-type`
matt.fellows
2022-10-05 21:30
Another concern is how you missed the error code uploading. Did you do it by hand or via a pipeline? If the latter it seems you might not be checking the exit status
kenneth.krause
2022-10-06 16:58
Hi. We have SSO using AzureAD and I am trying to figure out where our problem is. I have removed users in AzureAD from the pactflow group, but the users are not getting removed from pactflow user list. So we are showing more users have access than our external IdP. Can you advise where the problem may lie? Adding new users is not a problem--they have access within minutes, so the ADD integration is working.
rachelxelizabethh
2022-10-06 18:30
Hi! I'm hoping this is the right channel to ask this question ... we are using the "Hosted Stubs" feature of PactFlow and are having some trouble coming up with regular expressions for dynamic paths that have path parameters followed by other information. For example, we have two endpoints "/accounts/{accountId}" and "/accounts/{accountId}/transactions" that we would expect two different responses.
Currently we are using the regex expressions "\/accounts\/[A-Za-z0-9]+" and "\/accounts\/[A-Za-z0-9]+\/transactions" to try and accomplish this, which works fine for "/accounts/123", but "/accounts/123/transactions" is returning the same response.
Anyone have any guidance on how to get around this? It looks like somehow the stub is returning the first path that *contains* that regex, and does not match it exactly.
I have already tried "^\/accounts\/[A-Za-z0-9]+$" but it seems like the ^ and $ are being taken as the characters themselves, not the beginning and end.
Any help would be appreciated :)
yousafn
2022-10-06 19:04
Hey hey, this channel is grand. Hard to choose sometimes, we get the same here at the Pact/Pactflow team
I think you want an optional capture group on the `/transactions`
https://rubular.com/
`\/accounts\/[A-Za-z0-9]+(?:\/transactions)?`
yousafn
2022-10-06 19:06
> For example, we have two endpoints "/accounts/{accountId}" and "/accounts/{accountId}/transactions" that we would expect two different responses.
Oh do you want a capture group that only captures
`/accounts/123/transactions` but excludes `/accounts/123`?
yousafn
2022-10-06 19:07
in that case you can drop the optional group
```\/accounts\/[A-Za-z0-9]+\/transactions```
yousafn
2022-10-06 19:13
for just matching `/accounts/123`
and excluding anything with an additional forward slash after your alpha/digit capture group
`\/accounts\/[A-Za-z0-9]+(?!\/|[A-Za-z0-9])`
rachelxelizabethh
2022-10-06 19:39
Basically, there are two interactions that I need to account for, so I need both scenarios described :slightly_smiling_face: I would expect /accounts/123 to return one response but /accounts/123/transactions to return another.
But that last one may be what I'm looking for! Thanks, let me give it a try
omarsaddaoui
2022-10-07 07:30
Hello,
Quick question, I have an ICP-Postman provider that had 2 contracts with 2 versions 33azz2 and b7a25b45. I deleted the 33azz2 version with the endpoint https://xxxx.pactflow.io/pacticipants/ICP-postman/versions/33azz2.
When I run can-i-deploy ICP1, corss contract verification is fail with 33azz2 although I had deleted it.
Do you have any idea pls ?
matt.fellows
2022-10-07 14:02
Removing them from AD won't remove them automatically from Pactflow until our SCIM API is implemented and in use by your IDP (Q4 roadmapcitem currently in progress). You'll need to disable them from our UI in the meantime
rachelxelizabethh
2022-10-07 17:07
Yes that ended up working! The key was to exclude the "/" in the first call. Appreciate the help!
rachelxelizabethh
2022-10-07 17:45
One last update to this - I ended up switching approaches and just adding a "$" at the end to make "\/accounts\/[A-Za-z0-9]+$" etc.. For some reason if I don't set an example the Generex treats the $ as its own character and not the end of the string and throws an error like the below:
`Error - value to generate "/accounts/el$" does not match regular expression /\/accounts\/[A-Za-z0-9]+$/`
But, if I pass in an example myself that actually fits the regex (ex. /accounts/123) this approach works great.
noor.hashem
2022-10-11 15:52
Hello,
Does anyone know how I can set the branch name of a consumer driven contract testing provider? On the consumer I just set the branch name in the github workflows when it publishes the pact but on the provider i am not sure where I can pass in the branch name.
yousafn
2022-10-11 16:06
In your providers verification task - which language are you using for verifying?
noor.hashem
2022-10-11 17:35
Hi! Are there some kinds of before and after for states on the provider side? How can we clean up the database in between different states?
matt.fellows
2022-10-11 22:08
this is probably a language specific answer. But the simplest, is to clean the database as needed in the provider state callback itself
noor.hashem
2022-10-12 02:20
Oh yes so there is a provider state callback? Sorry I must have missed this, do you mind sending the docs or PoC that has this step?
matt.fellows
2022-10-12 08:41
example state handlers in NodeJS: https://github.com/pact-foundation/pact-js/blob/master/examples/e2e/test/provider.spec.js#L36-L60
matt.fellows
2022-10-12 08:42
> Oh yes so there is a provider state callback? Sorry I must have missed this, do you mind sending the docs or PoC that has this step?
the language I used there might have been misleading. I?m simply suggesting that the provider state handler be responsible for setting up its state, and not worrying about the state from others.
noor.hashem
2022-10-12 13:30
Oh so I looked at the example you gave and I already have state handles in my code to populate the database, so are you suggesting that in each state handler, before I populate the test db with the specific data I can clear out the database first then populate what I need, correct?
matt.fellows
2022-10-12 21:50
There should be before/after hooks though as Pact JS has that. I?m not really across the NestJS package, but presumably you should be able to access them, if not, it should be easy enough to expose through the API I imagine
aforeman
2022-10-13 01:19
Need some help bringing others along with you on your contract testing journey?
My new blog makes the case for why contract testing should be a vital part of your API testing strategy - not a nice to have - and shows where it fit in. Have a https://pactflow.io/blog/contract-testing-vs-integration-testing/ and let me know what you think :slightly_smiling_face:
cyrus.devnomad
2022-10-13 11:53
Hello
I have a provider with an API method which accepts a POST query with the following request body:
```{
"recipients": [
{
"firstName": "string",
"lastName": "string",
"emailAddress": "string"
}
],
"subject": "string",
"content": "string"
}```
And I have a consumer which sends a request to the provider with following request body:
```{
"Content": "<h1>test</h1>",
"Recipients": [
{
"EmailAddress": "",
"FirstName": "Test",
"LastName": "User"
}
],
"Subject": "Test subject"
}```
I'm trying to have Pact based bi directional contract testing working between these two apps.
When I publish both the provider OpenAPI definition and the consumer pact, Pactflow complains about the consumer pact being incompatible with the provider. The message is like this:
```Incompatibility With Provider Contract
Request Body Is Incompatible
Request body is incompatible with the request body schema in the spec file: should NOT have additional properties
Mismatched at:
[root].interactions[0].request.body```
I manually changed the pact file to have properties starting with lower letters and then published the manually modified pact file and then Pactflow was happy and there were no more errors about incompatibility.
My question is, can this case sensitive behavior be disabled by some configuration option?
Thanks
matt.fellows
2022-10-13 22:20
We don't have an option, no. Case is important, is it normal that your code send requests with the wrong case?
cyrus.devnomad
2022-10-14 11:55
Well the request body is generated from a class with that same structure. And because its a C# class it is upper camel case. On the consumer side the class is converted to Json and the upper camel case name remains like that. But the swagger.json API defintion on the provider side generates lower case properties. I guess thats normal behavior of Swashbuckle package that generates the swagger.json API definition.
Of course its not a big issue, I can declare the properties as lower case like this:
```[JsonProperty(PropertyName = "subject")]
public string Subject { get; set; }```
But because the communication is not affected by these case differences, the real communication between the provider and consumer works despite different case types, I wondered whether it would be possible to make Pact a bit less restrict.
Maybe Swashbuckle could be configured to stop changing the case type of properties involved.
nfinley
2022-10-14 21:18
Hello,
I am using Pactflow and specifically Bi-directional testing and I am running into an interesting error during contract comparison. The error I am running into is:
```Validator Error
can't resolve reference #components/schemas/class from id #```
Here is the api spec:
```openapi: 3.0.0
info:
title: "API"
version: "1.0.0"
components:
schemas:
class:
type: object
properties:
attempted:
type: boolean
paths:
/api/v1/instructor/me:
get:
tags:
- Instructor
description: This will fetch a users classes.
responses:
200:
description: You have successfully found all classes and their status for the current user
content:
application/json:
schema:
type: object
properties:
Classes:
type: array
items:
$ref: '#components/schemas/class'
Elem:
type: array
items:
$ref: '#components/schemas/class'
Trainings:
type: array
items:
$ref: '#components/schemas/class'
Sela:
type: array
items:
$ref: '#components/schemas/class'
SelaFT:
type: array
items:
$ref: '#components/schemas/class'
401:
description: Unauthorized```
And here is the consumer contract:
```{
"consumer": {
"name": "Consumer"
},
"provider": {
"name": "Provider"
},
"interactions": [
{
"description": "A request for a single user classes",
"providerState": "User exists with no classes",
"request": {
"method": "GET",
"path": "/api/v1/instructor/me",
"headers": {
"Authorization": "Bearer undefined",
"Accept": "application/json, text/plain, */*",
"Cache-Control": "no-cache"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json; charset=utf-8 "
},
"body": {
"Classes": [
],
"Elementary": [
],
"Sela": [
],
"SelaFT": [
],
"Trainings": [
]
}
}
}
],
"metadata": {
"pactSpecification": {
"version": "2.0.0"
}
}
}```
Hoping to get some guidance as to why I might be receiving this error. I have validated the Spec. One thing I was curious about is I am using the the pact specification of `2.0.0` on the consumer side. Should I be using `3.0.0`?
Thank you!!!
yousafn
2022-10-14 21:29
Hey @nfinley
Top marks for providing your spec + pact!
Your OAS is incorrect, just popped it into https://editor.swagger.io/
Your refs should be
`$ref: '#/components/schemas/class'`
The pact will return an error, as it specifies the field `Elementary` but your spec states `Elem`
Best of luck in your bi-directional journey buddy
yousafn
2022-10-14 21:30
reference docs on ref usage :slightly_smiling_face: https://swagger.io/docs/specification/using-ref/
nfinley
2022-10-14 21:34
Its always the smallest errors that create the biggest thorns :slightly_smiling_face:
yousafn
2022-10-14 21:45
`EmailAddress` != `emailAddress` though?
What would your provider do if it got a POST request containing
``` {
"firstName": "string",
"lastName": "string",
"emailAddress": "string",
"EmailAddress": "anotherstring",
"eMailAddreSS": "someotherstring"
}```
or
```
{
"firstName": "string",
"lastName": "string",
"emailAddress": "string",
"eMailAddreSS": "someotherstring"
}```
Your OAS that is generated should match that of your application, if it isn't you should configure an option in it, or programatically change it after, there are plenty of tools designed to manipulate an oas spec, otherwise you could easily write a tool to do so. Otherwise your clients e
This is a topic on the swashbuckle repo on the subject
https://github.com/domaindrivendev/Swashbuckle.AspNetCore/issues/383
You shouldn't be changing pact files manually, as they match your clients issued request, and expected responses which are important. Therefore it is equally important that your provider spec, matches it's implementation
yousafn
2022-10-14 21:47
I wrote a blog post about not trusting an OpenAPI spec until you've ran it through it an validator, the online swagger editor is a nice quick go to :slightly_smiling_face: glad you are sorted for now
noor.hashem
2022-10-18 21:40
Hi all, QQ, if a consumer uses several providers and then changes their expectations for each one of those providers, in the consumer driven contract testing would that mean that now in the consumer PR we need to checkout each one of those providers and run the verification?
I have a flow where when the consumer makes a changes the provider is checkout out in github workflows and the tests are run, then if they pass they can merge to master.
matt.fellows
2022-10-19 04:22
> would that mean that now in the consumer PR we need to checkout each one of those providers and run the verification?
you normally wouldn?t checkout the provider in the consumer?s build, no (unless you?re doing the webhookless flow).
You would normally have the webhook setup to trigger a verification of the pact
matt.fellows
2022-10-19 04:23
You would use the `contract_requiring_verification_published` event https://docs.pact.io/pact_broker/webhooks#using-webhooks-with-the-contract_requiring_verification_published-event
noor.hashem
2022-10-19 12:48
I was using the webhookless approach and did show it to someone from pact a while back and they mentioned it was fine. But now i'm thinking is the webhooks approach better. And in what way??
noor.hashem
2022-10-19 13:05
I originally saw two approaches one with webhooks and one without. I planned on trying out both, I arbitrarily began with the webhookless flow thats
matt.fellows
2022-10-19 13:40
I personally don't like the idea of checking out my provider's code in my consumers build. In some cases / orgs that might be very hard to achieve. If you have transitive dependencies that could get gnarly.
matt.fellows
2022-10-19 13:41
It feels a violation of some sanctified boundary. But I haven't thought too deeply on why I recoil at it by default
noor.hashem
2022-10-19 13:53
Yes I definitely see what you mean. However, I am running into a big problem with this flow. Essentially this is my exact current flow of things:
1. Consumer edits/creates pact spec with expectations in branch, pushed code to PR
2. github workflow first runs the pact file and creates a pact out of it if it, and then publishes the pact to pactflow
3. It then checks out the provider branch and runs the verification there and publishes the results
4. It then runs can-i-deploy
I see two issues with this flow.
1. The first is, say the consumer developer made a mistake and asked for a field that the provider does not provide. But now the verification failed on the PR and the results were published. Now on the provider side the verification fails and the provider repo is getting failures because the consumer made a mistake. How can we avoid this..
2. Creating a new contract would fail due to the provider not having verified it yet, and so how could I merge my PR to master with a failing test on my PR?
cyrus.devnomad
2022-10-19 14:45
Aha, so it should be possible to resolve the issue on the Swashbuckle side. I have to give it a try. Thank you very much for the link you posted.
tjones
2022-10-20 03:37
I would definitely not check out the provider's code in the consumer's build. It's unrelated - you don't want the consumer's build to fail if there is some compilation issue (eg a broken master) in another repo. You also don't want the consumer's build to fail if you're not passing provider verification.
thomas.koppensteiner
2022-10-21 11:01
Hello,
Pactflow shows `head` as branch and not `master` (as I expected).
Is this the expected behaviour or is something wrong in my pushing script?
thomas.koppensteiner
2022-10-21 11:27
OK, then something is wrong with the git command I'm using in script.
Thank you.
fquijada
2022-10-21 14:13
Hello, Could you please show me an example in js to make my first connection with pactflow, to be able to generate the pac and send it to the pacbroker and make the complete flow between the consumer and the provider?
yousafn
2022-10-21 14:25
https://docs.pactflow.io/docs/workshops/ci-cd/
Would recommend our ci/cd workshop :+1: it uses pact-js
fquijada
2022-10-21 14:40
```hello, where can I find the broker username and broker password in pactflow?```
yousafn
2022-10-21 15:05
https://docs.pactflow.io/docs/user-interface/settings/api-tokens
Click on the cog icon the top right hand side
you use a bearer token, PACT_BROKER_TOKEN, rather than a username/password (which are used in the open source broker)
brendan.j.donegan
2022-10-21 15:38
I encountered a number of errors today when trying to use the API in particular, just a heads up
matt.fellows
2022-10-21 21:33
Ok I saw a few statement timeouts, looks like it might be a matrix query. We'll investigate next week
fquijada
2022-10-23 01:06
Hello, how to use in windows pac cli? I download this package but I don't find instructions about how to use it on windows, I want to publish a contract from the consumer to pacflow, thanks for your help
fquijada
2022-10-24 01:51
Yes, thank you very much for your help, unzip and execute the pacbroker command giving it the address of the pact that I wanted to publish in pacflow and it worked, thanks @matt.fellows
noor.hashem
2022-10-24 13:44
Hello, I got the attached error message when I was merging my webhookless consumer into master, i've already merged the provider side with no errors. The step it failed on was publishing the pact step where I run this command: `npx pact-broker publish ./pacts --branch=$GITHUB_HEAD_REF --broker-base-url=$pact_broker --broker-token=$pact_broker_token --consumer-app-version=${version} --tag testing_tag` . The github workflows always passes on the PR, but for some reason it failed when I tried merging it. I have never see this error before on my PR. The only place I see `consumer_version_number` is here, https://docs.pact.io/pact_broker/api/webhooks#triggered-webhooks-for-pact-publication but i'm working with the webhookless approach. I took a look at the example webhookless consumer on github that you guys have but I also don't see a consumer_version_number in the workflows anywhere.
Please let me know if you'd like me to provide any more information.
noor.hashem
2022-10-24 19:05
Hi pactflow team, i'm currently having trouble implementing the webhookless approach to pactflow. In this query i'll try and provide as much information and files as possible. I've attached both the github workflow files as well as the consumer and provider tests. Right now I am facing a few problems. The first is the one I mentioned right in the above post ^. The second is that if the test fails on the consumer PR then it blocks the provider, now I know that this is covered in the docs via pending pacts. I tried using the pending as well as WIP features to solve this but clearly I think something is wrong with my flow. I feel that i'm close but might just be missing a few small pieces here and there. Please let me know if you would like me to provide any more information.
matt.fellows
2022-10-25 01:09
My suggestion is to echo out the commands with dummy values, my guess is that the variables aren?t being properly interpreted
matt.fellows
2022-10-25 01:09
From memory, you need to specify at the step level which env vars pass through to the step (for security reasons I think)
noor.hashem
2022-10-25 01:55
Well it is seeing the same error that the consumer build gets. So for example if the consumer asks for a first name and in their PR the provider runs and fails because the master version of the provider does not have a first name field for that particular api then on the provider side if I run the test on the PR they also fail. So I think im doing something wrong here in the sense that the new or updated consumer expectation in their own PR is stopping the provider.
noor.hashem
2022-10-25 01:55
Oh interesting okay I will try this. It is odd though that I don't get any errors when the github workflow runs on the PR only when I merge to master.
matt.fellows
2022-10-25 05:46
Debugging Github actions can be challenging. There was a project I recall that ran them on your machine locally as Docker images. I had a really hairy problem a few years ago (it didn?t help) but was a cool project
rachelxelizabethh
2022-10-25 15:32
Hi, I also put this in pact-jvm channel but I'm unsure if that's correct or if this would be the right channel ... we are trying to get Pact JVM's matchQuery() to work with PactFlow's Hosted Stubs. Because matching rules are a requirement for us, we are using Pact v2 contracts.
In our efforts to try and get dynamic query parameters working, we are facing the below error:
```Error from Pactflow stub: Pact::Error - Value to generate \"[\"string\"]\" does not match regular expression /[A-Za-z0-9]{1,}/```
For reference, we have tried both of the two ways (one including a custom example, and the other not):
```// Example not included (Pactflow generates randomly)
pactDslRequestWithPath = pactDslRequestWithPath.matchQuery(parameter.getName(), "[A-Za-z0-9]{1,}");
// Example included
pactDslRequestWithPath = pactDslRequestWithPath.matchQuery(parameter.getName(), "[A-Za-z0-9]{1,}", "string");```
Here is the relevant part of the contract that was generated:
```"request": {
"method": "GET",
"path": "<path>",
"query": "response_type=string&client_id=string&redirect_uri=string&state=string&scope=string",
"matchingRules": {
<path>,
"$.query.response_type": {
"match": "regex",
"regex": "[A-Za-z0-9]{1,}"
},
"$.query.client_id": {
"match": "regex",
"regex": "[A-Za-z0-9]{1,}"
},
...
}```
We were facing similar issues with machPath() when we were *not* including the example, but including the custom example worked in that case.
Has anyone got this feature working with v2 contracts? Am I missing something?
yousafn
2022-10-26 01:16
https://docs.pact.io/pact_broker/advanced_topics/pending_pacts
you don't have enablePending set to true, so you are seeing expected behaviour
yousafn
2022-10-26 01:18
``` PactProviderModule.register({
...
publishVerificationResult: true,
enablePending: true,
}) as any as Type<unknown>,```
in `providerTest-guest-users.pact.test.ts`
yousafn
2022-10-26 01:23
you also have no consumer version selectors in this build (which would be triggered when the provider codebase changes?), so I assume it is just grabbing the latest pact on your branch to verify, you should have some
https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors
You would have another build that would be triggered by the URL of a changed pact, normally done by webhooks, but I believe you want to checkout your providers codebase and use a webhookless flow?
If you could draw out a picture of what you are trying to attempt, I'll run through the workflows tomorrow
yousafn
2022-10-26 01:39
line 18 of your consumer action file
``` version: ${{ github.event.pull_request.head.sha }}```
line 41 of your consumer action file
``` run: npx pact-broker publish ./pacts --branch=$GITHUB_HEAD_REF --broker-base-url=$pact_broker --broker-token=$pact_broker_token --consumer-app-version=${version} --tag testing_tag```
your version number is only populated on pull requests but your workflow runs on pushes too, where `github.event.pull_request.head.sha` won't be populated
```on:
push:
branches: [master]
paths:
- 'packages/ab-payments/__tests__/**/*.pact.test.js'
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
paths:
- 'packages/ab-payments/__tests__/**/*.pact.test.js'```
yousafn
2022-10-26 01:40
as before a full reproducible example is the most helpful thing you can do, ideally in github in a public repository
yousafn
2022-10-26 01:46
had a look at the files, and pretty sure I've sussed out your issues
Running the provider verification task in your consumer build
``` Verify:
needs: Test
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
strategy:
fail-fast: true
env:
pact_broker: example
pact_broker_token: exmaple
PACT_BROKER_PUBLISH_VERIFICATION_RESULTS: true
version: ${{ github.sha }}
GITHUB_REF: ${{ github.ref }}
GIT_COMMIT: ${{ github.sha }}
application_name: 'platform_payment_service'
steps:
# checking out the provider so that we can run their pact tests on the consumer side
- uses: actions/checkout@v3
with:
repository: 'applyboard/platform_payment_service'
# some steps in the middle that i removed as they are confidential but essentially just setting up the provider test database and authentication etc.
- name: run pact Test and publish results
run: npm run pact:test```
the following values are from your consumer build and shouldn't be used by your provider, and in-fact aren't picked up by your provider build which uses these two
```const commitHash = execSync('git rev-parse HEAD')
.toString().trim()
const branch_name = execSync('git rev-parse --abbrev-ref HEAD')
.toString().trim()```
the type of provider build you are running, should normally be run in the providers pipeline when the provider build changes, it should have
1. consumer version selectors
2. enablePending true
3. includeWIPPacts
it would then pick up your pacts, and not fail the build
There is an additional type of provider verification that can be run, based on a changed pact URL, this URL, is output when your consumer contract is uploaded, and dependant on the event (pact content changed or contract requiring verification published), you could then verify just that Pact.
this provider verification build would only need the the pact url, over the above options.
drakulavich
2022-10-28 07:48
Hi there!
What happened with official twitter page? https://twitter.com/pactflow
yousafn
2022-10-28 10:51
Hey @drakulavich
SmartBear made the decision to retire our commercial tool social handles for in favour of a single handle http://twitter.com/SmartBear
You can still find our open source handle http://twitter.com/pact_up
Would love to hear the community?s thoughts on this!
drakulavich
2022-10-28 10:59
Hi @yousafn!
Sorry to hear it.
btw, it?s still on the website.
I personally don?t see the logic behind shutting down pactflow handle. SmartBear has many products, but customers might want to follow only specific updates. If they want to notify all, they can easily retweet product updates.
bas
2022-10-28 12:23
Hey all, I love how easy it is to unsubscribe for the Pactflow POST newsletter, but how do I sign up? Seems I didn't receive the latest one sent yesterday on my regular email address, but I saw it appear on another one I used before, and now I want to sign up with my regular email address again. You've hidden that feature pretty well, though!
bas
2022-10-28 13:27
Well, the email address I use now, where I didn't get the newsletter, has an active Pactflow instance assigned to it. The email address where I did receive the newsletter hasn't..
bas
2022-10-28 13:28
And they're two totally different domains and companies that happen to have the same person pushing the buttons
yousafn
2022-10-28 13:29
Yeah that was the thought pattern of a few of the engineers and they provided examples of other tech companies that were similar like AWS,
So would @SmartBearPactflow, @SmartBearBugsnag allow for both association with SmartBear and filtering for users down to the level of granularity that they want, when they need it
kedar.ghate
2022-10-31 14:10
Hi Pactflow team, firewall team in my project looking for IP of https://vfuk-digital.pactflow.io/ (port 443) to establish connectivity from our source systems in private network to pactflow, could you please help to get that?
bas
2022-10-31 15:16
Hey @kedar.ghate, is this what you were looking for? https://docs.pactflow.io/docs/webhooks-help/ (the list of public IPs is at the bottom of the page)
Oh wait, that's outbound.. But that same page says there's no static IP list for inbound traffic, so that's an answer, too.
sashi.kandru
2022-10-31 16:36
This seem to be happening when using the https://github.com/pact-foundation/pact-ruby-standalone and interacting with 1.21.0 version
yousafn
2022-10-31 17:06
that is a very old version from 2017
https://github.com/pact-foundation/pact-ruby-standalone/releases/tag/v1.21.0
is that the correct version, and can you provide reproduction steps, the pacts you are publishing, and command executing
shuying.lin
2022-11-02 03:43
:questionmario: *Help wanted: Pactflow contract test onboarding survey* :questionmario:
Hello @here, This is Shuying from Pactflow team. First of all, I would like to thank you for being with Pactflow :pactflow-platypus-slack:.
My team and I are looking to learn more about how you started your first contract test on Pactflow and what types of onboarding and learning experience you want to see on Pactflow. This will help us make improvements to existing onboarding experience. The survey should only take 5 minutes.
:link: Survey is https://forms.office.com/r/wpAbqXZsb8
You can only take the survey once, but you can edit your responses until the survey is closed on November 11, 2022.
If you have any questions about the survey, please email us: or DM me on Slack.
We appreciate your input! :bow:
thomas.koppensteiner
2022-11-02 08:17
@jwang I can't remember exactly. Just make sure your OAS file is valid. If possible avoid references for parameters and defined them directly for the API endpoint.
kedar.ghate
2022-11-02 10:51
Hi Pactflow team,
we are doing POC for contact based testing where Jenkins pipeline publishes contracts to pact flow (free tier).
Once POC is successful, we are keen to go for enterprise version. In this POC, pre requisite is to establish connectivity from
Our Jenkins server (in private network) to pact flow https://vfuk-digital.pactflow.io/ (port 443). When referred to pact flow documentation
(https://docs.pactflow.io/docs/webhooks-help/https://docs.pactflow.io/docs/webhooks-help/) , it says "The Pact flow application runs behind an AWS Load Balancer,
with a dynamic IP range. We don't currently provide static IPs for ingress whitelisting." The security team is looking for IP address or dynamic IP range of pact flow so they can allow the flow from source to destination. As we don't have the IP or IP range, we are blocked. Looking for your support assistance to overcome the blocker.
matt.fellows
2022-11-02 13:26
Hi Kedar, we don't have a fixed set of IPs or a range. We run on AWS so the range would be quite broad and not very useful.
I believe there is a backlog item but that is unlikely to occur this year.
matt.fellows
2022-11-02 13:26
You could consider our on premises application if security is a big concern
drakulavich
2022-11-03 17:35
Published the demo from MoT Meetup where I?m showing the easy way to practice BDCT: https://www.youtube.com/watch?v=0J4I8xxX3Do
Would appreciate any feedback :pray:
yousafn
2022-11-03 18:38
Dude! You are smashing it :raised_hands: Cannot wait to watch this tomorrow, you'll be accompanying me with my morning covfefe
eddie
2022-11-03 20:55
If I want to be sure which version of a given provider is currently deployed to each environment, what's the easiest way to achieve that with PactFlow?
noor.hashem
2022-11-07 18:37
Hi all, quick question, I see that in the provider repo https://github.com/pactflow/example-provider/blob/master/.github/workflows/contract_requiring_verification_published.yml on line 40 it uses `- run: docker pull pactfoundation/pact-cli:latest` but I have been using a command that looks like this: `npx jest --testMatch '**/*.pact.test.ts'"` does it matter which one I use? So far npx has been fine for me
matt.fellows
2022-11-07 22:47
They do two different things.
The first command is pulling in a docker CLI (to be used later), the second is running JS tests.
I think you mean you?re using `npx` to execute the CLI? If so, that?s totally fine
stefan.tertan
2022-11-08 12:22
Hi, I'm having some issues with bi-directional contract testing. I initially posted that here -> https://pact-foundation.slack.com/archives/C9VBGLUM9/p1667571726865759. The 2 issues I encountered were:
? issues with Authorization header not being present on negative tests (testing for 401 response codes) - *the resolution for that is that Pactflow does not support negative tests so I removed them*
? the other issue is with the response body.. it seems that the consumer pact wraps the response body and adds some additional fields, which cause the diffing to fail
stefan.tertan
2022-11-08 12:24
```"response": {
"body": {
"content": [],
"contentType": "application/json",
"encoded": false
},
"headers": {
"Content-Type": [
"application/json; charset=utf-8"
]
},
"status": 200
},```
noor.hashem
2022-11-08 16:10
Hi, i'm trying to find a way to send the results of the `Contract published that requires verification` webhook to the consumer PR that triggered it. I see on this page http://Contract%20published%20that%20requires%20verification there is a link to the `publishing-pact-verification-results-to-github` but the link does not work. Is there any other resource teaching how to do this?
matt.fellows
2022-11-08 20:40
Thanks for the detailed question! I also pasted in the other channel, but for posterity:
At the moment, we don?t support testing negative scenarios in this way because - as you?ve discovered - we validate the request and response bodies, and the request bodies won?t be valid.
We have a backlog item to consider ways to address this use case - perhaps by ignoring the request bodies for tests that have negative status expectations.
I?d suggest for now not including these in your consumer tests, or at least not having them written / persisted in your pact tests (one idea could be to simply post-process the pact file and remove entries that have negative status codes.
This way, at least you have some confidence your API client can handle the status codes.
noor.hashem
2022-11-08 21:19
Hi, when a consumer publishes a new expectation on a PR and a webhook triggers the provider verification to run a github workflow I am getting this error message `Must provide both provider and pactBrokerUrl if pactUrls not provided.` on that github workflow. But I configured the webhook exactly as shown https://docs.pactflow.io/docs/workshops/ci-cd/set-up-ci/configure-webhook. I have attached the provider verification file that is giving the error.
stefan.tertan
2022-11-09 10:40
@matt.fellows I removed the tests for 401s (no auth header) .... the other interactions still fail.... the issue is due to pact adding additional fields to the response body -> `contentType` & `encoded`... while wrapping the original returned body under `content` ... the OpenAPI spec says the response body is an array, but the response body in the contract is an object (the array data is put under `content`)
stefan.tertan
2022-11-09 10:43
See the 2 messages above (on this thread): what the response looks like in the pact & screenshots with the diffing errors
noor.hashem
2022-11-09 15:00
I'm looking over the article, so is they are using the `contract_published` and the `provider_verification_published` events. I was more looking for the two events in the attached image. My question essentially is, is this article the way for me to communicate the results of of the provider verification workflow that kicks off back to the consumer PR?
tjones
2022-11-09 23:24
``` pactBrokerUrl: ,
pactBrokerToken: ,```
^ This part won't compile, which won't be helping
matt.fellows
2022-11-10 06:53
Hi Stefan, apologies I haven?t had a chance to review this today. I?m off for the weekend so won?t get a view until next week.
I think to understand why the contract is being serialised not inline with your expectations, we?d need to see the test code that generates it also.
If you could share that, it would be great.
stefan.tertan
2022-11-10 10:42
sent you an invite with read access to the repo. the file is here - https://github.com/FundingCircle/stefan-cdct-mvp-frontend-consumer/blob/SRE-1222--pact-setup-30/src/utils/api/profile/profile-api.pact.test.ts
nathan.deamer
2022-11-10 11:34
Hi All,
Looking into webhooks (again).
When I publish a consumer in a branch, my webhook triggers to run the provider. But I seem to be getting 2 webhooks firing.
```[2022-11-09T18:47:44Z] DEBUG: Webhook context {"base_url":"https://gocity.pactflow.io","consumer_version_branch":null,"build_url":null,"consumer_version_tags":[],"event_name":"contract_requiring_verification_published","provider_version_number":"0609085aced24bfeaac13a660b9878230d44d197","provider_version_branch":"main","provider_version_descriptions":["latest from main branch"]}```
```[2022-11-09T18:47:47Z] DEBUG: Webhook context {"base_url":"https://gocity.pactflow.io","consumer_version_branch":null,"build_url":null,"consumer_version_tags":[],"event_name":"contract_requiring_verification_published","provider_version_number":"562fd7aaa5c45882a8121c9968a32de10c02f983","provider_version_branch":"main","provider_version_descriptions":["deployed in dev","deployed in ext","deployed in production"]}```
The different seems to be the `provider_version_descriptions` -is there any documentation on this?
noor.hashem
2022-11-10 13:54
Oh im so sorry if I was not clear, yes i know I want that one now, but I just was wondering if there were any specific instructions to how to set that one up for someone that has not dealt with wehbooks before.
noor.hashem
2022-11-14 04:59
Hi all does anyone have any experience on how to set this webhook up? https://docs.pact.io/pact_broker/webhooks/template_library#github---publish-commit-status
I tried setting it up in pactflow but am not entirely sure how. What I want is that when the provider verification github workflow that is triggered by the consumer PR either fails or passes, it sends some response to the consumer PR saying that the verification step either passed or failed but having a hard time setting it up.
noor.hashem
2022-11-14 05:00
Could it be that one is for the provider in the main branch and the other for the provider that is deployed to an environment such as production?
On the pact docs it states that the `contract_requiring_verification_published`
""""
triggers once for each of the following provider versions that are missing a verification result for the newly published pact:
? the latest version from the provider's main branch
? any version currently deployed to an environment
"""
bethskurrie
2022-11-15 04:39
I think it?s using a git command to determine the SHA that is returning `HEAD`
bethskurrie
2022-11-15 04:42
```0609085aced24bfeaac13a660b9878230d44d197```
is latest from main branch
bethskurrie
2022-11-15 04:43
```562fd7aaa5c45882a8121c9968a32de10c02f983```
is deployed in dev?,?deployed in ext?,?deployed in production?
bethskurrie
2022-11-15 04:43
You need a build that checks out the correct git SHA now, to run the pact tests.
bethskurrie
2022-11-15 04:45
Docs https://github.com/pact-foundation/pact_broker/blob/master/lib/pact_broker/doc/views/webhooks.markdown#dynamic-variable-substitution
? `${pactbroker.providerVersionDescriptions}`: The descriptions of the provider version(s) for which the contract_requiring_verification_published webhook has been triggered. Only populated for the contract_requiring_verification_published event.
noor.hashem
2022-11-15 04:53
Well i've followed the docs https://docs.pactflow.io/docs/workshops/ci-cd/set-up-ci/configure-webhook on how to set up the `Contract published that requires verification` webhook and that works great. But for the other webhooks i'm unclear on a few things:
1. Which webhook is the correct one for me to use if I want some kind of status on the consumer PR after the webhook triggers a provider verification github workflow. I was thinking to use the `Verification results published (with any status)` but i'm not sure if this is the correct one for what I want because I also see others like this one https://docs.pact.io/pact_broker/webhooks/template_library#github---publish-commit-status which sound like they do what i want "Report the pact verification status back to the consumer project in Github."
2. After finding the correct webhook, some of them as for a username and password, i'm not too sure what that username and password should be.
I think my first step is to find the correct webhook for the job. Does there exist a webhook that actually sends back some kind of status on the consumer PR after the verification step is finished?
dpal
2022-11-15 10:34
Hi All, I'm trying to publish a provider using `pactfoundation/pact-cli:latest` docker image. There I can get only `pact-provider-verifier` . But I want to publish the provider contract similarly how it get publish by `pactflow publish-provider-contract` by using the standalone pact cli. Anyone please help me. Thanks
aaron.m.williams24
2022-11-15 11:27
I normally use a flag when the contract is being verified? is this what you mean?
`publishVerificationResult: true`
matt.fellows
2022-11-15 11:28
the latest cli should have the `pactflow` command. What version of the CLI are you on?
matt.fellows
2022-11-15 11:29
> I normally use a flag when the contract is being verified? is this what you mean?
I think @dpal is asking about this command
matt.fellows
2022-11-15 11:29
```? docker run --rm pactfoundation/pact-cli:latest help pactflow <aws:pactflow-prod-admin>
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Commands:
pactflow help [COMMAND] ...
pactflow publish-provider-contract CONTRACT_FILE ... --provider=PROVIDER -a...```
dpal
2022-11-15 11:39
`docker run --rm pactfoundation/pact-cli:latest version`
`WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested`
`0.50.0`
dpal
2022-11-15 11:40
yes I'm looking for `pactflow` command. Which it seems not available on this docker image.
dpal
2022-11-15 11:51
I'm getting `ERROR: "pactflow publish_provider_contract" was called with arguments ["provider openapi specification file", "--tag-with-git-branch"]`
dpal
2022-11-15 11:52
even though I did not tag the branch I need to publish the openapi specification file. How can I do that? What is the parameter for that?
yousafn
2022-11-15 13:32
```docker run --rm pactfoundation/pact-cli:latest pactflow publish-provider-contract```
plus your opts
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow
can you provide your command you are using
elluru.kri.mohan
2022-11-15 18:18
HI All, I am trying to setup bi-directional testing using my existing pact-server without using pactflow. is it feasible? This is for an in-house experiment. I am publishing different fragments from the producer and consumers also producing their subset to the pact broker. And I need a way of identifying all the consumer published pacts from provider., which is what I believe PACTflow with bi-directional contract testing is doing.. Sorry for any incorrect understanding..
matt.fellows
2022-11-15 21:17
The open source pact broker doesn't support the bidirectional flow. Can you elaborate a bit more in case I'm misunderstanding?
bethskurrie
2022-11-15 21:57
You?ll want to trigger it for contract published and verification results published, because you won?t always get verification results published if the pact hasn?t changed.
bethskurrie
2022-11-15 21:58
Unfortunately, you?ll need to use a personal access token to authenticate with the Github API, because we haven?t added first class support for integrating with Github yet.
matt.fellows
2022-11-16 01:34
> I?m getting `ERROR: "pactflow publish_provider_contract" was called with arguments ["provider openapi specification file", "--tag-with-git-branch"]`
That says the arguments you are giving it are wrong
matt.fellows
2022-11-16 01:35
```docker run --rm pactfoundation/pact-cli:latest pactflow help publish-provider-contract```
noor.hashem
2022-11-16 04:57
Okay so i'd use the two checked in the image attached? And will those send the result back to the consumer PR?
bethskurrie
2022-11-16 04:59
from memory, there are some edge cases around the names of the github status check
bethskurrie
2022-11-16 04:59
I remember there being some issues with it, but I haven?t re-tested it since adding support for branches.
noor.hashem
2022-11-16 05:07
Okay, so i'm currently trying to use this here: https://docs.pact.io/pact_broker/webhooks/template_library#github---publish-commit-status
And I have configured it with the image below. But getting the error in the file attached. So I think there might be two issues. The first is that the url they are asking for is this:
`"https://api.github.com/repos/<organization>/<project>/statuses/${pactbroker.consumerVersionNumber}"`
I am not too sure what `<project>` would be in my case, from researching I just put the repo name.
But the main issue is this error:
```"message":"No commit found for SHA: 8700f0d",```
I think that might be because the commit is on the consumer PR not on master of the consumer repo so it cannot locate it?
bethskurrie
2022-11-16 05:10
Can you see if the SHA exists in github by going to
https://github.com/<ORG>/<repo>/commit/8700f0d
noor.hashem
2022-11-16 05:13
So the page does not exist but I think it is because the url needed is: `https://github.com/ApplyBoard/platform_monorepo_ui/pull/4500/commits/8700f0d74b44d2af3d7c6d27e2f0e5dbec74b598`
noor.hashem
2022-11-16 05:15
That link ^ does work, it does not take me to the actual commit itself but takes me to a page attached. This is a PR from a regular repo at our company so no not a fork
noor.hashem
2022-11-16 05:18
Oh no, the branch has a name of `testing_pactflow_webhooks_flow` I am going to try and use the full git hash just in case that is causing any issue
bethskurrie
2022-11-16 05:19
noor.hashem
2022-11-16 05:20
Yes so I got it, my url had `/commits` instead of `/commit` so now i get back the actual commit
bethskurrie
2022-11-16 05:21
bethskurrie
2022-11-16 05:22
So, if I use the short URL I get https://github.com/pact-foundation/pact_broker/commit/6169441
noor.hashem
2022-11-16 05:22
Yep exactly, so this link here gives me the attached image: https://github.com/ApplyBoard/platform_monorepo_ui/commit/8700f0d74b44d2af3d7c6d27e2f0e5dbec74b598
noor.hashem
2022-11-16 05:26
Im testing it out now and can let you know if it works in 3 mins :slightly_smiling_face:
noor.hashem
2022-11-16 05:40
So looks like it is working, going to test it a bit more but I can see this message on the PR. this is just one thing. So if I change the consumer expectation it makes a new contract and the can-i-deploy rightfully fails because there is no verification for that version of the pact until the webhook triggers the provider verification github workflow, then we get back the green status show in the image attached on the PR but the can-i-deploy still shows a failing sign on the PR so kind of contradicting.
noor.hashem
2022-11-16 05:40
I know that might be a little confusing ^ so let me know if I can clarify :slightly_smiling_face:
bethskurrie
2022-11-16 05:41
I think you?re going to want to put the provider branch name in the context
bethskurrie
2022-11-16 05:43
I?m not at all 100% that you?ll get the expected results unfortunately, because it really needs a proper github aware integration, rather than a generic webhook. but see how you go.
noor.hashem
2022-11-16 05:50
Oh okay I will test it out tomorrow as it is late in my timezone! Will update the threads here so that others can know how to do it as well.
elluru.kri.mohan
2022-11-16 16:43
we do have one pact broker internally, how to confirm if it supports or not? is there a version or details I can check on my server, if it supports bi-directional? We don?t have any plan of using pactflow saas service for bi-di CT.
yousafn
2022-11-16 16:47
The OSS broker doesn't support it as a feature, as Matt mentioned
You can use the open source tool https://github.com/pactflow/swagger-mock-validator to validate your specs against your pact files, and publish the verification results manually back to your broker, similar to this example
https://github.com/pactflow/pactflow-jsonschema-example#provider-test
But for support as a first class feature, it is unlikely to make it into the OS broker.
ryanthomas840310
2022-11-17 23:32
@elluru.kri.mohan
Another alternative for implementing the schema based provider verification yourself would be to leverage the new plugin interface.
https://github.com/pact-foundation/pact-plugins/blob/main/docs/writing-plugin-guide.md#plugin-interface
ryanthomas840310
2022-11-17 23:41
Hi pactflow channel.
With bi-directional contract testing, is it possible to define the request path for the consumer contract in a case insensitive way OR to configure the pactflow broker to use a case insensitive compare between the consumer contract path and the open api schema path?
Our providers are primarily .net apps which use case insensitive routing out of the box and our open api schema specs are generated from the code using swashbuckle which outputs the swagger.json spec using pascal cased paths for the operations.
This is causing pact verification failures due to case differences in the paths that don't matter for our apis.
yousafn
2022-11-17 23:46
Hey,
you can change it in Swashbuckle
https://pact-foundation.slack.com/archives/CLS16AVEE/p1665783956109099?thread_ts=1665662018.461889&cid=CLS16AVEE
ryanthomas840310
2022-11-17 23:57
I'm specifically asking about the request paths only, not the request body.
The below paths are all valid to our provider.
```GET https://something.contoso.com/api/v1/something/{somethingId}
GET https://something.contoso.com/api/v1/Something/{somethingId}
GET
https://something.contoso.com/api/v1/SOMETHING/{somethingId}
GET
https://something.contoso.com/api/v1/SomeThing/{somethingId}```
Specifying that the operation paths are case insensitive is not something that can be done at the open api schema level.
Both of these are valid and will work for our api.
```// Consumer A test case
pactProvider
.given('some case')
.uponReceiving('request description')
.withRequest({
method: 'GET',
path: `/something/1`,
headers: { ... expected request headers ... }
})
// Consumer B test case
pactProvider
.given('some case')
.uponReceiving('request description')
.withRequest({
method: 'GET',
path: `/Something/1`,
headers: { ... expected request headers ... }
})```
ryanthomas840310
2022-11-17 23:58
But the pactflow comparison of the pact .json and the open api schema currently says they're incompatible when the open api schema case and the pact .json case don't match
matt.fellows
2022-11-18 03:33
Unfortunately OAS is case sensitive and we interpret the OAS with that same case sensitivity
matt.fellows
2022-11-18 03:35
One approach if you can?t get consumers to be consistent, could be to post-process any pact files to downcase the paths (or the reverse, as the case may be).
matt.fellows
2022-11-18 03:35
If they are consistent, the simplest thing to do is update the OAS to be consistent with the consumers.
ryanthomas840310
2022-11-18 15:29
Thanks Matt.
So far my POC test cases have found consumers which are not consistent. Not sure how prevalent those inconsistencies are.
For context:
I've doing the POC work in order to build a detailed picture of what would be required to adopt Pactflow bi-directional testing. The POC details are so I can attempt to sell the business and fellow engineers on adoption.
I could recommend we change the consumers to require consistency. But there's a bit of drawback to that recommendation; more required changes needed for the tools to work becomes a harder sell for internal buy-in.
I'm trying to minimize any such required changes, even just small stuff like case changes in paths.
ryanthomas840310
2022-11-18 15:30
I appreciate the suggestion to post process the files.
I could probably provide a cross-cutting script for this which each team could utilize to bypass the need for more changes.
ryanthomas840310
2022-11-18 16:33
Would SmartBear/pactflow team consider adding a feature to allow opt-in behavior for pactflow users to make just the paths comparison case-insensitive?
I understand the need to be selective in regards to feature support. I would make the case that this is a fairly standard use case that could warrant opt-in support.
Many major APIs follow the practice of url paths being canonically lower-case and case insensitive (e.g. StackOverflow apis).
And as I mentioned this is the default behavior provided out-of-the-box for .NET apis.
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/routing?view=aspnetcore-7.0#route-templates
wng
2022-11-18 17:32
Does the email have filters that prevent messages from being sent? I've been trying to submit a ticket for an issue we're seeing in our pactflow instance, but my email keeps getting bounced as spam
tony.nguyen
2022-11-18 19:58
Has anyone seen this issue with Pactflow Login when trying to sign in with SAML? `Error logging in. invalid_ticket. Error reference: tGSLqEmNbU`
yousafn
2022-11-18 21:13
Hey Wayne,
Sorry about this, I've just tried and am having the same issue. I've raised it with the team and hope to have it sorted for you soon.
Thanks for letting us know!
yousafn
2022-11-18 21:15
Hey Tony!
I haven't, I would say drop the team an email via but our support email is currently down, should be back up shortly and I will let you know!
I assume this is an on prem instance?
yousafn
2022-11-18 21:17
Thought so, as it was SAML, is this preventing you from logging in fully, or just an intermittent issue?
Did you set the SAML integration up yourself or was it an administrator?
tony.nguyen
2022-11-18 21:19
This is happening to our system account. Normal users are able to access via SAML. I have verified with my SAML team if all the attributed needed (name, email etc) exist for this system account. Pactflow logs havent been very helpful with the root cause of the issue.
noor.hashem
2022-11-18 21:44
Hey all, does anyone know how to add multiple pact tests to a consumer as well a provider? Meaning, if a provider services several different consumers how would that look like? And if a consumer has several different providers how would that look like?
hazem
2022-11-18 21:58
Generally speaking, a consumer can initiate a pact with any provider. The provider can decide or not decide to verify. So if consumer A and consumer B use provider Z api for GetUser, then great, now provider Z knows how its api is being used by consumers. And it might possibly be the same exact usage, but provider Z is at least aware.
When defining a pact test as a consumer, one of the arguments is the ProviderName. So consumer A can have a pact with provider Z and provider Y. If using a broker, these pacts are published and discoverable to the provider. They are also pending until they are ultimately implemented and verified by the provider
Just in case you are new to pact, I would suggest going thru some tutorials if you have not yet. There is lots of great stuff documented that helped me. I found the pact-go tutorial super easy to follow.
wng
2022-11-18 23:12
Has anyone run into an issue with Pactflow where in Pactflow, the contract compatibility returns :white_check_mark: , but can-i-deploy fails? I get the generic failure message, but on verbose, it says verificationStatus failed. Clicking the link to the failure though brings me to the Pactflow verification results, that say it passed. The matrix view also shows no failures. Strangely though, no contract changes have been made either. Just randomly blocking our pipeline starting today
hazem
2022-11-19 00:23
Just some thoughts, maybe the consumer selectors is a good start if this is a provider. Are you doing can-i-deploy with tags or environment? The hal browser might also help determine what fails the check.
matt.fellows
2022-11-19 07:17
Exactly!
So basically on the consumer side it's just another set of tests for the consumer to write, but in the setup you specify a different provider.
On the provider side, it's just a matter of selecting pacts from the broker. The CI/CD workshop should give advice on this. Usually you would discover all pacts for consumers deployed to production, and the latest version associated with their main branches.
This way new consumers can come on board without needing to change config
noor.hashem
2022-11-19 20:47
Yes so I do understand all of that. But what i'm unclear about is if I have a consumer that uses 10 different endpoints from the same provider, how would we have something like that work? Would there be multiple pact files on the provider side to separate each endpoint being tested?
I guess i'm just a little unclear on the actual folder and file structure of what happens when we are testing a microservice with many api endpoints and there are several different consumers that each use several of those apis. On the consumer side I think i'm clear that they simply make the pact test and set the provider name, but on the provider side im not really sure how that looks like.
I assumed that a provider would be the entire microservice.
hazem
2022-11-19 22:51
I?m guessing if you are interested in organizing pact files for a provider, then you?re not using a broker such as pactflow. I haven?t done that since the tutorial, but if I recall the generated file is [consumer]-[provider].json. A flat folder structure seems sufficient, and all pacts for the consumer?s integration with the provider end up in the same file. Keep it simple.
noor.hashem
2022-11-20 02:42
Oh no I am using pactflow. I just mean, right now I have one contract with 2 interactions. That one contract is between one consumer and one provider and for one api endpoint. If I wanted to add another api endpoint to test, from my understanding I would need to create another consumer pact test (which make sense to me) but then what do I do on the provider side? Make another pact file that verifies that endpoint? I can send actual files etc. soon
matt.fellows
2022-11-20 03:01
The consumer makes the pact file. I think you mean on the provider side do you need a separate test file?
No, it should just be a single verification test suite that collects all pacts for all consumers
noor.hashem
2022-11-20 16:15
Ah okay but if I have a microservice set as a provider, wouldn't that one file be extremely long to be able to verify all kinds of different apis with different states for each and every api? Maybe i'm overthinking it..
eddie
2022-11-20 20:53
Hey,
Is there any way using the https://docs.pact.io/pact_broker/client_cli/readmeto exchange the long-lived bearer token for a short-lived access token?
matt.fellows
2022-11-21 06:31
We don?t support short lived tokens at this time. Is there a particular workflow you?re aiming for here?
Whilst it?s not what you?re asking, System Administrators can set API tokens to expire via the Preferences page.
matt.fellows
2022-11-21 07:00
> Ah okay but if I have a microservice set as a provider, wouldn?t that one file be extremely long to be able to verify all kinds of different apis with different states for each and every api? Maybe i?m overthinking it..
You might be! I?d say start simple and see how far you go. If it gets unweildly, you can always split the states out into other files to help manage it.
matt.fellows
2022-11-21 07:01
You can definitely create a separate verification suite per consumer if you want, but I?d advise against it.
matt.fellows
2022-11-21 07:04
It certainly seems more common in the .NET ecosystem (where file systems are case insensitive by default), and not the case in linux based systems, which are correct :laughing:
matt.fellows
2022-11-21 07:06
It wouldn?t be hard to support at the comparison level, but exposing that toggle to users is currently not something we can easily do - so likely it would be an all or nothing change, and the consequences of such a change would require consideration
ryanthomas840310
2022-11-21 14:12
> where file systems are case insensitive by default)
Don't know what you mean, .NET is cross platform.:stuck_out_tongue_closed_eyes:
> but exposing that toggle to users is currently not something we can easily do
Just out of curiosity, what's the limiting factor? Lack of support for account level configuration at the pactflow implementation of the broker?
wng
2022-11-21 15:10
```Computer says no ¯_(?)_/¯
CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
------------|------------|--------------------|------------|----------|--------
api-gateway | cf9445f... | payment-processing | 3fe42ae... | true | 1
api-gateway | cf9445f... | rates-engine | 3ab4234... | false | 2
VERIFICATION RESULTS
--------------------
1. https://uship.pactflow.io/contracts/bi-directional/provider/payment-processing/version/3fe42ae73ad7ee326ea772f6c3d31b06609ea6ed/consumer/api-gateway/version/cf9445f6c6fd57e7735ee2c45443633ceb92ce49/cross-contract-verification-results (success)
2. https://uship.pactflow.io/contracts/bi-directional/provider/rates-engine/version/3ab42341e5be6236f6f7f8938ebd433d283f058e/consumer/api-gateway/version/cf9445f6c6fd57e7735ee2c45443633ceb92ce49/cross-contract-verification-results (failure)
The cross contract verification between the pact for the latest version of api-gateway (cf9445f6c6fd57e7735ee2c45443633ceb92ce49) and the oas for the version of rates-engine currently in dev (3ab42341e5be6236f6f7f8938ebd433d283f058e) failed```
Using HAL browser, the failure link returns a 404 for BOTH ironically
The other weird thing I'm seeing is if I purge the contracts and reupload them via our pipelines, the reverse order happens. The payment processing contract fails but the rates-engine contract passes
wng
2022-11-21 15:17
```I also see:
1. https://uship.pactflow.io/contracts/bi-directional/provider/rates-engine/version/3ab42341e5be6236f6f7f8938ebd433d283f058e/consumer/api-gateway/version/79d67e9839b7cbec13d5a94de788937ad9ab10f1/cross-contract-verification-results (success)
All required verification results are published and successful```
but clicking on that link returns this:
ryanthomas840310
2022-11-21 22:17
> currently not something we can easily do - so likely it would be an all or nothing change, and the consequences of such a change would require consideration
Definitely would NOT make sense as an all or nothing change.
Treating an api that does use case sensitive request paths as though it were case insensitive would definitely be the worse of the two options.
If opt in isn't possible at this time then it only really makes sense as a feature request for some (hopeful) future point when opt-in is possible.
wng
2022-11-21 23:00
```CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
------------|------------|--------------------|------------|----------|--------
api-gateway | fca0c9f... | rates-engine | 28be355... | true | 1
api-gateway | fca0c9f... | payment-processing | 838e679... | false | 2
VERIFICATION RESULTS
--------------------
1. https://uship.pactflow.io/contracts/bi-directional/provider/rates-engine/version/28be355f626442bc5740048ed09a650c0ffe66bd/consumer/api-gateway/version/fca0c9f34255d12343c6a996c24523b1f9846e6d/cross-contract-verification-results (success)
2. https://uship.pactflow.io/contracts/bi-directional/provider/payment-processing/version/838e679b59b1e183a9c7dc654bc041c829a2a37b/consumer/api-gateway/version/fca0c9f34255d12343c6a996c24523b1f9846e6d/cross-contract-verification-results (failure)```
It just seems like now it's whichever the 2nd pact test is in the list fails. This one also returns a failure after we deleted everything and reset the contracts but following the link shows success
bethskurrie
2022-11-21 23:29
?invalid_ticket? is just the generic error, but there should be more detail in the logs. Try turning on debug logs too.
matt.fellows
2022-11-21 23:42
Hi Wayne.
We saw the bounced support ticket over the weekend. We?ll find a way to transfer this to ensure it doesn?t get lost. Please expect a contact from us shortly
ilia
2022-11-22 00:44
Hi Wayne, I am the Engineering Manager from Pactflow, I can see the issue you found has come thru our support system to our team, we will start looking into it, and advise soon (this conversation has all been recorded in our ticketing system).
Just want to apologies for the delay, and the bounce backs you have received in the last few days
jpullifrone
2022-11-22 17:29
I'm having the same issue as well. Using Can I Deploy in the Pactflow UI shows verification failed, but if you click on it, it takes you to a successfully verified pact.
abdel.akkoub
2022-11-22 19:48
Hello, we're currently attempting to validate a bi-directional contract using can-I-deploy. The problem we're running into is a mismatch between what PactFlow is showing, and what the can-I-deploy tool is returning. Using the same versions and tags on both pacticipants, PactFlow shows the consumer and provider OAS contracts are compatible with one another. However, when running the can-I-deploy, which is part of the CICD process when merging consumer from feature branch to main branch, can-I-deploy returns false. Even when we follow the link that shows the failed interaction that is displayed in the response of can-I-deploy, we see that the contracts pass and are compatible with each other. We attempted deleting consumer contract, and restarting the process, but we run into the issue with the can-I-deploy failing which is preventing us from being able to merge to main (without modifying the pipeline). Any advice on where to go from here?
eddie
2022-11-22 19:57
Sorry for the delay in responding :man-bowing:
> Is there a particular workflow you?re aiming for here
Admittedly it's an "us" problem, however due to how our CI/CD pipeline works it's difficult to stop the token being exposed in the logs.
If we could quickly exchange the long-lived token for a short-lived access token, that would (somewhat) mitigate the risk
bethskurrie
2022-11-22 22:30
@abdel.akkoub can you check the results of the provider?s self verification for that contract?
abdel.akkoub
2022-11-22 22:50
It shows success for the self verification results: ClientSpendAuthority self verification logs
```<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="client-spend-authority-service" tests="23" time="8.988">
<testsuite name="Currency / Currency List" id="865f664e-fcd9-46fd-adc2-f6a924445a7d" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="3.760">
<testcase name="[GET]::/currencies - Status code is 2xx" time="3.760" classname="ClientSpendAuthorityService"/>
<testcase name="[GET]::/currencies - Returned value length is 20" time="3.760" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Currency / Currency" id="07845dd0-518a-4ce5-a0e7-6e2143b9f3ea" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.105">
<testcase name="[GET]::/currencies/:currencyCode - Status code is 2xx" time="0.105" classname="ClientSpendAuthorityService"/>
<testcase name="[GET]::/currencies/:currencyCode - Correct response" time="0.105" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / GET User Spend Authority List" id="332a0e38-9ea8-494e-93ee-27726ef653b6" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.353">
<testcase name="[GET]::/spend-authorities/ - Status code is 2xx" time="0.353" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - manager Single" id="0dc36de1-d8eb-479d-971b-e41cbf873e52" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.364">
<testcase name="[POST]::/spend-authorities/add - Status code is 2xx" time="0.364" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - duplicate Spend" id="81ef02a5-8d0b-42e2-93ab-86f0664d712a" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.092">
<testcase name="[POST]::/spend-authorities/add - Status code is 400" time="0.092" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.092" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - manager Multiple" id="9dfdf9f8-def7-45a4-bdcd-290a5e89905c" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.028">
<testcase name="[POST]::/spend-authorities/add - Status code is 2xx" time="0.028" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - financial Single" id="f18c543e-0d86-459c-884c-ebdc6e431910" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.013">
<testcase name="[POST]::/spend-authorities/add - Status code is 2xx" time="0.013" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST Add Spend Authority - Negative amount" id="87b36a3e-9c61-4483-888f-8fd72afcca74" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.018">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.018" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.018" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST Add Spend Authority - Invalid spendtype" id="2691a0d4-c636-4289-bb3a-4eef101d03af" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.007">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.007" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.007" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - Invalid currency code" id="6a9ca6d8-e3dd-42cd-b8ad-553c722db392" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.015">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.015" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.015" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - Amount not numeric" id="9acf0744-e775-440b-9783-4e3fd64b25c3" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.016">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.016" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.016" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - Amount contains decimal" id="5afc1da1-1cd3-4df9-9758-e58ba73981c8" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.006">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.006" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.006" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - missing UserIds" id="0accac12-52c0-456d-9415-7a2a5cffb251" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.008">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.008" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.008" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - empty UserIds" id="46042ea6-c580-4bce-9f53-86cfd2788133" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.010">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.010" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.010" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST ADD Spend Authority - invalid UserId" id="356f037c-e53d-4464-91e3-5ee9562b5f68" timestamp="2022-11-22T22:08:32.821Z" tests="2" failures="0" errors="0" time="0.023">
<testcase name="[POST]::/spend-authorities/add - Response status code is 400" time="0.023" classname="ClientSpendAuthorityService"/>
<testcase name="Test Response Body" time="0.023" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST- Remove Spend Authority-Invalid currency" id="61bb3ff1-7135-4fef-aa08-a34d21f63da9" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.024">
<testcase name="[POST]::/spend-authorities/remove - Response status code is 400" time="0.024" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST- Remove Spend Authority-Invalid UserID" id="a58235dc-8ea3-4e8b-bba9-39edd8208a86" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.026">
<testcase name="[POST]::/spend-authorities/remove - Response status code is 200" time="0.026" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST- Remove Spend Authority-Invalid Spendtype" id="49269c29-ea32-4262-91a2-d179fff2391a" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.007">
<testcase name="[POST]::/spend-authorities/remove - Response status code is 400" time="0.007" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST- Remove Spend Authority- 401 Unauthorized" id="72e65478-440b-4e2b-b513-29c67fb7270f" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.010">
<testcase name="[POST]::/spend-authorities/add - Response status code is 401" time="0.010" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST- Remove Spend Authority - manager Single" id="0dd371ce-ac74-4769-843e-d9ef084145f0" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.311">
<testcase name="[POST]::/spend-authorities/add - Response status code is 200" time="0.311" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST Remove Spend Authority - manager Multiple" id="647c2a69-d414-422a-9809-474ac1b80613" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.012">
<testcase name="[POST]::/spend-authorities/remove - Response status code is 200" time="0.012" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Spend Authority / POST- Remove Spend Authority - financial Single" id="8fd0498f-be0c-45fe-b54f-bc089e154309" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.009">
<testcase name="[POST]::/spend-authorities/add - Response status code is 200" time="0.009" classname="ClientSpendAuthorityService"/>
</testsuite>
<testsuite name="Api Health Check" id="a411b49a-02fb-4d50-ab3c-bed0e12f0fa4" timestamp="2022-11-22T22:08:32.821Z" tests="1" failures="0" errors="0" time="0.011">
<testcase name="Status Code is 200 Ok" time="0.011" classname="ClientSpendAuthorityService"/>
</testsuite>
</testsuites>```
matt.fellows
2022-11-22 23:03
Hi @abdel.akkoub thanks for that. Could you please raise a support ticket via so we can keep track of this request?
matt.fellows
2022-11-22 23:06
hmm one option I can think of:
1. Create a new role and system account for rotating API tokens
2. Create a small service (e.g. a lambda aPI) that can regenerate an API token via the API
3. Your CI/CD job can fetch a dynamic API token from this service, and use during the the job.
It?s a bit complicated, but that could do
matt.fellows
2022-11-22 23:07
Thanks. Please mention that you are discussing the bug with the Pactflow dev team on slack so it?s routed correctly
abdel.akkoub
2022-11-22 23:44
Attempted to send few support emails, but kept getting "_550 5.7.360 Remote server returned message denied by administrative policy -> 550 Administrative prohibition - envelope blocked"._ Is there another email address or another way to create a support ticket?_I_
bethskurrie
2022-11-22 23:45
We?ve just migrated our mail server Abdel, and there have been a few issues. I?ll get someone on to this ASAP.
julian.alvarezv
2022-11-23 10:00
Hi all ..
Thank you very much in advance for the help.
We are having the following situation:
Consumers are correctly publishing contracts to pactflow, since the content of the contract has not changed we expect the contract to be marked as verified by the provider which is already deployed to a given environment.
But, this is not the case, the contract shows as not verified at all. Any ideas?
matt.fellows
2022-11-23 10:25
One common cause is the use of matchers/generators in your code that generate new values for the example (this seems to be common in the Java DSL where you can leave out an example and it generates a random value)
julian.alvarezv
2022-11-23 10:41
Thanks a lot, we will take a look to see if this is the case. Regards.
matt.fellows
2022-11-23 11:00
FYI I?m told the support email came through despite the bounce looking emails
julian.alvarezv
2022-11-23 12:29
like this one?
```.stringMatcher("clientSubType", joinEnumValues(SimplePartnerSearchFilterResponse.ClientSubTypeEnum.values()))```
Thanks
matt.fellows
2022-11-23 12:57
then run it again, and compare the pact - check to see if it?s different
julian.alvarezv
2022-11-23 14:05
That was actually the problem. Solved!
Now, if we use this event trigger:
Contract published that requires verification
We don't understand why the verification is triggered if the content has not changed and the contract shows as verified.
Thanks a lot!
abdel.akkoub
2022-11-23 16:41
Another thing to add. We're trying to ignore the pacticipant that's is causing the failure in Can-I-Deploy, but that also doesn't seem to work. Please see screenshot. Not sure if this is related or needs to be addressed separately. We did post a message in the broker channel, but wanted to share that incase it is related:
bethskurrie
2022-11-24 23:56
@julian.alvarezv it will be triggered if the *provider* version in one of master/any deployed environment has changed
bethskurrie
2022-11-24 23:56
It?s about getting any missing verifications that would stop you being able to deploy the new consumer version.
bethskurrie
2022-11-24 23:58
@abdel.akkoub the ignore for BDC is not yet implemented. We have a card for it, and it will be picked up in our next phase of BDC work.
bethskurrie
2022-11-24 23:59
You can use `PACT_BROKER_CAN_I_DEPLOY_DRY_RUN=true` if you need a ?break glass? way of getting a deployment out.
https://docs.pact.io/pact_broker/client_cli/readme#can-i-deploy
wng
2022-11-28 16:42
Looks like whatever new build was pushed out recently fixed it. Things are back to normal for us again at least
sasankdts
2022-11-29 04:31
Hi I created a project from create sample project from pactflow for bidirectional and getting errors on npm test gem bundler ruby gems loader molinillo etc
matt.fellows
2022-11-29 04:36
Sorry could you please provide more information? Ideally how to reproduce the issue and we get errors you are seeing
sasankdts
2022-11-29 04:39
Created a sample project with bidirectional provider javascript node provider mocha as template
matt.fellows
2022-11-29 04:54
What errors are you seeing?
If you can please keep the conversation contained to this thread, that will really help
matt.fellows
2022-11-29 05:18
Is there a reason why you?re taking screenshots? Is it possible to get the output pasted in code gates?
matt.fellows
2022-11-29 05:19
It looks like you are hitting the windows long path issue: https://github.com/pact-foundation/pact-js/issues/237#issuecomment-433591393
sasankdts
2022-11-29 11:02
Can we write provider tests to generate oas.yaml instead of using dredd postman and restassured and consumer without nock msw wiremock and mounterbank?
sasankdts
2022-11-29 11:03
Thanks in advance from create project i didn't see bidirectional pattern just downloading as normal
noor.hashem
2022-11-29 17:23
Hey Beth! Sorry for such a late reply. Took some time off. So things do seem to be working fine with just the integration that you've sent over, i have added the provider branch name to the context. So far I have not run into any issues but that is mainly because I work on one PR and one branch at a time.
yousafn
2022-11-29 17:49
You can generate OAS from your provider code for use in BDCT verifications
You still need to generate a pact file as an output of your consumer test.
As you whether you use the Pact framework directly (preferred) or adapt your existing mocks, to the pact specification, the choice is yours.
yousafn
2022-11-29 17:52
Out of interest, if you are on a windows machine, what shell are you using? and are you able to run the make commands, or are you trying to run the commands directly?
I know depending on how one develops in windows will determine the approach and we don't cater well in our demos, as we use macs or linux as our main dev boxes. I only pull out my windows laptop occasionally to test things out cross plat if I get time, otherwise I use it in CI, and always revert to using bash in window
bethskurrie
2022-11-29 22:48
Sorry, I just noticed that you said ?This is happening to our system account.? System accounts can?t log in to Pactflow. Only ?regular users? can.
bethskurrie
2022-11-29 22:48
The only way a system account can authenticate to Pactflow is with a bearer token to the API. They can?t use the UI.
sasankdts
2022-11-30 08:26
I am on windows and not able to run Make commands, yes running commands directly
noor.hashem
2022-11-30 14:20
Let me review the docs and see where/if there is room for improvements for this step
tony.nguyen
2022-11-30 14:37
@bethskurrie @sashi.kandru how can we assign a Pactflow bearer token to the system user? Does that not require the account to log into the UI once? Reason why we need this System account to have access to Pactflow is to automate our testing of new Pactflow releases. It is crucial for us to test new Pactflow docker releases before deploying it to our production environment for enterprise users. These automation should not be tied back to a regular user account.
bethskurrie
2022-11-30 22:41
Yes, the system account has a bearer token. It?s designed for use in CI, so you don?t tie your pipeline to a particular user. You log in to Pactflow as a user with the system account manage permission, and grab the token.
julian.alvarezv
2022-12-01 15:57
Hi! I got another doubt .. we have a service which is provider and consumer ... when we request can-i-deploy through the api:
https://pactflow.xxx.io/can-i-deploy?pacticipant=yyyyyy&version=zzzzzzz&environment=dev
The response is true ... all good, we are happy ... but when we do the verification with pact-broker command line tool from Jenkins it says no ...
```pact-broker can-i-deploy --pacticipant=yyyyy--broker-base-url=https://pactflow.xxxx.io --broker-token=**** --version=zzzzzzzz --to-envionrment=dev```
```CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
----------------------|------------|-----------------------|------------|----------|--------
wwwwwwwwwwwwwwwwwwww | e22e444... | yyyyyyyyyyyyyyyyyyyyy | b5b0a6c... | true | 1
yyyyyyyy | b5b0a6c... | kkkkkkkkkkkkkkk | ??? | ??? | ```
Any ideas what could we be missing?
Thanks a lot in advance, much appreciated the help. Regards!
brendan.j.donegan
2022-12-01 16:23
```
[2022-12-01T16:11:52.816Z] Error retrieving matrix. PactBroker::Client::Hal::ErrorResponseReturned - Error making request to https://hmhco.pactflow.io//matrix status=404
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/hal/entity.rb:197:in `assert_success!'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/hal/link.rb:45:in `get!'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/matrix/query.rb:11:in `call'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/base_command.rb:13:in `call'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/can_i_deploy.rb:107:in `block in fetch_matrix'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/retry.rb:22:in `while_error'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/can_i_deploy.rb:107:in `fetch_matrix'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/can_i_deploy.rb:111:in `fetch_matrix_with_retries'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/can_i_deploy.rb:34:in `call'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/can_i_deploy.rb:23:in `call'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/cli/matrix_commands.rb:36:in `can_i_deploy'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/cli/custom_thor.rb:23:in `start'
[2022-12-01T16:11:52.834Z] /pact/lib/pact/cli.rb:46:in `broker'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
[2022-12-01T16:11:52.834Z] /pact/bin/pact:15:in `<top (required)>'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/cli/exec.rb:58:in `load'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/cli/exec.rb:58:in `kernel_load'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/cli/exec.rb:23:in `run'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/cli.rb:483:in `exec'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/cli.rb:31:in `dispatch'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/cli.rb:25:in `start'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/exe/bundle:48:in `block in <top (required)>'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
[2022-12-01T16:11:52.834Z] /usr/lib/ruby/gems/3.0.0/gems/bundler-2.3.13/exe/bundle:36:in `<top (required)>'
[2022-12-01T16:11:52.834Z] /usr/bin/bundle:25:in `load'
[2022-12-01T16:11:52.834Z] /usr/bin/bundle:25:in `<main>'```
bethskurrie
2022-12-01 21:38
```export PACT_BROKER_BASE_URL="https://test.pactflow.io"
export PACT_BROKER_USERNAME="dXfltyFMgNOFZAxr8io9wJ37iUpY42M"
export PACT_BROKER_PASSWORD="O5AIZWxelWbLvqMd8PkAVycBJh2Psyg1"
docker run --rm \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_USERNAME \
-e PACT_BROKER_PASSWORD \
pactfoundation/pact-cli:0.51.0.0 \
pact-broker can-i-deploy \
--pacticipant docker-example-consumer \
--latest```
bethskurrie
2022-12-01 21:38
```export PACT_BROKER_BASE_URL="https://test.pactflow.io/"
export PACT_BROKER_USERNAME="dXfltyFMgNOFZAxr8io9wJ37iUpY42M"
export PACT_BROKER_PASSWORD="O5AIZWxelWbLvqMd8PkAVycBJh2Psyg1"
docker run --rm \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_USERNAME \
-e PACT_BROKER_PASSWORD \
pactfoundation/pact-cli:0.51.0.0 \
pact-broker can-i-deploy \
--pacticipant docker-example-consumer \
--latest```
bethskurrie
2022-12-01 21:42
hi @julian.alvarezv there?s nothing obviously wrong in what you?re doing. You should have the same results. I can?t identify the problem based on the information you?ve given.
You can raise an issue by sending an email to . Please include the full HTTP logs of both scenarios. You can turn on the HTTP logs for the can-i-deploy command by adding `--verbose`. If you are using the onprem product, then you?ll need to also include the server side logs for each of those requests - you?ll be able to identify them using the X-Request-Id response header that you?ll see in the HTTP logs.
matt.fellows
2022-12-01 23:18
Does that mean there was a behaviour change between the two versions that perhaps stripped/canonicalised the URL?
bethskurrie
2022-12-01 23:19
Ah, it was probably this one https://github.com/pact-foundation/pact_broker-client/commit/d21efe61edee43e78b4c1d81f59fba79284a6e59
bethskurrie
2022-12-01 23:20
I?ve made an issue for it https://github.com/pact-foundation/pact_broker-client/issues/121
werner.lauterfeld
2022-12-07 09:59
Hello,
is there an API endpoint available to delete 'the integration'?
I found the piece of documentation where it states how to do it with the GUI and I wondered whether it may also be available using the API.
matt.fellows
2022-12-07 10:15
Yes there is. The same API the UI uses should work. I can dig it up when next at my computer but I think you can navigate the Ahal relations to the "integration" relation, and then just issue a delete to the integration you want to remove
werner.lauterfeld
2022-12-07 10:41
Thanks Matt - for some reason I did not see "integration" in the HAL-Browser. :facepalm-animated:
Thanks a lot - no digging necessary.
Have a great day :)
yousafn
2022-12-07 14:35
Hey you can do something like this from your CLI
`curl -v -X DELETE -H "Authorization: Bearer $PACT_BROKER_TOKEN" "https://saflow.pactflow.io/integrations/provider/http-provider/consumer/http-consumer-1"`
cyrus.devnomad
2022-12-07 17:54
Hello,
I have a swagger.json and a pact file (see attachments). If I publish them to Pactflow, I get the following strange error (see attached screenshot):
Validator Error
Cannot read properties of undefined (reading 'type')
This error does not seem to be one of those "Contract Incompatibility Messages", which are listed here:
https://docs.pactflow.io/docs/user-interface/bi-directional/
Could it be that this might point to an exception thrown inside the Pactflow? Is something wrong with one or both of my files which causes Pactflow to throw an exception or something?
To reproduce the error, just publish both of the files to Pactflow server.
bethskurrie
2022-12-07 20:09
I'd say you are right. Can you run your oas through a validation tool to check its not missing a field?
cyrus.devnomad
2022-12-08 10:07
I just checked it with the http://editor.swagger.io online editor and validator and the syntax of my swagger file has indeed an error (see the screenshot). I have the following section in the file:
"security": [
{
"OAuth2": [
"OAuth2"
]
}
]
But my file misses the "secritySchemes" section that defines the "OAuth2" security scheme.
So, if I add the following section to my file, the file is shown as valid and also Pactflow no longer reports that error "Cannot read properties of undefined (reading 'type')".
Thank you for the hint
matt.fellows
2022-12-08 11:25
Thanks for confirming - we have added an item to the backlog to help differentiate OAS validation errors vs comparison errors
kedar.ghate
2022-12-09 09:46
Hi team,
we (vodafone) have sent email to in October 2022 for security assessment for pact flow and we didn't hear back on it. Could you please share right contact to follow up with it?
matt.fellows
2022-12-09 10:42
Hi Kedar. I can follow up for you. Could you please DM me the name/email of the right business and security contacts?
sasankdts
2022-12-09 12:34
What it means unknown status in pactflow though pact are compatible in bidirectional
sasankdts
2022-12-12 07:28
Github actions and workflow configuration not displaying for my private repositories
matt.fellows
2022-12-12 07:29
So the answer is right there - you need to set a status for the self verification
matt.fellows
2022-12-12 07:31
see these flags:
``` --verification-exit-code=EXIT_CODE \
--verification-results REPORT_PATH \
--verification-results-content-type REPORT_CONTENT_TYPE \
--verifier VERIFIER```
sasankdts
2022-12-12 10:24
Without ci/cd pipelines without make file need to pass these from commandline
matt.fellows
2022-12-12 10:56
Sorry, I?m not sure what you?re asking? These are flags on the CLI command, you can call that command through any means (makefile, bash script, docker command etc.)
sasankdts
2022-12-12 11:27
I am on private repositories and trying to config github actions not displaying workflows as in public repositories
matt.fellows
2022-12-12 20:58
Thanks. How are you calling it from your terminal? It seems you might be missing some arguments, which is why the status is ?unknown?
matt.fellows
2022-12-12 20:59
If you set the status to passing or failing, that would change what you see in Pactflow
matt.fellows
2022-12-12 20:59
Currently, it says ?the documents are compatible, but we don?t know if the provider implements the OAS?
sasankdts
2022-12-13 03:33
I need exactly where our verification tests between consumer contract and provider oas/yaml file happening and arguments that return pass/ fail after verification
matt.fellows
2022-12-13 03:35
The verification happens when you publish the provider contract. The cli I shared above has flags you need to set when you do that.
matt.fellows
2022-12-13 03:38
It also happens when can I deploy is called for comparisons that werent previously pre computed.
The point is, when you publish the pact file it the verification results should happen then. In your case you are generating the OAS and not testing it with a tool (you should probably also do that), so you can just set the result to success and specify an empty file as the results
sasankdts
2022-12-13 04:08
In your example of bidirectional with postman as provider where test is happening with a tool please point me with in your example
sasankdts
2022-12-13 04:10
Also please let us know additional features and support we get from pactflow on licensed veraion
matt.fellows
2022-12-13 04:44
Test step: https://github.com/pactflow/example-bi-directional-provider-postman/blob/master/Makefile#L19
Step where publish happens (note the flags): https://github.com/pactflow/example-bi-directional-provider-postman/blob/master/Makefile#L55
For features/support on plans, you can see a comparison table here: https://pactflow.io/pricing and our features summary here: https://pactflow.io/features.
nerea.tamayo
2022-12-15 09:53
Hey team! Just to confirm.... we have a consumer which is an iOS application where we can't apply bi-dirctional contract testing because none of supported tools are being used, and then we have a provider where we can apply bi-directional feature because Postman and SwaggerHub are used. So is it compatible to apply bi-directional feature in the provider side with a non-bi-directional (iOS app) in the consumer side? thanks in advance
yousafn
2022-12-15 10:36
Hey hey,
For BDCT, you currently need an OpenAPI definition on one side (plus verification results) as your provider contract (more specification def types to be supported in the future)
Consumer side you need a consumer contract, which is a Pact file.
You can generate this with a client library, there is part dart consumer, and pact-swift depending.
Also remember the bi-directional is a trade-off are you are not directly verifying the pact contents against a running provider, as you would do in CDCT, so it is worth considering and trialling out both options.
yousafn
2022-12-15 10:42
We can use the pact provider verifier standalone or in a client library for CDCT, from the provider side for issuing the requests made in the consumer contract to the provider.
You could use this additional layer somewhere to augment BDCT depending on your flow, to give you added security.
Thinking more widely about the pact provider verifiers role, currently it only reads pact files.
If it could read an OAS and issue those requests, it could be the one to verify that the provider contract is valid (as in the implementation matches the definition) rather than currently offsetting that the providers functional testing tool.
That was a conscious design choice as it allows users to present a definition only, without valid test results from a _real_ or even mock implementation, to facilitate the collaborative nature of design first development
yousafn
2022-12-15 10:43
That's been on our mind for a while https://github.com/pactflow/roadmap/issues/31
matt.fellows
2022-12-15 12:20
> Hey team! Just to confirm.... we have a consumer which is an iOS application where we can?t apply bi-dirctional contract testing because none of supported tools are being used,
what do you mean by this Nerea? So long as you can produce a pact file it should be compatible with the backend (with the caveats Yousaf has mentioned)
yousafn
2022-12-15 14:04
Hey Sanank,
You will find better support if you ask a _good_ question. You can find some guidance here.
https://docs.pact.io/help/how_to_ask_for_help#4-ask-a-good-question
matt.fellows
2022-12-16 01:59
Hi Stefan, I have reproduced the problem and have raised an internal ticket to fix (PACT-458).
In the mean time, I think the issue is that the specification serialisation is Pact V4 and there is a bug in how we compare a V4 pact to the OAS. For now, I?d suggest explicitly setting the specification version on the test to V2 or V3.
matt.fellows
2022-12-16 02:13
I?m very sorry for the delay in responding, sometimes issues like this can get lost in slack (especially ones that require more effort to debug).
matt.fellows
2022-12-16 02:14
I should have requested you submit a support ticket to . If the resolution above doesn?t sort you out, please do submit a ticket to the email above so the team will look into it and it won?t get lost in this noise.
stefan.tertan
2022-12-16 11:16
thanks, I will try this workaround and get back to you :+1: (and email support if not resolved)
joao
2022-12-20 11:44
Bi-directional verification
Hi folks,
I am trying to implement a bi-directional verification, but I am cracking my head with an error message
regarding the path.
Apparently the consumer contract and the Swagger spec are specify the same API method: POST /executeFlow/orderlookup/V1
But Pactflow says the paths are incompatible.
Any thoughts?
Note: The Swagger spec provided to me has a mismatch on the response body - this I know. I am confused about the error
on the paths.
joao
2022-12-20 13:27
Hi Yousaf,
Unfortunately the complete files have some information I would prefer to keep private.
I am sharing the top part of each file, where the paths are specified.
Let me know if you would like to see some other part of the files.
yousafn
2022-12-20 13:33
Appreciate that you want to keep the files confidential, but it will be much slower, and difficult to help. I have very limited time today before I go away on holiday for xmas,
If you don't wish to provide via here, feel free to raise an email to
joao
2022-12-20 13:48
Same error when trying to simply appending or joining them without duplication.
I will get back to the team that built the Swagger for clarification, maybe they specified it wrongly (manually generated).
Would you know if Pactflow looks at both basePath and path in the OAS or just the path, when comparing to the path in the consumer contract?
yousafn
2022-12-20 13:50
I don't think it considers the basePath, but would have to test. I would check the validity of the OAS document via something like http://editor.swagger.io
The underlying tooling is here if it makes it easier to reason about during your debugging
https://github.com/pactflow/swagger-mock-validator
joao
2022-12-20 13:57
Awesome, I will take a look and see what the other team says as well. Thanks Yousaf.
matt.fellows
2022-12-20 22:58
> Would you know if Pactflow looks at both basePath and path in the OAS or just the path, when comparing to the path in the consumer contract?
Pactflow doesn?t consider the base path (because each server may have a different base path).
We have a backlog item to consider how we might support it though.
lewiscowles
2022-12-21 15:39
I Should have been less of a whinge on this. I have left more polite suggestions for improvement today. Sorry about that Matt; it was very frustrating.
sasankdts
2022-12-21 16:28
How to create pact files if i have 100+ requests, do I have to create 100 pact files?
matt.fellows
2022-12-21 22:41
so if the 100+ requests are between Application A and B, then you?ll have 1 pact file
sasankdts
2022-12-22 02:07
I tried with 5 requests only last request generating pact file with last interaction
matt.fellows
2022-12-22 02:08
can you please share your code? Perhaps you?re clearing out the pact file after each test?
abubics
2022-12-23 01:32
Maybe this kind of thing? https://www.toolnb.com/tools-lang-en/jsontopostman.html
matt.fellows
2022-12-23 08:15
Yeah, they've been pretending for a while now :rolling_on_the_floor_laughing:
tjones
2022-12-23 10:57
I don?t really have any context here, but I did find this, which is worth being aware of: https://github.com/microsoft/playwright/discussions/11168
sasankdts
2022-12-26 08:04
For mismatch in key/propertyname assetoupid instead of assetcontrolgroupid below error displaying which is confusing
tjones
2022-12-28 23:07
Is there documentation on how verification failure details are sent to the pact broker? I tried the HAL browser on my hosted pactflow instance, but couldn?t find any extra details inside publish verification results
(also, it would be great if all that documentation were available without the HAL browser - it took me about 30 minutes of mucking about to find a resource that had publish verification results as an option)
sasankdts
2022-12-30 01:48
Could you please clarify the error like why it is displaying should not have additional properties instead of mismatch in properties
aparnachaudhari
2022-12-30 20:22
Is there a way to view the contract as JSON with the latest UI changes? i.e Pactflow Version: 1.21.1 I do not see this option any more
vtft1988
2023-01-04 14:15
Hi all, I have a problem with pact *can_i_deploy* feature. When I?m clicking the second link(failure) broker is showing: contacts are compatible? Could you explain the reason please, why verification results are called two times?
tjones
2023-01-05 10:20
What are you trying to do with can-i-deploy? From the warning messages it looks like Pact isn?t sure.
andrew.favaloro
2023-01-05 19:04
I have a question about the Matrix Filters on http://pactflow.io - are the consumer and provider sides flipped? When I search for a consumer branch, I get matching results in the provider side instead of the consumer side
ilia
2023-01-05 22:23
Hey Andrew, thanks for letting us know, it does look like a bug, we will investigate - just want to confirm (before i log this issue) if you haven't report this issue to ?
andrew.favaloro
2023-01-05 22:25
I have not reported the issue to the support email address. Should I?
ilia
2023-01-05 22:27
No, all good, just wanted to ensure there is no duplication :slightly_smiling_face:
yousafn
2023-01-06 13:30
provide the version number of the application you wish to deploy and which environment to which to deploy it too
yousafn
2023-01-06 13:31
also how have you got two entries, on the same version numbers with successful and failed results
yousafn
2023-01-06 13:32
Could you provide more context via the matrix screen to show which versions you have uploaded and with what metadata
vtft1988
2023-01-06 14:21
@yousafn probably i know the reason, this is because Provider self-verification is needed.
yousafn
2023-01-06 14:26
nothing is deployed to any env.
so a consumer wouldn't be able to deploy to env xyz
the provider would be able use can-i-deploy with version `e8884d6c` to a specified environment, which should return a success as the provider is compatible
matt.fellows
2023-01-09 01:00
Thanks all for your help. Vasil - best to also share the flags you are passing to `can-i-deploy` so we can advise in the future.
matt.fellows
2023-01-09 01:04
I don?t recall you being able to see the JSON directly, except in the case where we were unable to properly render it. You can get to the raw JSON quickly via:
1. Copy the URL to current pact
2. Go to the HAL browser (the ?API? link at the top)
3. Pasting the URL to the pact in the HAL address bar and hit ?Go!?
matt.fellows
2023-01-09 01:06
This is just the way the errors are presented. It uses the language of OAS and JSON schema, which is the underlying comparison technology used, and should allow better googling etc.
Additional properties are a really key concept in JSON schema, but also very complicated.
Basically, it means you are requesting _additional properties_ in the pact file - i.e. json fields - that aren?t present in the schema (OAS)
vika.karman
2023-01-09 09:11
Hello. I've got a question regarding can-i-deploy step. I'm using it 2 times in my CI/CD workflows. One - when I verify whether I can merge the code, in this case I verify pact against provider's master branch, the other one - when I verify whether I can deploy the code, in this case I verify pact against provider's env. So my script looks this way:
```#!/usr/bin/env bash
echo "running can-I-deploy with the following params""
PACT_BROKER_BASE_URL: $PACT_BROKER_BASE_URL
PACT_BROKER_TOKEN: $PACT_BROKER_TOKEN
consumer: $consumer
version: $version
provider: $provider
to_environment: $to_environment
"""
if [ -z "$version" ]; then
echo "ERROR: version cannot be empty to run can-i-deploy verification"
exit 1
fi
command=
if [ -z "$to_environment" ]; then
command="--pacticipant $provider --branch master"
else
command="--provider $provider --to-environment $to_environment"
fi
echo "command = $command "
docker run --rm \
-w "${PWD}" \
-v "${PWD}":"${PWD}" \
-e PACT_BROKER_BASE_URL="$PACT_BROKER_BASE_URL" \
-e PACT_BROKER_TOKEN="$PACT_BROKER_TOKEN" \
pactfoundation/pact-cli:0.50.0.32 \
broker can-i-deploy \
--pacticipant "$consumer" \
--version "$version" \
$command```
I've found that if I want to use `--branch master` in my command I need to use `--pacticipant` , if I want to use `--to-environment` I need to use `--provider` otherwise it returns an error...Am I missing something here? If not...why cannot it be the same (provider or pacticipant) in both cases?
matt.fellows
2023-01-09 10:48
hmmm something else must be the matter. All of our examples use the format
```pact-broker can-i-deploy \
--pacticipant ${PACTICIPANT} \
--version ${GIT_COMMIT} \
--to-environment ${ENVIRONMENT}```
vika.karman
2023-01-09 12:00
Right, the first pacticipant is consumer. What if I want to specify the provider name in the command?
klaudijus.rulys
2023-01-09 13:35
Hello :blob-wave:
Is there anyone that could help reset the pact broker user passwords in our hosted broker? I think we're using an old version of the broker, so have not found a way to do it myself :slightly_smiling_face:
ilia
2023-01-10 00:21
@andrew.favaloro would you be able to provide your tenant, or url? we are trying to investigate the issue at the moment and want to reproduce on our side
matt.fellows
2023-01-10 00:31
Why are you specifying two `--pacticipant` values in this case? I think that?s your issue
matt.fellows
2023-01-10 00:31
The simplest and most normal use case, is to deploy one application at a time
matt.fellows
2023-01-10 00:32
If you want to deploy the consumer, use the consumer name in the `--pacticipant` flag. And vice versa
vika.karman
2023-01-10 08:44
Ok my case is here: we have a multi-tenant consumer with single-tenant providers. So when I run can-i-deploy step, I need to verify pact between the consumer and different providers. Some providers are deployed to all envs, some providers are deployed only to some envs. What I want to make sure when I deploy my consumer is that a 'valid' version of the provider is deployed to env1, env2, env3.
I am going to use strategy.matrix for this:
```strategy:
matrix:
provider: [<provider1>]
env: [
"<env1>", "<env2>", "<env3>"
]```
and I expect my command call to looks this way (in a loop for each env):
```broker can-i-deploy \
--pacticipant "$consumer" \
--version "$version" \
--provider "<provider1>"
--to-environment "<env1>"```
vika.karman
2023-01-10 08:46
So I understand the idea when you specify only one pacticipant (the one you are about to deploy) and verify all pacts from pact broker. But what if I want to explicitly verify compatibility of my consumer with `exact` providers
matt.fellows
2023-01-10 10:03
I'm still not getting why you need to specify the providers in the command tho. by asking can I deploy and giving a specific application and environment, it will tell you if there are missing dependencies, or if those dependencies are incompatible.
vika.karman
2023-01-10 11:22
ok lets I have a consumer C and it has 2 providers P1 and P2.
? C is a multi-tenant service and gets deployed to a single env `prod` .
? P1 is a single-tenant service and gets deployed to client1_prod, client2_prod, client3_prod envs.
? P2 is a single-tenant service and gets deployed to client1_prod, client2_prod envs
I will record deployment for
? `P1` envs `client1_prod`, `client2_prod`, `client3_prod`
? `P2` envs `lient1_prod`, `client2_prod`
Now I need to verify if I can deploy C. I need to verify pacts
C - P1 env `client1_prod`
C - P1 env `client2_prod`
C - P1 env `client3_prod`
C - P2 env `client1_prod`
C - P2 env `client2_prod`
If I don't specify the provider, I need to verify <all pacts for consumer C> for envs `client1_prod`, `client2_prod`, `client3_prod`
But verification C - P2 env `client3_prod` will fail, because this service will never be there
jrede
2023-01-11 18:23
Hi Team, Im a QE thats started implementing Pactflow with the https://github.com/pactflow/pact-cypress-adapter However we?re running into issues with oneOf, allOf, ect? Pactflow expects the response to look like this:
```"response": {
"status": 200,
"headers": {
"content-type": "application/json",
"access-control-allow-origin": "http://exampleapi.com"
},
"oneOf": {
"body": {
"id": "string",
"email": "string",
"group": "string",
"isOnboarded": true,
"organizationId": 0,
"createdAt": "string",
"updatedAt": "string",
"contactType": "string",
"devices": ["string"],
"notificationPreferences": { "email": true, "sms": true },
"phone": "string",
"isDeactivated": true,
"deactivatedAt": "string",
"entityName": "string"
}
}
}```
However its impossible to create a response like this with cypress. Cypress?s intercept can only return a body, not a oneOf. How can we resolve this?
matt.fellows
2023-01-11 21:25
What makes you think Pactflow expects the body to have the top level property `oneOf`?
matt.fellows
2023-01-11 21:26
I?m guessing you?re using our BDCT feature with an OAS.
`oneOf` is a logical keyword in JSON schema, and not something that should appear in an actual body.
jrede
2023-01-11 21:26
since manually changing the contract to the above format, makes the contract work as expected
matt.fellows
2023-01-11 21:27
are you doing BDCT (i.e. publishing an OAS) or standard CDC (verifying with Pact)?
jrede
2023-01-11 21:28
heres what the provider looks like:
```"responses": {
"200": {
"description": "The found contact if they exist",
"content": {
"application/json": {
"schema": {
"oneOf": [
{ "$ref": "#/components/schemas/IndividualContact" },
{ "$ref": "#/components/schemas/AgentContact" },
{ "$ref": "#/components/schemas/EntityContact" },
{ "$ref": "#/components/schemas/LenderContact" },
{ "$ref": "#/components/schemas/InternalContact" }
]
}
}
}
},```
matt.fellows
2023-01-11 21:30
Worth having a quick read of https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support#oneof-anyof also if you haven?t already
jrede
2023-01-11 21:50
I dereferenced the oneOf schema (based on suggestions in that page) and unfortunately that didn?t do the trick.
jrede
2023-01-11 21:52
pactflow is still saying that the properties are all additional and its not matching the oneOf exactly. not sure what to make of it:
jrede
2023-01-11 23:20
no, your right. It was in the consumer request body but the provider did not specify it in the schema. However, I just removed it from the consumer and still get an abundance of issues:
jrede
2023-01-11 23:21
I also made all the properties required based on what I read on that page, still same issue. Heres the updated provider:
matt.fellows
2023-01-11 23:51
Any chance you could please share the OAS/pact file? if not here, please send to (along with your account information) so the team can review
hazem
2023-01-12 23:09
curious as to how this failed pact verification occurs, any ideas? details to follow within thread:
hazem
2023-01-12 23:14
The consumer has added a header to the pact interaction, and the provider does not yet use the new header. So I expect the interaction to be forwards compatible.
The webhook fires for event `Contract published that requires verification` and all signs point to the provider build to verify as passed. However it somehow gets this overall error?
hazem
2023-01-12 23:15
It's worth noting (or not) that there's a WIP/Pending pact failing but that is irrelevant to this integration.
hazem
2023-01-13 00:33
I'm using pact-go verifier. I have a webhook that passes `PACT_URL=` into the build, but i dont see the url in the logs:
```executing verifier FFI with args [--consumer-version-selectors {"mainBranch":true} --consumer-version-selectors {"matchingBranch":true} --consumer-version-selectors {"deployedOrReleased":true} --hostname localhost --port 56162 --state-change-url http://localhost:56162/__setup --broker-url https://redacted.pactflow.io --provider-version 227d33ea99083b0a2aa9e5f17db735c31ce913d3 --provider-name RedactedService --publish --provider-branch main --loglevel trace --enable-pending --include-wip-pacts-since 2022-04-15T00:00:00Z]```
hazem
2023-01-13 00:45
ah - maybe i should explicitly pass it in then. i recall a handful of ruby cli automatically reading some pact env vars. i'll be that is what's impacting this :q-scratch_one-s_head:
hazem
2023-01-13 00:58
this should probably move over to pact-go channel at this point. although i still dont quite understand how the original screenshot shows red but all interactions are green
buckley
2023-01-13 20:09
Hello, I'm sifting through the pactflow documentation in an effort to publish some provider contracts which should be defined by OAS. I'm failing to fully understand how to use the `verifier` and related cli flags. I've run through the killercoda Bi-Directional Contract Testing workshop, and understand how dredd is being used, but how would one provide the same information to the tool when their generating their code from the spec (option 1 workflow from step 3)?
TLDR;
1. Is there a more detailed description for what the `--verifier` (and related) flags do in the `pactflow publish-provider-contract` command?
2. How should this be used when code is being generated from the spec instead of the spec being written afterward?
ilia
2023-01-16 00:39
heya @andrew.favaloro, apologies this took longer than expected, a few us were on leave due to xmas period, we looked at the code, and found that the UI could be improved since the reason that the tag is filtering on the ?provider? is because that application is both a consumer and a provider of the other application - the table is just not showing that detail. We will be reviewing this and potentially fixing the UI to make it more understandable in the future.
varnit.garg2424
2023-01-16 07:22
Hi, I?ve got a question about pactflow Pricing & Plans.
What does it mean by 10 users vs 25 users? Can?t all the teams use the same user account - is that not advisable and what are the limitations?
Also, trial version says that it supports unlimited users and 5 integrations? Does that mean 5 pairs of provider-consumer services?
dpal
2023-01-16 10:02
@matt.fellows I was interested about message based BDC testing. I was looking for AsyncAPI support for provider. But you have mentioned that Q1 2023 is the current target for that. Is there a specific date you have when I can get that? Thanks
matt.fellows
2023-01-16 10:46
1. Yes, that?s right. It?s not advisable to share accounts i.e. user/passwords. If you use SSO it would be very difficult to achieve also
2. That?s right. Our free tier is really for your team to experiment and use it as you need, up to 5 consumer/provider pairs, at which point you would need to upgrade to a paid plan
matt.fellows
2023-01-16 10:50
That flag is really to capture the tool/process used to generate and test the contract. If you are simple generating the contract, then you can put whatever you want in there e.g. ?swashbuckle? or whatever tool you used to generate the OAS.
Just note there are circumstances where generating the contract from code is still unsafe (e.g. if you have an unimplemented controller, you can stilly easily generate an OAS that looks ok but is not actually doing anything). Personally, I would recommend against relying solely on this way of doing things, and would want to see at least some tests checking your endpoints return the right shaped data as per our examples.
matt.fellows
2023-01-16 10:52
Sorry I missed this - your question timing was impeccable :stuck_out_tongue:
matt.fellows
2023-01-16 10:52
I think it?s this: https://github.com/pact-foundation/pact_broker/discussions/425
matt.fellows
2023-01-16 11:23
Yes, you might be right there. A feature request to https://github.com/pact-foundation/pact-reference/issues would be great if you want that!
bethskurrie
2023-01-16 23:15
ugh, I don?t actually know. the ruby impl still doens?t use the proper format.
bethskurrie
2023-01-16 23:17
sorry, of all the crap documentation for the pact broker, this is the most crappily documented.
tjones
2023-01-17 00:18
I?m writing an alternative contract verifier (for non-pact contracts), and I want to tell the broker about the verification results in a way that will give the most information to users
tjones
2023-01-17 00:19
but the real question is ?how do I tell a Pactflow broker about detailed verification results?? (and perhaps also ?is that different to the OSS broker??)
bethskurrie
2023-01-17 00:21
i?m trying to find an example of a pact-jvm verification publication but all our internal verifications are still using the ruby impl
bethskurrie
2023-01-17 00:21
> So the differences for verification results are in the UI?
well.. there is no UI for verificaiton results in the Pact Broker
tjones
2023-01-17 00:22
I?m happy to go to the source, but only if there isn?t already documentation for it
tjones
2023-01-17 00:23
No stress. When I get to this (in a week or two I think) I?ll see if I can make some doco
bethskurrie
2023-01-17 00:23
the ruby still uses the original format and the UI just munges it into the new format.
bethskurrie
2023-01-17 00:24
the HAL docs source is here https://github.com/pact-foundation/pact_broker/tree/master/lib/pact_broker/doc/views
tjones
2023-01-17 00:29
My last team (genuinely) had someone else called Timothy Jones apply. One of the managers had to check with me to make sure I wasn?t trolling them.
leon.luu2
2023-01-17 07:27
Hello!
When I "Delete all pacts" it delete all pacts but once I publish a new contract again, I could see old pacts immediately. This doesn't seems correct as they are suppose to be deleted. Can you confirm this?
dyptorden
2023-01-17 07:57
I personally do not have experience with Pactflow - only with Pact Broker. But I find it quite similar (for your question at least). So my opinion is that you are not in the Contracts list but in the Matrix - Matrix being the "Run History List". Thus, because you re-published a contract that previously existed, you are being displayed the previous run history.
matt.fellows
2023-01-17 09:55
If you choose to ?delete the integration? it should remove the integration related data, I believe
leon.luu2
2023-01-17 09:56
If I remove the integration then all the webhooks connected to it will also be deleted I believe
matt.fellows
2023-01-17 09:57
I think you just want to delete the test and contract data, right? I?ll check with the team tomorrow, I don?t believe we have a button to just remove that
leon.luu2
2023-01-17 09:58
It works some months back I think which now has been changed to this behaviour.
matt.fellows
2023-01-17 10:01
I?ll follow up tomorrow Leon. I don?t think deleting all pacts removed test data along with the contract versions, but I may be mistaken
matt.fellows
2023-01-17 10:02
(I have set a reminder. The development team is currently in an all day workshop today/tomorrow so we might be a little long to get back)
buckley
2023-01-17 15:33
let me re-iterate though. The contract itself is not being generated. The contract (OAS) defines the interface (in code) to be generated. But it sounds like your recommendation is to test that the same way regardless
matt.fellows
2023-01-17 23:41
Correct, either way you can have a non-implemented controller.
You may well have other important checks and balances that would prevent this, so of course it?s up to you!
matt.fellows
2023-01-17 23:43
If you?re confident that the OAS represents your code, then just put in ?n/a? or whatever you need. It might be helpful to track this over time as we store it in Pactflow
hazem
2023-01-17 23:50
Just to follow up on the red/green in the screenshot. What does it mean for the overall to show red, but the only interaction to be green?
bethskurrie
2023-01-17 23:59
the verifications remain, and when you publish a pact with exactly the same content, it identifies that it was the same content as was previously published, and hence, the verifications are joined up.
bethskurrie
2023-01-18 00:01
it?s best to have your webhook creation under source control for this reason.
bethskurrie
2023-01-18 00:02
if there are only a few provider versions (as in, the application versions) then if you delete the version resource, the verifications will be deleted too.
bethskurrie
2023-01-18 00:04
> It works some months back I think which now has been changed to this behaviour.
There has been no change in this behaviour for many years.
bethskurrie
2023-01-18 00:04
you were probably just publishing different contracts, so it didn?t hook them up to the previous verifications.
cgoodison
2023-01-18 00:27
Hi Jacob, I replied to your email but thought I would here as well, just for posterity. After dereferencing and inlining the oneOf and anyOf keywords, I notice all the errors were related to the oneOf used for the 200 response to path `/v2/contacts/{contactId}`.
Digging into this the issue here is that the 2 objects provided are too similar for our tool to match the response from your pact file to one of these objects. This is because one has properties that are the subset of the other, and no unique properties of its own.
In this case we recommend combining the objects into one schema rather than using the oneOf keyword, and making the required list contain the fields that your consumers are using.
matt.fellows
2023-01-18 00:51
> deleting an integration does delete the webhooks
I should have read the popup, it does say that
matt.fellows
2023-01-18 06:55
what version on Pact Go are you on? There was a bug in the core that was recently fixed (in Dec, I think)
eugene.malihins
2023-01-18 10:53
Hello! I'm testing Pactflow bi-directional contract testing. I published Provider contract (OAS), and produced a Consumer contract via msw adapter, and have it published too. I need some help with understanding verification status.
I verified that contracts are compatible with `swagger-mock-validator` locally, and UI says `Contract Comparison Status Compatible` , but overall status is `Unknown` and matrix says `verification failed` ? `can-i-deploy` says no too
matt.fellows
2023-01-18 10:59
Please check if you've published provider self verification results
This is a flag on the provider contract publish cli
eugene.malihins
2023-01-18 11:01
right, I haven't - i was hoping that since we auto-generate OAS from the code this wouldn't be required (cli allowed to not publish it)
eugene.malihins
2023-01-18 11:13
does self-verification result contain a list of OAS endpoints and other info, or can it be a generic json that says verification :white_check_mark: ?
eugene.malihins
2023-01-18 13:41
assuming that OAS is always correct, this hack of a verification result works and Pactflow shows :white_check_mark:
```{
"success": "true"
}```
aforeman
2023-01-18 14:06
*Unveiling the SwaggerHub and PactFlow integration* :new:
You?ve seen it on the PactFlow roadmap and now we are excited to bring you the news of our first major release for 2023 ? PactFlow?s integration with https://swagger.io/tools/swaggerhub/, the single source of truth for API development.
API designers and developers can now get instance confirmation that their API specification is compatible with consumers as they write, or make changes to, an OAS directly in the SwaggerHub Editor.
Using a design-first API contract testing practice, in the click of a button, see how the consumers of an API will respond to changes and whether it result in a break or not. And for new APIs, gate a consumer from deploying to production until they are compatible with the API specification.
The integration is built on PactFlow?s https://docs.pactflow.io/docs/bi-directional-contract-testing feature suited to organisations following a design-first approach to API development. Design-first API contract testing significantly increases the visibility of how APIs are being consumed allowing teams to catch breaking changes during design, prevent drift and reduce their time to market.
_*Do you follow an API design-first approach?*_
Take a look at the https://docs.pactflow.io/docs/integrations/swaggerhub with the integration.
_*Learn more about design-first API contract testing*_
Read the https://smartbear.com/blog/api-contract-testing-for-a-design-first-world/ or join one of our upcoming https://smartbear.com/resources/webinars/unveiling-of-the-swaggerhub-pactflow-integration/?utm_medium=email&utm_source=marketing&utm_campaign=SwaggerHub+PactFlow+Integration&utm_content=webinar to learn about the value of design-first API contract testing and see a live demo of the integration in action ? from getting started to using the compatibility check.
matt.fellows
2023-01-19 08:37
Yep! It just works over API. Is there something worrying you in particular?
sasankdts
2023-01-19 12:21
Github actions displaying differently for private repositories, I mean workflow configuration part
john295
2023-01-19 23:26
Hi! I successfully worked my way through this tutorial (thanks for writing it!): https://docs.pactflow.io/docs/workshops/bi-directional/step1
I am using the dredd provider and mountebank consumer. I got everything working, including github automation and making contract-breaking changes to see the errors. I then decided to rebuild both projects with the latest version of npm modules since they were a year or two out of date. I succeeded with one exception:
When I run can-i-deploy on the provider I get
`No pacts or verifications have been published for version 9c9a30-master+9c9a30.SNAPSHOT.mika of pactflow-example-bi-directional-provider-dredd`
I get this same message running locally or on Github. Strangely, Pactflow show that the provider was updated and I get a green check-mark
I doubt this has anything to do with my npm module update. Rather I am really new to this and probably just making a noob mistake. I tried republishing the consumer, and setting the provider version in package.json to match. I still get the same thing.
sasankdts
2023-01-20 08:05
For in Birectional provider swagger.yml I have 10 properties/attributes/fields wheras consumer pact has only 9 fields but in pactflow displaying as pass instead of fail as 1 field missing in response
matt.fellows
2023-01-20 11:19
It doesn't break the consumer (remember this is not a functional test of the provider)
david342
2023-01-20 15:51
Please ,can you explain what a PactFlow application is , is it the pact between producer and consumer which is uploaded to PactFlow ? Thanks
matt.fellows
2023-01-23 01:02
My guess is one of the following:
1. You aren?t setting the flag to say the verification passed
2. Some version is not getting aligned (i.e. the version you?re checking is slighly different to the version you?re publishing (it looks like a SHA so probably not)
Flags for (1)
``` [--verification-success], [--no-verification-success] # Whether or not the self verification passed successfully.
[--verification-exit-code=N] # The exit code of the verification process. Can be used instead of --verification-success|--no-verification-success for a simpler build script.```
matt.fellows
2023-01-23 01:04
> does self-verification result contain a list of OAS endpoints and other info, or can it be a generic json that says verification
It?s whatever makes sense for your context to demonstrate the verification was successful. It can be a string that says ?YOLO? or a full markdown/json/other document with the test execution output etc.
eugene.malihins
2023-01-23 13:06
hello, seeing a 502 response when trying to retrieve an integration in http://Pactflow.io and also in `can-i-deploy` for the newly deployed Consumer Contract version
eugene.malihins
2023-01-23 13:56
ok matrix was updated and i get 200 back now for the subject actions. i think it just takes a while (~5min or so?) to compare a Pact with ~70 interactions :thinking_face:
john295
2023-01-23 14:09
HI Matt, I didn't change the sample code between it working and not working, but I do see a line in buid.yml for "publish" that has this flag --verification-exit-code=0
john295
2023-01-23 14:10
I still can't figure out how the producer decides which consumer pact version to compare
werner.lauterfeld
2023-01-23 14:19
Hi,
what is the recommended way to publish pacts in a CI-pipeline?
Publish a pact during development on a feature-branch or should it only be published once the PR is initiated?
kind regards
matt.fellows
2023-01-23 20:59
Hmm `502`s are never expected. I?ll raise with the engineering team shortly
matt.fellows
2023-01-24 00:43
That happens in a background process, the simple explanation is that it checks against deployed/released versions of consumers, as well as the main branch. I believe it does some others, but these are likely to cover the majority.
When you run `can-i-deploy` you can be more explicit about what consumer pact versions to compare to
fabio.rodrigues
2023-01-24 11:32
Hi, with the new login page, how one should login using SAML (azure group)
matt.fellows
2023-01-24 11:53
We haven?t rolled out a new login page. My guess is that SAML has been recently added to your account?
fabio.rodrigues
2023-01-24 11:55
one colleague of mine was using the default pacflow login page, and not the federated one
eugene.malihins
2023-01-24 14:10
also noticing that verifying a Provider Contract (OAS) against a Consumer Contract with 4 interactions is really fast, but when i upload a Consumer Contract with ~70 interactions it is not verified after ~30min. this is when i observe 502s as well
eugene.malihins
2023-01-24 16:58
still experiencing schemas not being validated, one of them published 3h ago. Are there any limits on Provider schema/Consumer Pact size that impacts this?
bethskurrie
2023-01-24 22:16
There are limits to the number of OAS/pact comparisons that get done when you publish.
bethskurrie
2023-01-24 22:17
When you publish an OAS or a pact, it generates comparisons between the latest pacts/oas for the most recent 100 branch heads.
bethskurrie
2023-01-24 22:18
That `-` there may indicate that there is no self verification uploaded. Can you check that for me?
bethskurrie
2023-01-24 22:20
What is your pactflow tenant name @eugene.malihins? We?ve been noticing some timeouts for a few matrix and dashboard queries that are hitting the request length limit because there are too many oas/pact comparisons trying to be run synchronously within the request. We?re in the process of updating that so it returns a ?comparison in progress? status and runs the comparison in a background process.
bethskurrie
2023-01-25 02:17
I think I?ve found your logs. The OAS/pact comparison is taking 53 seconds to run.
sasankdts
2023-01-25 07:10
In bidirectional contract testing example, newman results, swagger.yml and pact.json deletes existing for every run
matt.fellows
2023-01-25 07:44
Are you referring to this step: https://github.com/pactflow/example-bi-directional-provider-postman/blob/master/package.json#L8
Is there a reason why you wanted to know that?
matt.fellows
2023-01-25 07:45
You should always generate a new pact contract each testing session, so that you don?t have old/outdated interactions (test) in them
eugene.malihins
2023-01-25 10:23
i published Consumer pact `1.0.1-sameschema-dereferenced` last night, along with Provider OAS (verified) `1.0.1-dereferenced` - however this morning still i don't see a comparison.
I pushed the same `1.0.1-dereferenced` provider OAS once again this morning, and it got verified against an older Consumer `1.0.0-sameschema-dereferenced` , but still not against the latest consumer `1.0.1-sameschema-dereferenced` :thinking_face:
eugene.malihins
2023-01-25 10:25
53 seconds isn't too bad for now tbh if it can gate schema breaking deployments :slightly_smiling_face:
eugene.malihins
2023-01-25 10:27
however whenever i see a Provider<>Consumer pair not getting verified as above/currently - can-i-deploy and dashboard Overview return 502
eugene.malihins
2023-01-25 11:29
pushed (verified) `1.0.1-dereferenced` Provider OAS ~10min ago, it doesn't show up in the matrix
eugene.malihins
2023-01-25 11:38
trying to publish it again tells me `status=409 {"errors":{"content":"A provider contract for sequence-api version 1.0.2-dereferenced already exists with different content and may not be modified. You will need to delete this resource and re-publish it` , but it's still not in a matrix :confused:
yousafn
2023-01-25 13:37
I publish on every commit, and also at PR time, (where PR is rebased from master/main), that gives me the quickest feedback as a consumer, as each of these will trigger pact verification jobs.
If the pact content hasn't changed it will be pre-verified, if the pact content changes, people are made aware and it can be verified on the commit or PR time.
Some may choose to only do it at PR time, thats fine, there is no _right_ answer just a recommended one. Our CI/CD workshop publishes on every commit
xi.luo
2023-01-25 14:51
@yousafn Thanks for the reply. In your way (publish on every commit), there's high chance to have flooding pacts on pactflow. is there a way provided to automatically clean up old pacts?
yousafn
2023-01-25 15:28
I would expect that not to be the case, what is your throughput of commits? Provider verifications are only triggered if the Pact content changes, but the broker determines this, so it requires the contract from a known point in time, denoted by a version number (which we often use as a commit sha, because of its point in time guarantees)
We got asked this question recently, here was @matt.fellows's answers
> How many contracts can PactFlow handle?
The load/performance characteristics of the system are correlated to number of contracts, but it?s not the only factor. We have many customers on hundreds of contracts without performance related issues.
> Is there a way to clean up the environment and remove old contract testing information?
We run data cleaning on all tenants, documentation for this will actually be published soon. See https://docs.pact.io/pact_broker/administration/maintenance#recommended-starting-configuration-for-keep-selectors for a bit more information on the process, the exact values may be tuned by Pactflow based on performance, load and other characteristics.
eugene.malihins
2023-01-25 15:46
@cgoodison could you help me understand `inlining the oneOf` keywords please? I'm observing issues very similar to what Jacob posted here - this is after i dereferenced all `$ref` , and in our schema we also have a bunch of `oneOf` responses.
`json-schema-merge-allof` seems to not affect `oneOf` keywords at all :thinking_face:
bethskurrie
2023-01-25 22:03
Well, today is a public holiday in Australia, but I'll be working on it tomorrow!
sasankdts
2023-01-26 02:02
While generating swagger file from postman collection can we specify required fields and optional fields
sergewar
2023-01-26 10:35
Hey. Could anyone advice why I can?t upload/publish provider contract to my own local pact broker server?
It return 404 response
`Error making request to http://127.0.0.1:9292/contracts/provider/KAPI/version/5015 status=404`
But when I?m trying to upload to my own pact broker on http://pactflow.io all work as expected (using token)
Pact broker docker-compose config
```version: "3"
services:
postgres:
image: postgres
healthcheck:
test: psql postgres --command "select 1" -U postgres
volumes:
- postgres-volume:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
POSTGRES_DB: postgres
pact-broker:
image: pactfoundation/pact-broker:2.105.0.1
ports:
- "9292:9292"
depends_on:
- postgres
environment:
PACT_BROKER_BASIC_AUTH_ENABLED: "true"
PACT_BROKER_BASIC_AUTH_USERNAME: "un"
PACT_BROKER_BASIC_AUTH_PASSWORD: "pwd"
PACT_BROKER_ALLOW_PUBLIC_READ: "false"
PACT_BROKER_BASIC_AUTH_READ_ONLY_USERNAME": "unr"
PACT_BROKER_BASIC_AUTH_READ_ONLY_PASSWORD: "pwdr"
PACT_BROKER_DATABASE_URL: ""
PACT_BROKER_DATABASE_CONNECT_MAX_RETRIES: "2"
PACT_BROKER_BASE_URL: "http://localhost"
PACT_BROKER_BASE_URLS: "http://localhost:9292 http://127.0.0.1:9292 https://localhost https://127.0.0.1"
PACT_BROKER_PORT: "9292"
PACT_BROKER_USE_HAL_BROWSER": "true"
volumes:
postgres-volume:```
uploading script
```for file in "$PWD"/*.yaml
do
PROVIDER_NAME=$(yq eval .info.x-pact-name "$file" )
if [[ "$PROVIDER_NAME" != null ]]
then
echo "Publish '$PROVIDER_NAME' from $file"
pactflow publish-provider-contract \
"${file}" \
--content-type=application/yaml \
--provider="$PROVIDER_NAME" \
--provider-app-version="$RANDOM" \
--broker-base-url="http://127.0.0.1:9292" \
--broker-username="un" \
--broker-password="pwd"
fi
done```
yaml file
```openapi: 3.0.3
info:
title: k API
version: "0.1"
description: k API
x-pact-name: KAPI
paths:
/users/{userId}:
get:
summary: Returns user
parameters:
- in: path
name: userId
description: User ID
schema:
type: string
required: true
responses:
'200':
description: OK
'401':
description: Unauthorized```
dimundo
2023-01-26 11:08
so, for money only ?
and then it doesn?t matter if it is self-hosted PactFlow or not ?
yousafn
2023-01-26 11:10
The self hosted PactFlow broker is part of a paid plan yes
The feature is expected as part of your free starter plan which is a cloud hosted version of PactFlow. On prem is part of the enterprise plan I believe
dany.marques90
2023-01-26 15:41
Hi there,
I have a question regarding PactFlow. I uploaded the consumer and the provider contract to Pactflow and simulated a deployment on an environment for both consumer and provider.
Now I?m trying to use the `can-i-deploy` utility but it always fails with: `The provider contract for the version of my-provider currently in test (f02f154) has no verification results` .
Can someone help me? It looks like I?m missing something
eugene.malihins
2023-01-26 19:52
I think we have 3 issues with trying to validate Pacts against OAS, after merging/dereferencing all $refs:
? `oneOf` keywords inside the schema with Properties, each of the `oneOf` schemas also having Properties. Errors of the kind `should NOT have additional properties` - not sure how to work around it if there's no option to allow additional properties? Something along https://ajv.js.org/faq.html#additional-properties-inside-compound-keywords-anyof-oneof-etc?
? if i set `--additionalPropertiesInResponse true` in swagger-mock-validator ^ goes away, but two more issues occur:
? `Response body is incompatible with the response body schema in the spec file: should match exactly one schema in oneOf` and `Request body is incompatible with the request body schema in the spec file: should match exactly one schema in oneOf`
What's strange looking to me about the last two errors are these two comparisons:
```Comparing Mock
value: { day: 5, interval: 2, month: 4, type: 'YEARLY' }
With OAS
value: [ 'daysOfWeek', 'interval', 'type' ]```
and
```Comparing Mock
value: { daysOfMonth: [Array], interval: 1, type: 'MONTHLY' }
With OAS
value: [ [Object], [Object], [Object], [Object] ]```
In the first case it compares mock response with a non-matching `oneOf` schema (while there is a matching one, but it doesn't error). And in the second case it compares request with a literal `[Object]` ?
Any help/pointers are appreciated :pray:
bethskurrie
2023-01-26 21:59
@dany.marques90 can you please provide the URLs to each of the contracts, and the full steps to reproduce the error, and send them to
abdel.akkoub
2023-01-26 22:52
Thank you so much Beth and Matt! This issue was resolved after we reported it 2 months ago. However, it seems that the same issue has re-surfaced today. Do we need to open a separate ticket?
dpal
2023-01-27 12:01
Hi There, I'm looking for Async API support for provider side. When is that going to be live on Q1 2023? Thanks
sergewar
2023-01-27 14:02
Hi. I noticed that I can?t remove pact ending on `.yaml` Loaded accidentally with this name (full path to yaml file) and now can?t remove
yousafn
2023-01-27 14:26
Hey, we haven't committed to a date yet
Please feel free to add any input on what you are expecting to see in our roadmap placeholder if you so wish
https://github.com/pactflow/roadmap/issues/57
yousafn
2023-01-27 14:33
Hi Mark,
Best asking PactFlow specific questions in our dedicated channel.
You need to add a content type for the document you are uploadeding, `application/json` in your respect as its a JSON document, although application/yaml will work, the parser will accept both
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#docker-1
yousafn
2023-01-27 14:41
It's okay, we've all done it! And saweet that you are sorted, happy tinkering
sergewar
2023-01-27 14:49
something changed
now it show `%2F` instead of `/` in pact name
the error was
```No pacticipant with name
'pactname/but/without/yaml/at/the/end/openapi/tt/preferences' found```
sergewar
2023-01-27 14:51
and again changed %)
after my attempt for removing it doesn?t show full path
cgoodison
2023-01-29 22:57
Hi @eugene.malihins I'm not 100% sure why you are having issues with the properties in your oneOf section, however it's not recommended to turn on additional properties as it can cause other unexpected issues with the oneOf, allOf and anyOf keywords. Write up about it can be found here: https://bitbucket.org/atlassian/swagger-mock-validator/src/master/FAQ.md
In your OAS spec are you including a 'required' section for the objects? The tool uses the required fields to know which specific fields to compare with the values in the pact file.
It's hard for me to tell exactly what is causing these errors, Any chance you could please share the OAS/pact file? if not here, please send to (along with your account information) and I can have a bit more of a look for you :slightly_smiling_face:
eugene.malihins
2023-01-30 09:24
Hi @cgoodison, thanks for your response! I've figured out most of the oneOf issues by now - biggest issue was that properties contained in oneOf objects weren't actually present in the parent object itself. Required section existed already for both the parent object (requiring child object with oneOf) and inside the oneOf array. I am now hitting one more issue with oneOf, and i think it might come down to the either msw responding with incorrect object, or pact-msw-adapter capturing incorrect response. I'll dig a bit more and will send an email to support@ if i cannot figure it out. Thanks!
matt.corby-eaglen
2023-01-30 11:52
Hi - I am having some trouble removing a pact integration; any help would be welcome!
matt.fellows
2023-01-30 12:08
Hi Matt, leave it with us - we?ll take a look tomorrow when the engineering team is about. It looks like a query is timing out (we just saw the error in a monitoring channel).
If it?s urgent, do please raise a ticket at so we can track it and keep in contact with you.
matt.corby-eaglen
2023-01-30 14:48
its not urgent, but thanks for looking into it :slightly_smiling_face:
bethskurrie
2023-01-30 21:24
@matt.corby-eaglen there's a delete query that performs badly because it's using standard SQL instead of vendor specific PostgreSQL. I'm going to have a look at using some driver specific code in this case, so can you raise that ticket if you haven't already please.
bethskurrie
2023-01-30 21:41
Can you please make a new one, because the old one will be resolved @abdel.akkoub
bethskurrie
2023-01-31 04:39
Actually @matt.corby-eaglen it may turn out to be simpler than I thought. I?ve identified an index that may help, and I?ve added it. Can you try again and see if it works?
eugene.malihins
2023-01-31 15:46
how should one read this `swagger-mock-validator` output? What confuses me is comparison of mock response body to `value: [ [Object], [Object] ]`. It's not clear what is incompatible. I've omitted a bunch of fields in the mock for brevity.
```1 error(s)
response.body.incompatible: 1
0 warning(s)
{
warnings: [],
errors: [
{
code: 'response.body.incompatible',
message: 'Response body is incompatible with the response body schema in the spec file: should match exactly one schema in oneOf',
mockDetails: {
interactionDescription: '6cee5dad-fe47-4ab7-9384-d53d5260427a',
interactionState: '[none]',
location: '[root].interactions[27].response.body',
mockFile: './msw_generated_pacts/dashboard-web-api-1675179158185.json',
value: {
createdAt: '2022-10-30T00:00:00Z',
"...": "...",
status: 'DRAFT',
}
},
source: 'spec-mock-validation',
specDetails: {
location: '[root].paths./api/invoices/{id}/payment-status.put.responses.200.content.application/json.schema.oneOf',
pathMethod: 'put',
pathName: '/api/invoices/{id}/payment-status',
specFile: 'dereferencedSchema.json',
value: [ [Object], [Object] ]
},
type: 'error'
}
]
}```
bethskurrie
2023-01-31 19:50
Step 1 is done, which is executing the comparisons asynchronously. This should have fixed the 502s. Step 2 will come soon, which will be showing which comparisons are currently in progress.
cgoodison
2023-01-31 22:36
Hi again Eugene, it looks like the objects in your oneOf section do not have required fields marked on them. Without required fields the tool does the comparison on the type only, in this case [Object]. Since both of the options in the oneOf are that type it doesn't match to only 1.
Try marking the fields that are different between the objects as required, then it will do a deeper comparison on the objects and hopefully match to just one as intended.
There are some docs on it here: https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support#oneof-anyof
cgoodison
2023-01-31 22:37
If you don't have any luck with it happy to have more of a look if you want to share the contracts here or via the pactflow support email :slightly_smiling_face:
eugene.malihins
2023-02-01 09:01
thank you for the context on how comparison works @cgoodison, i think it might be helpful to others too :+1:
In our case the oneOf schemas were pretty much the same, which is an oversight on our end. Fixed :slightly_smiling_face:
eugene.malihins
2023-02-01 10:33
also sent an email to support@ for one last remaining oneOf bit that i cannot figure out. thank you!
jhordies
2023-02-02 13:31
Hello, I'm testing a pact against an open api specification, when my 3rd pact interaction is incompatible with the OAS, all my interactions are marked as failed
keith.w
2023-02-02 16:44
Hi all! I'm have some confusing (to me) information in pactflow/can-i-deploy. can-i-deploy is failing with a "There is no verified pact between version ..... and the version of thing-model-service currently in Dev....." but when I go to the matrix tab it shows a status of "Verified".
What am I missing or doing wrong? :sweat_smile:
matt.fellows
2023-02-03 06:30
Can you please raise an issue to
? Account name (e.g. http://xyz.pactflow.io)
? Link to integration that is failing
We might need to look into the data for you
joao
2023-02-03 09:45
Hi folks,
Regarding bi-directional verification, where the Contract Verification Status is Compatible (On the "View Contract" page,
but in the Overview page I see the integration status as Unknown, not Success as it happens when a consumer-driven integration is verified.
When I run can-i-deploy, it says that there is no verification results for this integration.
Is this expected? Am I misinterpreting something here?
werner.lauterfeld
2023-02-03 13:56
Hey guys,
we are having a look into stubbing and found the pact-stub-server that is provided by the pact foundation.
The docker image gives the option to provide a header to select provider-states and get an answer based on that.
We wondered whether there is an option to use this against the Pactflow instance as well.
Do you have a default header that we can set to use the stubbing service of PactFlow to select the response based on a provider state, or is this not possible at all?
Kind regards,
Werner
eugene.malihins
2023-02-03 15:15
Pipelines for both our products (API & Dashboard) are now fully integrated with Pactflow bi-directional contract testing :rocket:
I will continue keeping an eye on channels here for updates & provide feedback as it arrives :slightly_smiling_face:
eugene.malihins
2023-02-03 15:18
? OAS publishing
? Consumer Contract generation with `openapi-backend` + `msw` + `pact-msw-adapter`
? `can-i-deploy` on pre-merge checks and pre-deploy jobs for every env for both products
? `record-deployment` on post-deploy jobs for every env for both products
Thank you for the support we have received to take it online!
ulises.cervino
2023-02-03 15:30
quick question: which pact selectors do you use for verifying contracts, e.g. from the provider side of things?
eugene.malihins
2023-02-03 15:48
great question from the Advanced Topics category @ulises.cervino, which I haven't touched yet :smile: what am I missing out on?
eugene.malihins
2023-02-03 15:53
quick scan of the docs suggest that this allows to limit which provider<>consumer contracts are verified? looks like a good way to reduce the load on Pactflow / potentially speed up verification / reduce the amount of lines in the Matrix? Where does this config live?
ulises.cervino
2023-02-03 16:17
you can do this in a couple of ways, but the only way I managed to get it working was with a `@Selector` or whatever in pact-jvm
ulises.cervino
2023-02-03 16:17
and the idea is that, for instance, a provider only tries to verify consumer pacts for versions that have been rolled out/released and not for absolutely every branch
ulises.cervino
2023-02-03 16:17
I _think_ the default behaviour is only verify pacts for the `master` branch, which might not be what you want at the beginning actually
eugene.malihins
2023-02-03 16:59
would be great for someone from Pact team to elaborate on this :pray:
I currently only care about verifying OAS against Consumer Contract that is recorded to be deployed in `dev/test/staging/prod`
Same goes for verifying Consumer Contract against OAS recorded to be deployed in `dev/test/staging/prod`
So `main` branch for both
matt.fellows
2023-02-04 22:08
When you publish your provider contract, you must say the verification status to successful
matt.fellows
2023-02-04 22:12
At the moment this isn't possible. I'll add a note to our backlog. May I ask a bit more about your use case for the wtib server?
matt.fellows
2023-02-04 22:12
How would you know when to set the correct header to select the right interaction?
matt.fellows
2023-02-04 22:21
In this workflow you don't specify any selectors as there is no provider verification, unlike in regular Pact. Pactflow automatically pre verifies several consumer versions against the uploaded provider contract (this is what you see on the matrix page) and additionally it will dynamically compare the necessary versions during can I deploy.
matt.fellows
2023-02-04 22:22
If you think about it it makes sense. The provider publishes what it can do, the consumer publishes what it needs and when it comes time to deploy (or simply run can-i-deploy) it will tell you if they're compatible.
This is what makes the flow much more simple and decoupled
matt.fellows
2023-02-04 22:23
By default I believe it compares against main branches of consumers and deployed/released versions
myao
2023-02-06 06:36
Hi Pact gurus,
Currently we are working on BDCT and trying to publish our contract to Pactflow. I am using Docker commands but a bit confused with the doc examples and the command helper. What I am trying to do is to manually upload the contract to test the integrations on the CI pipeline, with the minimum command arguments and configurations. But I am getting the errors `Error making request to https://{PACTFLOW_HOST} status=400 {"errors":{"verificationResults.success":["success is missing"],"verificationResults.content":["content is missing"],"verificationResults.contentType":["contentType is missing"]}}` . So obviously, some of the params are mandatory that I missed, but the doc didn't clearly say which ones are and I am trying to figure it out.
The followings are the references and the examples I am following, with my scripts
matt.fellows
2023-02-06 07:11
I?m not sure why it doesn?t complain about all of the fields that are required when not given. i.e.
```? pactflow publish-provider-contract
No value provided for required options '--provider', '--provider-app-version'```
Should presumably list a bunch more items. There might be a good reason why the CLI doesn?t do that, I know it works within the framework of thor (the gem).
matt.fellows
2023-02-06 07:12
So one of these two is required:
``` [--verification-success], [--no-verification-success] # Whether or not the self verification passed successfully.
[--verification-exit-code=N] # The exit code of the verification process. Can be used instead of --verification-success|--no-verification-success for a simpler build script.```
matt.fellows
2023-02-06 07:13
You should provide at least the following also:
``` [--verification-results=VERIFICATION_RESULTS] # The path to the file containing the output from the verification process
[--verification-results-content-type=VERIFICATION_RESULTS_CONTENT_TYPE] # The content type of the verification output eg. text/plain, application/yaml
[--verification-results-format=VERIFICATION_RESULTS_FORMAT] # The format of the verification output eg. junit, text
[--verifier=VERIFIER] # The tool used to verify the provider contract```
ulises.cervino
2023-02-06 08:22
this is something I am actually wondering: I've specified no consumer selectors, hoping that my provider tests would try and verify all pacts, however I must ask: would this verify older pacts (for the same consumer/version/environment combination)? I suspect not? If I only specify the consumer in my provider test, will it try to verify all version/branch/environment pacts?
myao
2023-02-06 09:55
Just followed the above suggestions and I can see most of the errors are gone. But it still has one required field `verification-results-content` that I can't find from the doc. What's the expected value?
yousafn
2023-02-06 14:02
`--verfication-results` which is the path to the file, that contains the content. The CLI tool will read in the file, and send that as the `content`
yousafn
2023-02-06 14:04
you can see in the 2nd screenshot it is pointing to a verification results file, and then specifies the content type of the verification results file beneath it
eugene.malihins
2023-02-06 15:06
getting 502 from pactflow when running can-i-deploy :confused:
```ERROR: Error making request - Net::ReadTimeout Net::ReadTimeout with #<TCPSocket:(closed)> /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/hal/http_client.rb:80:in `block (2 levels) in perform_request', attempt 1 of 5
ERROR: Error making request - Net::ReadTimeout Net::ReadTimeout with #<TCPSocket:(closed)> /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/hal/http_client.rb:80:in `block (2 levels) in perform_request', attempt 2 of 5
ERROR: Error making request - Net::ReadTimeout Net::ReadTimeout with #<TCPSocket:(closed)> /usr/lib/ruby/gems/3.0.0/gems/pact_broker-client-1.66.0/lib/pact_broker/client/hal/http_client.rb:80:in `block (2 levels) in perform_request', attempt 4 of 5
Error retrieving matrix. PactBroker::Client::Hal::ErrorResponseReturned - Error making request to ***/matrix status=502 <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
</body>```
matt.fellows
2023-02-06 21:04
Thanks Eugene. We?ll take a look at your data to see what?s going on. We can see the timeouts on your account, and appreciate you raising a support request
eugene.malihins
2023-02-06 21:42
Thanks @matt.fellows. I think this happened when a bunch of Provider PRs were auto-merged, so I guess it flooded pactflow with lots of verification requests
matt.fellows
2023-02-07 02:53
It?s just the mime type of the verification results. e.g. `text/plain` or `application/html`
matt.fellows
2023-02-07 02:55
Possibly. I?ll leave that to the engineering folks to dig into, but thanks for the context!
matt.fellows
2023-02-07 03:44
> verify all pacts
when you say ?all? what is your definition? In the case of some customers, that could literally be hundreds of thousands of pacts.
I should also - I?m assuming you?re asking about regular Pact, and not BDCT right?
myao
2023-02-07 04:54
Is there a example of the `--verification-results` file? I'd like to see if I can skip it and just put some dummy text to pass my pipeline test
ulises.cervino
2023-02-07 08:02
caveat: I'm still trying to figure things out. Ref regular Pact vs. BDCT, I don't know the actual differences. We have consumer tests (written by other teams) and provider tests (written by us). In principle, when I verify contracts (provider tests) I'd like to aim for coverage (until it becomes a problem). That's what I mean by "all": if there's a branch `B` of consumer `C` that published a contract, when I run my provider tests I'd like to pick up the latest version of it. This is for all branches and consumers.
matt.fellows
2023-02-07 09:28
cool. Have you run the CI/CD workshop before? That?s worth doing, because I think it will cover what you are wanting to do
2023-02-07 09:28
Here are a number of useful hands-on labs that teach all of the key concepts: https://docs.pactflow.io/docs/workshops and https://docs.pact.io/implementation_guides/workshops
matt.fellows
2023-02-07 10:16
Yep, here you go?.
```echo 'generated by <program/process>' > results.txt```
The set the flags:
``` --verification-results=results.txt \
--verification-results-content-type=text/plain \
--verification-success \
--verifier=<program/process> ```
[3:07 PM] The point of the verification results is to provide traceability to future team members to understand what level of confidence they should have in the contract itself.
Given you?re just testing for now, anything is fine.
There is no format for the results, it?s the output file from the process that does the verifying, and what makes sense to a user later on. I believe we render various content types in the PactFlow UI (e.g. Markdown) also.
yousafn
2023-02-07 11:47
you can just send the openapi spec as the verification results and a nice easy way (or just a text file with anything in it)
ulises.cervino
2023-02-07 12:29
let me double-check, but worth asking: is this the cycle in which consumers publish pacts and via webhooks verifications are triggered, providers verify contracts on pushes to branches, and eventually it all converges on consumers/providers all checking `can-i-deploy` before rolling out? (and when rolling out recording the versions deployed)
myao
2023-02-07 22:48
Thanks both, really glad I've got the provider contract published via the CI pipeline now :smile:
matt.fellows
2023-02-08 00:19
It talks about how to map the tooling onto your CI/CD processes basically, and answers your question how to validate ?all pacts?
ulises.cervino
2023-02-08 07:53
I should watch it then, for the "validate all pacts" bit (the rest of the pipelining is in place)
ulises.cervino
2023-02-08 07:53
now, quick question: is the "validate all pacts" to do with the consumer pact selectors?
ulises.cervino
2023-02-08 07:55
and actually, regarding selectors, maybe I missed it: are these the recommended guidelines still https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors#recommended ?
matt.fellows
2023-02-08 10:11
TL;DR - wip is probably what you need to ensure you validate all the current work in progress branches/features, alongside the important ones you specify in selectors
ulises.cervino
2023-02-08 10:33
right, at first I thought checking only deployed/released is what I want, but then I thought that that might be premature optimisation, which made me think "verify ALL pacts", including wip
ulises.cervino
2023-02-08 10:33
as long as we're vigilant and aware that if wip fails that's fine?, then it should be all right?
joao
2023-02-08 12:35
Ah, got it. Thanks Matt.
Now I am facing another problem.
When trying to re-publish the OpenAPI file, I am facing a 404 error "The requested document was not found on this server".
Either using the same version or a different verion.
I don't get exactly why a 404 if I am trying send data to the server.
Any ideas?
matt.fellows
2023-02-09 06:14
note how there are two `//` in the URL (if you set `--verbose` on you?d see the request path in more detail).
werner.lauterfeld
2023-02-09 10:25
Hey Matt,
the pact-stub-server allows to set a header to determine the answer that you would get based on provider state.
With that you could always built certain scenarios e.g. for prototyping something.
Easiest example I can come up with would be a get endpoint returning 404 (no item), then you put something there and on next request to get endpoint would yield the item.
As stated earlier, just wondered whether you have something similar in place for Pactflow.
For us this is not an important feature looking at PactFlow, since we can always use the pact-stub-server for local development.
Thanks a lot for your answer.
Kind regards,
Werner
matt.fellows
2023-02-09 11:05
oh right, I understand about provider states and the stub server, but was just trying to understand how it would be used.
In most cases I?ve seen from people, the stub server is designed for uses in test cases (usually e2e UI type tests - such as Cypress). So to be able to make use of the feature, the API client would need to be told by the test runner to ?add this header?. It?s not impossible, but certainly makes writing those kinds of tests more difficult/cumbersome.
I think we should support it, or something like it if we continue to invest in the hosted stub feature.
A bigger issue at the moment is that it only supports V2 pacts, which is more important to address I think
priyanka.soni
2023-02-09 13:17
Hi Community, I have a question about BDCT.
On *Consumer* side we have an iOS app implementing *traditional PACT.*
On our *provider side*, we have OAS available and we think that we could generate the PACT file by implement BCDT using RestAssured. Is this compatible?
In the documentation we saw:
Compatible with Consumers
This project is currently compatible with the following consumers(s):
? https://github.com/pactflow/example-bi-directional-consumer-nock
? https://github.com/pactflow/example-bi-directional-consumer-msw
? https://github.com/pactflow/example-bi-directional-consumer-wiremock
? https://github.com/pactflow/example-bi-directional-consumer-mountebank
Does it mean Bidirectional feature is not compatible with PACT contracts?
yousafn
2023-02-09 13:29
Hey hey,
Each of those consumers, use consumer testing tools native mock capabilities, and then converts successful matches to Pact specification format files.
If you already have a consumer generating consumer pact files via a Pact client library framework, this is the ideal scenario.
You can indeed use BDCT to compare that consumer pact file with the OAS of your provider.
Are you currently doing provider verification with a Pact client library, for for provider?
BDCT will not utilise provider states, or matchers so that is one caveat to be aware of, but these won't stop you from doing the pact -> oas comparison
tommy.chen
2023-02-10 01:11
Hey team, this is tommy from culture amp. I seem to get a ssl issue. Here is the thread where I initially asked the question https://pact-foundation.slack.com/archives/C9VPNUJR2/p1675989683954029
joao
2023-02-10 11:01
Ah, got it. I was able to upload now.
One question Matt: Currently I am dealing with a provider that doesn't have any validation of the OpenAPI
spec, like Dredd. So, when I send only --verification-success, I get a "verifier is missing".
I see Pactflow will accept even fake report, but I would prefer to not have this in the pipeline.
Is it possible to send only --verification-success, with the verifier and the report?
matt.fellows
2023-02-10 12:15
It is not sorry. The report can be anything that describes why you think it is correct
How do you know it's safe then if you're not testing it
noor.hashem
2023-02-10 14:43
Hi pact team, i'm trying to access the subscription portal in pactflow but I have not received a one time password and have requested again but nothing has come in for a while.
matt.fellows
2023-02-10 22:31
That might be a security feature of our subscription platform (Chargebee). Are you the billing admin? If you aren't it might not tell you that. Feel free to DM me with your details and I can take a look, or drop a support ticket to and they'll help you out
matt.fellows
2023-02-10 22:32
Please share the subscription id (you can get this from within Pactflow) and your email address
buckley
2023-02-13 22:58
Is there a required workflow order needed in order to tell the broker that contracts should exist? I'm attempting to spin up some provider tests but `pact-provider-verifier` keeps returning with something similar to the following: (organization and project have been masked)
```/usr/lib/ruby/gems/3.1.0/gems/pact-1.63.0/lib/pact/hal/entity.rb:102:in 'assert_success!': Error retrieving https://${organization}.http://pactflow.io/pacts/provider/${project}-provider-golang/for-verification status=404 {"error":"The requested document was not found on this server."} (Pact::hal::ErrorResponseReturned)```
matt.fellows
2023-02-13 23:00
can you please share a bit more to help us diagnose what?s going on here?
e.g. have you published your contract? What arguments are you passing to the CLI?
bethskurrie
2023-02-13 23:02
This workshop explains the process https://docs.pactflow.io/docs/workshops/ci-cd
buckley
2023-02-13 23:03
For Matt's concerns, it would appear that I have NOT published a contract yet since I started with the provider rather than the consumer. That'll be my next step.
buckley
2023-02-13 23:05
As far as CLI goes, I'm using the golang testing tools (`go test ./...`) with environment variables to configure the `VerifyRequest` object used by `VerifyProvider`
matt.fellows
2023-02-13 23:36
Right. Pact is a consumer driven tool, so usually best to start with the consumer.
You can do provider first of course, but you won?t have anything to validate
bethskurrie
2023-02-15 21:21
Sorry for slow reply. I'll have a look at that error reference. We may be working out way up the error stack!
bethskurrie
2023-02-16 06:28
@matt.corby-eaglen deployment is running right this moment. Can you try the deletion again when you have a minute?
matt.fellows
2023-02-16 10:50
I?m glad we were able to find you a https://www.youtube.com/watch?v=Nj4q4rfDcNw
matt.corby-eaglen
2023-02-16 10:51
haha you have no idea how much fan email I get (or used to get) on my gmail account from him lol I still get a bit now and again
jwang
2023-02-16 23:56
In Pactflow, I am getting a Validator Error due to my provider contract specifying "items": { "type": "number", "format": "int32" }. This seems to also break the consumer tests in a way that they all give false positives. I don't see anything offspec with our implementation but if I remove the "format": "int32" from my property the consumer contract properly validates with confirmed errors.
Is there a known bug related to specifying format in a property in the provider contract?
thomas.koppensteiner
2023-02-17 11:42
Hello,
I encountered the following error.
The producer specifies an API path similar to `GET /users/{user_id}/settings` `{user_id}` is defined as `string` with a regex to validate the format in detail (int, uuid or other formats).
The consumer calls the API with `GET /users/1234/settings` and the pact check fails with `"No request method is found /users/1234/settings" => "Path or method not defined in spec file: GET /users/1234/settings"`.
When I remove the regex check from the producer OAS file, then the pact check works.
matt.fellows
2023-02-17 11:55
Hi Jay, not that I know. Mind sharing a bit more about your setup so we can take a look?
matt.fellows
2023-02-17 11:57
Can you please share your OAS? Does it pass basic validation? (E.g. from the SwaggerHub editor)
thomas.koppensteiner
2023-02-17 12:09
The OAS is closed source, but both Stoplight and openapi_rspec didn't noticed an validation error.
thomas.koppensteiner
2023-02-17 12:14
Hmm "Stoplight Studio" (desktop versions) doesn't report errors, "Stoplight" (online) reports an error. I'll double check that.
thomas.koppensteiner
2023-02-17 12:36
It might be related to the usage of
```\A Start of string
\z End of string```
instead of
```^ Start of line
$ End of line```
Build is running.
noor.hashem
2023-02-17 19:59
Hi there, does anyone have a pactflow example of a ruby provider verification?
I'm not sure how to set up the *pact_broker_client* task if im using pactflow
```require 'pact/provider/rspec'
require "./spec/pact/provider_states_for_order_item_categories"
Pact.service_provider "monolith" do
honours_pact_with 'testDemo_platform_payment_service' do
pact_broker_base_url = ''
pact_broker_client = PactBroker::Client.new(base_url: pact_broker_base_url)
pact_url = pact_broker_client.pact_url('monolith', 'testDemo_platform_payment_service')
pact_uri pact_url
publish_verification_results true
enable_pending true
include_wip_pacts_since "2020-01-01"
end
end```
iamchughmayank
2023-02-17 22:24
Hello! :wave:
I posted https://pact-foundation.slack.com/archives/C9VBGLUM9/p1676663254541939 in #pact-js but as I dig in more, I feel that as per https://pactflow.io/blog/verification-results/, my implementation should have worked. Hence, X-posting it here to see if I can get some help with PactFlow as my pact broker?
matt.fellows
2023-02-18 03:18
I've just responded. I think it's a pact JS issue/question so let's keep the chat there. Thx
matt.fellows
2023-02-18 03:21
Ideally the OAS, or an OAS that exhibits the same issue.
Does it pass validation by tools like the SwaggerHub editor https://editor.swagger.io/
noor.hashem
2023-02-21 15:33
Hi there i'm using pactflow on a provider ruby on rails application and on the verification step i am getting a 401 unauthorized error.
See error below:
`Error retrieving https://applyboard.<name>.io status=401 Unauthorized. Please see https://docs.pactflow.io/docs/login-help for more information. (Pact::Error)`
I have already followed the steps outlined here: https://docs.pactflow.io/docs/login-help/#getting-a-401-unauthorized-when-publishing-or-verifying-pacts
Package should be up to date, i'm using pact gem version 1.32. Using the read/write token from pactflow. And I followed the provider code in pact_helper.spec here: https://docs.pactflow.io/#configuring-your-api-token
matt.fellows
2023-02-21 21:22
Can you please set log level to debug (or verbose to true) and share a redacted version to us?
matt.fellows
2023-02-21 23:04
Interesting. I can?t see any `Authorization: Bearer xyz` header in there
noor.hashem
2023-02-21 23:52
Oh do I need to somehow manually add that Authorization header?? Let me try to get you some more information.
matt.fellows
2023-02-22 00:10
I don?t know the Ruby implementation all that well and why it?s not working though
noor.hashem
2023-02-22 15:45
Here is a little more information, we using ruby on rails with a rails version of 4.2. I have pact in the Gemfile.Lock with a version of 1.32.0. I'm using the regular `bundle exec rake pact:verify` to try and verify pacts. In the Rakefile i've added : `require 'pact/tasks'`
My pact_helper.rb file looks like:
```require 'pact/provider/rspec'
require "./spec/pact/provider_states_for_order_item_categories"
Pact.service_provider "monolith" do
app_version "7c264ed3c871a8d34e7f9e56cd18aec441e0be06"
publish_verification_results 'true'
honours_pacts_from_pact_broker do
verbose true
pact_broker_base_url "https://<ORG>.http://pactflow.io", { token: "<READ/WRITE TOKEN>" }
end
end```
Then in the `spec/pact/provider_states_for_order_item_categories.rb`
I have:
```equire "rails_helper"
Pact.provider_states_for "testDemo_platform_payment_service" do
provider_state "All order item categories are present" do
set_up do
student = create(:student)
sign_in student
end
end
end ```
matt.fellows
2023-02-22 22:13
That gem predates PactFlow, so it?s not surprising it doesn?t support token based auth! :stuck_out_tongue:
noor.hashem
2023-02-22 23:06
Ohhh loool okay I see, so could I try using a newer Gem then maybe? But would that work with such an old version of ruby on rails
matt.fellows
2023-02-23 02:58
hmm that I couldn?t tell you, but I guess a `bundle update` will tell you how far off the mark you might be
bethskurrie
2023-02-23 03:12
It?s going to mostly depend what version of RSpec you?re running. It does not care about rails itself.
j3rry.wan9
2023-02-27 07:34
Hi, engineering teams at my company are using the https://github.com/pact-foundation/pact-ruby-cli (via its Docker image https://hub.docker.com/r/pactfoundation/pact-cli) for publishing pacts to PactFlow. However, a recent internal security audit flagged 10+ high or critical security vulnerabilities as part of the latest tag i.e. `pactfoundation/pact-cli:0.51.0.0` . The audit was done by running the https://github.com/aquasecurity/trivy security scanner and result can be easily reproduced locally. As the latest tag is published 3 months ago, I'm wondering if there is a plan to release a newer image to patch those vulnerabilities?
matt.fellows
2023-02-27 10:50
You?re best raising (and checking) an issue here: https://github.com/pact-foundation/pact-ruby-cli/issues
We also use trivy, so i?m surprised we haven?t caught that yet. Or maybe we have, and just haven?t looked at them yet
j3rry.wan9
2023-02-27 16:17
Thanks Matt! I created https://github.com/pact-foundation/pact-ruby-cli/issues/91 just now.
noor.hashem
2023-02-28 20:45
Hi there, are there any examples of how to set up a https://github.com/heartcombo/devise/blob/41e2db212065f2a75a5d73e76d6eb8ecd53a6acf/lib/devise/test/integration_helpers.rb#L37-L41 with pact ruby? Trying to use sign in but I think because pact is not of type request then it does not work.
In the rails_helper we have this line that is needed, i've tried using type to be pact but still no luck.
`config.include Devise::Test::IntegrationHelpers, type: :request`
This is the code I have atm:
```require "rails_helper"
Pact.provider_states_for "testDemo_platform_payment_service" do
provider_state "Order items exist" do
set_up do
user = User.create!(
username: "admin1",
email: "",
role: "admin",
password: "example",
password_setup: true,
blocked: false,
has_accepted_terms: false,
has_accepted_privacy: false,
)
sign_in user
end
end
end ```
nfinley
2023-02-28 21:03
Hello! We are using http://pacflow.io to house all of our consumer and provider contracts and we are seeing a really strange issue. The issue is we have an interaction listed as `???` and we can't click into it to delete it or anything (see image below). When we do we get a blank screen. Is there a way to delete this via api or otherwise? We'd really like to remove this from our dashboard. Thank you so much.
ilia
2023-02-28 23:11
Hi Nigel, thanks for letting us know, we will investigate and come back with a solution, would you be able to send a support request to and we can take it from there, thank you!
thomas.koppensteiner
2023-03-01 12:15
Hello,
I encountered a response-body validation error.
The server/producer defines the "400 Bad Request" response body as follows:
```responses:
error-400-bad-request:
description: ''
content:
application/vnd.api+json:
schema:
type: object
properties:
errors:
type: array
items:
type: object
properties:
source:
type: object
properties:
parameter:
type: string
pointer:
type: string
code:
type: string
enum:
- BAD_REQUEST
- UNPROCESSABLE_ENTITY
status:
type: string
enum:
- '400'
- '422'
details:
type: string
required:
- code
- status
required:
- errors
examples:
Example 1:
value:
errors:
- source:
parameter: id
detail: is in invalid format
code: BAD_REQUEST
status: '400'```
Example:
```{
"errors": [
{
"source": {
"parameter": "id"
},
"detail": "is in invalid format",
"code": "BAD_REQUEST",
"status": "400"
}
]
}```
The client/consumer used the following mocked response body:
```{
errors: [
{
status: "400",
code: "INVALID_QUERY_PARAMETER",
source: {
params: "foobar"
}
}
]
}```
It expected the undefined `params` instead of `parameters` , but pactflow didn't detect an error.
thomas.koppensteiner
2023-03-01 15:07
@bethskurrie, @matt.fellows is this a know issue or should I open a bug ticket somewhere?
pitas01_lope
2023-03-01 16:30
Hello, I have a provider which get verified against multiple consumers using the traditional way, a new API request has been added to represent a new consumer can I use Bi-directional contract testing for the newly developed API requests only?
bethskurrie
2023-03-01 22:12
99% of unexpected behaviour seems to be because of that being set incorrectly!
bethskurrie
2023-03-01 22:13
I haven?t done any work on the oas/pact verifier logic though (we?re currently using the swagger mock validator under the hood https://www.npmjs.com/package/swagger-mock-validator) so I?m not the best person to advise you on this.
bethskurrie
2023-03-01 22:41
I?ve just checked with the team. Apparently the additionalProperties is set by default now, so ignore that comment. Please do send an email to and provide the relevant pact and OAS.
iamchughmayank
2023-03-02 10:23
Hiya folks :wave:
I looking for some help with the webhooks in PactFlow.
The issue I am facing is that `contract_requiring_verification_published` is being fired even when there is no change in the contract from last ?latest? published contract
iamchughmayank
2023-03-02 10:23
This is what I am trying to do:
1. Consumer runs the contract tests
2. Consumer publishes the contract tests with `--consumer-app-version=${{git.sha}} --broker-token=${PACT_BROKER_TOKEN} --branch=${{ github.head_ref || github.ref_name }}`
3. Provider runs the verification by pulling `latest: true contracts`
4. Provider publishes the contract verification results with `providerBranch: ${PACT.mainBranch}`
iamchughmayank
2023-03-02 10:27
Post the initial verification,
1. consumer creates another branch and does not change the contract (some README changes)
2. Consumer publishes the contract tests again with the same params. The `consumer-app-version` changes as per new Git commit and `--branch` also changes
The following events are still fired:
`Events detected: contract_published, contract_requiring_verification_published, contract_content_changed (pact content has changed since previous untagged version)`
`Webhook "Contract verification required " triggered for event contract_requiring_verification_published.`
iamchughmayank
2023-03-02 10:32
I followed the documentation https://docs.pact.io/pact_broker/webhooks . As per the documentation:
> ? If the relevant consumer version has any tags, then for each tag, check if the content is different from the previous latest version for that tag. It is ?changed? if any of the checks are true. One side effect of this is that brand new tags will trigger a pact changed event, even if the content is the same as a previous version.
> ? If the relevant consumer version has no tags, then check if the content has changed since the previous latest version.
>
I was expecting the `contract_content_changed` event to not fire because even though I have not supplied any ?tags?, the content is same as the previous latest (and verified) pact.
iamchughmayank
2023-03-02 10:34
I intend to setup the webhook on `contract_requiring_verification_published` event to trigger the provider tests. However, since the event is fired even when the contract content is not changed, it defeats the purpose in my implementation.
iamchughmayank
2023-03-02 10:34
I am hoping to get some help in debugging where I could be wrong or what part I could be missing
iamchughmayank
2023-03-02 10:35
Another note:
the `mainBranch` for provider and consumer has been set as `trunk`. using `pact-broker create-or-update-pacticipant`. The last verified pact was from `providerBranch = trunk` and the content of consumer contract did not change in the next publish
matt.fellows
2023-03-02 11:53
I wonder if this statement needs to be expanded to include *branches:*
> If the relevant consumer version has any tags, then for each tag, check if the content is different from the previous latest version for that tag. It is ?changed? if any of the checks are true. One side effect of this is that *brand new tags will trigger a pact changed event, even if the content is the same as a previous version.*
matt.fellows
2023-03-02 11:54
@bethskurrie are you able to confirm if branches will behave like tags in this instance?
iamchughmayank
2023-03-02 12:25
Hey folks! I missed something while debugging this. It turns out that in my test, the sample response was using faker library to create the `id` field in the response.
It was an oversight on my part. My apologies :sorry:
However, this makes me think that using fixture generators to create the sample response could be a popular use case. I am sure a lot of teams would like to use the same response as test data in their unit/functional and contract tests to reduce maintainability.
I wonder how can Pact Broker be further improved to consider:
```Like({ id: integer(10) }) === Like({id: integer(11)})```
and not trigger `contract_changed` event. Or, maybe trigger `contract_changed` but does not trigger `contract_requiring_verification_published` event
matt.fellows
2023-03-02 12:26
It?s something we have discussed before, I think. You could raise a request at http://pact.canny.io and see where it lands.
matt.fellows
2023-03-02 12:27
The issue is at the moment, the Pact Broker is not that aware of the contents of a pact file (it?s not entirely true, but mostly true). It certainly isn?t aware of matching rules, generators etc.
It could be made to be aware, it?s just not at the moment
iamchughmayank
2023-03-02 12:46
Thanks, Matt! I will raise a ticket here :wink:
I have another question but I feel that it is more suited for #pact-js. :slightly_smiling_face:
dominik.chmielarz
2023-03-02 17:08
Hey everyone,
Is it possible to get API tokens of some other user through API or TF having sufficient permissions? I couldn't find it in the documentation.
matt.fellows
2023-03-02 23:11
I think it is possible via the APIs. We haven?t published our APIs yet, but you could probably see the API calls from the XHR calls in the UI
dominik.chmielarz
2023-03-03 08:04
Thanks, I'll look into it. Do you have any estimate when it will be implemented? Ideally in TF provider
yousafn
2023-03-06 14:22
not specifically but you can utilise any consumer side ruby examples to create a consumer contract (pact file)
from the provider side, assuming you have an OAS, you can utilise that plus the functional test results from your ruby provider code.
yousafn
2023-03-06 17:09
Do you use AsyncAPI in your organisation?
Would you like to be involved at an early stage, in the design of support for AsyncAPI as part of PactFlow's Bi-Directional contract testing offering?
If so, please drop us an :blob-wave:, a reply in a thread :thread: , or join our https://github.com/pactflow/roadmap#developer-preview-programs :laptop_parrot:
mwilso29
2023-03-07 15:20
Hi All. Why does my provider's can-i-deploy step say successful when the status of the broker is 'Unknown' awaiting the consumer contract to be generated and published to the broker? The provider side is uploading an openapi spec and newman verification results. In this scenario the consumer contract is invalid so should break my pipeline but its passing way before then. If I rerun this can i deploy step after the consumer has published and the status is failed in the broker, the can I deploy steps works as expected. Any ideas?
yousafn
2023-03-07 16:22
If you?ve deployed the provider, and then the consumer contract comes along and it is wrong, it would be picked up during the can-i-deploy check on the consumer side, stopping it from being deployed and triggering a conversation as to where the error lies
yousafn
2023-03-07 16:23
as per the recommended setup you should be
? uploading contracts with a branch
? using a version number associated with or including a git sha
? using the environment flag as suggested in the can-i-deploy check
mwilso29
2023-03-07 16:27
Would the same thing happen if the consumer was published to the broker with version 1 and the provider publish was version 2? Would the provider still be free to deploy?
mwilso29
2023-03-07 16:29
In my scenario I've deliberately made a breaking change to the provider (which causes issue with the consumer) at this instance it will go to production on the provider side and cause production issues for the consumer before any discussion can take place with the consumer side. What am I missing here?
yousafn
2023-03-07 16:30
you aren't using can-i-deploy correctly, you have not marked your provider as deployed to production and your screenshot shows no consumers
yousafn
2023-03-07 16:31
If your provider was marked as deployed to production, incompatible consumers would not be able to deploy to production.
if your provider and consumer were compatible and both deployed to production, a breaking provider change against a deployed or released consumer would stop the provider deploying.
yousafn
2023-03-07 16:32
I would recommend going through the bi-directional workshop
https://docs.pactflow.io/docs/workshops/bi-directional-contract-testing
if you haven't already
mwilso29
2023-03-07 16:37
Why does the provider side can i deploy work differently to the consumer? When I have no integration in the broker. I deploy my consumer pipeline which uploads the contract and calls the webhook to deploy the provider pipeline. The can-i-deploy step in the consumer pipeline waits as expected for the provider contract/openapi spec to be published and then verifies against it. All I have done above is the same scenario but starting from the provider pipeline and it acts differently
yousafn
2023-03-07 16:38
until the consumer comes along, and then it is bound by its agreement, If the provider was already in production, the consumer must firstly honour the production provider.
yousafn
2023-03-07 16:38
A provider team has no job to do, in consumer driven contract testing, until a consumer pact has been published
yousafn
2023-03-07 16:39
bi-directional contract testing, allows a provider first aspect, whereby you may be designing your API via OpenAPI, and want Pact to protect you with can-i-deploy whilst you wait for consumers to come onboard
yousafn
2023-03-07 16:40
As you aren't publishing provider contracts with a branch, nor have them marked as deployed to an environment your can-i-deploy checks aren't deterministic (as they take the latest pact, rather than latest pact from a main branch - configurable, or the latest deployed/released)
yousafn
2023-03-07 16:46
Any time buddy, that's what we are here for! I would try out the workshop and let us know how you get on, in your own time
mwilso29
2023-03-07 16:48
I will take another look but I think with what you've described above I am very close to a working POC to move forward and demo to my team thanks
yousafn
2023-03-07 16:53
Awesome, good luck! Interested to hear your teams thoughts, and happy to work to iron out any reservations!
I assume your based in the UK due to the timezone? I'm over in Leeds
hazem
2023-03-07 17:03
hello - i need some assistance with pactflow. We're introducing a new integration and the provider main branch cannot detect any `for-verification` pacts, however we have a pending pact to be verified on a feature branch for both the consumer and provider.
hazem
2023-03-07 17:13
I've tried to debug myself and I can verify the HAL browser is not returning any `for-verification` results using these steps. Env is pact-go 2.x.x and using 3.0 spec.
Steps to debug:
1. pact-go provider-verification, set log level to `DEBUG`, this gave the output:
``` Sending JSON to https://[redacted].http://pactflow.io/pacts/provider/[Redacted]Service/for-verification using POST: {"providerVersionTags":[],"includePendingStatus":true,"includeWipPactsSince":"2022-04-15T00:00:00Z","consumerVersionSelectors":[{"mainBranch":true},{"matchingBranch":true},{"deployedOrReleased":true}],"providerVersionBranch":"main"}```
2. Posted in the HAL Browser and received no results
3. Confirmed in the HAL Browser that pacts exist,
```https://[redacted].http://pactflow.io/pacts/provider/[Redacted]Service```
bethskurrie
2023-03-07 21:51
> @bethskurrie are you able to confirm if branches will behave like tags in this instance?
Wow. This is greatly embarrassing. I think that logic needs updating for branches.
bethskurrie
2023-03-07 21:53
I?m amazed and appalled that I haven?t noticed this before. I?ve made an issue here https://github.com/pact-foundation/pact_broker/issues/602
matt.fellows
2023-03-07 22:16
Hi Haz, could you please raise a ticket at ? We?ll need the specific provider details and of course your account so we can check for you.
Just in case, the logic for detecting ?wip? is outlined here: https://docs.pact.io/pact_broker/advanced_topics/wip_pacts#technical-details. Worth double checking that the pact you expect to verify matches that criteria
hazem
2023-03-07 22:16
This is for documentation purposes on this workflow I follow. FWIW it feels like a regression since this has been how I've introduced new consumer driven pacts in the past.
1. Define a new pact for consumer "FooService" on consumer branch `feature/fun-123` with a new provider "BarService".
a. BarService does not have any real pacts on main branch
2. Use CI to publish the pact with `branch=feature/fun-123`
3. New integration exists between FooService :pact-open-link: BarService
a. The newly published pact in unverified, so I'd call this `pending`
4. BarService implements the verification on like named branch, eg `feature/fun-123`
5. Provider CI executes on feature branch
a. Provider Verification Configuration
i. ConsumerSelectors
1. mainBranch = true
2. matchBranch = true
3. deployedOrReleased = true
ii. IncludePendingStatus = true
iii. IncludeWipPactsSince = yep
b. `for-verification` api returns the new pact
c. Verification is success.
d. can-i-deploy is a success
6. Provider merge to main
a. Repeat 5a and 5b on main branch
i. `for-verification` does not find any pacts to verify
b. can-i-deploy fails due to no successful verifications on main branch
At 5c, I would think the pact although first verified is still consider pending and therefore should be returned in `for-verification` api given the provider verification configuration. Even if it's WIP at this point it is still pending or at least I thought.
I think my work-around is to delete the verification on the initial provider feature branch to put the pact back in unverified. I did this with a clone existing and used the same query on a provider main branch and was able to get back a pact requiring verification.
bethskurrie
2023-03-07 22:59
Oh! I remember now. The branches work with the new event, `contract_requiring_verification_published`
hazem
2023-03-07 23:18
Curious as to how/when pacticipant gets their main branch set. Eg, I noticed another provider recently onboarded has this set using the cli describe-pacticipant
```Main Branch: main```
abubics
2023-03-08 00:31
To put it another way (in case it helps), the consumer-provider pairing is a direction dependency link. The consumer strictly depends on the provider (needs a compatible one to exist). The provider doesn't care if it has no consumers.
hazem
2023-03-08 01:31
Interesting, so had my main branch verification found a for-verification pact and verified then this would get set.
matt.fellows
2023-03-08 01:35
> Interesting, so had my main branch verification found a for-verification pact and verified then this would get set.
only on the provider
To set it on the consumer, it?s when the contract is published (or if explicitly set)
matt.fellows
2023-03-08 01:36
Once the ticket hits the L3 team they?ll dive into the data for you on the PactFlow side
mwilso29
2023-03-08 11:48
@yousafn Yeah I am based in Manchester. Just demoed the bi directional stuff to our teams today actually haha. Think my line manager will most likely be in touch to arrange for you guys to do a talk in the near future if possible
marckpuiu
2023-03-08 13:10
Hi. I posted a message on #pact-js and I got the suggestion to ask you if you are still using pact + cypress after pact-web was deprecated? Here is the question for reference https://pact-foundation.slack.com/archives/C9VBGLUM9/p1678280003634509 . Thank you
yousafn
2023-03-08 15:19
Oh :nice: !
I used to live in Manc but hopped across the pennines now to Leeds. Sure thing, we've got some spot on solution engineers to help you guys out!
matt.fellows
2023-03-09 00:48
Our Cypress tests are actually still on 9.x.x, but we only need the extract payload function
michael.branders484
2023-03-13 13:45
I would like to try out the bi directional function of pactflow. I generate my OAS with annotations in my code so i do not have a YAML or JSON file to upload as this is generate on the swagger endpoint when the app is running. How do i then upload the correct OAS spec to pactflow. It is shown how to do it if you have a OAS spec first but not how you do it code first and document the spec in your code. Any help?
yousafn
2023-03-13 13:50
you should be able to build your code and generate the OAS file as an artefact.
If not you could spin your app up, grab the payload from the swagger endpoint
yousafn
2023-03-13 13:52
We mention it in our guide here
https://docs.pactflow.io/docs/workshops/bi-directional/step3#verifying-the-provider-contract
> 1. Generate the OAS from code. This is the most reliable, because whenever the implementation changes, the OAS will change with it. Tools like Spring Docs (Java) and Swashbuckle (.NET) will do this for you.
The actual mechanism depends on your tool
michael.branders484
2023-03-13 15:06
I see an example for gitaction https://github.com/pactflow/actions/tree/main/publish-provider-contract
Are there plans to add support for azure devops and pipelines there?
yousafn
2023-03-13 15:51
It's not on our roadmap but you can add it as an issue if its something you want to request https://github.com/pactflow/roadmap/projects/1
it should be trivial, as it simply wraps the CLI tool (which we advise using directly)
Those actions were created by a community member @roy_collings (which we are very grateful for!) and transferred over, so we would happily support someone who wanted to.
sugi
2023-03-13 21:15
I was looking for github workflow example to publish & run can-i-deploy...Found this: https://github.com/pact-foundation/pact_broker-client/blob/master/.github/workflows/test.yml
michael.branders484
2023-03-14 07:33
Well examples like that is what making implment the flows easyer. Now its looking stuff up online on how to get it working. Dont get me wrong, your documentation is very good but it still is lacking on some parts like how to implement the full ci/cd flow.
yousafn
2023-03-14 12:11
the more examples we build the greater the cost of maintainence. everyone?s ci process is slightly different so we aim on teaching the general concepts, and as part of ci cd workflows albeit with github actions, the principles for the recommended workflow will be largely the same in a ci agnostic way.
if you wish to get recommendations for your current pipeline and deployment process, please feel free to provide details and we can assist where possible :)
yousafn
2023-03-14 12:12
the ci/cd workshop for consumer driven contract testing is here
https://docs.pactflow.io/docs/workshops/ci-cd
yousafn
2023-03-14 12:13
bi directional workshop including ci/cd
https://docs.pactflow.io/docs/workshops/bi-directional-contract-testing
tjones
2023-03-16 05:08
Two cool things about this google search:
1) Pactflow?s article about CDC is above Martin Fowler?s original article
2) `consumer-driven contract testing pact` is the top link in ?people also search for?
tjones
2023-03-16 05:13
Correction: Martin Fowler didn?t write that article(!) it?s just on his website. I have been attributing this wrong *for years*!
tjones
2023-03-16 05:14
Update: There are two Martin Fowler contracts articles there. Here?s one he did write, which isn?t the one in the screenshot:
https://martinfowler.com/bliki/ContractTest.html
yousafn
2023-03-16 08:16
> Correction: Martin Fowler didn?t write that article(!) it?s just on his website. I have been attributing this wrong *for years*!
yes! that would be Ian Robinson from Thoughtworks at the time that wrote in 2008! i am pretty sure it was Thoughtworks who ran with Pacto, which generally pre dated my Pact experience which started about 2017 i think
tjones
2023-03-16 14:23
> Pacto
I?d never heard of this! Here?s a https://github.com/thoughtworks/pacto. Looks like they retired it and recommend Pact instead :heart:
gsouza
2023-03-16 18:42
hey #pactflow
i'm facing this *Mismatches message error* after run the provider tests:
```Body:
$: Expected body type of 'application/json' but received 'text/html'
Header:
Content-Type: Mismatch with header 'Content-Type': Expected header 'Content-Type' to have value 'application/json' but was 'text/html'```
where should I manage that? at service? at provider file?
gsouza
2023-03-16 19:07
is this issue related and still in progress?
https://github.com/pact-foundation/pact-js/issues/575
matt.fellows
2023-03-17 23:13
The problem is your provider is responding with HTML and not json. The issue is with your provider or your test is wrong (E.g. is hitting the wrong endpoint)
matt.fellows
2023-03-20 02:06
My theory is that it was some clone/copy of when Pact was created. If you recall, Pact was created at REA where a bunch of TW?s worked - including some on the initial implementation! TW were always very supportive/proponents of Pact, so I wonder if this was a nod to it. It would be a little too coincidental that another contract testing tool cropped up just after Pact, with almost the same name from a company that had a hand/early visibility on the other
matt.fellows
2023-03-20 02:08
Back to the point - thanks Tim! I?m in two minds about that article. I really want to improve it and make it the tome it needs. But I think part of the value is that it?s short and digestible in terms of length. We probably need both
matt.fellows
2023-03-20 12:35
sorry Sasank, that?s really hard to see. It looks like you have some error with the OAS rendering? does it render in https://editor.swagger.io/? That?s usually a good litmus test for valid OAS files
vika.karman
2023-03-20 14:12
hello! I have a question regarding webhook configuration in pactflow.
What is Authentication field for? As I understand, if I specify `Authorization` header with proper token it should be enough...
with Authentication setup with my creds I get 404 when I test my webhook.
without it I get 401
Could you please help me understand this?
NOTE: token is fine, webhook works via curl
gsouza
2023-03-20 19:09
hey @matt.fellows i have checked and i'm hitting the correct endpoint, the error is related to *Mismatches*
I updated the service and consumer to send the request without Content-Type and Accept, it worked and I can run the consumer and then the publisher step!
However the provider step is still failing for one of my three services that i have and is about the Content-Type which looks confusing because i'm not sending this header at all and in the consumer spec i'm not expecting for any header as well
gsouza
2023-03-20 19:25
looking at terminal i could see this:
```2023-03-20T19:18:33.500771Z INFO ThreadId(12) pact_verifier::provider_client: Sending request HTTP Request ( method: POST, path: /dynamicApi, query: None, headers: Some({"Content-Type": ["application/json"], "Authorization": ["Bearer undefined"]}), body: Present(523 bytes) )
2023-03-20T19:18:34.129861Z INFO ThreadId(12) pact_verifier::provider_client: Received response: HTTP Response ( status: 200, headers: Some({"connection": ["close"], "x-powered-by": ["Express"], "x-amz-version-id": ["12OAnMDcwjX61drz1pjWwYOPF4..klSN"], "via": ["1.1 http://e8eec15d9551dd475d4c478f9fbb5f04.cloudfront.net (CloudFront)"], "x-amz-cf-pop": ["IAD12-P3"], "accept-ranges": ["bytes"], "last-modified": ["Tue", "14 Feb 2023 20:52:17 GMT"], "x-amz-cf-id": ["Xjw15a5yV5gWhs_wHi8mGd67o4fbSCh4kzF5t1D-lJesk-aiymZ29g=="], "x-cache": ["Error from cloudfront"], "date": ["Mon", "20 Mar 2023 19:18:38 GMT"], "x-amz-server-side-encryption": ["AES256"], "server": ["AmazonS3"], "etag": ["\"f522e4542a43a286da7783b5f70d983f\""], "content-type": ["text/html"], "content-length": ["37937"]}), body: Present(37937 bytes, text/html) )```
*""x-cache": ["Error from cloudfront"]"*
matt.fellows
2023-03-20 21:23
A 404 suggests you are authenticating ok, but the resource you're trying to reach is not found
anshu.ranjan1002
2023-03-20 21:32
Hi all, I was able to publish Open API spec to my (trial) broker, but now when I publish it, in the UI, I see this error: The requested document was not found on this server.
Even though on the command line, it says successfully published. Could someone help pointing the issue here? Thank you in advance!
matt.fellows
2023-03-20 22:10
Hi Anshu, what do you mean by ?trial? - is this an on-prem trial or cloud trial?
matt.fellows
2023-03-20 22:11
> error: The requested document was not found on this server.
could you please elaborate a bit more on this? What URL are you on when you see this error?
anshu.ranjan1002
2023-03-20 22:47
This is a cloud trial if I understand correctly. I suppose the on-prem trial will be the one inside the compang
anshu.ranjan1002
2023-03-20 22:48
On the cmd line it says: Successfully published provider contract for pactflow-example-bi-directional-provider-postman version 215d0f1-master+215d0f1.SNAPSHOT.LAPTOP-CG5C2KOB to Pactflow
anshu.ranjan1002
2023-03-20 22:48
But I do not see the contract in the UI, it instead gives the error I mentioned
matt.fellows
2023-03-20 23:50
Can you please share more about how you have published the contract? It looks like some key info is missing. Also, note the version is `undefined`
anshu.ranjan1002
2023-03-20 23:51
> I did this:git clone https://github.com/pactflow/example-bi-directional-provider-postman.git
anshu.ranjan1002
2023-03-20 23:52
root@LAPTOP-CG5C2KOB:/home/n/workspace/example-bi-directional-provider-postman# echo $PACT_BROKER_BASE_URL
https://jpmc.pactflow.io
matt.fellows
2023-03-20 23:54
I?d like to see the log output though please (if you can redact any secrets)
anshu.ranjan1002
2023-03-21 00:09
Here is the log output:
https://gist.github.com/aranjan1002/75c5c4b167c5aab0082298930c2d10f4
matt.fellows
2023-03-21 00:34
any chance you could please add the `--verbose` flag to the publish command?
```publish_provider_contract: .env
@echo "\n========== STAGE: publish provider contract (spec + results) ==========\n"
${PACTFLOW_CLI_COMMAND} publish-provider-contract \
${OAS_PATH} \
--provider ${PACTICIPANT} \
--provider-app-version ${GIT_COMMIT} \
--branch ${GIT_BRANCH} \
--content-type application/yaml \
--verification-exit-code=${EXIT_CODE} \
--verification-results ${REPORT_PATH} \
--verification-results-content-type ${REPORT_FILE_CONTENT_TYPE}\
--verifier ${VERIFIER_TOOL}\
--verbose```
matt.fellows
2023-03-21 00:47
thanks. One theory is that it?s something strange in the version number (there is a `%` , it should be escaped, but I wonder if that?s the issue)
matt.fellows
2023-03-21 00:47
see if you can hard code `GIT_COMMIT` in the make file to something like `1.0.0`
matt.fellows
2023-03-21 00:57
No, I mean these:
```GIT_COMMIT?=$(shell git rev-parse --short HEAD)
GIT_BRANCH?=$(shell git rev-parse --abbrev-ref HEAD)```
anshu.ranjan1002
2023-03-21 01:01
I do not see these lines in Makefile: https://raw.githubusercontent.com/pactflow/example-bi-directional-provider-restassured/master/Makefile
matt.fellows
2023-03-21 01:03
wait, you said the example you were using was postman: https://pact-foundation.slack.com/archives/CLS16AVEE/p1679356307896119?thread_ts=1679347977.841209&cid=CLS16AVEE
anshu.ranjan1002
2023-03-21 01:05
My bad, I pasted the wrong URL. But it is not there in postman makefile either
anshu.ranjan1002
2023-03-21 01:14
I have to go out in a bit...
Please leave the message regarding your findings.
matt.fellows
2023-03-21 02:20
looks like the version is the issue, I suspect the `%` is causing the problem. It?s a UI only bug, because the `can-i-deploy` check works as you?d expect. I?ll raise a defecct
matt.fellows
2023-03-21 05:00
The problem is that you have characters that are not URL escaped (In your case, my guess is `+`). Can you please try a scheme that doesn?t include those for now
j3rry.wan9
2023-03-21 08:12
Hi, how can I go about debugging incompatible contract comparison when using BDCT? I'm facing the following issues:
matt.fellows
2023-03-21 09:27
Hi! Please check this article on `allOf` support first: https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support
cyrus.devnomad
2023-03-21 13:24
I have a consumer deployed to environment env1. The consumer (ver-x, branch-x) contract expects a GET query like this:
```{
"param1": [
"2"
],
"param2": [
"abc"
]
}```
I have a bi-directional setting between my provider and my consumer.
I have intentionally changed the provider (ver-y, branch-y) API to not be compatible to the consumer (ver-x, branch-x) contract by offering only the parameter param1.
But when I publish the provider swagger.json file, the interaction consumer contract (ver-x, branch-x) vs provider (ver-y, branch-y) is shown as compatible and green. And when I execute can-i-deploy provider (ver-y, branch-y) to env1, the command passes and allows me to deploy the provider.
Shouldn't a provider API which misses some of the parameters the consumer expects, be considered as incompatible to that consumer? What could be the reason why my interaction is evaluated as compatible by Pactflow?
anshu.ranjan1002
2023-03-21 16:09
Oh right, when I removed the + and - from the version, it worked fine. Thank you for the help.
anshu.ranjan1002
2023-03-21 16:09
I like that I can now communicate on Slack. So I will be posting any questions that I may have over here.
vika.karman
2023-03-21 20:07
once I left Authentication values blank and left only github token in headers it started to work. But now at my company I am told I cannot store token in pactflow. They say I should use github app..
matt.fellows
2023-03-22 02:58
Hi Cyrus,
Currently this is considered a ?warning? but that warning is not shown in the PactFlow UI. It is because query parameters are a special case.
The original authors of the tool we use in the BDCT workflow reasoned as follows:
> The reason we didn?t do this in the first place is it becomes a blocker for fast 5 style changes where the consumer deploys first. For example if we wanted to add a new query param to an API in a world where unknown query params were disallowed then the provider has to change the Swagger file as the first step. Our thinking was that in general more teams would prefer the flexibility to deploy in any order over the strictness.
We have created a tracking item to revisit this decision, in light of this report and real-world usage.
I?m assuming you have found this behaviour surprising?
Might I ask a follow up question. In your case, what is the impact of the provider _not_ supported that query param? Does it simply filter the response or does it change the behaviour of the API in a way that would result in breaking the client?
matt.fellows
2023-03-22 03:22
FYI we?ve raised a bug and it will be fixed in the next sprint or so. Thanks for drawing attention to it, i?m surprised it wasn?t picked up when the new versioning tool was used. It obviously works on the server side, so that is probably why it wasn?t noticed.
matt.fellows
2023-03-22 03:22
If we ever take too long to respond or miss the question and it?s a bug, do raise with
matt.fellows
2023-03-22 03:23
No worries. We are looking at automatically inlining `allOf` in a future release. See also https://github.com/pactflow/roadmap/issues/85
vika.karman
2023-03-22 09:44
@matt.fellows what approach would you advice as an alternative to webhook configured in pactflow with a token stored in pactflow?
matt.fellows
2023-03-22 09:46
Hmm my only suggestion would be to build an app outside of Pactflow that can proxy webhook executions from Pactflow to then communicate to your secure targets
vika.karman
2023-03-22 10:11
yeah....this is the way I am thinking of...I thought maybe there is something more simple))) because it looks like contract testing is turning out to be more complicated than the service creation :)))
matt.fellows
2023-03-22 10:17
We do have longer term plans to create a first class SCM/Github integration, but that is unlikely to happen in the timeframes you?ll want it
matt.fellows
2023-03-22 10:18
What?s the reason behind not being able to store the token in PactFlow? Actually, I realised I assumed github but that was probably anohter conversation. What is the target here?
vika.karman
2023-03-22 10:20
I guess they don't want to store a github token in pactflow due to security reasons.
matt.fellows
2023-03-22 10:26
OK, so github have pretty good tokens - you can make them fine grained so that they only work for a given repo, with a given set of perms (e.g. triggering a build, updating statuses).
But understandable, if that?s the case we can?t do much about that for you sorry!
tjones
2023-03-22 14:06
No, that means that Cloudfront has cached an S3 response error. I don?t know why it?s coming back as a 200 - I would guess some configuration error in the way cloudfront is set up on your provider, causing it to mis-report errors.
tjones
2023-03-22 14:07
My guess is that your test is not set up correctly, causing there to be no authentication token (perhaps this `undefined` made it in to the contract during the consumer test)
tjones
2023-03-22 14:13
> confusing because i?m not sending this header at all
You are sending a content-type header, because http requires it if you have a body. Pact knows this, and will match on the content types. As Matt said above, you?re experiencing errors here because the content that is returned by your provider during the test is text/html (as you can see in your snippet).
fabio.rodrigues
2023-03-22 14:19
Hey guys, quick question
We're testing 2 services using bi-directional contratc tests.
One of them is keycloak and another one is a external service (in that it is outside of the service cluster).
Due to that some address that the external service access need to be 'translated' to internal ones. case in point, the external service has no definition of realmid that Keycloak uses in all endpoints.
The external service uses `<tenantname>.<company>.com/.well-know/configuration` but keycloack expects a `realms/<realmid>/.well-know/configuration`. Thus the formats of urls are different (although they end in the same endpoint).
We have a network address table that moves the request to the correct one (translates `<tenantname>.<company>.com/.well-know/configuration` to the `realms/<realmid>/.well-know/configuration`) . Is there any way to make this pass with bi-directional contract tests (Keycloak is an external component we don't control) and the external service don't has any initial access to know the realmid.
yousafn
2023-03-22 14:31
Hey., there is no current capability built into PactFlow.
You could transform your OAS prior to uploading to PactFlow, or your pact file. The OAS would probably be easier (and safer) to parse/transform with existing tooling
You could additionally annotate your document to state that these have been rewritten from x source
cyrus.devnomad
2023-03-22 14:33
Hi Matt, thank you for your explanations.
Do I understand you correctly: You refer to query parameters as a special case among the following ways parameters can be passed from a consumer to a REST API provider: header parameters, cookie parameters, path parameters, query parameters, body parameters, matrix parameters, form parameters, and plain parameters?
Yes, I found this behavior surprising because if I remember correctly, the last time I was experimenting with Pact tests involving missing parameters on the provider side, Pact would not accept such cases. I'm pretty sure this was the case in a consumer-driven test setting. However, I was under the impression that I also saw the same behavior in a bi-directional test setting a few months ago, but I'm not sure about that. I might be hallucinating.
But in our case, if the consumer were to send a request with two parameters to the provider that only supports the first parameter, the provider, as it is right now, would return error 500. I guess that wouldn't be the filtered response you were referring to. However, our system is in the prototype stage, and I don't think the behavior of our provider could be seen as particularly mature at this stage.
gsouza
2023-03-22 17:46
> `"Authorization": ["Bearer undefined"]` <-- this is very suspicious
sorry but i don't know the reason for that, i'm sending the token at Auth header and i saw that but i couldn't figure out the reason!
the confusing thing is that for those others services that i have works correctly and specially for this one don't
Anyway i'll review each file and each test
anshu.ranjan1002
2023-03-22 18:23
Good to know. I was told that the primary mode for communication should be what company provides. So I will be using mostly the email for now.
gsouza
2023-03-22 18:27
> My guess is that your test is not set up correctly, causing there to be no authentication token (perhaps this `undefined` made it in to the contract during the consumer test)
if was the case i shouldn't see any FAIL message from pact, right? I mean after run the consumer tests i should at least see a warning or failed result
here is the output after run the consumer tests:
```2023-03-22T17:59:01.165083Z INFO tokio-runtime-worker pact_mock_server::hyper_server: Received request HTTP Request ( method: POST, path: /dynamicApi, query: None, headers: Some({"host": ["127.0.0.1:53884"], "content-length": ["523"], "content-type": ["application/json"], "accept": ["application/json", "text/plain", "*/*"], "accept-encoding": ["gzip", "compress", "deflate", "br"], "user-agent": ["axios/1.3.1"], "authorization": ["Bearer undefined"], "connection": ["close"]}), body: Present(523 bytes, application/json) )
2023-03-22T17:59:01.165192Z INFO tokio-runtime-worker pact_matching: comparing to expected HTTP Request ( method: POST, path: /dynamicApi, query: None, headers: Some({"Content-Type": ["application/json"], "Authorization": ["Bearer undefined"]}), body: Present(523 bytes, application/json) )2023-03-22T17:59:01.166133Z INFO tokio-runtime-worker pact_mock_server::hyper_server: Request matched, sending response HTTP Response ( status: 200, headers: Some({"Content-Type": ["application/json"]}), body: Present(3247 bytes, application/json) )2023-03-22T17:59:01.169841Z WARN ThreadId(02) pact_models::pact: Note: Existing pact is an older specification version (V3), and will be upgraded```
@tjones
gsouza
2023-03-22 19:11
> You are sending a content-type header, because http requires it if you have a body. Pact knows this, and will match on the content types. As Matt said above, you?re experiencing errors here because the content that is returned by your provider during the test is text/html (as you can see in your snippet).
i get that but i'm not getting understanding the reason why is turning to text/html after run the provider tests....
here is my request at pactflow / expected response
tjones
2023-03-22 21:22
Your provider is returning text/html, not pactflow.
It is probably doing that because your consumer test has a broken auth header
matt.fellows
2023-03-23 00:28
I?m sorry Sasank, we?ll need a wider view (not just visually, but information wise) to be able to help here.
Do you have the OAS or a minimal version of an OAS that reproduces that problem?
matt.fellows
2023-03-23 00:31
Just for clarity, the address is really for issues that customer care can help with (account issues, product bug reports etc.). Your account manager might be able to help with certain queries, but for general Pact/PactFlow advice this is the community for that.
matt.fellows
2023-03-23 01:48
Thanks for the explanation, this will be helpful as we consider the implementation
matt.fellows
2023-03-23 01:51
I refer to it as a special case, in that query parameters aren?t really seen as the body of a request or part of a resource itself (i guess it?s in the name ?query?), so the original authors made some concessions about how concerned they should be about differences in consumer vs provider behaviour.
They erred on the side of ?we only want to fail a check if it?s _definitely_ going to break things? instead of ?might?. I think we ought to sway towards ?we only want to say it?s safe it we are _as sure as we can be_ that it won?t break things?
cyrus.devnomad
2023-03-23 09:42
Than you very much for your explanations. By the way, is there any way to see/get that warning you mentioned before?
matt.fellows
2023-03-23 09:45
It's on that same internal tracking issue for us to review how to convey warnings (and indeed check the warnings it produces)
anshu.ranjan1002
2023-03-23 18:04
I see, if I understand correctly, for Pact/Pactflow advice I should direct email to . Is that right? I am not sure who is my account manager, but I am new the team so Rejeesh might know.
anshu.ranjan1002
2023-03-23 21:54
I just posted multiple questions regading birectional testing to the support email and cc'ed it to your email.
matt.fellows
2023-03-24 00:51
> I see, if I understand correctly, for Pact/Pactflow advice I should direct email to . Is that right? I am not sure who is my account manager, but I am new the team so Rejeesh might know.
Jason Stahl is your account manager.
But more generally, I would use this community for general ?advice? and use the email address for technical issues/bug reports etc.
tjones
2023-03-24 13:12
```const token = process.env.ACCESS_TOKEN;```
^ This might be undefined. In general, you shouldn?t rely on an environment variable during tests. More importantly, you shouldn?t write a real access token into the contract.
Usually, you would do something like `const token = "someToken"` and then have a state like `"someToken is a valid token"` and stub out your authentication. Alternatively, you can use `fromProviderState` to get the token from the provider state, and use a dummy one during the contract definition.
tjones
2023-03-24 13:12
To be clear, that line might be undefined - and definitely was at the time you wrote the contract that you?re having trouble verifying.
anshu.ranjan1002
2023-03-24 15:32
I was wondering when should I expect to hear back for my questions? Is there an SLA?
matt.fellows
2023-03-27 02:37
Hi Anshu, I don?t believe there are SLAs attached to support tickets but usually I would expect a response within 1-2 business days.
gsouza
2023-03-27 14:44
@tjones I've update my consumer tests with a token valid just to check the results at pactflow and here is the result:
even using a valid token i'm still facing the *Mismatches* error:
```Body:
- Expected body type of 'application/json' but received 'text/html'
Header:
- Content-Type: Mismatch with header 'Content-Type': Expected header 'Content-Type' to have value 'application/json' but was 'text/html'```
as you can see below i've sent the token instead undefined
matt.fellows
2023-03-27 22:10
So PactFlow is just showing you what the provider test got. You?ll need to look at the logs of your provider test build to see why you are getting this `text/html`. It seems like it?s returning an http 200 because there are no errors about the status (that seems concerning to me also) but it?s definitely getting back something that?s not JSON. Perhaps you could share your provider test? Are you running it against a live system or a local build?
matt.fellows
2023-03-29 01:20
Thanks. A few things
1. You seem to be using a fixed environment. This will make supporting provider states difficult, as well as authorization. You will need to either hard code credentials that don?t expire into the consumer test, or I can suggest you consider using a `requestFilter` to inject one a correct one at runtime (see Step 10 of https://github.com/pact-foundation/pact-workshop-js/)
2. You shouldn?t specify `pactUrls` to a hard coded `latest`. You should use consumer version selectors for this purpose. I would highly recommend running the above workshop and also consider the CI/CD workshop (see howtolearn :point_down: )
2023-03-29 01:20
Here are a number of useful hands-on labs that teach all of the key concepts: https://docs.pactflow.io/docs/workshops and https://docs.pact.io/implementation_guides/workshops
tjones
2023-03-29 01:42
Also, it?s best not to hard code your pactBrokerToken (it should come from an environment variable), and it?s also not good to share it on slack - you may want to change it now.
tjones
2023-03-29 01:43
Putting your access token in the contract isn?t what I advised - you should follow the auth approach in the docs. Probably what is happening is that your auth token is not valid at the time of the verification.
tjones
2023-03-29 01:43
The error you have looks like it comes from cloudfront - I would start there looking for the misconfiguration
matt.fellows
2023-03-29 07:05
You can?t have a request body for a GET. Once you change that to e.g. a `POST` there are other errors. I?d use http://editor.swagger.io as a guide to get it right. Once it properly renders and validates there, you can try uploading to PactFlow.
PactFlow uses a similar UI library component (but doesn?t have the advanced validation/error messages of SwaggerHub).
sasankdts
2023-03-29 07:41
I did validated and no errors displaying now but displaying No API definition provided after uploading to pactflow
gsouza
2023-03-29 19:11
I'll review and if necessary apply your thoughts, thanks!
I do thing is something related to cloudfront as well @tjones but i'm not aware how to investigate that....
@matt.fellows how can i grab or generate the output log?
bethskurrie
2023-03-30 00:47
@hazem sorry it?s taken so long to get back to you on this. I?ve reproduced your scenario, and this is what the WIP explanation logs say:
bethskurrie
2023-03-30 00:47
```2023-03-30 11:42:44.975904 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Potential WIP pacts for provider branch main created after 2020-01-01T00:00:00+00:00 by consumer branch -- ["FooService 2 (branch feature/fun-123)"]
2023-03-30 11:42:44.976124 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Potential WIP pacts for provider branch main created after 2020-01-01T00:00:00+00:00 by consumer tag (none)
2023-03-30 11:42:44.976474 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Ignoring pacts explicitly specified in the selectors (none)
2023-03-30 11:42:44.976793 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Ignoring pacts successfully verified by this provider branch when not WIP (none)
2023-03-30 11:42:44.977383 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Ignoring pacts successfully verified by another provider branch when not WIP -- ["FooService 2 (branch feature/fun-123)"]
2023-03-30 11:42:44.978735 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Ignoring pacts explicitly specified in the selectors (none)
2023-03-30 11:42:44.979048 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Ignoring pacts successfully verified by this provider branch when not WIP (none)
2023-03-30 11:42:44.979434 D [37431:13260] [wip] PactBroker::Pacts::PactsForVerificationRepository -- Ignoring pacts successfully verified by another provider branch when not WIP (none)```
bethskurrie
2023-03-30 00:48
It is NOT considered WIP on your main branch because you successfully verified it with an explicit selector on the matching branch.
bethskurrie
2023-03-30 00:48
```#!/usr/bin/env ruby
# The content_id is used to deterministically generate pact content. It doesn't matter what the value is.
# If you want to simulate publishing a pact that has the same content as a previous pact, use the same ID.
# If you want to simulate publishing a pact with different content, use a different ID.
#!/usr/bin/env ruby
# The content_id is used to deterministically generate pact content. It doesn't matter what the value is.
# If you want to simulate publishing a pact that has the same content as a previous pact, use the same ID.
# If you want to simulate publishing a pact with different content, use a different ID.
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem "faraday"
end
begin
$LOAD_PATH << "#{Dir.pwd}/lib"
require "pact_broker/test/http_test_data_builder"
base_url = ENV["PACT_BROKER_BASE_URL"] || "http://localhost:9292"
CONSUMER = "FooService"
PROVIDER = "BarService"
td = PactBroker::Test::HttpTestDataBuilder.new(base_url)
td.delete_pacticipant(CONSUMER)
.delete_pacticipant(PROVIDER)
.create_pacticipant(CONSUMER, main_branch: "main")
.publish_contract(consumer: CONSUMER, consumer_version: "2", provider: PROVIDER, content_id: "111", branch: "feature/fun-123")
.get_pacts_for_verification(
provider: PROVIDER,
enable_pending: true,
provider_version_branch: "feature/fun-123",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ mainBranch: true }, { matchingBranch: true } , { deployedOrReleased: true }]
)
.verify_pact(
index: 0,
provider_version_branch: "feature/fun-123",
provider_version: "2",
success: true
)
.can_i_deploy(pacticipant: PROVIDER, version: "2", to_environment: "test")
.get_pacts_for_verification(
provider: PROVIDER,
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ mainBranch: true }, { matchingBranch: true } , { deployedOrReleased: true }]
)
rescue StandardError => e
puts "#{e.class} #{e.message}"
puts e.backtrace
exit 1
end```
bethskurrie
2023-03-30 00:49
If I remove the `{ matchingBranch: true }` selector, and only rely on WIP to bring in that pact, then I still get the pact as WIP on the main branch.
bethskurrie
2023-03-30 00:50
```#!/usr/bin/env ruby
# The content_id is used to deterministically generate pact content. It doesn't matter what the value is.
# If you want to simulate publishing a pact that has the same content as a previous pact, use the same ID.
# If you want to simulate publishing a pact with different content, use a different ID.
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem "faraday"
end
begin
$LOAD_PATH << "#{Dir.pwd}/lib"
require "pact_broker/test/http_test_data_builder"
base_url = ENV["PACT_BROKER_BASE_URL"] || "http://localhost:9292"
CONSUMER = "FooService"
PROVIDER = "BarService"
td = PactBroker::Test::HttpTestDataBuilder.new(base_url)
td.delete_pacticipant(CONSUMER)
.delete_pacticipant(PROVIDER)
.create_pacticipant(CONSUMER, main_branch: "main")
.publish_contract(consumer: CONSUMER, consumer_version: "2", provider: PROVIDER, content_id: "111", branch: "feature/fun-123")
.get_pacts_for_verification(
provider: PROVIDER,
enable_pending: true,
provider_version_branch: "feature/fun-123",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ mainBranch: true } , { deployedOrReleased: true }]
)
.verify_pact(
index: 0,
provider_version_branch: "feature/fun-123",
provider_version: "2",
success: true
)
.can_i_deploy(pacticipant: PROVIDER, version: "2", to_environment: "test")
.get_pacts_for_verification(
provider: PROVIDER,
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ mainBranch: true }, { deployedOrReleased: true }]
)
rescue StandardError => e
puts "#{e.class} #{e.message}"
puts e.backtrace
exit 1
end```
bethskurrie
2023-03-30 00:50
This is the tool I used to reproduce the issue https://github.com/pact-foundation/pact_broker/blob/master/ISSUES.md
hazem
2023-03-30 00:52
Yes, this is my question exactly. I feel that this particular case is easily produced when on boarding a first time provider. I get everything working by using the matchbranch selector. .. Yes it is technically a wip pact as I expect, but it is also a pending pact. And would be ideal in this case to verify it on provider main
hazem
2023-03-30 00:53
My only way around this was to create a new consumer branch to get the pact back into unverified in order to get the provider main branch to pass
bethskurrie
2023-03-30 00:53
The answer is: you don?t need the { matchingBranch: true } if you?ve got wip pacts turned on.
hazem
2023-03-30 00:55
Ah, I see. Let me go look at the doc again. I feel I got these selectors from a doc
bethskurrie
2023-03-30 00:57
I would either use { matchingBranch: true } or wip, but don?t use both if you want it still to be wip on your main branch after successful verification on the feature branch.
bethskurrie
2023-03-30 00:58
Here?s a repro of your issue in a build https://github.com/pact-foundation/pact_broker/actions/runs/4559431887/jobs/8043376957#step:3:1404
bethskurrie
2023-03-30 00:58
In the second scenario where I have removed { matchingBranch: true }, the pact is still wip on main https://github.com/pact-foundation/pact_broker/actions/runs/4559431887/jobs/8043376957#step:3:1496
hazem
2023-03-30 00:59
Interesting. I?ll take a look probably on Friday at this point. Thanks for the follow up
bethskurrie
2023-03-30 00:59
Here is the builder code that reproduced your particular issue https://github.com/pact-foundation/pact_broker/blob/issue/missing-pacts-for-verification-00544115/script/data/reproduce-issue.rb
gsouza
2023-03-30 01:02
Actually in my company there is no provider team, so i don't know where to start to look this misconfiguration....probably backend team might know something
matt.fellows
2023-03-30 02:06
If it?s 9.x.x of Pact, then it looks like it should be in `./logs/pact.log`. If it?s 10.x.x or later please set the log level to `DEBUG` and attach as a file here (please redact any sensitive info, such as credentials)
jyothy18
2023-03-30 13:37
I uploaded my provider and consumer contract and see the above. can someone please explain about the paths I see in both consumer and provider
hazem
2023-03-30 17:01
i think we can clarify this doc https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors for https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors. it feels that the wip and pending pacts support is a far more valuable feature when enabled for main branch development.
matchingBranch selector and enabling wip/pending seem to be mutually exclusive features???
matt.fellows
2023-03-31 01:04
Your consumer contract has a field `cdsld` but the OAS doesn?t have that property in the JSON schema for that endpoint
fabio.rodrigues
2023-03-31 12:54
Hello all
How does pactflow contracts deal with api versioning. For example a service has a v1/endpoint and v2/endpoint with slight modifications. Both with consumer-driven contract tests and bi-directional ones?
matt.fellows
2023-03-31 22:16
It should handle them, how do you version - with a path prefix or otherwise?
sugi
2023-04-03 09:15
Hello all, Question on recording deployment: We're using BDCT and using github actions to record deployment of provider after the deployment to environment workflow.... We have test , sandbox environment and production environment and using only master branch. When the PR gets merged to master it automatically deploys to staging & sandbox and the production deployment is a manual trigger which happens later. In the overview page in Pactflow, when the new master is deployed to staging & sandbox, I could see the version (b3333) and environment staging & sandbox but master in production is of different commit (a22222) and I don't see it in the overview....I can't see the matrix at the moment since we are still working on consumer contract.....Just wanted to check when I run Can-I-deploy against production on Consumer , would it verify against commit a22222?
jyothy18
2023-04-03 09:46
Hello all, I have a question regarding checking the compatibility of my swagger yml file directly from swagger. I tried uploading my consumer contract to pactflow. After that I went into swagger and checked compatibility from there. As part of it I have entered my plactflow login details, put the read wrote token and selected the consumer contract from the list and validated it. The result came as contract is compatible. I have 2 issues here 1. When I expand the compatibility I see that swagger is not linked to the consumer contract and I still see response as compatible 2. I never get to select a different contract from pact flow when I try to check compatibility again. Without taking any details swagger says that my swagger yml is compatible
matt.fellows
2023-04-03 10:18
That's correct. When you say "can I deploy to environment X" Pactflow will find the version of the app you are deploying in the target environment, and then check all collaborators for compatibility
matt.fellows
2023-04-03 10:21
The default check is against consumers deployed/released, or the main branch of the consumer.
So if no consumers are showing its because of this.
We are looking at ways to be able to let users configure from the SwaggerHub UI
See also https://docs.pactflow.io/docs/integrations/swaggerhub#contracts-covered-by-the-check
yousafn
2023-04-03 10:46
the application name should be that of your provider in PactFlow :+1: - the provider may not be created (if no consumer contracts involving the provider have been uploaded)
matt.fellows
2023-04-03 12:09
From memory, the variable part needs to be a whole path segment, and not just a sub part.
That is, the `v` should also be part of `{version}`. I think you could then apply a regex constraint on that path variable
jeremy.chao
2023-04-04 14:53
:wave: is there a way to reset password from an admin account in pactflow? When i press forgot password for my user account it doesn't send me the verification code via email - not in spam or anything :disappointed:
jyothy18
2023-04-04 16:30
I am also providing you a screenshot to say that I have both provider and consumer contract from pactflow and it says it is compatible
yousafn
2023-04-04 17:04
The SwaggerHub integration will search for the latest on the main branch, or deployed or released consumers.
I can see that there is no branch property on your consumer, and I assume it isn't deployed to an environment and marked as such in PactFlow
yousafn
2023-04-04 17:05
> Consumer contracts must be using the https://docs.pact.io/pact_broker/branches and https://docs.pact.io/pact_broker/recording_deployments_and_releases feature - it is not compatible with the use of https://docs.pact.io/pact_broker/tags.
> The compatibility check uses the following selection criteria:
> ? The latest version for each consumer's main branch
> ? Any consumer version that is deployed or released to an environment
> In selector terminology, this would be:
> ? `{ "mainBranch": true }`
> ? `{ "deployedOrReleased": true }`
> This means each application (pacticipant) should explicitly set their main branch (e.g. `main`, `developer` or `master`) and publish contracts for that branch, as well as using the `record-deployment` command.
> See the documentation on https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors for additional information on how these work.
matt.fellows
2023-04-05 01:07
Yes, the simplest way is for your admin to re-invite you. That should unlock your account and allow you to try logging in again with the temporary password
matt.fellows
2023-04-05 01:07
usually this happens if you were invited, but waited a long time before actually attempting a login
matt.fellows
2023-04-05 01:08
we use AWS Cognito under the hood, and it has a security policy that transitions the user to a locked status if they don?t login within a certain time
nerea.tamayo
2023-04-05 06:57
Hi team! We are experiencing issues with Verification Results in different projects, Verification stage is failing due to an erroneous validation error. We get "No pact files were found to verify", but we can see the pacts on the PactFlowUI. I saw that a similar issue happened in March (https://status.pactflow.io/incidents/ccnx506zn4m4). Is there any issue related to that? Thanks in advance
matt.fellows
2023-04-05 13:20
Not that I'm aware of Nerea. That incident was purely a display issue in the UI.
How do you know it's erroneous? If you could share some details (cobdoguro/expected behaviour) we could look into it for you
thomas.koppensteiner
2023-04-05 15:08
Hello,
does pactflow support path suffixes like `.json` as part of the path?
Example: `/v1/user/925888/products.json`
matt.fellows
2023-04-06 00:30
I don?t see why it wouldn?t, but couldn?t tell you off the top of my head - have you tried?
thomas.koppensteiner
2023-04-06 13:20
It works if the suffix is hard coded, like `.json`.
The path doesn't match, if it uses a variable (path param), like `.{format}`.
thomas.koppensteiner
2023-04-06 13:50
Another question:
> Response body is incompatible with the response body schema in the spec file: should be object
I think there is something missing in the error message? What `should be object` ?
thomas.koppensteiner
2023-04-06 13:53
Expected response:
```Status:200
Headers:
{
"Content-Type": "application/json"
}
Body:
{"foo":true}```
API:
```{
"foo": true
}```
thomas.koppensteiner
2023-04-06 14:00
pact
```{
"consumer": {
"name": "gem"
},
"provider": {
"name": "service"
},
"interactions": [
{
"description": "A request to fetch the user's example",
"providerState": "the required headers are passed to the request",
"request": {
"method": "get",
"path": "/v1/user/925888/example.json",
"headers": {
"Accept": "application/json"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": "{\"foo\":true}"
}
},
],
"metadata": {
"pactSpecification": {
"version": "2.0.0"
}
}
}```
thomas.koppensteiner
2023-04-07 05:25
We are using `pact (1.63.0)` for ruby on the consumer side. The above pact is a bit simplified.
thomas.koppensteiner
2023-04-07 07:48
Hello,
something is wrong here and I don't know what.
Here is a full example:
OAS file:
```openapi: 3.0.0
x-stoplight:
id: zri1hpqxajhkj
info:
title: internal.v1
version: '1.0'
description: Simply dummy `application/json` API.
servers:
- url: 'http://localhost:3000'
paths:
'/v1/users/{userId}':
parameters:
- schema:
type: integer
name: userId
in: path
required: true
description: Id of an existing user.
get:
summary: Get User Info by User ID
tags: []
responses:
'200':
description: User Found
content:
application/json:
schema:
$ref: '#/components/schemas/User'
examples:
Example:
value:
id: 4711
first_name: Bob
email:
email_verified: true
create_date: '2023-04-07'
operationId: get-users-userId
description: Retrieve the information of the user with the matching user ID.
x-internal: true
components:
schemas:
User:
title: User
type: object
x-examples:
Example:
id: 4711
first_name: Bob
email:
email_verified: true
create_date: '2023-04-07'
x-internal: true
properties:
id:
type: integer
description: Unique identifier for the given user.
example: 4711
first_name:
type: string
example: Bob
email:
type: string
format: email
example:
email_verified:
type: boolean
description: Set to true if the user's email has been verified.
default: true
create_date:
type: string
format: date
description: The date that the user was created.
example: '2023-04-07'
required:
- id
- first_name
- email
- email_verified
x-internal: true```
Pact:
```{
"consumer": {
"name": "sandbox-gem"
},
"provider": {
"name": "sandbox-service"
},
"interactions": [
{
"description": "A request to fetch a dummy user",
"providerState": "and the service returns successful response",
"request": {
"method": "get",
"path": "/v1/users/925888",
"headers": {
"Accept": "application/json"
}
},
"response": {
"status": 200,
"headers": {
},
"body": "{\"id\":4711,\"first_name\":\"Bob\",\"email\":\"\",\"email_verified\":true,\"create_date\":\"2023-04-07\"}"
}
}
],
"metadata": {
"pactSpecification": {
"version": "2.0.0"
}
}
}```
Pactflow validation result
```response body is incompatible
Response body is incompatible with the response body schema in the spec file: should be object
Mismatched at:
[root].interactions[0].response.body```
thomas.koppensteiner
2023-04-07 09:03
Using jsonapi (`application/vnd.api+json`) doesn't lead to the above error!
OAS:
```openapi: 3.0.0
x-stoplight:
id: zri1hpqxajhkj
info:
title: internal.v1
version: '1.0'
description: 'Simply dummy `application/vnd.api+json` API. https://jsonapi.org/'
servers:
- url: 'http://localhost:3000'
paths:
'/v1/users/{userId}':
parameters:
- schema:
type: integer
name: userId
in: path
required: true
description: Id of an existing user.
get:
summary: Get User Info by User ID
tags: []
responses:
'200':
description: User Found
content:
application/vnd.api+json:
schema:
$ref: '#/components/schemas/User'
examples:
Example:
value:
data:
id: 4711
type: user
attributes:
email:
email_verified: true
create_date: '2023-04-07'
operationId: get-users-userId
description: Retrieve the information of the user with the matching user ID.
x-internal: true
components:
schemas:
User:
title: User
type: object
x-examples:
Example:
data:
id: 4711
type: user
attributes:
email:
email_verified: true
create_date: '2023-04-07'
x-internal: true
properties:
data:
type: object
required:
- id
- type
- attributes
properties:
id:
type: integer
example: 4711
type:
type: string
enum:
- user
example: user
attributes:
type: object
required:
- email
- email_verified
properties:
email:
type: string
format: email
example:
email_verified:
type: boolean
default: true
create_date:
type: string
format: date
example: '2023-04-07'
required:
- data
x-internal: true```
Pact:
```{
"consumer": {
"name": "sandbox-gem"
},
"provider": {
"name": "sandbox-service"
},
"interactions": [
{
"description": "A request to fetch a dummy user",
"providerState": "and the service returns successful response",
"request": {
"method": "get",
"path": "/v1/users/925888",
"headers": {
"Accept": "application/vnd.api+json"
}
},
"response": {
"status": 200,
"headers": {
},
"body": "{\"data\":{\"id\":4711,\"type\":\"user\",\"attributes\":{\"email\":\"\",\"email_verified\":true,\"create_date\":\"2023-04-07\"}}}"
}
}
],
"metadata": {
"pactSpecification": {
"version": "2.0.0"
}
}
}```
thomas.koppensteiner
2023-04-07 09:17
For me it looks like, that `application/hal+json` also works, but I haven't investigated that in depth.
matt.fellows
2023-04-11 00:29
Hi Thomas. Something doesn?t look right to me though. The `response.body` property is escaped JSON, but it shouldn?t be if it is supposed to be a type of JSON
thomas.koppensteiner
2023-04-11 06:11
> The `response.body` property is escaped JSON, but it shouldn?t be if it is supposed to be a type of JSON
Thank you for the hint. It wasn't clear for me, that this could be the issue.
I'll try to generate a non-escaped version.
matt.fellows
2023-04-11 06:15
Yeah, that?s a tricky one. It?s a legit possibility to have escaped JSON, and because it?s a string, it?s also valid JSON
thomas.koppensteiner
2023-04-11 06:24
> How are you generated that?
We use https://github.com/pact-foundation/pact-ruby for testing the consumer contract. There I used a ruby hash for the response body and called `.json` on it. This resulted in the escaped JSON string in the pact.
I would expect a warning from `pact-ruby` in this case.
nerea.tamayo
2023-04-11 06:25
Hey Matt!! Sure let me collect the details from the teams to provide you what makes sense. thanks a lot for your support!
matt.fellows
2023-04-11 06:32
Thanks. Perhaps it?s worth raising an issue for it? (or a PR if you have a good suggestion)
thomas.koppensteiner
2023-04-11 07:21
Now I understand the original error message:
> Response body is incompatible with the response body schema in the spec file: should be object
For me
"Response body is incompatible with the response body schema in the spec file: The response body should be a JSON object, but is as (JSON) string"
would be more clear and helpful.
noor.hashem
2023-04-11 19:47
Hey all, when using pactflow i've created a team using the dashboard. But how can I assign pacts via the javascript package to automatically be assigned a specific team? I've seen a few things online but did not get it to work. This is the version: "pact-foundation/pact": "^9.18.1".
tjones
2023-04-12 00:09
According to this, https://docs.pactflow.io/docs/user-interface/settings/teams/ I think you just add the application to the team on that UI panel
matt.fellows
2023-04-12 03:34
That?s right. I _believe_ there is a feature request somewhere where you can specify the team on publish, but I seemed to have misplaced it. I will dig it up or create a new one
matt.fellows
2023-04-12 03:35
The simplest option, is to pre-create all applications and teams, and do the assignments at that time.
e.g. using something like https://github.com/pactflow/terraform-provider-pact
matt.fellows
2023-04-12 03:35
We?ll also be publishing our APIs later in Q2 (possibly early Q3) so you could build around that also
noor.hashem
2023-04-12 13:26
Oh I see okay so I need to do it through the actual UI and support for doing it programmatically may come in the future. Thank you both! :raised_hands:
matt.fellows
2023-04-13 07:11
I?d like to see it in our CLI tools also, but I don?t see it being part of our JS (or any other client languages) any time soon, if I?m honest
tjones
2023-04-13 07:54
You _could_ support this server side with no changes to tools, if you treated provider names that deserialised to an object differently at the pactflow end:
```provider: JSON.stringify({ "team": "MyTeam", providerName: "MyProvider" })```
or something
nerea.tamayo
2023-04-13 11:10
Hi Matt!!Sorry for my late response but we identified something in common in all cases we were getting the issue, and finally we fixed it. Sorry it was due to an issue from consumer side. Thanks a lot for your support!
nickwilliamsqa
2023-04-13 17:08
Hi, moved this here from general. I'm experiencing difficulties in validating a consumer against the provider contract generated from an OpenAPI spec. The OpenAPI spec has references, inheritance, `oneOf/allOf` keywords and some circular references. The comparison is failing in several places due to "x should NOT have additional properties".
I have followed the guidance given https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support/, but even after dereferencing and inlining the same errors still occur. I have eventually been able to get this working by further manually wrangling the spec, so could potentially write a script that would take care of this for me. However, this feels a bit hacky and is not ideal.
Are these more advanced OpenAPI spec features not supported? Is there any other guidance on what steps we can take without having to resort to intervening and amending the spec this way, or does anyone have any advice on how best to tackle this?
I am using pact-python for the consumer, and pact-php for the provider. Will comment on this thread with some excerpts of the OpenAPI spec.
Thanks in advance.
nickwilliamsqa
2023-04-13 17:08
Example of inheritance & references in the spec:
```"Parent": {
"description": "Parent schema",
"type": "object",
"required": ["id", "activities"],
"properties": {
"id": {
"type": "integer",
"example": 3,
"enum": [3, 4, 5, 6, 7, 8, 9, 16],
"nullable": false
},
"activities": {
"type": "array",
"nullable": false,
"minimum": 1,
"items": {
"type": "string",
"example": "6AA2A3B9-66BD-447C-90C8-A350F60D592B"
}
}
}
},
"Child": {
"description": "",
"allOf": [{
"$ref": "#/components/schemas/Parent"
}],
"type": "object",
"properties": {
"id": {
"type": "integer",
"enum": [3]
},
"activities": {
"type": "array",
"items": {
"type": "string",
"enum": [
"value",
"another value"
]
}
}
}
}```
Example of keyword & circular references:
```"properties": {
"project_type": {
"type": "object",
"nullable": true,
"oneOf": [{
"$ref": "#/components/schemas/Category"
}]
}
}
....
"schemas": {
"Category": {
"type": "object",
"required": ["short_name"],
"properties": {
"short_name": {
"type": "string",
"nullable": false,
"example": "test"
},
"child": {
"nullable": true,
"allOf": [{
"$ref": "#/components/schemas/Category"
}]
}
}
}
}```
matt.fellows
2023-04-14 04:09
Thanks Nick. When you say ?manual wrangling? what did you have to do to get it working?
I?d need an engineer on the team to look into it. As you?ve noticed, the keyword support comes with caveats due to the way the schemas get applied and merged
yousafn
2023-04-14 11:44
As above, would be good to know the exact steps of manual wrangling?
Also is it possible to have a repro pact/openapi spec and we can do some digging
nickwilliamsqa
2023-04-14 13:23
Thanks both. So it was a combination of things - firstly de-referencing and then using the json-schema-resolve-allof library as described in the docs.
This however resulted in an invalid schema in our case, specifically due to the inheritance enums in the example above resolving strangely which leads to errors in PactFlow due to duplicate IDs (one of many of the same error):
`schema is invalid: data.definitions['Projects'].items.properties['props'].properties['thing'].oneOf[0].properties['primaries'].items.oneOf[1].properties['id'].enum should NOT have duplicate items (items ## 0 and 1 are identical)`
However these do not appear to actually be duplicated:
```"oneOf": [{
"description": "",
"type": "object",
"properties": {
"id": {
"type": "integer",
"enum": [3, 4, 5, 6, 7, 8, 9, 16],
"example": 3,
"nullable": false
},
"activities": {
....
}
},
"required": ["id", "activities"]
},
{
"description": "",
"type": "object",
"properties": {
"id": {
"type": "integer",
"enum": [4, 4, 5, 6, 7, 8, 9, 16],
"example": 3,
"nullable": false
},
"activities": {
....
}
},
"required": ["id", "activities"]
}
...
]```
There is definitely an issue here with the dereferencing, as the enum values have been listed incorrectly (I assume due to inheritance), but I am unsure why PactFlow is identifying those as duplicates, as `properties.id.enum` values are different between the two.
After manually fixing this (so each child contains a single enum value e.g. `properties.id.enum: [3]`), the dereferencing has fixed a bunch of errors - but unfortunately there are still a large number of `should NOT have additional properties` errors, where we have `oneOf`.
In some situations, where we only have a single reference within the `oneOf`, I was able to get this working by removing the `oneOf` array. For the example in the `schemas.category` above, this became the following which works:
```"properties": {
"project_type": {
"type": "object",
"nullable": true,
"required": ["short_name"],
"properties": {
"short_name": {
"type": "string",
"nullable": false,
"example": "test"
},
"child": {
"nullable": true,
"type": "object",
"required": ["short_name"],
"properties": {
"short_name": {
"type": "string",
"nullable": false,
"example": "test"
},
"child": {
"nullable": true
}
}
}
}
}
}```
But has meant the loss of the circular reference for the nested `child` property.
Where multiple entities exist under `oneOf` and so couldn't just be removed, I added `additionalProperties: true` wherever the keyword was used, which seems to be the nuclear option I'd much rather avoid. E.g
```"properties": {
"metrics": {
"type": "array",
"nullable": true,
"items": {
"additionalProperties": true,
"oneOf": [{
...
}, {
...
}
...
]
}
}
}```
nickwilliamsqa
2023-04-14 13:24
Am just checking with the powers that be that I'm ok to send over a copy of the spec/pact - will do this asap. Thanks again.
yousafn
2023-04-14 13:27
no worries chap, thanks for all the additional detail
It would be great if you could raise the issue against https://github.com/pactflow/swagger-mock-validator as this is the underlying tool we use, if you can build a non client specific example, if would be great to get in as a test case, unit or e2e, so we have a good base for investigations
nfinley
2023-04-14 13:44
Hello! I wondering if there is a way to add a second user as a billing administrator within the Pactflow portal? I am currently the only billing admin but I need to add/change to another person so we can purchase a subscription. Any help would be much appreciated. Thank you!!
matt.fellows
2023-04-15 05:32
Thanks Yousaf.
Actually you can only have a single billing admin, you can change that by logging into the subscription platform here: https://docs.pactflow.io/docs/billing/
Alternatively, support can definitely help you out
cameron.allan853
2023-04-16 22:13
Hello, I signed up to Pactflow and recieved my registration details. When I try to follow the URL (`Here is the URL for your shiny new toy (your credentials will be sent in a separate email):`) I see this message. I'm seeing this across browsers/devices/internet connections
matt.fellows
2023-04-16 23:43
I?m going to DM you. I can see a registration under your name, and that is not the link in the email
matt.fellows
2023-04-16 23:49
As discussed, it looks like an email security tool that rewrites URLs e.g. https://www.securityhq.com/services/antigena-email-as-a-service/
I know SmartBear uses mimecast which does the same thing
sugi
2023-04-17 15:36
Hi All, Question regarding usage of anyOf keyword and de-referencing it in schema: We are using BDCT in Ruby and using rswag to generate the OAS...I've de-referenced the anyOf in the schema definition and do I need to de-reference it in the response as well ?
sugi
2023-04-17 15:38
```responses:
'200':
description: User found
content:
application/vnd.api+json:
schema:
"$ref": "#/components/schemas/UserResponse"
schemas:
UserResponse:
type: object
properties:
data:
"$ref": "#/components/schemas/User"
included:
type: array
items:
anyOf:
- type: object
properties:
id:
type: string
type:
type: string
-type: object
properties:
name:
type: string
type:
type: string
required:
- data```
sugi
2023-04-17 15:38
UserResponse schema has anyOf and it is been de-referenced and User doesn't have the keyword
yousafn
2023-04-17 16:17
I would say yes, you would have to. There is also an issue with anyOf and the difficulties managing it.
Your example is showing anyOf for the response being
`[{"id":"foo","type":"bar"]`
or
`[{"name":"foo","type":"bar"]`
If you have a client generating a pact file that only tests for one of those cases `[{"id":"foo","type":"bar"]`, they many never received the other anyOf object in this case as we wont have verified it`[{"name":"foo","type":"bar"]`
in regular consumer driven testing for pact, you would use provider states and an actual verification of the provider to assert that when the provider was capable of providing both types of responses that consumer has required
Have you seen any issues in verifying the contract, and how are you deferencing your OAS
sugi
2023-04-17 17:02
Looking for some ruby libraries to dereference the OAS...Do you have any recommendations?
nickwilliamsqa
2023-04-17 17:15
Thanks Yousaf - will try and make the time to raise the issue sometime this week.
Have got the all-clear to send over the (anonymised somewhat) copies of our Open API spec/Pact contract. Will DM them to you directly
yousafn
2023-04-17 17:16
not in ruby no, I scripted up a javascript package to do it which can be used on the command line and executed with `npx` https://github.com/YOU54F/oas-merge
thomas.koppensteiner
2023-04-18 06:43
Hello,
does pactflow currently support "JSON:API" form https://jsonapi.org/?
If not, could you please look at https://github.com/pactflow/roadmap/issues/97 and https://pact.canny.io/feature-requests/p/support-for-jsonapi?
matt.fellows
2023-04-18 07:50
I don?t see why you couldn?t test JSON API now with Pact. It?s JSON, after all
matt.fellows
2023-04-18 07:52
With our BDCT feature (which I think is what you are after?), it should actually support it now
matt.fellows
2023-04-18 07:52
I believe there is currently a known bug in relation to other JSON types
thomas.koppensteiner
2023-04-18 07:58
Yes, this looks exactly as the issue we have right now.
I'll link it on github.
matt.fellows
2023-04-18 08:04
btw, you are more than welcome to contribute a PR to address the issue. We haven?t dug into the problem, it?s possibly something quite straightforward
yousafn
2023-04-18 23:40
you either verify by url, or by selectors, not both. if your job sets both, i don?t think the ffi will warn you.
tjones
2023-04-19 10:29
Hello! I?m getting:
```* Configure separate http response provider pact verification build and webhook to trigger it when the pact content changes. See https://docs.pact.io/go/webhooks```
from Pactflow, even though the contract is verified. I think this is a bug, but it might be because I?m invoking it wrong, so I?m reporting here before opening an issue. I?ll put my request details in a thread:
tjones
2023-04-19 10:30
```POST: https://MY_DOMAIN.pactflow.io/contracts/publish
{
"branch": "main",
"contracts": [
{
"consumerName": "http request consumer",
"content": "<..BASE64 ENCODED CONTRACT...>",
"contentType": "application/json",
"providerName": "http request provider",
"specification": "pact",
},
],
"pacticipantName": "http request consumer",
"pacticipantVersionNumber": "0.7.0-main+75.72e1e6d.SNAPSHOT.Rogue",
"tags": [],
}```
meghaagr
2023-04-19 10:32
hi Team,
We are doing a POC using Pactflow in dotnet core application, we would like to know what other API testing tools we can use apart from schemathesis(licensed version). We are looking for free version.
matt.fellows
2023-04-19 13:19
Yep! Pretty much any API testing tool - RestAssured, Dredd, Postman are all options.
matt.fellows
2023-04-19 13:21
good question. I think this endpoint is in the Pact Broker - so I think OSS
ravinder.kadiyan
2023-04-20 10:10
Hi Team,
We are trying to use the pactflow bidirection feature and struck on provider side verification result... We don't want to use any other external tool like schemathesis or newman... is it possible to write provider test using xunit (dotnet6) which can also generate the verification result. (provider test similar to whitebox component test, where we start the server inside the test and run test corresponding to it... )
bas
2023-04-20 10:37
Hi @ravinder.kadiyan, just checking: are you sure you aren't mixing up things here? With bidrectional contract testing, both provider and consumer publish their own version of a contract, which will then be uploaded to Pactflow for comparison. There's no such thing as a 'provider verification result' in BDCT.
bas
2023-04-20 10:41
Unless you're talking about the part where you can publish additional verification results? That can be 'anything' really, including any report produced by xunit, as far as I know.
bas
2023-04-20 10:44
As for the actual provider-side contract, I _think_ that still has to be an OpenAPI spec at the moment, but I could be wrong.
ravinder.kadiyan
2023-04-20 11:27
Usage:
pactflow publish-provider-contract CONTRACT_FILE \
--broker-token=BROKER_TOKEN
--broker-base-url=BROKER_BASE_URL
--provider PROVIDER \
--provider-app-version PROVIDER_APP_VERSION \
--branch BRANCH \
--content-type CONTENT_TYPE \
--verification-exit-code=EXIT_CODE \
--verification-results REPORT_PATH \
--verification-results-content-type REPORT_CONTENT_TYPE \
--verifier VERIFIER
I am referring to '--verification-results REPORT_PATH' here.. swagger is still the same way as in bidirection
ravinder.kadiyan
2023-04-20 11:32
Instead of using schemathesis or newman, can we write test using xunit at provider side and generate the report..
bas
2023-04-20 11:35
Sure, why not. One typical report to include could be a JUnit XML report for example, see https://xunit.net/docs/format-xml-v2. That'll be written to a file, if you read and base64 encode that, it should work.
If you're looking for a library to write those tests, allow me to do a shameless plug for https://github.com/basdijkstra/rest-assured-net
matt.fellows
2023-04-20 12:40
Of course. We don?t care what tool you use, there are _so_ many good API testing tools - as long as your team is happy using a tool, and are comfortable that your API is well tested
ajaydiwakar.ka
2023-04-20 18:10
Is this the right values to provide for xunit as verifier ? @matt.fellows @bas
```--content-type application/xml \
--verification-results ./report.xml \
--verification-results-content-type application/xml \
--verifier xunit```
Getting an error like "status=400 {"errors":{"verificationResults.success":["is missing"]}}" although the tests are passed.
bas
2023-04-20 18:16
This blog post contains an example where I upload the provider contract via the PactFlow API: https://www.ontestautomation.com/an-introduction-to-contract-testing-part-6-bi-directional-contracts/ (almost at the end of the post).
As you can see the verification results contain a success: true entry. You need to specify something similar. PactFlow isn?t going to parse your results file.
bas
2023-04-20 18:21
Reading https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas
I think you need to pass either ?verification-exit-code=0 or ?verification-success as a flag to resolve that error
matt.fellows
2023-04-21 02:05
PactFlow doesn?t know how to interpret the test results, it needs to be told if it?s passing or failing. As Baz said, you need to pass the flag missing in the error
eddie
2023-04-23 23:57
Hey,
Having some trouble publishing pact verification results.
```Publishing of verification results failed with an error: Link/Resource was not found - Request to pact broker path '/pacticipants/xyz.BFF' failed: 404 Not Found. URL: 'https://mycompany.pactflow.io/pacts/provider/xyz.BFF/consumer/prq.SPA/version/a29c9c63b2fc1cae4b8488b322b6fc7b02cd5ca1'```
Code in question looks like this:
```verifierSource = provider.WithUriSource(_executionConfig.Pact.IncomingWebhook.PactUrl,
webHookOptions => webHookOptions.TokenAuthentication(_executionConfig.Pact.Broker.Token)
.PublishResults(optionalProviderVersion.IsSome,
optionalProviderVersion.Match(v => v, string.Empty),
options => options
.WithProviderBranchIfSuppliedAndValid(_executionConfig)
.WithTeamcityBuildUrlIfSuppliedAndValid(_executionConfig.Teamcity))
);```
*Q1*: Is the `PactUrl` somehow wrong? It's like this currently `https://mycompany.pactflow.io/pacts/provider/xyz.BFF/consumer/pqr.SPA/version/a29c9c63b2fc1cae4b8488b322b6fc7b02cd5ca1`
*Q2*: What is the error message (`Link/Resource was not found`) telling me?
*Q3:* How do I arrange for this to throw an exception if publishing results fails rather than just logging this error?
matt.fellows
2023-04-24 04:34
I?m wondering if perhaps it?s the full stop in the URL? (It should be supported, but that?s one thing that jumps out at me)
matt.fellows
2023-04-24 04:35
I?m not quite following the `webHookOptions` though and the `WithProviderBranchIfSuppliedAndValid` things - these don?t look part of the Pact .NET API. Is this some internal wrapper?
ajaydiwakar.ka
2023-04-24 13:06
API contract test with pactflow now works fine. Now we wanted check with Kafka as well. I could see samples for Java https://github.com/pactflow/example-provider-java-kafka/ Is there any samples in dotnet that we could refer to ? @matt.fellows @bas
kerrypmckeever
2023-04-24 17:46
Hey there! I have a new integration, and can see in Pactflow that my consumer and provider integration are added (albeit unverified, since only the new consumer pact has been uploaded to Pactflow so far). However, when I execute `/pacts/provider/:provider/for-verification` endpoint, it shows no pacts to verify. Does anyone know what could be causing this? I have verified the provider I'm passing in for `:provider` matches the actual provider shown in Pactflow:
kerrypmckeever
2023-04-24 18:17
If it's helpful, I've also included the debug logs where it is failing to find the pacts from the broker in our CI here:
yousafn
2023-04-24 18:32
I assume you have no branch or tags applied to the published consumer?
```Sending JSON to ***/pacts/provider/auth-api/for-verification using POST: {"providerVersionTags":[],"includePendingStatus":true,"consumerVersionSelectors":[],"providerVersionBranch":"15696/merge"}```
you have no consumer version selectors here, so currently you would be need to verify your pact by URL, provider verification tasks will traditionally pick up deployed or released consumers, plus those against the configured main branch (main/master if published will automatically set it as the main branch)
vandana.aveva
2023-04-24 18:34
Hi,
I'm trying PactFlow on my local machine. But I'm not able publish my contract. The publish command: npm run pact:publish, throws error '.' is not recognized as an internal or external command
kerrypmckeever
2023-04-24 19:00
Ah, yep. That would make sense - thank you for reminding me of the thing that trips me up every time I onboard a new provider. :pray:
kerrypmckeever
2023-04-24 20:52
@yousafn does adding `enablePending: true` to the provider verification options not allow passing if no pacts from the target branch are found? I thought it did.
yousafn
2023-04-24 21:36
no enable pending stops the provider build from failing. it?s not a work in pending pact so including wip pacts doesn?t work as it?s not associated to a branch or tagged, i?d recommend reading the recommend guidance on publishing, pending pact/ and work in progress pacts in the docs
noor.hashem
2023-04-25 03:09
Hi all! Quick question, in this repo for https://github.com/pactflow/example-consumer in the makefile on line 9, where do I find the GITHUB_WEBHOOK_UUID. As well in the example-provider on lines 3 and 4 in the makefile where do I find the PACT_CHANGED_WEBHOOK_UUID and the CONTRACT_REQUIRING_VERIFICATION_PUBLISHED_WEBHOOK_UUID?
eddie
2023-04-25 22:36
`can-i-deploy` question:
First:
? We create a build of our consumer at version `consumer-v1` and publish the contracts
? The provider verifies those contracts :white_check_mark:
? `can-i-deploy` returns true
? We release consumer version `consumer-v1` to the environment
`
Then:
? We create a build of our consumer at version `consumer-v2` and publish the contracts
? These contracts are slightly different to the previous version from `consumer-v1`
? The provider verifies those contracts :white_check_mark:
? `can-i-deploy` returns true
? We release consumer version `consumer-v2` to the environment
Later:
? We revert the changes from the consumer, producing a new consumer version `consumer-v3` and publish the contracts
? *The JSON for these contracts is identical* to those published by `consumer-v1`
*Should `can-i-deploy` return `true` in this instance* (assuming the provider hasn't been deployed)? Or does the provider need to explicitly verify this consumer version?
tjones
2023-04-26 03:51
> (assuming the provider hasn?t been deployed)?
I think this breaks your example, yes? Because if the provider has never been deployed, all can-i-deploy queries should return false.
Anyway, to your question:
> Or does the provider need to explicitly verify this consumer version?
No, the provider doesn?t need to re-verify the same contract with a new version. The broker hashes the contract contents, and knows that it is already verified. See https://docs.pact.io/getting_started/versioning_in_the_pact_broker for more details.
tjones
2023-04-26 03:55
It?s common (and probably helpful at first) to think of each version as a new contract, but that?s not actually how it works. Each version is a pointer to a contract.
eddie
2023-04-26 03:55
Thanks for confirming. I should have been more precise - I meant "assuming the provider hasn't been deployed *again since*" :wink:
> No, the provider doesn?t need to re-verify the same contract with a new version
That was my understanding. I was trying to figure out why it wasn't working here (given my understanding of how the hashing works). It turns out the other team *had* released the provider subsequently. They don't have `can-i-deploy` integrated into their pipeline yet and hadn't checked the current contracts (only the new contracts we'd asked them to check)
tjones
2023-04-26 03:57
Ah! That?s going to be a problem, because can-i-deploy needs to know what is and isn?t deployed in order to reason about it. So, your can-i-deploy check will be untrustworthy if the provider isn?t doing record-deployment (etc)
eddie
2023-04-26 03:59
My team were early-adopters of Pact at my organisation, the other team is still on the implementation journey and we were their guinea-pigs :wink:
They'll be integrating `can-i-deploy` and checking contracts for consumers in the environment as part of the CI/CD pipeline soon.
eddie
2023-04-26 03:59
I was just having a panic-moment that everything I knew about Pact was wrong for a minute :smile:
tjones
2023-04-26 03:59
> It turns out the other team had released the provider subsequently. They don?t have can-i-deploy integrated into their pipeline yet
FWIW, this problem is the other reason I wrote https://www.npmjs.com/package/absolute-version - so (when integrated) I could query a health endpoint and get back exactly what was deployed by the other team.
tjones
2023-04-26 03:59
> everything I knew about Pact was wrong for a minute :smile:
Not even for a moment.
thomas.koppensteiner
2023-04-26 11:03
Please review or pickup my PR: https://github.com/pactflow/swagger-mock-validator/pull/20
sashi.kandru
2023-04-28 17:35
Hi Team, Does bi-directional testing support event-driven applications as well?
nickwilliamsqa
2023-04-28 21:35
Finally managed to get to this - I have created an issue here: https://github.com/pactflow/swagger-mock-validator/issues/21
Included a simple example OAS spec & PactFile. Thanks!
matt.fellows
2023-04-28 22:26
Not yet sorry, but would love to chat with you more on your use case. Could you elaborate a bit on the use case / needs?
matt.fellows
2023-04-28 22:29
For some prompts
? Do you use AsyncAPI?
? If so, who/what role authors that doc?
? What's the tech stack?
? What problems are you trying to solve and how have you tried to solve them (presumably unsuccessfully) to date?
milda.abromaviciute
2023-05-02 08:40
hi all, can someone please tell me the broker api endpoints (HAL) for uploading open api specs and recording deployments? can't seem to find docs on that. Thanks! :thankyou:
matt.fellows
2023-05-02 11:32
Hi Milda, we haven't actually published them (either as HAL relations or docs) as there is some changes coming. We aim to publish in Q2
Can I ask what you're aiming for here? I'm assuming you can't use the CLI tools for some reason?
One way to get the endpoints is to use the CLI and set it to verbose. All the queries will show there.
milda.abromaviciute
2023-05-02 11:38
Yep we can't use the CLI tools currently, so trying to get a workaround :slightly_smiling_face: Wanna have a service running accepting input at all times, writing up a service wrapper for the provided api. Will run some verbose commands in the cli, thanks so much :heart:
vandana.aveva
2023-05-02 22:40
Hi,
I'm trying to implement a test flow for birectional testing and I'm following the dotnet project example:
https://github.com/pactflow/example-bi-directional-provider-dotnet
I've created my own api using .net web api template for minimal webapi option. I'm not able to run verify_swagger step in there.
I'm getting error:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='host.docker.internal', port=9000): Max retries exceeded with url: /swagger/v1/swagger.json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe8c34912d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
I'm new to docker, do I need to replace url http://host.docker.internal:9000/swagger/v1/swagger.json with my localhost url?
I tried that as well, and I'm still getting same error error.
Any idea what can be done here? requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=7039): Max retries exceeded with url: /swagger/v1/swagger.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f47ec6bf460>: Failed to establish a new connection: [Errno 111] Connection refused'))
When I tried to run publish EXIT_CODE=0 make publish_provider_contract step,
I'm getting error
PactBlogsApi/swagger.json \
--provider "blogs-provider" \
--provider-app-version f4c421b-main+f4c421b.SNAPSHOT.USSCOL28H27D3 \
--branch main \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results report.txt \
--verification-results-content-type text/plain\
--verifier schemathesis
docker: Error response from daemon: the working directory 'C:/repos/pact-research-api/pactblogsapi' is invalid, it needs to be an
absolute path.
See 'docker run --help'.
make: *** [Makefile:57: publish_provider_contract] Error 125
Can you suggest what can be done here?
cody.jenkins
2023-05-03 05:13
I'm seeing some strange behaviour running Pact in my CI. Everything was working fine yesterday, but for some reason today the client is not able to fetch the pacts from the broker. It's coming back with a 400 response. Logs in the thread :thread:
cody.jenkins
2023-05-03 05:13
```
[04:06:49.444] INFO (1864): pact@11.0.2: Verifying provider
[04:06:49.470] INFO (1864): pact-core@13.13.6: Verifying Pacts.
[04:06:49.473] INFO (1864): pact-core@13.13.6: Verifying Pact Files
2023-05-03T04:06:49.508954Z INFO ThreadId(03) pact_verifier::pact_broker: Fetching path '/' from pact broker
2023-05-03T04:06:50.589353Z INFO ThreadId(03) pact_verifier::pact_broker: Fetching path '/pacts/provider/ShippingServiceHttp/for-verification' from pact broker
2023-05-03T04:06:51.211098Z ERROR ThreadId(03) pact_verifier: Failed to load pact - Could not load pacts from the pact broker '************************'
2023-05-03T04:06:51.211158Z WARN ThreadId(03) pact_matching::metrics:
Please note:
We are tracking events anonymously to gather important usage statistics like Pact version and operating system. To disable tracking, set the 'PACT_DO_NOT_TRACK' environment variable to 'true'.
Failures:
1) Failed to load pact - Could not load pacts from the pact broker '************************'
There were 1 pact failures
[04:06:51.232] ERROR (1864): pact-core@13.13.6: Verification unsuccessful
FAIL src/shared/pact/__test__/ShippingServiceHttp.producer.pact.test.ts (9.121 s)
? Shipping Service HTTP ? provides expected responses
Verfication failed
at node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.ts:52:20```
cody.jenkins
2023-05-03 05:15
Has something changed which would prevent the pact client from accessing the pacts stored in the pactflow broker?
cody.jenkins
2023-05-03 05:17
I thought I could get the redacted values quickly but it's hidden away somewhere, just trying to source them
matt.fellows
2023-05-03 05:20
If you could set the log level to `DEBUG` you should be able to see the request/response (it might be trace). The response _should_ contain (what I would guess are validation) error messages, that appear to be suppressed.
cody.jenkins
2023-05-03 05:21
one of the other devs turned on debug and got this. I haven't run myself yet but will try to asap:
```2023-05-03T04:24:50.589390Z DEBUG ThreadId(16) pact_verifier::pact_broker: error Response for pacts for verification ContentError("Request to pact broker URL 'https://lynk.pactflow.io/pacts/provider/ShippingServiceEvent/for-verification' failed - HTTP status client error (400 Bad Request) for url (https://lynk.pactflow.io/pacts/provider/ShippingServiceEvent/for-verification)") ```
cody.jenkins
2023-05-03 05:23
trace like set log level to trace or attach to the running process and trace?
matt.fellows
2023-05-03 05:23
My _guess_ is that a blank value might be coming through for one of the selector properties
matt.fellows
2023-05-03 05:41
Looks like these args came through:
```{
"providerVersionTags": [],
"includePendingStatus": true,
"consumerVersionSelectors": [{ "latest": true, "branch": "master" }],
"providerVersionBranch": ""
}```
matt.fellows
2023-05-03 05:42
The error is that the version branch is empty:
``` "errors": {
"providerVersionBranch": [
"when pending or WIP pacts are enabled and there are no tags provided, the provider version branch must not be an empty string, as it is used in the calculations for WIP/pending. A value must be provided (recommended), or it must not be set at all."
]
}```
matt.fellows
2023-05-03 05:42
I?m going to raise a defect to report the error, it?s clearly not coming out
cody.jenkins
2023-05-03 05:45
ah ok, good that you found that because I couldn't get much more out of the trace logs for some reason :think_ie:
cody.jenkins
2023-05-03 05:56
So for this, are we doing something wrong by not setting the providerVersionBranch? I'm still coming back up to speed on this project (was off it for 8 months) so I'm struggling to remember how we've configured it.
Or is this something that the pact client should send differently and we'll need to wait for a patch and upgrade?
matt.fellows
2023-05-03 05:58
You should set it, because (as per the error) it?s needed for the pending status calculation (and you have pending set to `true` )
cody.jenkins
2023-05-03 05:59
ok, cool, I'll have to see what possibly changed on our side because it's happening sort of out of the blue.
matt.fellows
2023-05-03 05:59
It?s also the same value used when you publish results, so you should set it, because then PactFlow can track the branches in use (see https://docs.pact.io/getting_started/conceptual_overview#branches--environments and https://docs.pact.io/pact_broker/branches)
matt.fellows
2023-05-03 06:00
I know we?ve recently upgrade our validation with a Ruby 3 upgrade, so this might have actually been a change or bug that was fixed (by accident or on purpose) - but what it?s doing now is highlighting a problem that was possibly masked before
cody.jenkins
2023-05-03 06:00
I've reset back to a commit that previously ran successfully. This commit fails now
cody.jenkins
2023-05-03 06:01
so something changed on the pactflow side that causes the failure to happen now. But yeah maybe we've been doing it wrong this whole time but didn't realize it until now
matt.fellows
2023-05-03 06:03
thanks, I?ll share the feedback. The lead is off at the moment, but I think it was something that was known/watched with a close eye.
I think the lack of a visible error message (probably missing in the core) is part of the problem too.
matt.fellows
2023-05-03 06:09
Welcome. I?ve added a https://github.com/pact-foundation/pact-reference/issues/273. Solving it here will fix the error in most client libraries.
milda.abromaviciute
2023-05-03 08:59
standalone ruby version too low, want something common across teams rather than individual calls that are language dependent (bit tech stach in the company), also need a running service rather than using the docker image provided to spin up pods for each request
sashi.kandru
2023-05-03 14:29
HI Matt, I will find out more, but here are the high level details. We have multiple event driven applications (Async style) That use SQS, Kafka etc.. the case we wanted to evaluate is for event message consumers that have multiple event producers... similar to APS where one to many relationships for event space. would it BDC reduce the toll on message consumers?
vandana.aveva
2023-05-03 16:29
Hi,
I'm trying to publish my contract using local cli
../pact/bin/pactflow.bat publish-provider-contract \
swagger.json \
--broker-base-url <<My Pacthflow host>> \
--broker-token "<<Token>>" \
--provider "pactblogsapi" \
--provider-app-version 3a0994c \
--branch pactblogsapi-main \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results report.txt \
--verification-results-content-type text/plain \
--verifier Schemathesis
I'm getting error making request to https://compworld.pactflow.io/contracts/provider/pactblogsapi/version/3a0994c status=400 {"errors":{"verificationResults.content":["must be
filled"]}}
The file report.txt contrains verification result. Any suggestion to fix this?
yousafn
2023-05-03 16:38
Hey Vandana, that looks sensible, here is one I have from my history
```pactflow publish-provider-contract \
products.yaml \
--provider test \
--provider-app-version 1.0.1-some-sha \
--branch test \
--content-type application/yaml \
--verification-exit-code=0 \
--verification-results products.yaml \
--verification-results-content-type application/yaml \
--verifier test```
I'm cheating here and just sending the OAS as the verification report.
I assume the folder you are in, contains both the swagger.json and and report.txt?
I wonder if the pact ruby standalone is looking in its path where it lives, rather than where your pact file lives, and needs an absolute path, I wouldn't have thought that would be the case but will test out as I have a windows machine here
Also top tip ` ` ` three backs (without the spaces) will provide a code block for you, that you can paste code snippets into :slightly_smiling_face:
Makes it nicer on everyones eyes
yousafn
2023-05-03 16:40
I wonder if
`../pact/bin/pactflow.bat`
means its looking outside the folder
or did you mean to write `./pact/bin/pactflow.bat`
with your code being in x folder and `pact` being in the same folder as your code?
yousafn
2023-05-03 16:42
if you do `dir` in your console, do you see your `pact` folder and the `swagger.json` and `results.txt`?
yousafn
2023-05-03 16:46
from my testing, passing `application/yaml` is fine for a valid json document. Feel free to private message me your documents and I'll test on my windows and mac machines.
Could you move the `pact` folder into the folder where your code is (where the `swagger.json` and `results.txt` reside)
vandana.aveva
2023-05-03 16:55
I was able figure out my issue, my report.txt is blank at the location passed to the command
adam.cox
2023-05-05 08:33
Hi, Is it possible from the pactflow UI to get contact details for a given participant? When contract verfication fails developers on either team would like to easily be able to find out who they should talk to.
I had a quick look around but I could not see anything in there that would support this at the moment.
yousafn
2023-05-05 09:02
I don't believe so but pretty sure @shuying.lin had it as part of the UI redesign.
If not i'd recommend dropping a request here
https://github.com/pactflow/roadmap/issues
This could vaguely attribute to it, contract insights, but a new card is probably worthwhile
https://github.com/pactflow/roadmap/issues/29
adam.cox
2023-05-05 09:29
https://github.com/pactflow/roadmap/issues/99
Let me know if you need more details than this
matt.fellows
2023-05-05 09:38
Thanks! I just saw this request, we'll take this feedback on board with an upcoming design refresh
matt.fellows
2023-05-05 11:37
there is a pre-release of a late version of Ruby :point_right: that might work: https://github.com/pact-foundation/pact-ruby-standalone/releases/tag/v2.0.0
There is a bug in the mock server, but if you?re just publishing pacts/provider contracts it should work I believe.
matt.fellows
2023-05-05 11:37
Not sure why you need a running service to publish pacts? sorry I?m obviously confused!
matt.fellows
2023-05-08 11:25
That might be a default in the swagger ui component, if none is specified. What?s the OAS file?
milda.abromaviciute
2023-05-09 09:03
is the update of the standalone still using the traveling ruby? or has it been ditched? as that's what was causing the security issues for us previously I believe
haiyang.huang
2023-05-09 09:52
Hi everyone, quick question for you, is there a limit on the number of environments in pactflow? Thanks!
ravi_shankar_lakkims
2023-05-09 14:44
Hi All, would like to know that.., when we go for on premises PactFlow do we need to setup our own broker??
Can I have a comprehensive differences among on premises Vs SASS
yousafn
2023-05-09 14:47
Hey,
All the info to answer your questions are in the provided links :slightly_smiling_face:
https://pactflow.io/features/
https://docs.pactflow.io/docs/on-premises
yousafn
2023-05-09 14:47
> when we go for on premises PactFlow do we need to setup our own broker??
yes
matt.fellows
2023-05-10 01:01
Yep! on-premises means you need to run it on your own kit. Whether cloud, vm or bare metal - you?ll have to install and maintain it.
I?d highly recommend SaaS if you can get it through your GRC/Security team. It?s not a complicated setup, but everything you don?t need to run yourself is saving you time/$$
yousafn
2023-05-11 20:38
Hey,
Please note the community is not a general helpdesk,
take the time to raise a suitable question please, the time of others is valuable, so help them help you
https://docs.pact.io/help/how_to_ask_for_help
You have been asked several times in this channel
can you provide your script, how do you determine your environments, do you have this information available to you when performing the call, if not, you need to look at providing that information to your build
jonatan.jaworski
2023-05-12 01:01
Hi! Does anyone know what the Secrets API contracts look like? There's no documentation in HAL for this endpoint, and I want to see what the contracts look like in order to programatically update tokens and store them in secrets. I know specifying teams is something that can be done because the https://registry.terraform.io/providers/pactflow/pact/latest/docs/resources/secret uses the API under the hood.
jonatan.jaworski
2023-05-12 01:08
Looking forward to https://github.com/pactflow/roadmap/issues/94, by the way!
tjones
2023-05-12 01:52
I'm not a pactflow person, and I don't know the answer to your question for sure - but I've been looking at the pact broker API (which also isn't fully documented). The best approach at the moment is to look at the clients.
Looking at that terraform code, I found this client:
https://github.com/pactflow/terraform-provider-pact/blob/master/resource_secret.go#L89
Where I _think_ this is the payload:
https://github.com/pactflow/terraform-provider-pact/blob/master/broker/secret.go#L3
and this is the HTTP call:
https://github.com/pactflow/terraform-provider-pact/blob/master/client/client.go#L336
(a POST to `/secrets`)
and this is the response:
https://github.com/pactflow/terraform-provider-pact/blob/master/broker/secret.go#L12
You can also see all of the contract that the terraform provider defines https://github.com/pactflow/terraform-provider-pact/blob/master/client/client_pact_test.go#L332
This isn't as good as documentation, of course, as you don't know what else it might support / whether it promises to support these fields etc.
tjones
2023-05-12 01:53
That last link is probably the most important one, so I'll link it again:
https://github.com/pactflow/terraform-provider-pact/blob/master/client/client_pact_test.go#L332
jonatan.jaworski
2023-05-12 02:42
Oh, that's for the open source project actually, there should be one for Pactflow.
jonatan.jaworski
2023-05-12 02:43
A few links https://pactflow.io/pactflow-feature-roadmap/or else #feature-requests
jonatan.jaworski
2023-05-12 02:45
Yeah, it was pretty simple. In case someone else stumbles upon this, the `POST` request looks like this:
```{
"name" : "mySecret",
"description" : "Secret created via API",
"value" : "Super secret",
"teamUuid" : <team_uuid>
}```
yousafn
2023-05-12 03:09
nice work Tim! i just sleuth the network calls in chrome inspector after doing the ui action. looking forward to being able to share the public api ( and in OpenAPI form! )
matt.fellows
2023-05-12 04:46
Thanks! Detail will start to appear on that ticket soon, but feel free to add any comments/wishes :slightly_smiling_face:
tati.shep
2023-05-15 05:51
Hi! Is there a changelog on Pactflow releases? My team started getting some errors on cross verification results for bidirectional contracts, but all worked last week. I?m looking for the clues :pray:
matt.fellows
2023-05-15 06:22
Hi Tatiana, we don?t publish a changelog for all of the changes we make (lots), but usually for the big ones.
We have made changes to BDCT recently, including some bug fixes.
This is likely the one that is causing issues: https://github.com/pactflow/swagger-mock-validator/pull/25
tati.shep
2023-05-15 06:47
Thanks for your reply! Yeah, we started getting such errors on all our covered endpoints:
```Response Body Contains Unknown Information
No schema found for response body
Mismatched Pact Path:
[root].interactions[0].response.body
Mismatched Provider Path:
[root].paths./api/v1/variables.get.responses.200```
It feels like the validator stoped recognising out schemas or something. Let me have a look at the link you?ve shared
voon.wong
2023-05-15 06:56
Hey, I made those changes. There should be better content-negotiation now, and it should try to apply the correct schema. When we launched it, we expected that some number of users were incorrectly passing before, and should be correctly failing with this change. But, it's possible that there are bugs too! Keep us posted!
matt.fellows
2023-05-15 07:03
looking at the above:
> ```No schema found for response body```
I suspect you were a subset of our customers who did not have the correct content types set, and as such, were getting a false sense of security. The previous behaviour was to skip such checks, which is antithetical to the objective here
tati.shep
2023-05-15 07:09
I see! yeah, we wrote OAS spec ?manually? and relied on https://github.com/stoplightio/spectral validation to force us to specify all the fields, but apparently it didn?t.
matt.fellows
2023-05-15 07:17
Let us know how you go, and if you need any assistance in getting that lined up, happy to help
tati.shep
2023-05-15 09:48
UPD: It wasn?t an issue with OAS, it was that we sent the incorrect request `accept` header `"accept": "application/json, text/plain, */*"` in the consumer contract. Changing it to `"accept": "application/json"` helped.
Thanks for your help!
voon.wong
2023-05-15 11:51
:thinking: we didn't account for multiple `accept`. I'll add that to our backlog
matt.fellows
2023-05-15 12:44
That?s good to hear. It makes sense - if you say you can accept multiple content types, but there is only a schema for one of them, we could probably make the assumption that you can handle the one that?s in the OAS. But if there is a match for multiple, we?ll need to deal with that.
yousafn
2023-05-16 10:01
Using `pact-msw-adapter` in your PactFlow bi-directional tests?
You _may_ want to cast your eyes over these PR's
? https://github.com/pactflow/pact-msw-adapter/pull/92
? https://github.com/pactflow/pact-msw-adapter/pull/93
Thanks as always to our community contributors :hug-blob-but-its-okay-because-theyre-vaccinated:
tanyaryzhova93
2023-05-16 16:42
Hi, did you encounter such error when you try to record deployment in PactFlow?
```This version of Pactflow does not support recording deployments, or you do not have the required permission to read environments. Please upgrade to the latest version if using Pactflow On-Premises, and ensure the user has the environment read permission.```
Record deployment works perfectly fine for my other repos but for one repo I have this error.
tanyaryzhova93
2023-05-16 18:24
Tried to record deployment manually on my local machine, and it worked fine. But fails in ci/cd
tanyaryzhova93
2023-05-16 20:54
I reproduced the error and this behaviour looks weird to me. For example, I have the following ci/cd set up in GitLab where I specify `PROVIDER_VERSION` in variables and then I pass this variable to the record deployment job:
```verify contracts:
stage: validate
extends: .pact-aws-cli-v1
variables:
PROVIDER_VERSION: $CI_COMMIT_SHA
script:
- >-
pact-verifier
--hostname="app"
--broker-url="$PACT_FLOW_BASE_URL"
--token="$PACT_FLOW_KEY"
--loglevel="info"
--provider-branch="$CI_COMMIT_REF_SLUG"
--provider-version="$PROVIDER_VERSION"
--provider-tags="$CI_COMMIT_REF_NAME"
--consumer-version-tags="main"
--provider-name=dummy-microservice-provider
--publish
record deployment in pact:
stage: record deployment
extends: .pact-aws-cli-v1
variables:
ENVIRONMENT: $ENVIRONMENT
script:
- >-
pact-broker record-deployment
--pacticipant=dummy-microservice-provider
--version="$PROVIDER_VERSION"
--environment="$ENVIRONMENT"
--broker-base-url="$PACT_FLOW_BASE_URL"
--broker-token="$PACT_FLOW_KEY"
--verbose```
and in this case record deployment fails with the error above. If I provide `--version=$CI_COMMIT_SHA` in the `record deployment` job everything works. Why I can?t use `PROVIDER_VERSION` defined in variables?
And an interesting thing, it works for `can-i-deploy`
tanyaryzhova93
2023-05-16 22:00
well, specifying PROVIDER_VERSION in global variables helped:
```variables:
PROVIDER_VERSION: $CI_COMMIT_SHA```
matt.fellows
2023-05-16 22:21
Is it solved now? Looks like it was a variable in your pipeline that was causing the issue? My guess is that error is showing up because that value was blank and the CLI couldn't find the right resource to call
tanyaryzhova93
2023-05-16 22:23
Yes, the variable was causing the issue. I was confused because everything worked for can-I-deploy job. :woman-shrugging::skin-tone-2:
And yes, it looks like it?s fixed now.
I?ll be 100% sure in several hours :smile:
tanyaryzhova93
2023-05-17 13:14
Hi team, is there a way to download all provider contracts for a service from Pactflow?
aforeman
2023-05-18 02:33
ICYMI :book: :student: :pactflow:
https://pact-foundation.slack.com/archives/C5F4KFKR8/p1684312200649979
tanyaryzhova93
2023-05-18 11:08
download contracts and run provider verification against these contracts
yousafn
2023-05-18 12:23
Why are you not able to use the provider verifier CLI or a provider verifier in your codes native language?
srikanth_vemuri
2023-05-18 12:23
Hi,
I am trying to use `--filter-state` to allow 5 out of 6 interactions in a contract to be verified.
When I run the contract verification, I see only required 5 interactions are running, but he PactFlow website, shows all the 6 interactions with green check mark.
I expect the 6th interaction to be untouched since the verification is not ran against it.
Can someone explain this behavior?
Thanks.
cc: @adam.cox
yousafn
2023-05-18 12:24
you could easily script something up
this will get you the list of `pacticipants`
```curl -H "Authorization: Bearer $PACT_BROKER_TOKEN" "https://test.pactflow.io/pacticipants" | jq .```
but you could also avoid also that jazz, by just using the verifier cli
https://docs.pact.io/implementation_guides/cli#provider-verifier
yousafn
2023-05-18 12:25
the verifier cli accepts consumer version selectors for pact retrieval
https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors
tjones
2023-05-18 13:17
Not a pactflow person, but I don?t think you can publish partial verification. I believe filter state is only for debugging
konstantin.manna
2023-05-19 07:14
Following up on this from Tatiana: Our usecase is that when receiving a new consumer contract for a service that I develop and that is the provider, I do get these new contracts usually as files. When updating the provider-states in this service then locally I want to ensure that it works against the new contract and all the existing published contracts. What has been very helpful in the past (with pact-broker) was to download all contracts for this provider service locally to a folder, add the new contract-file and be able to debug with that (i.e. include, exclude, change) any of the contracts. How do you recommend to achieve this with pactflow?
konstantin.manna
2023-05-19 07:19
iiuc pactflow also internally still uses pact-broker to some degree or is at least compatible with its API, so I guess we could just use the same approach as before to achieve this? Let me know though if there is generally a better approach to achieve the same.
srikanth_vemuri
2023-05-19 12:33
On this topic: https://pact-foundation.slack.com/archives/CLS16AVEE/p1684412631122329
We have a contract that has multiple interactions (say 2 sets of interactions) and these needs to run on some dependent pre-conditions.
We set pre-conditions for set 1 and run those interactions using `--filter-state` for set 1 and the same thing for set 2.
Why does the pactflow makes all interactions green check mark, if I run only set 1 interactions?
Is that by design or a bug in pactflow?
@tjones: Thanks for your response, I don?t see any relation mentioned anywhere in docs on `--filter-state` and `publish`. We should be able to publish partial interaction in a contract.
matt.fellows
2023-05-19 13:35
Why are you doing it this way? Can't you just use regular provider states
matt.fellows
2023-05-19 13:47
I think if you can publish verification results for filtered interactions that's probably a bug on JVM oris a display issue in Pactflow (or both).
ariveros142
2023-05-19 15:56
Hi everyone, is there any docs or examples in JS/TS I can check about the integration of PACT with Swagger, thanks in advance!
matt.fellows
2023-05-19 23:31
I'd start here: https://docs.pactflow.io/docs/bi-directional-contract-testing/
matt.fellows
2023-05-22 07:39
Yep, PactFlow is a superset of the Pact Broker APIs. So you can carry on doing it just as you were
matt.fellows
2023-05-22 07:39
It sounds like the primary problem statement is making it easier for local development/debugging?
matt.fellows
2023-05-22 07:40
Perhaps a CLI command that allows you to download the contracts based on consumer selectors could be the way to go?
matt.fellows
2023-05-22 07:40
basically, if you ran `pact-provider-verifier` CLI with the `--verbose` flag, you could follow the API calls there and use that to download the contracts you needed
adam.cox
2023-05-22 12:47
I'll try to add some context and maybe there is a better way of trying to achieve what we are trying to do.
We have a provider that behaves differently depending on the hardware it is running on (these are embedded devices). If the device uses one chip one of the methods is not available and returns a `success: false` response.
The consumer has two routes in the code, one that uses the method and gets the success response and one that uses the method and gets a failure response and then calls a different method instead.
We have have a contract from the consumer with both the success response and the failure response. We are using provider state to capture whether or not we are expecting the method to be available.
Now we want to verify this contract on the provider side. The contract needs to be verified against two devices with different chips so we are using the provider state filter to get all the interactions for the first chip and then publishing the results. We will then run the verification on the other device with the inverse filter so we can get all of the interactions verified.
Basically, we want to be able to use provider state to know which chip to run the verification against.
We also considered having different provider names for each hardware but seeing as the consumer just thinks of this as a single provider API then it would push the responsibility of knowing which responses come from which hardware to the consumers.
Any advice on how best to approach this would be welcome.
robert.turner
2023-05-22 15:57
Hello, was wondering if someone can help me out. I cloned the https://github.com/pactflow/example-bi-directional-consumer-dotnetand provider repos for the bi-directional testing https://docs.pactflow.io/docs/examples/bi-directional/consumer/mountebank/. They both use a `make` file that references `pactfoundation/pact-cli`. But this does not exist in the repo so it fails. can't see any sign of this folder anyway. Anyone know what the deal is?
yousafn
2023-05-22 16:07
Hey,
Its a docker image, the makefile should pull it down if you have Docker installed on your machine
yousafn
2023-05-22 16:08
The makefile aliases `PACT_CLI`
https://github.com/pactflow/example-bi-directional-consumer-mountebank/blob/e63a9e7d43707b645d2e30b9d6703610ec711a01/Makefile#L10
for the docker run command for that image
robert.turner
2023-05-22 16:08
Hi, thanks - We can't use docker in my company and I use Podman instead
yousafn
2023-05-22 16:09
Just seen your screenshot, it will most likely be down to volume mount paths from windows into your docker container, in your case podman
yousafn
2023-05-22 16:10
you just need to adapt the commands to suit as per the pact-cli readme, you should be able to see the full makefile commands being run in the github actions suites, you'll just need to tailor them to suit for podman, I don't use podman personally
yousafn
2023-05-22 16:11
There are other CLI tools you can use other than via the docker image, its available as a standalone executable or as a ruby gem
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#publishing-the-provider-contract--results-to-pactflow
there is a list and some examples there hope that helps a bit.
yousafn
2023-05-22 16:15
I think the main is the volume mount path, I created an updated version of that Makefile to allow for cross testing with any of the Pact CLI tools
https://github.com/pactflow/example-bi-directional-consumer-cypress/blob/5bec0088a6bd6a28c14a2c9978265a780637c622/Makefile#L149
you'll note the volume mount paths here reference the root to `PWD` with a prefix `/`
`PACT_CLI="docker run --rm -v ${PWD}:${PWD} -e PACT_BROKER_BASE_URL -e PACT_BROKER_TOKEN pactfoundation/pact-cli"`
vs
`PACT_CLI_DOCKER_RUN_COMMAND?=docker run --rm -v /${PWD}:/${PWD} -w ${PWD} -e PACT_BROKER_BASE_URL -e PACT_BROKER_TOKEN pactfoundation/pact-cli:${PACT_CLI_DOCKER_VERSION}`
That seemed to work for me cross platform
robert.turner
2023-05-22 16:18
I think it's working now, thanks - but now failing for a different reason?
yousafn
2023-05-22 16:21
it hasn't been tested for windows, and you will just add another layer of complexity
yousafn
2023-05-22 16:22
check you can run the pact-cli via the docker command, outside the makefile
check you can publish the pact that is baked into the image (using your env vars)
check you can publish the pact from your machine by volume mounting into docker
yousafn
2023-05-22 16:23
probably first place to start is, it looked like the tests ran, is there a pact file somewhere generated :slightly_smiling_face:
yousafn
2023-05-22 16:25
right can you get podman working with a command like this
```docker run --rm \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
publish \
/pact/example/pacts \
--consumer-app-version foo-commit```
yousafn
2023-05-22 16:25
you will need to set the env vars for your shell, were you using cmd prompt? or pwsh?
yousafn
2023-05-22 16:28
wsl will be your easier bet all round, (1 or 2?)
however it _does_ work on windows, just not OOB with our instructions.
also git bash (and the bash prefix command in the different windows shells is odd) as it seems to use wsl, which confuses me and scripts :smile:
robert.turner
2023-05-22 16:29
Lol, just realised that's where that screen shot is from :facepalm: Not from my bash window
yousafn
2023-05-22 16:30
all good homie! I'm here for an hour or so before I grab some grub so you've got time to grab a brew
yousafn
2023-05-22 16:39
just testing out now,
which is your preferred windows shell?
? cmd prompt
? pwsh 5
? pwsh 7
? git bash
? wsl2
Happy to help you support any of them, for dotnet, we can probably update the repo to 6.x or 7.0, as 3.1 is old now
yousafn
2023-05-22 16:43
winner winner chicken dinner!
yeah `export PACT_BROKER_BASE_URL=https://foobar.pactflow.io`
nice, I can get you sorted on cmd prompt too, I stretched my legs on there back in the day before becoming a mac boi, and I recently found oh my posh, so all my windows shells are dead pretty so don't mind an excuse dipping back into to devving on a windows box
yousafn
2023-05-22 16:46
correct, you can try our public one out with this
```export PACT_BROKER_BASE_URL=https://test.pactflow.io
export PACT_BROKER_TOKEN=129cCdfCWhMzcC9pFwb4bw```
and check if they work?
yousafn
2023-05-22 16:48
okay cool so maybe we just need to either check your creds are correct set (you can use `echo $PACT_BROKER_TOKEN` in your shell to see it matches up
yousafn
2023-05-22 16:52
so the repo was recently updated to poll and wait for a verification result, as the provider side hasnt been published there will be no verification result so it will time out
yousafn
2023-05-22 16:52
I also note its using the pact net beta version, you can see the delta of moving to the latest pact-net 4.x in these PR's
https://github.com/pactflow/example-consumer-dotnet/pull/8/files
https://github.com/pactflow/example-provider-dotnet/pull/9
yousafn
2023-05-22 16:53
I can add that to an ever growing list of tasks hehe hopefuly that will sort you out, happy to accept any PR's back to the repo to make you or others lifes easier in the future :slightly_smiling_face:
yousafn
2023-05-22 16:54
even an echo statement saying if you havent published your provider yet, this will time out after x seconds, why not run your chosen provider ci now?
yousafn
2023-05-22 17:17
hey hey
so this seems to work for me
https://github.com/pactflow/example-bi-directional-provider-dotnet/compare/dotnet6?expand=1
yousafn
2023-05-22 17:21
you can just checkout that branch if you want for the moment, id rather get the CI passing before I merge :slightly_smiling_face:
yousafn
2023-05-22 17:31
so that script is going to try and run docker, which I assume you don't have in wsl
yousafn
2023-05-22 17:32
got it working in CI, needed to match the dotnet version because its set in global.json
yousafn
2023-05-22 17:33
so it runs `dotnet example-bi-directional-provider-dotnet/bin/Debug/netcoreapp6.0/example-bi-directional-provider-dotnet.dll` and grabs the pid
yousafn
2023-05-22 17:34
so now we've got
`docker run --net="host" schemathesis/schemathesis:stable run --stateful=links --checks all http://host.docker.internal:9000/swagger/v1/swagger.json > report.txt`
yousafn
2023-05-22 17:35
```?18:33:41 ? docker run --net="host" schemathesis/schemathesis:stable run --stateful=links --checks all http://host.docker.internal:9000/swagger/v1/swagger.json
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
======================= Schemathesis test session starts =======================
Schema location: http://host.docker.internal:9000/swagger/v1/swagger.json
Base URL: http://host.docker.internal:9000/
Specification version: Open API 3.0.1
Workers: 1
Collected API operations: 2
GET /Products . [ 50%]
GET /Products/{id} . [100%]
=================================== SUMMARY ====================================
Performed checks:
not_a_server_error 101 / 101 passed PASSED
status_code_conformance 101 / 101 passed PASSED
content_type_conformance 101 / 101 passed PASSED
response_headers_conformance 101 / 101 passed PASSED
response_schema_conformance 101 / 101 passed PASSED
Hint: You can visualize test results in http://Schemathesis.io by using `--report` in your CLI command.```
robert.turner
2023-05-22 17:35
do I need to run this in a new window with the eample app still running?
yousafn
2023-05-22 17:35
oh yes please, if you could run in a new wsl terminal :slightly_smiling_face:
robert.turner
2023-05-22 17:41
podman can be a bit eggy with docker links. stick http://docker.io/ in front an it will find em
yousafn
2023-05-22 17:42
so host.docker.internal allows a docker container to talk to the host platform and is required on mac/windows using docker because not linux
yousafn
2023-05-22 17:43
podman might not have that issue, or you might need to use a special format so localhost doesn't resolve to the container and instead resolves to your local machine which is running the provider
yousafn
2023-05-22 17:44
it relates to this workaround in docker
https://docs.docker.com/desktop/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host
yousafn
2023-05-22 17:45
this looks like your solution
https://github.com/containers/podman/issues/10878#issuecomment-877012025
yousafn
2023-05-22 17:48
yep that looks current from the docs
https://docs.podman.io/en/v3.3.0/markdown/podman-pod-create.1.html#network-mode-net
robert.turner
2023-05-22 17:50
Ah, I'm conscious that you said you had to go in an hour over an hour ago
tjones
2023-05-23 00:12
> so we are using the provider state filter to get all the interactions for the first chip and then publishing the results. We will then run the verification on the other device with the inverse filter so we can get all of the interactions verified.
This makes me nervous - how would you check that you have filtered the interactions correctly and not missed any?
tjones
2023-05-23 00:17
With Pact, I probably would write a script that:
? Runs one chip
? Runs the other chip
? Publishes verification success or failure
But that would have the problem I mentioned above.
I see that granular verification results could solve this, but I think it would be a risky feature to have
tjones
2023-05-23 00:20
> Basically, we want to be able to use provider state to know which chip to run the verification against.
This is a natural fit for state variables `fromProviderState` but I don't think Pact lets you use those in configuration
adam.cox
2023-05-23 08:55
Not HTTP contract, its WebSockets. We have made a plugin to support this.
Languages: C++ for both.
We are currently exploring an option where we use a proxy to sit in front of the devices. The proxy will have a registry of the states which need to be sent to which device. We can then point the verifier at the proxy and use the state-change-url to tell the proxy which device it should be proxying to (hope that makes sense!).
However, I am curious about this:
> With Pact, I probably would write a script that:
> ? Runs one chip
> ? Runs the other chip
> ? Publishes verification success or failure
Are the verification results being written to a file by the pact-standalone-verifier, such that we can combine the results from each chip? I thought the results were being held in memory and just sent up at the end of the process. tbh I haven't looked into the mechanics of the verification result publishing at all yet.
> This makes me nervous - how would you check that you have filtered the interactions correctly and not missed any?
It is a good question. We thought that if there were partial results then the contract would not be marked as verified until all interactions were green lit and therefore that would allow us to know that all interactions in a contract were verified and we hadn't missed any.
Hopefully we will be moving towards a system where the provider has consistent behaviour across all chips that are supported. However this a way off unfortunately and until we have that we are going to have to be diligent in ensuring our tests are covering all the right things.
robert.turner
2023-05-23 09:51
Hi @yousafn thanks a lot for yesterday. Really helpful. Had to iron through a few more issues but think I'm nearly there now.
Current status is that I can run the `make fake_ci` successfully for the provider, but the consumer still fails at the `can-i-deploy` step in it's `make fake_ci` step.
Not sure where to go from here because that's where the example documentation ends.
yousafn
2023-05-23 10:11
the makefile has a condition that will only do the deploy step from the main/master branch
yousafn
2023-05-23 10:11
> thanks a lot for yesterday. Really helpful. Had to iron through a few more issues but think I?m nearly there now.
Pleasure, glad you are cooking on gas
robert.turner
2023-05-23 10:21
On the side, I have another issue with the consumer tests. They always fail because they can't start the mock server unless I restart my PC. Obviously there is some sort of running process stopping it from being able to start. I don't suppose you know what that is?
yousafn
2023-05-23 10:45
The consumer side more than likely wants updating to latest pact net (non beta) once I?ve done that I?ll merge both the PR?s
ivan.mikhalka
2023-05-23 11:11
Hi, how ongoing changes in Pactflow free plan (1 July) will affect admins? Both users and admins going to have same access level?
yousafn
2023-05-23 13:11
Right updated both repos to .net 6 and pact-net 4.5.0 on the consumer side, all passing in CI.
Is your provider marked as deployed in production now?
robert.turner
2023-05-23 14:19
It is yeh. Let me just go get latest everything, get back on main for the provider and see where that gets me :+1:
konstantin.manna
2023-05-23 15:02
Yeah I've did it with that
```import json
import os
from pathlib import Path
import httpx
Path("provider_pacts").mkdir(parents=True, exist_ok=True)
headers = {"Authorization": f'Bearer {os.getenv("PACTFLOW_API_TOKEN")}'}
with httpx.Client(headers=headers) as client:
for pact in client.get(
"https://codility.pactflow.io/pacts/provider/task-service/latest",
).json()["_links"]["pb:pacts"]:
Path(f"provider_pacts/{pact['name']}_contract.json").write_text(
json.dumps(client.get(pact["href"]).json(), indent=4),
)```
robert.turner
2023-05-23 15:35
Sorry, had to go to a meeting. Something seems to have gone wrong when I pulled latest because now the provider project doesn't load in VS :disappointed:
yousafn
2023-05-23 15:37
can you check what version of dotnet with `dotnet --version` you may need to update the global.json to reflect your installed version
https://github.com/pactflow/example-bi-directional-provider-dotnet/pull/16/files#diff-8df3cd354bc584349d04ad5675b33c042d8b99b741b8b95af394c55e0f5001bfR3
robert.turner
2023-05-23 15:37
says `C:\dev\TestGround\PactflowStuff\contract-testing-poc\pactFlow\dotnet\example-apps\provider\example-bi-directional-provider-dotnet\example-bi-directional-provider-dotnet.csproj : error : The project file cannot be opened by the project system, because it is missing some critical imports or the referenced SDK cannot be found.` So I installed the latest SDK to see if that fixed it and it hasn't. Though I did just think I probs just need to restart my computer or something
yousafn
2023-05-23 15:37
> Though I did just think I probs just need to restart my computer or something
This is almost, always, never a bad shout
robert.turner
2023-05-23 15:58
Not sure which refs, it's only referenced in that global.json file. changing that to match my version of .Net didn't fix it
lori.maurais
2023-05-23 16:21
Hi, I?m just getting started on a pactflow poc. I have a simple contract (messaging) published on the consumer side (java), but when I try to run the provider test (node) I am unable to download that contract from the broker. Any help would be appreciated. Thanks! Pasting log in a thread
lori.maurais
2023-05-23 16:21
``` RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
[12:18:55.275] INFO (86088): pact@11.0.2: Verifying message
[12:18:55.299] INFO (86088): pact-core@13.13.8: Verifying Pacts.
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
2023-05-23T16:18:57.198115Z WARN ThreadId(20) pact_models::pact: No metadata found in pact file "https://linostest.pactflow.io", assuming V3 specification
2023-05-23T16:18:57.199289Z WARN ThreadId(20) pact_models::pact: No metadata found in pact file "https://linostest.pactflow.io", assuming V
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
2023-05-23T16:18:59.216868Z INFO ThreadId(20) pact_verifier::pact_broker: Fetching path '/pacts/provider/linos-backend/for-verification' fr
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
2023-05-23T16:19:00.123939Z ERROR ThreadId(20) pact_verifier: Failed to load pact - Could not load pacts from the pact broker 'https://linostest.pactflow.io'
2023-05-23T16:19:00.124025Z WARN ThreadId(20) pact_matching::metrics:
Please note:
We are tracking events anonymously to gather important usage statistics like Pact version and operating system. To disable tracking, set the 'PACT_DO_NOT_TRACK' environment variable to 'true'.
FAIL apps/core-service/src/tests/Contract/aapiContract2.spec.ts (13.017s)
message provider tests
send an event
? Damaged flowrack inventory exception (5024ms)
? message provider tests ? send an event ? Damaged flowrack inventory exception
: Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error:
69 |
70 | describe('send an event', () => {
> 71 | it('Damaged flowrack inventory exception', () => {
| ^
72 | return p.verify();
73 | });
74 | });
at new Spec (node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
at Suite.<anonymous> (apps/core-service/src/tests/Contract/aapiContract2.spec.ts:71:5)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 total
Snapshots: 0 total
Time: 13.097s
Ran all test suites matching /.\/apps\/core-service.+\aapiContract2.spec.ts/i.
Verifying a pact between consumer and provider
WARNING: Pact file has no interactions
Failures:
1) Failed to load pact - Could not load pacts from the pact broker 'https://linostest.pactflow.io'
There were 1 pact failures
[12:19:00.544] ERROR (86088): pact-core@13.13.8: Verification unsuccessful
/Users/m_884005/Documents/Lineage/repos/linos-backend/nestjs-monorepo/node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.js:62
reject(new Error('Verfication failed'));
^
Error: Caught error after test environment was torn down
Verfication failed
at /Users/m_884005/Documents/Lineage/repos/linos-backend/nestjs-monorepo/node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.ts:52:20```
matt.fellows
2023-05-23 21:47
Can you please share your provider code setup?
Looks like you're passing the Pactflow host as a pact file. You probably want to look at the pact broker / token options as well as selectors. See the examples (howtoexamples)
lori.maurais
2023-05-23 21:54
Hi Matt, Sure?
```const baseOpts = {
// logLevel: 'DEBUG',
provider: 'linos-backend',
providerVersion: '1.0', // process.env.GIT_COMMIT,
pactBroker: 'https://linostest.pactflow.io',
consumerVersionTags: [],
// providerVersionTags: process.env.GIT_BRANCH ? [process.env.GIT_BRANCH] : [],
// pactBrokerUrl: ,
pactBrokerToken: 'ogmEqX8PNkmfHnOJ4W16TQ',
};
// For builds triggered by a 'contract content changed' webhook,
// just verify the changed pact. The URL will bave been passed in
// from the webhook to the CI job.
const pactChangedOpts = {
pactUrls: ['https://linostest.pactflow.io'],
};
// For 'normal' provider builds, fetch `master` and `prod` pacts for this provider
const fetchPactsDynamicallyOpts = {
provider: 'linos-backend',
consumerVersionTags: ['master', 'prod', 'main'], // the old way of specifying which pacts to verify
// consumerVersionSelectors: [{ tag: 'master', latest: true }, { tag: 'prod', latest: true } ], // the new way of specifying which pacts to verify
pactUrls: ['https://linostest.pactflow.io'],
pactBrokerUrl: 'https://linostest.pactflow.io',
pactBrokerToken: 'ogmEqX8PNkmfHnOJ4W16TQ',
enablePending: false,
};
const opts = {
...baseOpts,
...(process.env.PACT_URL ? pactChangedOpts : fetchPactsDynamicallyOpts),
messageProviders: {
'Damaged flowrack inventory exception': () =>
// messageService.sendMissingFlowrackInventoryRequest('46',{
messageService.buildAAPIMessage({
type: 'MissingFlowrackInventoryRequest',
input,
facilityId: 'facilityId',
}),
},
};
it('Damaged flowrack inventory exception - message provider pact', () => {
const p = new MessageProviderPact(opts);
// let msg = messageService.buildAAPIMessage({
// type: 'MissingFlowrackInventoryRequest',
// input,
// facilityId: 'facilityId',
// })
// console.log("Message: " + msg)
console.log(`opts: ${JSON.stringify(opts, undefined, 2)}`);
return p.verify();
});```
lori.maurais
2023-05-23 21:54
here are what the options look like when running:
``` console.log
opts: {
"provider": "linos-backend",
"providerVersion": "1.0",
"pactBroker": "https://linostest.pactflow.io",
"consumerVersionTags": [
"master",
"prod",
"main"
],
"pactBrokerToken": "ogmEqX8PNkmfHnOJ4W16TQ",
"pactUrls": [
"https://linostest.pactflow.io"
],
"pactBrokerUrl": "https://linostest.pactflow.io",
"enablePending": false,
"messageProviders": {}
}```
lori.maurais
2023-05-23 21:58
I did try to remove the pactUrls (see options below), but got the same result - could not load pacts
``` opts: {
"provider": "linos-backend",
"providerVersion": "1.0",
"pactBroker": "https://linostest.pactflow.io",
"consumerVersionTags": [
"master",
"prod",
"main"
],
"pactBrokerToken": "ogmEqX8PNkmfHnOJ4W16TQ",
"pactBrokerUrl": "https://linostest.pactflow.io",
"enablePending": false,
"messageProviders": {}
}```
matt.fellows
2023-05-23 22:45
Can you please share the error/logs when you have removed the `pactUrls` item? That should be populated by values from a webhook, or if you have a direct URL to a pact you want to verify
matt.fellows
2023-05-23 22:48
If the second options can?t find a pact file, it?s probably because the tags don?t match what?s been published
matt.fellows
2023-05-23 22:49
But also, you shouldn?t be using tags - you should be using branches. See the CI/CD workshop here: https://docs.pactflow.io/docs/workshops/ci-cd/
(or copy how the examples do it)
Our killercoda workshops should also be using branches and environments - may I ask where you?re getting the tags example from?
matt.fellows
2023-05-23 22:50
Lastly, please regenerate your API token (you just published it in a public forum)
jonatan.jaworski
2023-05-24 03:55
Hi everyone - Does Pactflow have any sort of rate limiting for certain endpoints in their API? I'm trying to automate token rotation for webhooks, and what I have in mind will rotate auth tokens for all the teams in the organisation once a week; as a result I'll be firing a bunch of requests to update the value of a secret for every team, and I'm wondering if that's going to fail due to rate limiting.
matt.fellows
2023-05-24 03:57
We do, but we don?t publish the exact numbers. They are fairly generous, so I wouldn?t expect you to hit them. There is a current issue with the WAF we use that unfortunately doesn?t return a `429` for when you hit the limit and instead returns a `401` I think
This is actually going to be addressed in the near future (our security team is in the process of updating the WAF to a later version), but just so you?re aware of it
ravi_shankar_lakkims
2023-05-24 09:57
Hi Pactflow Team,
Need guidance on how to setup Pactflow on premises, had below questions. I will be glad someone shedding some light on those questions
1. While setting up Pactflow on premises, is that the whole Pactflow foundation that is going to be get setup on premises??
2. Is PactFlow Broker is an individual component of Pactflow which can be setup individually??
yousafn
2023-05-24 10:02
Hey,
I am going to assume you haven't read the on prem docs based on your questions, so I would suggest
https://docs.pactflow.io/docs/on-premises
yousafn
2023-05-24 10:03
1. While setting up Pactflow on premises, is that the whole Pactflow foundation that is going to be get setup on premises??
no
1. Is PactFlow Broker is an individual component of Pactflow which can be setup individually??
if your on prem, you set up the broker and the db, and all other related infra.
If your SaaS, you sit back and don't worry about any of that
ravi_shankar_lakkims
2023-05-24 10:06
Thank you so much Nabi for the quick response. and thanks for answering my questions as well. Let me deep dive and bounce back if in case If I have more questions.
matt.fellows
2023-05-24 11:30
> 1. Is PactFlow Broker is an individual component of Pactflow which can be setup individually??
PactFlow _is_ the open source Pact Broker with additional things. It?s the on-premises version of the SaaS product at http://pactflow.io
lori.maurais
2023-05-24 12:27
Good Morning, Matt. The example code I?m following is here: https://github.com/pactflow/example-provider-js-sns/blob/master/src/product/product.pact.test.js. I?ll be the first person to admit I don?t know what I?m doing, so I?m sure some of it seems crazy to you - I?ve been changing up options hoping something will work. Logs from the run without the pactUrls:
```> nestjs-monorepo@0.0.1 test:contract2
> jest --testTimeout 30000 --no-cache './apps/core-service.+\aapiContract2.spec.ts' --maxWorkers=4
jest-haste-map: Watchman crawl failed. Retrying once with node crawler.
Usually this happens when watchman isn't running. Create an empty `.watchmanconfig` file in your project's root folder or initialize a git or hg repository in your project.
Error: Watchman error: std::__1::system_error: open: /Users/m_884005/Documents/Lineage/repos/linos-backend/nestjs-monorepo/apps: Operation not permitted. Make sure watchman is running for this project. See https://facebook.github.io/watchman/docs/troubleshooting.
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
[08:25:37.718] INFO (10083): pact@11.0.2: Verifying message
console.log
opts: {
"provider": "linos-backend",
"providerVersion": "1.0",
"pactBroker": "https://linostest.pactflow.io",
"consumerVersionTags": [
"master",
"prod",
"main"
],
"pactBrokerToken": "xxxxxxx",
"pactBrokerUrl": "https://linostest.pactflow.io",
"enablePending": false,
"messageProviders": {}
}
at Object.<anonymous> (apps/core-service/src/tests/Contract/aapiContract2.spec.ts:76:13)
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
2023-05-24T12:25:39.785510Z INFO ThreadId(17) pact_verifier::pact_broker: Fetching path '/pacts/provider/linos-backend/for-verification' from pact
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
2023-05-24T12:25:40.553318Z ERROR ThreadId(17) pact_verifier: Failed to load pact - Could not load pacts from the pact broker 'https://linostest.pactflow.io'
2023-05-24T12:25:40.554256Z WARN ThreadId(17) pact_matching::metrics:
Please note:
We are tracking events anonymously to gather important usage statistics like Pact version and operating system. To disable tracking, set the 'PACT_DO_NOT_TRACK' environment variable to 'true'.
RUNS apps/core-service/src/tests/Contract/aapiContract2.spec.ts
Failures:
1) Failed to load pact - Could not load pacts from the pact broker 'https://linostest.pactflow.io'
There were 1 pact failures
FAIL apps/core-service/src/tests/Contract/aapiContract2.spec.ts (11.558s)
message provider tests
? Damaged flowrack inventory exception - message provider pact (3600ms)
? message provider tests ? Damaged flowrack inventory exception - message provider pact
Verfication failed
at node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.ts:52:20
Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 total
Snapshots: 0 total
Time: 11.636s
Ran all test suites matching /.\/apps\/core-service.+\aapiContract2.spec.ts/i.
(base) m_884005@AMBUS000029 nestjs-monorepo % ```
lori.maurais
2023-05-24 12:30
I?m not triggering any of this with CICD - I?m just trying to get the tests to run locally at this point - I?m at the beginning of this journey. The contract was published without any tags, or branches - just a vanilla publish from the maven plugin on the consumer side.
garry.wilson
2023-05-24 12:53
I'd like to some clarity on this too, please.
We have a small team using Pact, so we are all in a single team as `users`, but a few users are specifically `admins` too, and a bot user with the `CI/CD, SwaggerHub` roles. when the RBAC is removed as part of the the change to free plans, will every user become an `admin`? or something else?
tanyaryzhova93
2023-05-24 13:13
Hi Team!
We are in the process of rolling out Pact across our services and we would love to track the adoption. Do you have any features in the platform to show the statistics/usage data behind PactFlow? For example, we?re looking for:
? number of validations per service/per service couple per last X days
? heatmap of contract checks in the system (eg. similar to Network Diagram available in the app but with usage on top of it)
If not, do you have any suggestions on what PactFlow could offer us in this case?
yousafn
2023-05-24 13:21
Hey Ivan/Garry!
I've pinged this query onto the team, so they will endeavour to get back to you as soon as they can :+1:
yousafn
2023-05-24 13:24
NP, may be useful just to drop your customer accounts here (pactflow url) so we can tie you back to an account, and whoever reaches out can have an informed discussion with you. (Just thinking forward)
Edit: happy for it to be via PM :slightly_smiling_face:
matt.fellows
2023-05-25 07:04
Hi Tatiana,
We have reporting/insights as a Q4 roadmap item: https://github.com/pactflow/roadmap/projects/1
In the meantime, there is a `/metrics` endpoint you can query to get summary you they might find useful. Some of our customers hit this endpoint regularly, and ingest into tools like Splunk/ELK and visualise the trend from there.
You could potentially also scrape the https://pactflow.io/blog/audit-api/ and do a similar thing.
matt.fellows
2023-05-25 07:06
It?s because you?re telling Pact to find contracts by tag (`master`, `prod` and `main`) but none of the contracts you have published have those tags - hence it can?t find them
matt.fellows
2023-05-25 07:07
I?d suggest starting here: https://docs.pactflow.io/docs/tutorials#getting-started-with-pactflow
You can do a full end-to-end in the browser and then once you?ve got the gist of it, translate to code on your desktop
matt.fellows
2023-05-25 07:14
Writing pact tests is one part, and then being able to link them to both sides (consumer/provider) is the other key bit. We use things called ?consumer version selectors? on the provider tests to discover the pacts (the relevant integration points in different states of development/deployments) we care about. In your case, the tags don?t line up so it?s not finding anything to verify.
I think you could also just drop the `consumerVersionTags` property and it would just find the latest, which is probably fine for a POC
giulio.giovannini
2023-05-25 08:51
Hi! I am testing the bi-directional contract testing and I have a problem with pactoflow not matching the resource url between the consumer and the provider.
The consumer contract has a request to path `/mybaseurl/resource` while the provider Open API Spec yaml starts with a servers parameter like:
```servers:
- url: /mybaseurl```
and then defines the resource as:
``` /resource
post:
...```
Pactflow can not match the paths as it disregards the servers parameter. The open Api Spec is autogenerated by Springboot so I can not change it that much.
Is there a workaround for this?
yousafn
2023-05-25 09:20
> The open Api Spec is autogenerated by Springboot so I can not change it that much.
You can, that is the major selling point of using a machine readable specification
yousafn
2023-05-25 09:21
> Is there a workaround for this?
I would assume as an initial pass, it would be to add the path on servers.url[0] into the front of each of your path entries
giulio.giovannini
2023-05-25 10:03
Hi. Thanks for the info. So, you are saying to edit the AOS in order to add the basePath to each of the paths before publishing it to pactflow, correct?
yousafn
2023-05-25 10:11
I believe based on what you've said about your consumers requests that should satisfy it.
Worth a shot :slightly_smiling_face:
giulio.giovannini
2023-05-25 11:13
Yes, that will work and in my tests actually worked. But editing a OAS that was auto-generated to make it match the consumer contract looks a bit of a hack. Will do that for now. I will keep looking for a better solution.
yousafn
2023-05-25 11:39
schema compliance is only going to get you so far, performing full pact provider verification will offer more guarantees
yousafn
2023-05-25 11:40
> But editing a OAS that was auto-generated to make it match the consumer contract looks a bit of a hack.
You?d be surprised at the amount of auto-generated OAS?s that do need work to be usable/human readable
yousafn
2023-05-25 11:41
its also use cased based, you wouldn?t do it for every case, but if this case, you know it works, then its a valid solution, you don?t just completely modify the OAS to not reflect reality
yousafn
2023-05-25 11:42
I very rarely advocate for code generated OAS?s or if they are auto generated, to be ratified against a design first approach to an API (ie a hand crafted OAS based on actual expectations, rather than a result of the code)
yousafn
2023-05-25 11:43
ie how do you know the auto generated OAS completely reflects that of your code, and also completely reflects the requirements of your API in the first place
giulio.giovannini
2023-05-25 12:26
For this last question, I have to trust the OAS generating engine. In theory it implements the same rules that are used to marshal the incoming JSON to the classes the HTTP body is mapped in. I haven't checked but I tend to trust that it will reproduce what is in the code.
caitriona.gallagher
2023-05-25 13:03
@garry.wilson @ivan.mikhalka happy to assist you both! I will slack you both individually and we can take this offline :slightly_smiling_face:
lori.maurais
2023-05-25 13:20
Thanks, Matt. I?ve already done the in-browser tutorials in both Java and Node - they were very helpful. Now I?m trying to get this to work on our own code base, which is where I?ve hit a couple of hiccups:
srikanth_vemuri
2023-05-25 13:20
@tjones, @matt.fellows: For this use case, we logged partial interaction results to a file using `--json` option.
The plan is to combine the results from from 2 partial interaction verifiers into one .json file and post to pactflow.
Is there a way to post the .json verifier results to pactflow?
mike.key
2023-05-25 16:06
Is it expected behavior that a user with role Test Maintainer cannot filter on environments in the matrix?
lori.maurais
2023-05-25 18:13
I have published with tags and can now pull the contract on the provider side - so that is resolved.
matt.fellows
2023-05-25 22:25
I?ve created https://github.com/pactflow/swagger-mock-validator/issues/28 to track this Giulio.
There was an internal card for this already, but you can follow the progress above.
As there is a relatively simple workaround it?s not at the top of the pile, and when there are multiple servers provided the rules for validation become a little more messy.
matt.fellows
2023-05-29 00:38
the way to work it out is to add DEBUG logging to the verifier to see what it does, and then follow that
matt.fellows
2023-05-29 00:38
We?re in the process of documenting our APIs, but in the meanwhile that?s probably the fastest way forward.
matt.fellows
2023-05-29 00:39
The endpoint to post verification results to is a relation on the JSON response for the pact that is being verified, if I recall
jonatan.jaworski
2023-05-29 02:48
Just a heads up in regards to idempotence: that post request from above creates a secret and returns a uuid. Subsequent identical POST requests will result in 409, because the name of the secret must be unique within a team when creating it.
In order to update its value, the url must contain uuid of the secret (so, it'll look something like `/secrets/Lifqj_IK0GZXMB5BQFshNw` instead of just `/secrets`).
tanyaryzhova93
2023-05-30 18:39
Hi team! I have a question regarding tags in PactFlow. I have the following situation:
I publish contracts with `main` tag to PactFlow only when changes were introduced to the `app` directory. Also, I assign a contract version as a commit id.
However, I encountered a situation when `main` tag is unassigned from the contract. Contract has no tag and a new contract wasn?t published.
It happens when changes are made NOT in the `app` directory, changes merged to a main branch, and in the end `main` tag is unassigned from the contract somehow. I am confused here, why the contract loses the tag if another contract wasn?t published, we didn?t run contract publishing job.
Did you encounter such situation? Do you have any ideas why it may happen?
matt.fellows
2023-05-31 00:00
Hi Tatiana,
Sorry I can?t fully follow this scenario. Would you mind clarifying by perhaps stepping through the sequence of events (e.g. CLI calls and/or screenshots of each step)?
Tags shouldn?t disappear on their own, but I think this is what your question is?
You could also consider looking at the audit trail API, you might be able to see any calls that have modified/deleted resources relating to the integration.
tanyaryzhova93
2023-05-31 00:39
:one: Changes were introduced in the *`app`* directory and merged to the `main` branch.
Pipeline was created and contract tests were executed and a contract was published to the PactFlow with `main` tag and `commit id` as a version.
On screenshot you can see `main` tag was assigned. And you can see `publish contracts` job in the pipeline
Everything is ok here
tanyaryzhova93
2023-05-31 00:43
:two: Changes were made *NOT* in the `app` directory, in the `infra` directory for example. The changes from the `infra` directory were merged to the `main` branch.
According to the pipeline rules, we do not run `publish contracts` when changes were made in `infra`.
So, new contract wasn?t published to the PactFlow. But the latest published contract lost `main` tag (I don?t have screenshot) but there was `N/A` in the tags.
tanyaryzhova93
2023-05-31 00:44
yes, I am trying to understand why it disappeared.
Do I understand correctly that the tag should be overwritten only when a new contract version was published to the PactFlow?
tanyaryzhova93
2023-05-31 01:48
I am checking audit endpoint right now, and I see some events for the second changes made in the `infra` dir. I see the following events:
? branch_version
? pacticipant_version_tag
these events repeated several times
But why I see these events in the `audit` for the second change? I didn?t communicate with PactFlow in the Ci/CD when these changes were made?.I would expect to see NO events for the second change.
matt.fellows
2023-05-31 01:57
Thanks. Can you please help me understand the relevance of the `app` and `infra` directories?
matt.fellows
2023-05-31 01:57
Any chance you could share the URL of the contract with the tag on it and when it has been removed? We can look at our side and tell you what we see
tanyaryzhova93
2023-05-31 01:58
`infra` contains terraform files and everything related to the infra
`app` contains the code of the app (API for example)
tanyaryzhova93
2023-05-31 01:59
Let me ask my manager first if I can do that. If they allow me to share the contract, can I send you this info in the private message?
matt.fellows
2023-05-31 02:00
I think in this case, it might be best to raise a support ticket and this way we can track it.
2023-05-31 02:00
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
matt.fellows
2023-05-31 02:00
You can share a link to this conversation so you don?t need to repeat yourself, it will get escalated to the right people straight away
matt.fellows
2023-05-31 02:01
My only guess is that somebody else is publishing over the top of it and it?s a new version that is missing the tag (if you check the URL and note a different version then that could be it)
tanyaryzhova93
2023-05-31 02:07
but there is no new contract in the PactFlow, just a contract from the case 1 with `N/A` tag.
matt.fellows
2023-05-31 02:08
is it a new _version_ of the contract though? That?s one question we need to clarify
tanyaryzhova93
2023-05-31 02:13
ok, let me ask my manger if I can share data and I?ll create a request.
Also, my company uses Enterprise plan of PactFlow. Would it better to send an email to our supporting person from Pact or create a request as you mentioned above?
matt.fellows
2023-05-31 03:30
The support link above is for enterprise customers, so you should be good. But you can share the link to your account manager who can follow along
alexander.friesen
2023-05-31 07:49
Hi all!
I am working under "*Pactflow Enterprise - Cloud*" plan.
In https://pactflow.io/features/ I see, that "*SAML 2.0 Provider"* is supported, as an Authentication method.
When looking into "https://siemens.pactflow.io/settings/authentication" I see 2 Authentication methods
? Google Authentication
? GitHub Authentication
*Question*: where do I set up the SAML authentication?
nada
2023-05-31 10:42
Hi all!
I have one silly question :slightly_smiling_face: Did anyone here tried to generate the contract tests with chatGPT? I mean it would be fantastic that for the old services I can generate the contract test without to much of the effort by devs :slightly_smiling_face:
matt.fellows
2023-05-31 11:10
See the instructions here: https://docs.pactflow.io/docs/user-interface/settings/authentication#saml-support
You'll need to raise a support ticket when ready to setup
matt.fellows
2023-05-31 11:13
You can try, but the first example I saw resulted in bug raised on Pact JS which wasted a bunch of people's time as it was nonsensical.
I think it will be a little longer before it can generate from scratch but I reckon it'll be useful in augmenting.
Good reminder for me to test out codepliot too!
thomas.koppensteiner
2023-05-31 12:20
Hello,
I'm working with version `PactFlow Version: 0dde05985`, which fixes are included there?
Does it already include https://github.com/pactflow/swagger-mock-validator/pull/18 ?
yousafn
2023-05-31 13:00
Hey @thomas.koppensteiner
Looks like version v12.1.0 of swagger-mock-validator went in last week in release `57e3d7bea`
matt.fellows
2023-05-31 13:01
It should have all of them included up to #26. #18 was closed but not merged, the fix there is incorporated in the most recent merge, which contains numerous improvements to content-negotiation and content matching
mike.key
2023-05-31 16:29
I am just making sure. Why can test maintainer not see what is in each environment? What is the right role to give people that need to use the matrix based on environments?
eli.thorkelson
2023-05-31 18:16
Hi! I?m trying to test a new CI integration with PactFlow and my user account was just upgraded from `Viewer` to `Viewer, CI/CD`. I tried to generate a read/write API token for testing, but the Api Tokens page only shows me information about a Read only token. Anyone know how I should proceed?
simon.selvadurai
2023-06-01 03:59
Hi Eli,
Currently you have a Viewer, which has the ability to view the read-only token
To be able to view the read/write token you need the permission to `token:manage:own`
1. You need to be assigned a role that has that permission - such as the `User` role (https://docs.pactflow.io/docs/permissions/predefined-roles#user)
2. Create a new role that has the specific set of permissions they need, including `token:manage:own`
robert.turner
2023-06-01 13:18
Hi all, can PactFlow be used to test hidden endpoints. E.g. what if the provider has an endpoint markerd with `[ApiExplorerSettings(IgnoreApi = true)]` or the like?
tjones
2023-06-01 13:48
Are you asking if Pactflow?s swagger validation will work in a case like that? Or whether Pact in general will have any problems with such an endpoint?
robert.turner
2023-06-01 13:49
Is there a neat way around it or is it a case of if it's hidden from the swagger then it's just not possible
tjones
2023-06-01 14:38
I would assume it?s not possible if it?s not in the swagger, but you?ll have to wait for a pactflow person to answer (many of them are asleep)
thomas.koppensteiner
2023-06-01 15:09
Javascript error on http://pactflow.io
```Something went wrong. This error has been sent to the PactFlow team and will be investigated ASAP.
TypeError: n is undefined```
tranghetti
2023-06-01 18:49
Hi there :wave:!
I deployed both consumer and provider to the environment "dev" in Pactflow with a successful validation. Now, I'm trying to simulate a breaking change in the consumer API, I renamed an existing field. Using can-i-deploy doesn't allow me to deploy neither the consumer nor the provider with the new versions. Can you please guide me to the right direction here? Which side should I deploy first and how I do that when having these scenarios? Thanks in advance.
ilia
2023-06-01 23:15
@thomas.koppensteiner can you please provide more information? whats the url you are seeing this, what action are you performing?
matt.fellows
2023-06-02 00:23
Have you taken a look at the CI/CD workshop? (see link below - howtolearn :point_down: )
2023-06-02 00:23
Here are a number of useful hands-on labs that teach all of the key concepts: https://docs.pactflow.io/docs/workshops and https://docs.pact.io/implementation_guides/workshops
matt.fellows
2023-06-02 00:24
I suspect you?ll need to work on branches/tagging to prevent getting into this situation.
abubics
2023-06-02 02:01
Also, in case it's not clear from here, if you want to make a breaking change like that, you can make it safe by doing it in smaller steps: https://docs.pact.io/faq#how-can-i-make-a-breaking-change-to-a-provider
ilia
2023-06-02 04:02
@thomas.koppensteiner we found the issue in our logs, could you try again and see if the error happens?
tranghetti
2023-06-02 13:40
Thanks, guys. I understand the concept of the "expand and contract pattern," and it's really helpful.
However, I'm a bit confused about the statement regarding working on "branches and tagging to prevent getting into this situation". In my understanding, branches and tags function similarly. I currently use branches to publish the contracts. I can publish a new contract with a breaking change to a feature branch, update the provider to support this breaking change in a branch with the same name, and validate the tests. However, deploying a breaking change can still be challenging as it will block deployment to an environment. The only solution I see is to skip the `can-i-deploy` step in the consumer pipeline and coordinate a subsequent deployment of the provider with a code that supports the new contract.
Apart from that, I will recommend to my team that we always follow the "expand and contract pattern" as it seems to be a safer approach.
jegadeesan.ponnusamy8
2023-06-02 16:33
Hi all, I am trying to learn more on the Pactflow for functional testing. Could someone give some direction on this topic? I wanted to understand whether is it possible mock test data for a provider or consumer and run the functional testing using Pact Flow
matt.fellows
2023-06-03 00:00
Pact (and by extension, PactFlow) is not a functional testing solution. But you could use the mocks generated by Pact to use a stubs in your other tests
matt.fellows
2023-06-03 23:59
Yes, a breaking change is breaking regardless of the use of branches etc.
That was a poor suggestion on my part not fully seeing the real problem
sugi
2023-06-05 16:22
Hello there :wave: I have a question regarding keyword support `oneOf` in the schema...Using BDCT workflow and we have `oneOf` keyword in the schema and I had followed the instructions here : https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support/ and dereferencing and inlining $refs using `swagger-cli` before publishing to pact broker...When I set the `additionalProperties:false` to the oneOf `refs` as mentioned here: https://github.com/pactflow/bdct-oas-examples/blob/main/examples/oneOf/oneOf.oas.yml#LL44C7-L44C34 the contract is failing....
We also use these oneOf refs in the include param...Wondering if that could be the issue..any suggestions?
sugi
2023-06-05 16:28
To provide more context
say `/v1/students` endpoint which contains query param like `full_time`, `part_time`
In the response based on the query param include requested ,the response would contain the one of them fullTime or partTime object
When I set `additionalProperties:false` to `full_time` and say fullTime has properties `id, type, attributes` then the contract is failing
`Response body is incompatible with the response body schema in the spec file: should NOT have additional properties - id`
sugi
2023-06-05 16:30
One of the consumer is using the student endpoint with include param
`full_time`
`"query": "include=full_time",`
matt.fellows
2023-06-05 23:02
Have you added the `additionalProperties: true` bit as noted here: https://github.com/pactflow/bdct-oas-examples/blob/main/examples/oneOf/oneOf.oas.yml#L26
2023-06-06 11:13
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
matt.fellows
2023-06-06 12:49
Wait, sorry, `additionalProperties` needs to be on the property adjecent to `oneOf` (see screenshot). In your case, it looks to be much higher up the hierarchy, which may be invalid or be dangerously allowing additional properties where it shouldn?t (or just unrelated)
sugi
2023-06-06 15:32
Thanks Matt! I've created a support ticket with OAS + pact and also moved the additionalProperties to the property adjacent to `oneOf` as you have suggested...
noor.hashem
2023-06-07 13:52
Hi there! Does pactflow have an api endpoint for organizations to be able to pull their billing / invoice amounts for past or current months?
matt.fellows
2023-06-08 00:34
Not at the moment Noor. You can login to the portal to fetch it manually (e.g. via https://docs.pactflow.io/docs/billing/), but you may already know that
tjones
2023-06-09 07:22
Documentation bug: https://docs.pactflow.io/docs/examples/java/provider-springboot/ says it?s using a ?public tenant on pactflow?, which is technically true, but I think might be misleading - as that looks like the OSS broker not the pactflow broker.
You might want to either update the text to say that it?s a publicly hosted version of the OSS broker, or update the public one to be a real pactflow instance as an advert for the service (probably the text update is the easiest).
sugi
2023-06-09 15:32
:wave: Currently using BDCT and using github actions to do `can-i-deploy` Recently seeing an issue where the contract comparison takes longer and the build fails....Suggested fix :https://docs.pactflow.io/docs/bi-directional-contract-testing/deploying#polling was to use retries and wait time.....Wondering if these params are available via current github actions https://github.com/pactflow/actions/blob/main/can-i-deploy/canideployTo.sh
yousafn
2023-06-09 16:06
Hey @sugi
You are correct, it hasn?t been mapped yet, it was requested
https://github.com/pactflow/actions/issues/20
I don?t think we fancy the maintenance of individual actions, so may not update them in a timely manner, but happy to accept a pull request.
I?ve just updated the repo with a generic action which gives access to the full cli bundle via a single action, so you should be able to set parameters to your hearts content :slightly_smiling_face:
yousafn
2023-06-09 16:06
you should be able to add this in now
``` - uses: pactflow/actions@main
- run: pact-broker help
if: runner.os != 'windows'
shell: bash
- name: pact-broker.bat on bash
run: pact-broker.bat help
if: runner.os == 'windows'
shell: bash```
steven.wang06
2023-06-13 06:27
Hi? Is it possible to get the contract files( e.g. swagger files from provider side ) in pact-broker ?
matt.fellows
2023-06-13 06:31
You could probably get it from the APIs. We will be publishing our APIs in the next month or so, but for now, I?d look at the XHR calls from the PactFlow UI and use that URL.
What?s the use case?
steven.wang06
2023-06-13 07:50
We already collect all swagger files from provider side, and upload them to pact-broker. I'd like to expose them to developer for reference purpose.
matt.fellows
2023-06-13 08:09
Perhaps it helps for you to understand why I?m asking - you already have the OAS files (because you uploaded them to PactFlow). Why do you need to then get them _back_ from PactFlow - what additional information/detail/value are you hoping to get from this that you couldn?t do without?
rajat.chouhan
2023-06-13 10:20
Hello, I have a question regarding keyword support `oneOf` in the schema...Using BDCT workflow and we have `oneOf` keyword in the schema and I had followed the instructions here : https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support/ and we did dereferencing and inlining $refs with keyword `oneOf` pact results are red and without `oneOf` pact results are green.
Can we use `oneOf` keyword in schema to make pact green? Any suggestions?
Please refer to the attached observations.
matt.fellows
2023-06-13 11:08
Yes you can, please see https://github.com/pactflow/bdct-oas-examples/tree/main/examples/oneOf
jegadeesan.ponnusamy8
2023-06-13 15:53
Hi, Do we have Pactflow available for asynchronous message based testing for .Net? If so, could you please share some examples or documentation? I was going through the examples and documentation, but couldnt find anything seems to be available for Pactflow.
matt.fellows
2023-06-13 23:15
I don?t think so, but there is nothing specific/different to PactFlow needed to make the examples work
jegadeesan.ponnusamy8
2023-06-14 04:33
Hi Matt, Thanks for getting back. You mean that the messaged based testing is same in Pact and PactFlow?
matt.fellows
2023-06-14 05:32
Pact = the testing tools *
PactFlow = hosted version of the Pact Broker, with enterprise features
_`*` - we do have a different testing feature in PactFlow called bi-directional contract testing, but that is not applicable to your case_
prakhar.roy371
2023-06-14 05:36
Hi all, If we aquire a docker image for pact broker, I wanted to know if we get the same UI inside that docker image as we do in Pactflow free account. (Screenshot attached)
If possible can someone send image of how pactflow looks in the docker image. Thanks!
matt.fellows
2023-06-14 05:42
It looks more like what you see here: https://github.com/pact-foundation/pact_broker
matt.fellows
2023-06-14 05:42
Or from the other side, these are the key differences: https://pactflow.io/features
prakhar.roy371
2023-06-14 06:01
Thanks for the quick response! Would it be a correct statement if I say that the commercial PactFlow UI would have the base features that free account has and additional features on top of that?
prakhar.roy371
2023-06-14 06:04
So far, I followed the pactflow documentation.
https://docs.pactflow.io/docs/workshops/ci-cd/set-up-ci
This is for Consumer-Driven-Contract-Testing CI/CD.
I want to make sure that I would be able to do any and all configurations on the commercial account that I have done on my free acount.
matt.fellows
2023-06-14 06:12
> I want to make sure that I would be able to do any and all configurations on the commercial account that I have done on my free acount.
what do you mean by ?free account? in this context?
Assuming by free you either mean PactFlow?s free starter plan, or the open source pact broker, yes you definitely should
prakhar.roy371
2023-06-14 07:06
Hi all, If an organization wants to aquire a docker image from Pact broker to deploy on-premises. I had the following doubts
1. From where do we download the docker image? We have the on-premises licenses with us where we can apply and make things work.
My assumption here is that docker image is readily available for us to use, be it for open source or commercial account. Only catch is that applying the license key gives us full access for PactFlow. Kindly correct me if I am wrong here.
P.S. I am looking for a docker image for Pactflow .Net
prakhar.roy371
2023-06-14 07:08
Thanks, that is exactly what I meant by free acount! :slightly_smiling_face:
matt.fellows
2023-06-14 07:29
No, they are different.
Pactflow -> https://docs.pactflow.io/docs/on-premises/docker-image-registry
OSS Pact Broker -> https://hub.docker.com/r/pactfoundation/pact-broker/
kedar.ghate
2023-06-14 14:20
Hi team,
We use https://vfuk-digital.pactflow.io/ instance of pactflow. From local we are able to run pact-broker can-i-deploy command successfully. From CICD pipeline, it throws error - Error retrieving matrix. Net::HTTPClientException - 403 "Forbidden". I have verified the API token, its valid and working. Could you please suggest what could be the reason and the fix for that.
matt.fellows
2023-06-14 22:15
403 indicates the credentials lack the permissions needed. Check the permissions of the API Token used in your CI/CD pipeline
matt.fellows
2023-06-14 22:15
> P.S. I am looking for a docker image for Pactflow .Net (edited)
there is no specific image for .NET.
kedar.ghate
2023-06-15 08:22
token has read-write permissions, I tried same token from local, its working. Its giving error in CICD pipeline
kedar.ghate
2023-06-15 10:36
curl -v https://vfuk-digital.pactflow.io/ -H "Authorization: Bearer **************"
ilia
2023-06-16 00:00
This looks like its coming from your proxy, can you check with your IT department (or whoever is manages your proxy) we have information here on this https://docs.pact.io/implementation_guides/ssl#proxies
wvkehoe
2023-06-16 11:15
Does pactflow support uploading multiple JSON files in cases where the provider contract's base openapi.json file contains relative $ref references of the form `"$ref": "./shared-parameters.json#/components/parameters/myparameter"`"?
If I attempt to use pactflow to upload multiple files to pactbroker, I get the following error on the second file:
```A provider contract for <my-service> version <my-service=-version> already exists with different content and may not be modified.```
I guess the next obvious question is whether pact-broker even supports relative $ref references in the base openapi spec JSON file.
yousafn
2023-06-16 11:23
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#document-limitations
> ? OAS documents must not be split across multiple files. You should combine any documents together, using tools like https://github.com/robertmassaioli/openapi-merge or https://www.npmjs.com/package/speccy. That is, PactFlow can not resolve remote references to files, and will not resolve URL references
yousafn
2023-06-16 11:24
to the 2nd point
> ? It is recommended to programmatically dereference and inline `$refs` in the OAS document uploaded to PactFlow, as they can cause issues when validating `nullable` fields and nested `$refs` can not be accurately compared with a pact file. This can be accomplished using packages such as https://www.npmjs.com/package/json-schema-merge-allof and https://www.npmjs.com/package/json-schema-resolve-allof
alexander.friesen
2023-06-16 15:03
@matt.fellows
Something is wrong with our OpenAPIspec
containing "oneOf" statements.
How can we debug the reason of the mismatch, shown in http://pactflow.io?
matt.fellows
2023-06-18 23:45
Please see the examples in the repo above as to why your `oneOf` is not working.
matt.fellows
2023-06-18 23:47
In any case, you need `additionalProperties` set to `true` at the level of `oneOf` e.g.
matt.fellows
2023-06-19 12:29
We are considering/investigating doing the deref for you. I suppose we could merge, but there are many hazards there (e.g. if there are references to other non-local/relative components).
john.joel.thetla
2023-06-20 07:43
Hi all, If we acquire docker image for pact broker and run it on premises, will it have some kind of UI to interact, if it has that what type of interaction will it provide us.
matt.fellows
2023-06-20 08:34
Yes it will, it's not very sophisticated tho. See the pact broker GitHub repo for an example of what it looks like.
matt.fellows
2023-06-22 04:22
Hi @giulio.giovannini we have been looking into this change. It gets quite complicated quickly.
For example, it?s possible to have multiple servers, with conflicting base URLs:
```servers:
- url: https://hostname/v1
servers:
- url: https://hostname/api/v1```
We would need a way to resolve this reliably without making it confusing. You can also have not just ambiguity, but straight up conflicts.
e.g. it?s also possible to have conflicts at the `path` level, like so:
```paths:
'/fruits/apple'
servers:
- url: https://hostname
'/apple'
servers:
- url: https://hostname/fruits```
These two would theoretically be the same, as far as the ?resolved? path would be.
paul.caplan
2023-06-22 13:40
Hi All - Are there any recorded presentations of this slide deck available to watch?
https://docs.pactflow.io/docs/workshops/advanced
wvkehoe
2023-06-22 15:07
Does anyone know whether pactflow ~validates~ verifies that a provider actually supports all of the query string parameters sent by the consumer? I have a situation where that appears to not be the case.
paul.caplan
2023-06-22 21:48
Yep that?s the part I found thanks. :sweat_smile: Have not found videos for the other sections of the slides though ?
tjones
2023-06-22 22:41
If you don?t include all the parameters sent by the client in the contract, they?re ignored
If the provider doesn?t read one of the parameters, but it wasnt necessary to generate that response, the test could also pass
john.joel.thetla
2023-06-26 10:01
Hi all, If we acquire the on-premise enterprise docker image and on board on to our docker management systems, run our CI/CD pipelines along with pact broker, where can we see the pact results. How can we interact with the on-premise Pact broker?(I mean the UI dashboard)
tjones
2023-06-26 10:03
I?m not a pactflow person, but wouldn?t it depend on where you hosted the docker image?
wvkehoe
2023-06-26 13:08
The latter is the case.
> If the provider doesn?t read one of the parameters, but it wasn't necessary to generate that response, the test could also pass
And as far as I can tell, there is no way in OpenAPI to state that a provider should fail if it receives a query string parameter that it does not use.
prakhar.roy371
2023-06-27 07:21
Hi all, my organization procured a *"PactFlow on-premises license file"* and *credentials* for acccessing the docker image present on RedHat Quay Enterprise docker repository. But when we use those credentials, it is showing as invalid credentials.
Is this a common occurence that anyone has faced? Kindly let me know the solution to this. Thanks!
matt.fellows
2023-06-27 08:04
Hi Prakhar,
The credentials you were given were likely what are referred to (by http://quay.io) as Robot Tokens.
matt.fellows
2023-06-27 08:04
You won?t be able to use them to login to http://quay.io - but you don?t need to
matt.fellows
2023-06-27 08:04
You can simply authenticate to the http://quay.io docker registry (as you would with any docker registry) and then pull as needed
matt.fellows
2023-06-27 08:05
If you could please show how you?re trying to authenticate and the error you are getting, that would be helpful.
prakhar.roy371
2023-06-27 12:22
Hi @matt.fellows, I followed the doc sent above and ran the command with the credentials provided to me.
1. docker login -u="<username>" -p="<password>" http://quay.io
2. docker pull http://quay.io/pactflow/enterprise
I was able to pull my docker image, but I wanted to know if this is stored somewhere in my local in some form?
After I pulled the docker image, I was not able to see it anywhere
P.S. Sorry for the late follow up, got involved with some other work.
matt.fellows
2023-06-27 22:18
Hi Prakhar, it?s stored in your local docker setup (you could list all images with `docker images`). I?d suggest working with somebody that knows Docker from here to setup PactFlow as it will require decent knowledge of docker for any issues, and of course general ?DevOps? capability
prakhar.roy371
2023-06-28 11:03
Thanks for your help Matt! I will study further on docker and explore! Thanks for enabling me on the same :slightly_smiling_face:
boer.k
2023-06-29 10:26
Hi, I originally asked this question in the wrong channel: https://pact-foundation.slack.com/archives/C9VPNUJR2/p1687958938502709
I am curious if anyone would like to share an example of how the compatibility matrix of consumer and provider versions looks like for them? Were worried that even when you only have a few versions of a consumer and provider, that when you compare every consumer version to every provider version, that the compatibility matrix is going to explode. Were still in the early stages of working with contract testing and pactflow, and maybe were worried about something that isnt even a problem.
matt.fellows
2023-06-29 10:30
Yeah, it?s not really a problem. The current UI doesn?t help (and there is a project going on at the moment that we think will improve this DX significantly)
matt.fellows
2023-06-29 10:31
The retention rules will remove some of the old data, but potentially hundreds of redundant (i.e. outdated) pacts will be about that aren?t very useful.
matt.fellows
2023-06-29 10:31
We?d like to be able to setup some integrations with tools like GitHub, so that when a branch is merged/deleted etc., related contracts are also removed. You could look to set it up yourself
matt.fellows
2023-06-29 10:35
See https://pact-foundation.slack.com/archives/C5F4KFKR8/p1686867025780129?thread_ts=1686824483.876189&cid=C5F4KFKR8 on how to delete pacts for a branch
tjones
2023-06-29 11:15
In practice you don?t need to worry about the size of the matrix, because the relevant information will be at the top.
tjones
2023-06-29 11:15
It would matter if you were deploying versions at random from all history, but almost always people deploy recent versions.
tjones
2023-06-29 11:17
For the same reason, you don?t need to verify all contracts from all time. Usually you do something like `dev`, `test`, `prod`, `<all others less than X days old>`.
tjones
2023-06-29 11:18
Bonus subtlety - the pact broker thinks about _contracts_, which is different to _consumer versions_. Most consumer versions have a contract that is the same as a previous one.
tjones
2023-06-29 11:18
eg, if there are no changes to the consumer contract between `dev`, `test`, `prod` (and you select all of those at verification time), you?ll only get one contract to verify.
matt.fellows
2023-06-29 11:24
The matrix view in the Pact Broker/PactFlow is a bit of a bum-steer - who wants to see a wall of entries. It?s not that relevant, most of the time, you?re interested in the question ?what is this version of my app compatible with?? or ?can I deploy this version of the app to <env>?.
alexander.friesen
2023-06-30 07:00
Thnx @matt.fellows.
Do you know, how SAML integration affects the licens?
"Automated user deprovisioning (users will appear "active" and count toward user limits, although will not be able to login if disabled in the IdP)"
means that I will have to delete deprovisioned users - in the pactflow console, to remove em from the license quota?
jegadeesan.ponnusamy8
2023-06-30 15:36
Hi Team, I do have a provider service which publishes more than one openapi sec. So could you please tell me whether we can publish more than one openapispec contract for a provider? OR do you have any suggestion how to handle these scenario?
matt.fellows
2023-06-30 22:46
We only support one: https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#document-limitations
Can you please elaborate on your use case?
matt.fellows
2023-07-02 04:25
If they have been deprovisioned (disabled or deleted) they won't count toward limits
romuald.quantin
2023-07-03 11:14
Hi here, I may have posted that one in the wrong place:
https://pact-foundation.slack.com/archives/C5F4KFKR8/p1688382451821939
jegadeesan.ponnusamy8
2023-07-03 12:34
Hi @matt.fellows - We build API which can have different swagger endpoints for external(customer facing) and internal to the company.
jegadeesan.ponnusamy8
2023-07-03 12:36
Also, if the same microservice acts as an consumer and provider with different interaction types (synchronous and message based), can we use the same name for all these interactions OR do we need to specifiy different names.. any thought
prakhar.roy371
2023-07-03 13:10
Hi,
I am trying to run migrations on PostgreSQL DB for Pactflow in local. I am using the following doc for doind the same. https://docs.pactflow.io/docs/on-premises/upgrading/database-migrations/
While executing the below command,
docker run --rm \
--env PACTFLOW_DATABASE_URL=":port/database" \
--entrypoint db-version \
http://quay.io/pactflow/enterprise
I ended up with the following error :
bundler: failed to load command: rake (/home/pactflow/vendor/bundle/ruby/3.2.0/bin/rake)
/home/pact_broker_fork/lib/pact_broker/version.rb:2:in `RGLoader_load': RubyEncoder Loader - *This protected script requires the pactflow-onprem.lic license file in order to run. Contact the author of the script to get a license file.* Error code [13] (RGLoaderError)
We do have the license file with us, need to know how we can use it.
yousafn
2023-07-03 17:09
Hey,
You need to volume mount the license key, its shown in the next section for Rollback
`--volume $PWD/pactflow-onprem.lic:/home/pactflow-onprem.lic`
yousafn
2023-07-03 17:14
Hey,
How are you uploading the provider contracts, I note they don?t have a branch associated (which is part of the recommended setup)
It would be useful if you could provide the pact file and openapi specification as it would allow someone to test out the same scenario :+1:
matt.fellows
2023-07-03 22:23
We recently made some changes to allow negative scenario testing. But if you can share your contract we?ll take a look
matt.fellows
2023-07-03 22:58
You have a couple of choices, I think
1. Merge them together before uploading to PactFlow
2. Name them as separate providers in PactFlow (e.g. provider-internal and provider-public)
matt.fellows
2023-07-03 22:59
> Also, if the same microservice acts as an consumer and provider with different interaction types (synchronous and message based), can we use the same name for all these interactions OR do we need to specifiy different names.. any thought
BDCT does not support async, it only supports HTTP APIs
matt.fellows
2023-07-03 22:59
If you?re referring to Pact tests, in V4 you can combine them. Prior to that you need to name them separately
boer.k
2023-07-04 12:08
I was about to send you the contracts, but just to make sure I'd send you the right versions of the contracts, I decided to delete all integrations, and try to reproduce it again. Strangely enough, now it does fail as expected. I have no idea how what I did now was any different than before. I did it again with the same colleague who also saw the same false positive last week. And neither of us can spot any difference. I also didnt set a branch for the provider still as I didnt think that would matter in the comparison. But it fails as expected now.
matt.fellows
2023-07-04 13:00
:rocket: *Bi-Directional Contract Testing: Release announcement* :new:
Just checking in to let you know we have dropped a major update to our Bi-Directional Contract Testing feature.
As a summary, we have released support for:
? :swagger: OAS 3.1
? Full RFC 9110 semantics for media types. TL;DR - content negotiation and vendor specific media types will work a lot better!
? Pact V4 (HTTP interactions)
? Testing negative scenario e.g. checking behaviour for 4xx, 5xx and other error scenarios
Lastly - and most importantly - we have *improved how we support the https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support* *`anyOf`* *`oneOf` and `allOf`,* including for use cases in composition, inheritance and polymorphism. These changes improve the guarantees, including ensuring the use of `discriminator` is correct (instead of ignoring the invalid setup), and simplify the setup previously required to make this work the way you would expect.
:warning: _The validation has improved in this release which may cause some tests that were previously passing to now fail. This is a good-bad thing. Good, because we have improved the guarantees, bad because you will need to update your tests to cater for this. See the https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/changelog#july-2023 for advice on addressing these errors_ :warning:
For a full review of the OAS features we support, see :point_right: https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/features.
We have also made a number of updates to our documentation, which is much more comprehensive than before including improved ?how to fix? https://docs.pactflow.io/docs/bi-directional-contract-testing/compatibility-checks, detailed breakdown of our https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/features and an overview of how we support https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support (an area that seems to trip people up the most)
Lastly, we have created a https://github.com/pactflow/bdct-oas-examples of example OpenAPI documents and pact files that can be used to better understand how the features and capabilities translate to real life. This should help bring to life the theoretical documentation.
If you have any questions or comments please feel free to respond to this thread, we?d love to hear from you:ear:
adam.strawson
2023-07-04 14:59
Hi :wave:
We've using Pact bi-directional flows already, and have recently moved some of our frontend services into Vercel - I was wondering if anyone has any tips or examples with making Vercel and Pact work well together?
adam.strawson
2023-07-04 14:59
We ideally want to use Vercels git integration, but I think the only way we can see this working is to move the Vercel build/deploy steps into our GitHub Workflows, so that we can continue to have Pact work alongside Vercel, however we lose some of the nice Vercel feedback features.
jegadeesan.ponnusamy8
2023-07-04 15:58
V4 mean that Pact specification version.correct? We are using http://Pact.Net which doesnt support V4 yet. So I think, I may need to use different names for provider and consumer. Does this applicable to BDCT as well. I read somewhere that it is applicable for message based interactions? i.e. For BDCT, if microservice A acts as a provider (using BDCT) and also consumer of Microservice B, then do I need to use different names?
matt.fellows
2023-07-04 22:44
I don?t know Vercel very well. Are you able to elaborate on the challenges?
matt.fellows
2023-07-04 22:45
BDCT doesn?t support async/message based interactions, those requests are ignored
jegadeesan.ponnusamy8
2023-07-04 23:13
I meant for HTTP based interactions.Sorry if it is sounded for messaged testing....... if microservice A acts as a provider (HTTP based interaction using BDCT) and also consumer of Microservice B, then do I need to use different pacticipant name for microservice A i.e. like microserviceA-provider & microserviceA-Consumer..
matt.fellows
2023-07-04 23:15
> i.e. For BDCT, if microservice A acts as a provider (using BDCT) and also consumer of Microservice B, then do I need to use different names?
You shouldn?t need to, no
sathish.sundararajan
2023-07-05 04:59
Hello, We are in the process of configuring Pactflow webhooks to trigger github actions workflow. For authentication, I see there are two options where one is personal access token and the other is github app. Github app provides shortlived tokens whereas personal access token provides custom range for expiration. Is there a different way to have token created with custom expiration for entire org instead of using personal access token ? Interested to understand how everyone is using it
matt.fellows
2023-07-05 05:09
For now, you will need to use a PAT or at least, you will need to give PactFlow a token that it can use (as a secret).
There are APIs to manage secrets, so if you wanted to rotate them by some other process (e.g. a lambda / function that?s job is to create new tokens from GitHub, and update the secret in PactFlow), that could work
sathish.sundararajan
2023-07-05 05:12
Thanks Matt, we are trying to avoid PAT as this will be used by org level and will impose a security risk for the user. Can you provide the api info for updating secrets ?
matt.fellows
2023-07-05 05:21
We?ll be publishing our APIs in the next ~month or so, but from the PactFlow UI if you head to the secrets page, open the network tab and you should be able to copy the XHR calls. They are quite basic, from memory
matt.fellows
2023-07-05 05:22
You can also see the pact tests for the terraform client here: https://github.com/pactflow/terraform-provider-pact/blob/master/client/client_pact_test.go#L332-L418
prakhar.roy371
2023-07-05 05:50
Hi all,
I have obtained an enterprise pactflow docker image as well as a license file. In the below pactflow doc,
https://docs.pactflow.io/docs/on-premises/license/
There is a way to create a new image with the license file baked in it.
I am not able to understand the steps shown in that section. (As shown in screenshot)
So far what I have done :
1. Pulled Enterprise docker image using robot credentials.
2. Have put my license file in a directory. (C:\PactFlow)
I want to know what command I need to run to create a new image with license file baked in it.
tjones
2023-07-05 06:57
Create a Dockerfile with that content, then run the docker build command listed on your screenshot, in the directory with the Dockerfile
tjones
2023-07-05 06:59
You may need to update the path to your license file that is listed in your Dockerfile.
prakhar.roy371
2023-07-05 07:14
ARG ONPREM_IMAGE
FROM ${ONPREM_IMAGE}
COPY --chown=app:app pactflow-onprem.lic *C:/PactFlow/*pactflow-onprem.lic
*My license path :* C:/PactFlow
Is this the only thing I need to change in my Dockerfile for path (see highlighted part) ?
tjones
2023-07-05 07:20
I don't think that's right. The syntax for the copy command is source -> destination.
tjones
2023-07-05 07:23
I'm not a pactflow person (and I don't use docker very much), I'm just interpreting the docs for you.
adam.strawson
2023-07-05 08:56
Prior to Vercel, we ran our frontend service in Kubernetes, and had CI/CD handled within Github Actions. We had the calls to Pactflow as part of those workflows.
With Vercel, they manage the CI/CD process with their git integration, but they have no way to customise that process to tie in any of the Pactflow actions we had previously.
We do have the option of continuing to run our CI/CD in GitHub Actions still, which we can then use Pactflow how we did previously, but we lose some of the nice features of the Vercel git intergration.
I was hoping someone here might have had some nice working examples of Vercel and Pactflow working together, but I think the answer is really going to end up resulting in us having to have a tradeoff between changing how we use Pactflow, or losing some of the benefits of Vercel.
prakhar.roy371
2023-07-05 10:14
I am assuming here that I need to put a path in place of ONPREM_IMAGE.
Just not sure what I need to put.
1. I know where my license file exists in my local.
2. I don't know the location of enterprise docker image in my local. It is present in the docker desktop
tjones
2023-07-05 10:22
Pretty sure that?s the docker image ID/tag, not a path. Check the docker documentation
matt.fellows
2023-07-05 11:54
Could you link the license at runtime as a volume mount? Then you don't need to create a new pipeline to build the image, publish it to a registry etc
matt.fellows
2023-07-05 11:55
Yes I think so. If there are any hooks that can go into the process that would be the way to go.
Let us know where you land!
eddie
2023-07-06 09:05
Hey,
This page https://docs.pact.io/pact_broker/recording_deployments_and_releases#application-instances says that application instances shouldn't be used for blue/green or other long running deployments (eg canary).
But it doesn't say what should be used instead...
We're currently running with two consumers - Foo.Stable and Foo.Canary - but not sure if this is advisable or not
matt.fellows
2023-07-06 10:49
(sorry, I saw your other post earlier today but just didn?t get a chance to read the specific question and respond)
matt.fellows
2023-07-06 12:53
Check out https://pact-foundation.slack.com/archives/CLS16AVEE/p1688475600649069
There is a link to how to address those
prakhar.roy371
2023-07-06 13:40
@matt.fellows Thanks for your suggestion.
Also, Sorry for the late response, I am new to Kubernetes.
I created a yaml script for deployment. For volume mount, in my yaml file I am putting in the name and path. But I am not sure what to put there in path for this. (See screenshot below)
Can you suggest what to put there OR another way?
*What I have done so far :*
I am able to deploy my image (without license file) to PKS where its asking for a license file.
eddie
2023-07-06 20:52
So *TL;DR:* With blue/green it's normal to have two consumers e.g `MyConsumer-Blue` & `MyConsumer-Green` ?
matt.fellows
2023-07-06 23:26
You could also embed the binary contents this way:
```kubectl create configmap pactflow-license --from-file pactflow.lic```
then
```apiVersion: v1
kind: ConfigMap
metadata:
name: pactflow-license
namespace: pactflow
binaryData:
pactflow.lic: >- eAAAAE9GdbRGFS/CRDnABtJx...TRUNCATED...IGFueSBlZGl0b3Iu
```
then I add this to the deployment
```
spec:
containers:
- image: http://quay.io/pactflow/enterprise:1.11.1 # note, set version here
name: pactflow
resources: {}
volumeMounts:
- name: config-volume
mountPath: /home/pactflow-onprem.lic
subPath: pactflow.lic
volumes:
- name: config-volume
secret:
secretName: pactflow-license```
Again, if that doesn?t make sense, I recommend you work with somebody at your company that does understand it.
rajat.chouhan
2023-07-07 06:15
ok, i couldnt find something about nullable in their "helpful repository".
rajat.chouhan
2023-07-07 06:15
I am getting below validation error
"nullable" cannot be used without "type"
prakhar.roy371
2023-07-07 12:01
```kubectl create configmap pactflow-license --from-file pactflow.lic```
On running the above command, I am not getting any data inside my Config-map. (See Screenshot below)
Also, after deploying my image onto PKS using deployment script, I am getting the error saying :
*"Protected script /home/pact_broker_fork/lib/pact_broker/version.rb checksum error. The file has been modified. If this script requires a license file in order to run, this error may be caused by an invalid license file. Install the original unmodified file or contact the author of the script to get the original file or license file."*
I checked with the Kubernetes team on this, they are unable to figure out why we have no data coming up from license file.
Everything in the script looks good to them, but they are not sure how we can read from the license file.
prakhar.roy371
2023-07-07 12:24
We experimented with a plain text file with data and ran commands to create the config map. There we could see the data.
Hence, we changed the extension of license file to txt to experiment if the license file data comes into the config map or not. Even then we are not able to see the data in the config map.
matt.fellows
2023-07-07 22:33
It's an ajv validation error: https://github.com/ajv-validator/ajv/issues/1882#issuecomment-1030159275
matt.fellows
2023-07-09 11:45
I can't help much more here, sorry, they are the two main options that I know work, so it's probably something about how it's expected or your environment.
Maybe go back to baking the license file in
eddie
2023-07-09 23:52
But if we record a deployment to the environment, that removes the previously deployed version from the environment
matt.fellows
2023-07-10 00:05
Correct. Given the advice on the above page, you should only do that once the cutover is in place though. 1-6 hours is an awfully long time though.
eddie
2023-07-10 00:06
> 1-6 hours is an awfully long time though
It's enough time to see sufficent real-world usage and compare it to "stable" in parallel to be confident we haven't regressed :slightly_smiling_face:
matt.fellows
2023-07-10 00:07
> We?re currently running with two consumers - Foo.Stable and Foo.Canary - but not sure if this is advisable or not
How does the consumer know it?s hitting the stable or canary version of the provider? Usually, this is transparent to consumers
eddie
2023-07-10 00:07
Sorry - we're not canary releasing our provider. We're canary releasing the _consumer_
matt.fellows
2023-07-10 00:11
I think in your case you could consider it a ?long-running? deployment. But having two applications is probably not the end of the world either
prakhar.roy371
2023-07-10 05:47
The pactflow documentation was a little unclear to me on the steps to perform.
*All I did was :*
Create a Dockerfile with the content in section 1.
But what all we need to modify in that file, and what args to put in docker build command, need help on that. Can you explain what am I missing here?
Simply copy pasting that content in the dockerfile does not work when I run the docker build command.
tjones
2023-07-10 06:15
You?re going to need help from someone familiar with the docker syntax (this is why I linked you to the documentation)
tjones
2023-07-10 06:29
Someone familiar with the docker syntax should be able to read and understand the instructions there - for example, `ONPREM_IMAGE` is clearly something you'll have to provide yourself
tjones
2023-07-10 06:30
(I'm not a pactflow person, and I barely ever use docker, so I'm not the best person to help - but I would guess there's someone in your organisation that understands the docker commands. If there isn't, you can become one by reading the documentation for the Dockerfile syntax that I linked earlier)
tpaktopsp
2023-07-10 07:05
Hi there, Pactflow Team.
Hope you can help me to understand if this is an expected behaviour or bug.
I replicated these scenarios on example-consumer / example-provider pair from pactflow university https://docs.pactflow.io/docs/workshops/ci-cd.
I noticed two strange things:
1. When verification results of a pact contract are removed
? Contract is still displayed as valid
? And all interactions are green ticked even so there are no verification results
? *can-i-deploy* does the right thing and :x: fails
2. When verified :white_check_mark: contract is removed entirely
? Contract cannot be seen in the list of contracts for an integration
? If only run *can-i-deploy* it passes :white_check_mark: even so the contract supposedly has been removed :grimacing:
? When run full build and can-i-deploy it still passes but now contract reappears with verification results.
? My expectation in this case was that verification results would be removed when I remove the whole contract
tjones
2023-07-10 07:08
I'm not sure that deletion does delete all related resources - what's the use case you have for deleting contracts?
tpaktopsp
2023-07-10 07:11
We had some problems on provider side and tried to force revalidation of a bunch of contracts. For some reason, there were not revalidating and I think our team in struggle to fix it decided to just remove pacts. I guess, there were hoping that it would cause clean slate and keep with revalidation.
matt.fellows
2023-07-10 07:58
hmm it?s tricky. Which app were you expecting _not_ to be able to release, but could? It?s hard to tell if you?re considering a consumer or provider.
It?s tricky to say if it?s a bug or not, because there are a lot of complex relationships that are modelled
matt.fellows
2023-07-10 07:58
if you really need to reset things, deleting the integration would probably be the simplest
tpaktopsp
2023-07-10 12:33
so what is the purpose of remove ?pact? button anyway if it doesn?t really remove anything?
prakhar.roy371
2023-07-10 14:15
Hi,
So I followed the Pactflow doc for running my enterprise image on my local.
(https://docs.pactflow.io/docs/on-premises/docker-compose-example/)
I got a docker-compose script from the pact docs.
On placing that docker-compose.yml file with my pactflow.lic file I was able to run the enterprise image on my local.
NOW, I am trying to deploy it to Kubernetes via deployment scripts but I am facing the following issue.
*"The postgres container is up in my pod along with pactflow container, but the pactflow container is unable to talk to postgress"*
I am suspecting this has something to do with this docker-command (See screenshot below)
How can I apply the same configuration in my Kubernetes deployment yaml script?
(Attached the deployment script and logs given by my kubernetes team)
matt.fellows
2023-07-10 22:08
So that says ?wait for the `postgres` container on port `5432` to be available?
matt.fellows
2023-07-10 22:34
oh sorry, I definitely read your original question as ?deleted verification results?.
prakhar.roy371
2023-07-11 06:54
Thanks for your response Matt. Was able to figure it out and finally was able to see the UI.
However, the UI is not responding.
There is a broken "Pactflow Demo Auth" button which is not responding on clicking.
Can you please guide me on where I can find the root cause for this? Really greatful for your help so far!
matt.fellows
2023-07-11 07:07
It looks like the image is not loading properly. Are there any errors in the network console or PactFlow logs?
prakhar.roy371
2023-07-11 07:22
I am checking with the Kubernetes team for the latest logs. Will send it here for your reference in some time
matt.fellows
2023-07-11 07:33
So those are CSP headers, and are critical to ensuring your app is safe from common web-based attacks
matt.fellows
2023-07-11 07:34
the issue is (if you notice) it seems to be referring to `localhost` - is this intentional? My guess is that you still have references to localhost in your setup/config somewhere
rohit.m.patil27
2023-07-11 12:44
Can we implement pact flow in dot net pact code? If yes, where can I get a sample or code .
matt.fellows
2023-07-11 12:52
As discussed in the DM, what is it about the example (with linked code) here that doesn't meet your needs https://docs.pactflow.io/docs/examples/dotnet/consumer/ ?
prakhar.roy371
2023-07-12 07:12
Thanks a lot @matt.fellows!
You were right all our configurations were pointing to localhost. Finally the image is deployed on our PKS and up and running!
Finally! :partying_face:
marko.lamberg
2023-07-12 10:43
Hey guys,
a question about best practices of record-deployment and can-i-deploy.
We are using http://pactflow.io
We are using record-deployment, but we have up to dozen merges (pact published) to integrated environment and maybe one to prod.
When our consumer is using can-i-deploy they can't get the both environments from one version.
I was looking to use record-release for integrated environment and record-deployment for the prod. This would otherwise work but record-deployment is tagged as `environment` and record-release is tagged as `release environment`.
Would this work for can-i-deploy without changes, or do they have to do any modifications for their can-i-deploy call? (edited)
matt.fellows
2023-07-12 10:57
> We are using record-deployment, but we have up to dozen merges (pact published) to integrated environment and maybe one to prod.
what do you mean by this? Do you deploy multiple versions to the ?integrated? environment?
matt.fellows
2023-07-12 10:57
> When our consumer is using can-i-deploy they can?t get the both environments from one version.
why do you need to know this?
matt.fellows
2023-07-12 10:58
Is it a linear deployment process.
Merge PR 1 -> Deploy to integrated
Merge PR 2 -> Deploy to integrated
Merge PR n -> Deploy to integrated
<some process to agree on prod>
Deploy to prod
This type thing?
marko.lamberg
2023-07-12 11:26
Ours is not linear, dozen merges and deployment to integrated. But only one of them is going to prod:
PR 1 to PR 12 -> deploy to integrated
PR 11 -> deploy to prod
matt.fellows
2023-07-12 12:15
what I?m asking is that PR1 and PR12 (and the intermediates) - are they all deployed to the same environment at the same time, or does each PR override the previous one?
matt.fellows
2023-07-12 22:50
so it should be straight forward. Each time you release to the any environment, you just call `record-deployment` with the version that is deployed to it.
prakhar.roy371
2023-07-13 05:29
Hi all, my org has acquired a license for pactflow and all the setup has been done. The image is up and running on PKS.
But as this trail broker is allowing everyone without a password, I wanted to know more about how the contracts pertaining to a specific team would be maintained.
How are we going to manage user logins for this?
matt.fellows
2023-07-13 07:19
You?re currently using :point_right: https://docs.pactflow.io/docs/on-premises/authentication/demo/
For ?reals? you should use :point_right: https://docs.pactflow.io/docs/on-premises/authentication/saml
rajat.chouhan
2023-07-13 08:26
Hello, I am getting error "*must not have unevaluated properties*" in pact on validating against my schema.
Attached sample of my schema and error I am getting. Please suggest some solutions.
Thanks
matt.fellows
2023-07-13 10:33
tushar.adsul223
2023-07-14 15:33
do net pact how to publish pact contract to pact flow broker .Please send steps and code
drettie
2023-07-17 14:07
hi there, we noticed pactflow has a https://pactflow.io/partners/ and wondering if others have had success going with this route and could provide any feedback ?
we are London based and have an existing self hosted pact-broker implementation running to support 100+ microservices, we think our setup is not setup as it should be as failing contracts will sometimes get to production and have gaps in training as our developer count has grown in recent years.
also if any pactflow maintainers are here: the get in touch links seem to be broken on the above page
yousafn
2023-07-17 15:29
Hey hey,
Thanks for letting us know about the links, I?ll see if I can get them sorted.
I used for one of the listed partners, InfinityWorks, they have an office based in London.
We have great relationships with the others, Sngular /Kruezwerker & DiUS (who helped foster and support Pact, and also the birthplace of PactFlow).
They regularly put out content and are engaged with some of our clients. You should be able to get contact details on their main websites, but if you struggle I can see if I can connect you.
matt.fellows
2023-07-18 01:31
Thanks for picking this up David, it looks like an issue with the redirect service we used (possibly corporate IT has modified our DNS)
prakhar.roy371
2023-07-18 06:16
Hi all,
When the CICD Pipelines run the Consumer Test Cases and when there is *NO CHANGE* in the generated pact file and when we try to publish the pact to broker, will the unchanged pact get published or not ?
matt.fellows
2023-07-18 06:39
TL;DR - always publish from pipelines. This makes the pipelines simpler to reason about, the steps are always the same
matt.fellows
2023-07-18 06:39
If the content is not different, it will be considered ?pre-verified? (that is, verification results for the same contents can be re-used in `can-i-deploy` and similar calculations)
prakhar.roy371
2023-07-18 12:57
Thank you so much for the info!
The pact doc was really informative :slightly_smiling_face:
tanyaryzhova93
2023-07-18 15:18
Hello! Is it possible to extract and see a value from PactFlow secrets?
jegadeesan.ponnusamy8
2023-07-18 15:49
@matt.fellows - I was testing HTTP based and message based interaction for a microservice and facing the below issues. I am using PactNet (5.0-beta) and PactV4. Could you please review and provide your inputs, if I am missing something in the implementation? My test pactflow account is https://jp.pactflow.io/.
*Scenario:* I have "catalog" microservice which acts a provider (BDCT) with the pacticipant name "catalog-api". This same microservice will also acts as message based provider (for 2 event types - ServiceCreate & ServiceDelete). I planned to use the same pacticipant name "catalog-api" for message based interaction as well.
*Issue Description:*
1. When I use the same pacticipant name "catalog-api", the message based interactions are automatically getting verified based on the OAS document published as part of BDCT. But I was expecting that the message based interactions shouldnt be verified against OAS contract and wait until I run the Provider verifier tests for "catalog-api" events.
2. To work around the issue#1, I published the consumer contracts with a different pacticipant name for "catalog-api" i.e. "catalog". I couldnt include 2 different events (i.e. ServiceCreate & ServiceDelete) in the same consumer contract and run the Provider verifications tests. The tests are failing with the generic error message "Pact net verification failed". But when I include only one event type in the contract, the provider tests are running successfully without any issues.
matt.fellows
2023-07-18 22:59
> 1. When I use the same pacticipant name ?catalog-api?, the message based interactions are automatically getting verified based on the OAS document published as part of BDCT. But I was expecting that the message based interactions shouldnt be verified against OAS contract and wait until I run the Provider verifier tests for ?catalog-api? events.
>
Yes this is tricky. the BDCT verification ignores non-HTTP based interactions so it will be ?green? even though the messages aren?t included. This is because BDCT workflows need not have a Pact verification step.
matt.fellows
2023-07-18 23:00
Creating 2 pacticipants is probably the way to go. We have been thinking on the best way to resolve this, but we will need to as we add different contract types into the system
tanyaryzhova93
2023-07-19 00:53
I create a pact flow secret with terraform and wanted to see what value it puts. I thought maybe there is an endpoint to get the secret quickly. Ok, I?ll use terraform output then.
jegadeesan.ponnusamy8
2023-07-19 01:21
Thanks for your feedback. I will try creating a support request for further tracking on the issue# 1. Also, could you please provide some light on the issue# 2 i.e. I couldnt have more than 1 event type in the same interaction.
james_fraser
2023-07-19 14:19
Hey there :wave: I?m having some issues with a bi-directional contract
The OpenAPI Spec that?s uploaded is using OAS 3.0.1 and has components that specify `"exclusiveMaximum": false` (apparently booleans are allowed in 3.0 but it expects a number in 3.1)
Pactflow is failing the verification with a lot of errors like this:
```schema is invalid: data/definitions/ABC/properties/DEF/exclusiveMaximum must be number```
As far as I can tell this should be valid, but is anyone aware of something I?m missing?
matt.fellows
2023-07-19 22:27
This is an issue we recently discovered. We will be addressing it in the coming iteration (starting today).
It?s to do with a compatibility difference between OAS and JSON schema. OAS defines it as a boolean, JSON schema needs it to be a number - so we need to transform it.
I think it could be mitigated in the meantime in two ways
1. Dereference all schemas prior to uploading (my understanding is that the bug is in referenced components of the schema)
2. Remove that constraint prior to uploading to PactFlow temporarily
matt.fellows
2023-07-19 22:30
> 1. To work around the issue#1, I published the consumer contracts with a different pacticipant name for ?catalog-api? i.e. ?catalog?. I couldnt include 2 different events (i.e. ServiceCreate & ServiceDelete) in the same consumer contract and run the Provider verifications tests. The tests are failing with the generic error message ?Pact net verification failed?. But when I include only one event type in the contract, the provider tests are running successfully without any issues.
>
I?m not sure what the problem is from this description. It should work. So it?s either:
1. A bug in your setup
2. A bug in Pact .NET
To understand what the problem is, we?d need to a repro and a bug report on GitHub please
matt.fellows
2023-07-19 22:30
> I will try creating a support request for further tracking on the issue# 1
You could consider raising a https://github.com/pactflow/roadmap/issues. I also will add it on our internal insights board for consideration
james_fraser
2023-07-20 08:13
Ok cool thanks for the update Matt :smile: hopefully it gets sorted soon :crossed_fingers:
rajat.chouhan
2023-07-21 11:02
I have used unevaluated property as true as per documentation but still getting issue as it does not identify this additional property on validator. Please help.
yousafn
2023-07-21 11:05
RE: https://pact-foundation.slack.com/archives/C9VPNUJR2/p1689928386348559?thread_ts=1686740085.427319&cid=C9VPNUJR2
Hi Adra,
Are you logged in as a member of that team? Have you logged out after updating the team settings?
matt.fellows
2023-07-21 11:19
We don't allow unevaluated properties to be true. If we did, the validation would mean nothing
rajat.chouhan
2023-07-21 12:04
Get Endpoint
*Error* - must NOT have unevaluated properties
*Consumer*: [root].interactions[2].response.body.data.0.attributes = {"entity":{"type":"Equipment","id":"1234"},"isRoot":false,"name":"PointGroup Apiss 98"}
In below part of provider schema I am getting this error.
*Provider Schema sample*:
PointGroupAttributes:
additionalProperties: true
allOf:
- $ref: "#/components/schemas/PointGroupAttributesBase"
- type: object
properties:
entity:
type: object
required:
- id
- type
properties:
id:
type: string
example: "entityId"
type:
type: string
example: "Device"
adra_verma
2023-07-21 13:49
I wanted to create system account so that with single common username and password or token anyone can login to pactflow
adra_verma
2023-07-21 13:51
This I want to implement https://pactflow.io/blog/team-based-secrets-webhooks-and-system-accounts/ we've added the `system_account:manage:team` permission, to allow users to manage System Accounts that belong to their team. Users with this permission are able to create, read, update and delete a System Account associated with their own team.
matt.fellows
2023-07-22 00:18
System accounts don't have usernames/passwords and are not designed for use by users. They also dot have access to the UI.
Using them (or any user) in the way you intend would be a breach of our terms and conditions
matt.fellows
2023-07-22 02:35
How could I tell you if the snippet is compatible without the referenced schema?
matt.fellows
2023-07-24 12:33
Thanks, it?s still a little tricky with the data you have given. It might be ideal you raise a support ticket so we can get an engineer on the case
2023-07-24 12:33
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
matt.fellows
2023-07-24 12:45
It seems to be related to the `isRoot` property. There is hopefully a straightforward explanation for it, but it?s late and my brain isn?t working right now :wink:
prakhar.roy371
2023-07-24 14:51
Hi all, I have my pactflow on-prem setup in PKS. Currently it is in Demo-auth mode.
I wanted to know what changes I would need to do to make AUTHENTICATION ENABLED.
I have gone through docs and configurations of SAML-Pactflow doc.
Do I just need to add all configurations in below image (PKS Deployment script) ? (By all config i mean all variables starting with Prefix *PACTFLOW_SAML_*)
matt.fellows
2023-07-24 22:55
That?s correct. You don?t necessarily need all of the variables, are some (e.g. if the metadata URL is provided) automatically determine others.
see https://docs.pactflow.io/docs/on-premises/authentication/saml
matt.fellows
2023-07-25 00:00
Please do ensure you update some of these, e.g.
The encryption key, admin key and secrets.
requiring https should also be set to `true` in real life.
I also hope you can update the postgres database from the default username/password :scream:
adra_verma
2023-07-25 08:48
Hi Matt. We want something account/group level. So, that can be accessed within a single team and can be accessed with token or something.
matt.fellows
2023-07-25 09:47
What is your use case? Perhaps we start there and I can help.
For clarity, are you using PactFlow or the open source Pact Broker?
prakhar.roy371
2023-07-25 11:37
Hey @matt.fellows
We have acquired a DNS for our pact-onprem
http://pactflow.<mycompany>.com/
Wanted to knwo if I need to put the above value for the environment variable shown in screenshot.
matt.fellows
2023-07-25 11:41
It should be the URL of your IDP: https://docs.pactflow.io/docs/on-premises/environment-variables#pactflow_saml_idp_sso_target_url
adra_verma
2023-07-25 12:13
http://Pactflow.io for POC but we have license one also its undergoing security pipeline.
adra_verma
2023-07-25 12:17
Use case: We don't want single user login it should be group level and we can add multiple legitimate team members under that group.
matt.fellows
2023-07-25 12:24
So you would usually create a Team (in the Settings page) and assign users (and applications) to that team.
adra_verma
2023-07-25 12:49
I don't want login like , . etc .What I want group wise login it should show group username login instead of respective user.
adra_verma
2023-07-25 12:56
If you refer below link in this e.g is user group can similar way, we should create and login with same https://pactflow.io/blog/user-management-invite-users/
matt.fellows
2023-07-25 13:44
I'm still not following. If the user has an email, you can invite them as per the article shown. There is no such thing as a "group login". The article you linked is just adding single users ( might be confusing you?)
matt.fellows
2023-07-25 13:44
I'm still confused about the use case / problem you're trying to address
prakhar.roy371
2023-07-25 15:55
This is the deployment script for deploying to PKS. It is working with demo auth but when I configured it with SAML, it still shows demo auth?
Am I missing something here?
matt.fellows
2023-07-26 01:00
I would take a look at the boot logs of the application, ideally set to DEBUG mode. This will enable you to see what the application is getting during bootstrap.
Also, is there a reason you?re still using the trial license?
prakhar.roy371
2023-07-26 06:13
Do you mean with the trial license we cannot enable the authentication? We are still in DEMO phase, hence using trial license for now.
Also, where can I find the boot logs of THIS APPLICATION?
rajat.chouhan
2023-07-26 07:59
I am also thinking the same but it looks correct for me not sure why it is creating issue.
Created ticket for this *00560370*
matt.fellows
2023-07-27 06:55
Yes, we?ll respond to the ticket. TL;DR - nested `allOf`s seem to be problematic. We?re not sure if it?s an issue with AVJ (the validator we use) or something in JSON Schema
rajat.chouhan
2023-07-27 13:15
We are also facing one issue in pact related to oneOf in provider schema
*error*: must match exactly one schema in oneOf
*consumer response*: [root].interactions[9].request.body.data.attributes.configuration = {"binaryPointAlarmConfiguration":{"alarmValue":false}}
*Provider schema: (for full schema you can refer to schema I uploaded)*
Schema sample
BinaryPointAlarmConfiguration:
type: object
properties:
binaryPointAlarmConfiguration:
type: object
required:
- alarmValue
properties:
alarmValue:
type: boolean
matt.fellows
2023-07-27 13:23
Thanks. I'm actually wondering if we need to think about treating response bodies that use oneOf as any of because of this problem. I'll discuss with my colleague tomorrow
yousafn
2023-07-28 10:30
:blobwave:
Happy Friday PactFlow peeps.
If you?ve subscribed for our PactFlow POST, you should have Episode 29 landing in your inboxes today.
Not subscribed? Not to worry.
You can subscribe here :point_right: https://pactflow.io/subscribe-to-the-post/
Or read the PDF version :point_down:
thatrandybrown
2023-07-28 16:59
Hey everyone! :wave: I'm popping in to see if there's something I'm missing.
When I run `@pactflow/swagger-mock-validator` (versions attempted: `11.3` and `12.1` ) I get an error when it tries to identify my security scheme for some endpoints in my openapi.json; I get `cannot read property type of undefined at isApiKeySecuritySchemeInHeaderOrQuery` . This makes sense--some of my endpoints are public and don't have a security scheme definition associated with them.
My understanding may be incorrect, but I thought that public apis don't need to have an explicitly defined security scheme in openpai3. I'm unable to share the openapi def as it's work stuff.
Is this an issue in the code (I notice if I deep dive and just change `scheme.type` to `scheme?.type` on the apiKey and http securityScheme validators all is well with the world) or with my understanding of openapi?
If it's an issue with the code, I'm happy to submit a PR to update! Thanks!
yousafn
2023-07-28 17:08
Hey up Randy,
It would be worth raising an issue on the repo, and a pull request would be welcome.
May be best to start with a test case to show the issue, in the PR, in the existing test suite.
Appreciate you can?t share your own api.
We have friends in the the world of OpenAPI who can get eyes over the issue.
It?s the weekend now for our Aussie engineers, so you may have to wait until early next week for some input.
Hope that helps!
thatrandybrown
2023-07-28 17:12
@yousafn that makes sense, thanks! it's closing on the weekend for me (and vacation next week) so I'll try to kick off the convo next week, but may be when I return from vacation.
Definitely open to starting with a convo around a test case.
maxime.raverdy
2023-07-31 15:48
Hi everyone,
We are using pactflow in my company.
Do you know if it's possible to download and save the json contract from the pact broker? In order to debug the provider test?
Thanks a lot :pray:
matt.fellows
2023-07-31 22:29
Hi Maxime, you can download the contract but not via the current UI (in the next iteration that will be added)
You can see the contract used in the validation in the verification logs, and simply click on that URL / view the network tab on the page
matt.fellows
2023-07-31 22:29
You can also navigate to pretty much any resource via the HAL browser: https://docs.pactflow.io/docs/api/
maxime.raverdy
2023-08-01 14:42
Oh I just found another way :slightly_smiling_face:
Just add `.json` to url like:
```https://company.pactflow.io/pacts/provider/PROVIDER_NAME/consumer/CONSUMER_NAME/version/VERSION_NUMBER```
maxime.raverdy
2023-08-01 14:43
But thanks a lot :smile:
> (in the next iteration that will be added)
Nice can't wait for that :pray:
prakhar.roy371
2023-08-03 11:58
Hi all.
This is regarding the "process of using a dockerfile to bake the lic file into the pactflow image".
I was able to do it in my local, but now I want to automate the whole process.
I have written a job in gitlab. (See attached ss)
*I am facing 2 issues here.*
1. docker tag command (2nd line in script section of job)
2. docker build command (5nd line in script section of job)
For 1.
I am able to run this command locally and push to harbor. But when I am doing that from gitlab, I am getting error saying *"docker tag requires exactly 2 arguments"*
For 2.
I commented the first three lines, and directly pulled existing image from Harbor (one without lic file)
and tried to bake in the file and push it back to harbor with *tag = baselic*
*But when I do that, in Harbor I am not seeing the lic file which I was seeing when I did it from local. (See harbor ss)*
Image marked as 1 = via cicd (no folder icon next to it)
Image marked as 2 = via local (shows folder icon next to it denoting that lic file baked in successfully)
matt.fellows
2023-08-03 23:07
Hi Prakhar, this is a bit harder for us to help I?m afraid. If it works in local, but not in CI that is likely to indicate differences between the setups, and that will be hard for us to diagnose without access to your Gitlab environment.
> *?docker tag requires exactly 2 arguments?*
this can happen if you?re using variables as argument, but those variables aren?t set on the runner. It can also happen due to incorrect escaping
> and tried to bake in the file and push it back to harbor with *tag = baselic*
> *But when I do that, in Harbor I am not seeing the lic file which I was seeing when I did it from local. (See harbor ss)*
This sounds like you might not be properly mounting the license file into the right place for it to be available in the script
matt.fellows
2023-08-03 23:08
I think it best you pair with somebody on your team who has done similar work before so that you can get their advice/knowledge of the particular challenges in your CI environment
matt.fellows
2023-08-04 05:11
For now, we have provided a workaround for your `allOf` situation. We have some backlog items created to improve the experience, that we are reviewing in the next iteration.
You will be automatically contacted via support when those features are completed.
jegadeesan.ponnusamy8
2023-08-08 03:08
Hi,
We are using Pactflow in our CICD pipelines to record the deployment of a microservice. But we are getting the below error message, whenever I try to record the deployment for an environment using Pacttoken. I am pretty sure that, I am using the ReadWrite token for recording the deployment. So I am not sure why Pactflow is giving this misleading error message. Could someone help me to figure out the root cause of this issue?
Error making request to https://avevasoftwarellc.pactflow.io/pacticipants/platform-catalog-async/versions/2a557f724adad76caff2e2a9596659cd80624f67/deployed-versions/environment/f92b098b-73f7-48e4-9075-ea6153311f15 status=403 Forbidden. Either you are using a read only token for a request that requires a write token (the most likely cause), or you do not have the required permissions.
Error making request to https://avevasoftwarellc.pactflow.io/pacticipants/platform-catalog-sync-api/versions/2a557f724adad76caff2e2a9596659cd80624f67/deployed-versions/environment/f92b098b-73f7-48e4-9075-ea6153311f15 status=403 Forbidden. Either you are using a read only token for a request that requires a write token (the most likely cause), or you do not have the required permissions.
Error making request to https://avevasoftwarellc.pactflow.io/pacticipants/platform-catalog-sync-internal/versions/2a557f724adad76caff2e2a9596659cd80624f67/deployed-versions/environment/f92b098b-73f7-48e4-9075-ea6153311f15 status=403 Forbidden. Either you are using a read only token for a request that requires a write token (the most likely cause), or you do not have the required permissions.
Error making request to https://avevasoftwarellc.pactflow.io/pacticipants/platform-catalog-sync-operations/versions/2a557f724adad76caff2e2a9596659cd80624f67/deployed-versions/environment/f92b098b-73f7-48e4-9075-ea6153311f15 status=403 Forbidden. Either you are using a read only token for a request that requires a write token (the most likely cause), or you do not have the required permissions.
matt.fellows
2023-08-08 03:11
if you are using a read/write token, then you do not have the required permissions (the other part of the error)
matt.fellows
2023-08-08 03:12
Alternatively, there is an intermediate proxy, and you aren?t authenticated. You can confirm this by adding `--verbose` to the command and seeing what requests/responses are made.
james_fraser
2023-08-11 15:52
Hey there :wave: is there any documentation for what the `/metrics` endpoint response in the Pactflow api represents?
I?m curious about the values returned by fields like `verificationResults` and `providerContractSelfVerifications` and would like to know what the numbers actually mean, as they don?t seem to line up with what my assumptions would be
matt.fellows
2023-08-11 23:27
``` "verificationResults": { <- regular pact provider verification
"count": 11, <- total number of provider verifications
"successCount": 7, <- ...of successful ones
"failureCount": 4, <- ...of failed ones
"distinctCount": 11,
"first": "2019-11-10T22:40:22+00:00", <- first time a verification happened
"last": "2023-07-27T04:56:38+00:00" <- last time one happened
},```
matt.fellows
2023-08-11 23:27
PactFlow does clean tenant data, so old data can be removed that would make these numbers unreliable.
OSS Pact Broker has cleaning also
https://docs.pact.io/pact_broker/administration/maintenance
matt.fellows
2023-08-11 23:30
> providerContractSelfVerifications
these are, I think, effectively the number of provider contract publications with self verification results.
I?m also confused by this number in my tenant, because it differs from the `providerContractPublications` item, which seems unusual. It could be another artifact of the cleaning process.
matt.fellows
2023-08-11 23:31
We have insights and reporting as something on our backlog to significantly improve in PactFlow, because of the limitations of the above (and to be honest, they are fairly crude metrics)
james_fraser
2023-08-14 09:06
Thanks for the clarification Matt :simple_smile: yeah if cleaning can modify the metrics that probably means we can?t rely on them too much
Looking forward to seeing what you?ve got planned :smile:
ariveros142
2023-08-14 22:10
Hi everyone, do you know how to fix this error: pact-core@13.15.0: {supportedPlatformsMessage}
We detected your platform as:
- darwin-arm64
We looked for a supported build in this location /Users/Documents/Repos/service/prebuilds/darwin-arm64
Tip: check there is a prebuild for darwin-arm64
check the prebuild exists at the path: /Users/Documents/Repos/service/prebuilds/darwin-arm64
Wrong Path?: set the load path with PACT_PREBUILD_LOCATION ensuring that $PACT_PREBUILD_LOCATION/prebuilds/darwin-arm64 exists
- Note: You dont need to include the prebuilds/darwin-arm64 part of the path, just the parent directory
- Let us know: We can add more supported path lookups easily, chat to us on slack or raise an issue on github
[16:57:53.137] ERROR (22295): pact@11.0.2: The pact mock service doesn?t appear to be running
- Please check the logs above to ensure that there are no pact service startup failures
- Please check that pact lifecycle methods are called in the correct order (setup() needs to be called before this method)
- Please check that your test code waits for the promises returned from lifecycle methods to complete before calling the next one
- To learn more about what is happening during your pact run, try setting logLevel: ?DEBUG?
Thanks!
ariveros142
2023-08-14 22:24
it should be looking at the node module levels no? but is looking at the repo level
matt.fellows
2023-08-14 22:26
can you provide a little more info about your setup - is there a repro you can provide?
ariveros142
2023-08-14 22:32
it solves the first issue but still getting: [17:30:00.489] INFO (27853): 0.4.7: pact native library successfully found, and the correct version
[17:30:01.276] ERROR (27853): pact@12.1.0: The pact mock service doesn?t appear to be running
- Please check the logs above to ensure that there are no pact service startup failures
- Please check that pact lifecycle methods are called in the correct order (setup() needs to be called before this method)
- Please check that your test code waits for the promises returned from lifecycle methods to complete before calling the next one
- To learn more about what is happening during your pact run, try setting logLevel: ?DEBUG?
ariveros142
2023-08-14 22:33
Unfortunately I cant share code here, but I can send directly to you if that works let me know
matt.fellows
2023-08-14 23:18
sure, you can, ideally a minimal example with no work IP in it would be fine though
sashi.kandru
2023-08-15 12:12
@matt.fellows: Are there any alternatives you would suggest on how to approach this? Currently we have a separate OAS for each version of the code, Is BDC approach possible with separate OAS? https://pact-foundation.slack.com/archives/C5F4KFKR8/p1692042155313029
matt.fellows
2023-08-15 12:41
No, the OAS for a given version of a provider (version being an independently deployable thing) must fully describe it. If there are multiple APIs it supports spread across multiple OAS files, they must be merged. If they are separately deployed, they might be best modelled as different providers (i.e. unique name in Pactflow)
matt.fellows
2023-08-15 12:43
The workarounds are to merge them (there are plenty of tools that can do this, it's probably quite straightforward without a dedicated tool) or separate the apps out logically in Pactflow
adebakre
2023-08-15 15:27
Hi all. I'm quite new to pactflow and pact as well. I am trying to set up a poc for a .NET implementation using a free pactflow account. I can't seem to see any c# template on the pactflow UI. Can anyone direct me to where i can get steps to set this up? The documentation is not very clear. Thanks
yousafn
2023-08-15 15:30
linked from here
https://docs.pactflow.io/#consumer-driven
on the .net tab (which i appreciate is tiny)
prakhar.roy371
2023-08-16 13:22
Hi all,
My pactflow on-prem setup has been done on Kubernetes (PKS). But right now it is in demo auth mode. (See screenshot)
I was going through the SAML Auth documentation ( https://docs.pactflow.io/docs/on-premises/authentication/saml/ )
There is a SSO enablement team in my company that wanted the following info so that they can configure the on-prem broker with SSO.
1. ACS / Callback URL
2. SAML Subject ( username / nt id / email )
3. Metadata URL ( when I go to the one as per doc, it shows nothing -> http://pactflow.cre.pks.com/auth/saml/metadata )
matt.fellows
2023-08-17 00:39
> 1. ACS / Callback URL
https://docs.pactflow.io/docs/on-premises/authentication/saml/#assertion-consumer-url
The URL is `https://<your PactFlow host>/auth/saml/callback`.
> 2. SAML Subject ( username / nt id / email )
https://docs.pactflow.io/docs/on-premises/environment-variables#pactflow_saml_idp_id_attribute
> 3. Metadata URL ( when I go to the one as per doc, it shows nothing -> http://pactflow.cre.pks.com/auth/saml/metadata )
If you are unable to reach this endpoint, then something might be incorrect in your setup.
prakhar.roy371
2023-08-17 05:49
For Metadata URL, I have configured it in environment variables like this :
But it is throwing 404 when trying to access it, am I missing something here?
matt.fellows
2023-08-17 05:51
Your IDP should have a metadata URL that PactFlow can query to get the information it needs. If you don?t know it, you need to specify the other attributes in the guide (the metadata endpoint can be used to discover the other items)
prakhar.roy371
2023-08-17 06:01
So the devpactflow.cre.pks.__<http://.com|.com> is our IDP configured by our PKS team.
They said the vendor provides the metadata url. That is why I pinged here.
matt.fellows
2023-08-17 06:56
Yes, there are two metadata URLs: The IDP metadata URL (https://docs.pactflow.io/docs/on-premises/environment-variables#pactflow_saml_idp_sso_target_url) and the Service Provider one, which is the one available at `https://<your PactFlow host>/auth/saml/metadata` (https://docs.pactflow.io/docs/on-premises/authentication/saml#metadata-url)
prakhar.roy371
2023-08-17 07:01
So correct me if I am wrong here,
We JUST need to set the "PACTFLOW_SAML_IDP_SSO_TARGET_URL" (IDP Metadata URL).
The Service provider one is automatically set by default with value
*[https://<your PactFlow host>/auth/saml/metadata]* and will query from target url
matt.fellows
2023-08-17 07:09
You need to set all of the required ones here: https://docs.pactflow.io/docs/on-premises/environment-variables/#saml-authentication
If you set the metadata URL (`PACTFLOW_SAML_IDP_METADATA_URL`) , then the PACTFLOW_SAML_IDP_SSO_TARGET_URL and PACTFLOW_SAML_IDP_CERT_FINGERPRINT can be skipped.
matt.fellows
2023-08-17 07:09
> The Service provider one is automatically set by default with value
> *[https://<your PactFlow host>/auth/saml/metadata]* and will query from target url
correct
mahesh.damavarapu
2023-08-17 11:45
Hello, In the scope of bi directional contract testing, Will Pactflow do the verification/comparison of contracts after publishing both consumer contract and provider contract?
matt.fellows
2023-08-17 12:46
Yes. It needs both to compare and check if the parties are compatible. Is that what you're asking?
mahesh.damavarapu
2023-08-17 12:55
I mean are you saying pactflow will be doing the comparision and update the status accordingly ? If so, what is the frequency, is it configurable ?
matt.fellows
2023-08-17 22:47
> I mean are you saying pactflow will be doing the comparision and update the status accordingly ? If so, what is the frequency, is it configurable ?
If you think about it, it doesn?t make sense to check on a schedule. The only time the value will change is if the consumer publishes a new contract, or the provider does, or if either of them deploy to a different environment.
When you call `can-i-deploy` if there isn?t a comparison pre-calculated, it will calculate one.
aravind.pai
2023-08-18 06:26
*Possible to hide secrets in the pactflow web UI?*
We have some authentication API in our contract tests where we pass an authentication token. This appears in plaintext in the web UI under `Interactions`. To avoid revealing the tokens (for example, in customer demos), is it possible to mask this information in the web UI?
Similar to how Github and GitLab do in CI build logs by matching user-configured secrets.
matt.fellows
2023-08-18 06:31
Oh. Why are you using real tokens in contract tests?
I thought you were asking about PactFlow secrets, but these are separate things
aravind.pai
2023-08-18 06:39
We don't use real tokens in contract tests. But the same "fake" token needs to be configured on the provider for the authentication test to be successful (and hence, the contract to be verified). That could be a potential risk.
matt.fellows
2023-08-18 06:40
this is really designed for controlled test environments, but I can see that some people could run these against live environments (generally a bad practice)
aravind.pai
2023-08-18 06:44
Hmm, now that I think about it, forget everything I said. This is a non-issue (at least for us). We load our provider in a docker container for the contract verification, and dispose of it immediately after the contract verification is done. So as long as we don't use any token that's used in some real deployed instance, we should be fine I guess.
eugeniosaulo
2023-08-18 21:34
I have published my first contract to PactFlow. I am doing BiDirectional contract testing. I published the contract but PactFlow is complaining that I did not publish the self verification test result.
I am using Spring Cloud Contract to verify the contract before publishing. Can I use those results and push them to PactFlow or do I have to use Pact?
yousafn
2023-08-18 21:59
Hey bud,
yes you can post your test run output to PactFlow, and the success or failure of the test run. We normally connect these as an output of the test run, so they publish with the corresponding exit code from the tests.
https://docs.pactflow.io/docs/bi-directional-contract-testing/publishing#usage
These are the options you would need
1 of the following for your self-verification result.
``` [--verification-success], [--no-verification-success]
# Whether or not the self verification passed successfully.
[--verification-exit-code=N]
# The exit code of the verification process. Can be used instead
of --verification-success|--no-verification-success for a
simpler build script.```
and the following parameters for uploading the report file. Note the report file can any content, it's more of a convenience for users of your system so they can reference the test results from your providers own test suite (which isn't Pact)
``` [--verification-results=VERIFICATION_RESULTS]
# The path to the file containing the output from the
verification process
[--verification-results-content-type=VERIFICATION_RESULTS_CONTENT_TYPE]
# The content type of the verification output eg. text/plain,
application/yaml
[--verification-results-format=VERIFICATION_RESULTS_FORMAT]
# The format of the verification output eg. junit, text
[--verifier=VERIFIER]
# The tool used to verify the provider contract
[--verifier-version=VERIFIER_VERSION]
# The version of the tool used to verify the provider contract```
matt.fellows
2023-08-21 05:21
Just coming back to this one Rajat. the `oneOf` issues you?re having, have you looked at adding a `discriminator` keyword: https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support/#the-discriminator-keyword?
That would help narrow the chances of a schema conflict
mahesh.damavarapu
2023-08-21 15:04
Hello, I am doing Bidirectional Contract Test with GraphQL API. After publishing both consumer and provider contract, I am getting the following errors
```Response.Content-Type.Incompatible
Response Content-Type header is incompatible with the mime-types the spec defines to produce
[root].interactions[0].response.headers.Content-Type = "application/json"
Request.Content-Type.Unknown
Request content-type header is defined but the spec does not specify any mime-types to consume
[root].interactions[0].request.headers.Content-Type = "application/json"```
The headers are same in both contracts. Any help here please
matt.fellows
2023-08-21 21:27
It says the headers you have defined in the pact file don't exist in the OAS
mahesh.damavarapu
2023-08-22 00:30
Hi Matt, Thanks for reply. I see it's the issue of postman and postman2openapi npm package.
In postman, if we are doing Graphql and provide the query and variable seperately and try to transform into OAS file, the request body and response headers are missing. It needs to select as none body type and provide both query and variables in a single json and mime type as application/json, the conversion to OAS is properly working
prakhar.roy371
2023-08-22 06:54
@matt.fellows
I got the IDP metadata ( its in xml format - SEE SCREENSHOT)
I have a link that reaches this location. I have provided that in the environment variables.
*ISSUE*
When I click on login button, the button is not redirecting anywhere. Showing this while inspect. (See ss below)
matt.fellows
2023-08-22 07:03
Hi Prakhar, we?re going to need to know the values you have set in your configuration to help you. The CSP issue is because there is a misalignment between the host names you are allowed to post to, and what it is actually wanting to post to.
Best to raise a support ticket so the team can help you
matt.fellows
2023-08-22 07:04
Note how the form is trying to post to `http://devpactflow?` but in the metadata it wants to post to `https://pf-uat?`. Something is misaligned that is resulting in that setup
matt.fellows
2023-08-22 07:04
The logs of the PactFlow instance should tell you what values it sees for all of it?s configuration, please share those with the support ticket
2023-08-22 07:04
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
prakhar.roy371
2023-08-22 07:10
Okay sure, I can raise the support ticket and provide all the info.
One issue there is that its asking for Account URL.
But smart bear team would not be able to access that without our VPN.
Is there a way we can have a short live session on zoom etc for this?
matt.fellows
2023-08-22 07:14
The support ticket would be the starting point for that if it gets to it.
matt.fellows
2023-08-22 07:15
hmm no it doesn?t, you can just ignore that and I?ll ask to see if we can make it go away for those using on-prem
prakhar.roy371
2023-08-22 07:16
Since we are using the latest pact image available from quay, I provided the latest available version.
Shall I provide our internal DNS for account URL for now or leave it blank?
matt.fellows
2023-08-22 07:17
> Shall I provide our internal DNS for account URL for now or leave it blank?
It?s probably helpful context, but you can just provide that in the problem description
prakhar.roy371
2023-08-22 07:28
I have provided all the information along with the error i am seeing as well as the PKS deployment script where I have set up the envs variable for pactflow instance.
matt.fellows
2023-08-22 07:30
thanks! did you also attach the broker logs showing the configuration it sees?
matt.fellows
2023-08-22 07:32
That will be more important that the PKS configuration, because it shows exactly what the broker thinks was given to it
s.lohmeier
2023-08-22 09:09
We are getting 504 errors from the Pactflow API. Could anyone look into this?
matt.fellows
2023-08-22 09:20
Could you please raise a support ticket so we can track it?
howtopactflowsupport
2023-08-22 09:20
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
matt.fellows
2023-08-22 09:21
I don't see any issues with our monitoring could be specific to your account or something else.
s.lohmeier
2023-08-22 09:50
I would rather want to avoid creating another ticket with your customer service. I did that when we had a problem with 502 errors from Pactflow in the past and they were not able to solve the issue (to put it mildly).
matt.fellows
2023-08-22 10:24
what?s your tenant? Gateway timeouts are usually due to data complexity and certain queries timing out.
We are aware of some performance issues which we are working on (mostly related to matrix e.g. https://github.com/pact-foundation/pact_broker/pull/631)
Are you seeing them via certain API calls?
s.lohmeier
2023-08-22 11:22
Sent tenant and more details in a direct message. We got the 504 when loading a Pact before verification. Thanks for you help, Matt!
yousafn
2023-08-22 14:02
Always an exciting day at SmartBear! Super cool to see Stoplight join the fold. Spectral and Prism have long been part of my open source tool-belt, and can?t wait to help connect the Swagger / Pact / Cucumber & SoapUI communities together with Stoplight, and helps consolidate a fragmented API testing landscape.
https://smartbear.com/blog/elevating-api-quality-with-stoplight-and-smartbear/
Have you used Stoplight in the past? Any other open-source tools that you can?t live without? Why not let us know in the thread :point_down:
tati.shep
2023-08-24 07:23
Hi Pactflow team,
Our bi-directional contracts started failing with no changes on our side. I found that changes to https://github.com/pactflow/swagger-mock-validator/pull/38 got released recently. I don?t quite understand how to fix that on our side though. We don?t set `additionalProperties: false` in our schema, should we do it on each object type to be able to pass validation now?
tati.shep
2023-08-24 07:57
for more context:
we do have fields like `properties` or `metadata` where we expect any field to be send and we store the object as is later in the db.
Now all such fields trigger compatibility errors `must NOT have additional properties` even if I explicitly set `additionalProperties: true`
matt.fellows
2023-08-24 13:18
additionalProperties can also accept a schema of allowed types, have you considered this case?
It's a tricky problem. We've seen a lot of examples where it's set to true but for bad reasons.
We may need to consider allowing an application to configure that behaviour
tati.shep
2023-08-25 03:26
> additionalProperties can also accept a schema of allowed types, have you considered this case?
yeah, but it?ll be a temporary solution for us as if tests start sending different fields in the object (and we don?t care that they change), we will have to update provider contract again.
As I still want to test such a field and have a stable contract, I?d like to have a way to say ?ok, I know the risks, please apply additionalProperties: true?
prakhar.roy371
2023-08-25 12:29
Hi all,
We were using the on-prem license right now for Pactflow. But now we acquired the pact saas license now.
I wanted to know what would be the steps to enable company sso login on saas pactflow since everything is handled at Pactflow end.
I couldn't find any docs on that.
Another question is : Once any person logs in to pactflow, how will they know which team they will be a part of. Is that job of the admin/team admin?
(See screenshot below)
charlan.bettiol
2023-08-25 17:41
Hello everyone, good afternoon, how are you?
recently here at our company we bought pact-flow and we are implementing and assembling the flows and we faced some problems and I would like to know if we can help us?
The first moment would be, what is the ideal flow for using and creating contract tests?
The second would be: today we have the problem of being able to develop pacts locally using pact-flow but we have the following issue, the api tokens cannot be exposed in the code since we do not know how to do it at this point so that the local development is available using the online broker.
matt.fellows
2023-08-28 06:15
I think I?ve created a type that will work for you. I?ll publish an example shortly
matt.fellows
2023-08-28 06:44
I?ve also realised you?re sending this on the _request_ side - the change above probably is most impactful on the response side. The issue is it?s hard for us to detect if it?s in a `definition` - it could be used in both contexts.
matt.fellows
2023-08-28 06:45
See this example here: https://github.com/pactflow/bdct-oas-examples/tree/main/examples/additionalProperties
matt.fellows
2023-08-28 06:45
I think this expresses a JSON type that is equivalent to `additionalProperties: true` .
kedar.ghate
2023-08-28 07:41
Hi team,
Is there any way to export matrix data to file or get it via pactflow CLI command?
matt.fellows
2023-08-28 10:33
Can you please elaborate on specifically the data (use cases) you want for those reports? The matrix has a lot of data, I suspect you just want a subset of it
kedar.ghate
2023-08-28 11:04
Looking for columns captured in below screen shot for all records of a week
alexander.friesen
2023-08-28 12:47
Hi @matt.fellows.
I believe the issue in our "*oneOf*" case - was not described correctly.
Let me try to describe it.
Here the starting point:
*We have a producer-contract (OAS)*
with endpoint *POST /partition/{partitionId}/points/{pointId}/alarm-configurations*
using the "*oneOf*" statement
*with 3 DISJUNCT schemas. (*So the schemas do not overlap.)
We have written a consumer-contract
where we provide a POST body, which matches *exactly ONE of the 3 schemas*
listed in "oneOf"
See the screenshot below, where the "binaryPointAlarmConfiguration" property is available
in exactly one schema and in our consumer-contract body
alexander.friesen
2023-08-28 12:51
*The problem statement:*
When comparing the OAS and the consumer-contract body
the http://pactflow.io console complains
`"Request body is incompatible with the request body schema in the spec file: *must match exactly one schema* in oneOf`
`[root].interactions[7].request.body.data.attributes.configuration = {"binaryPointAlarmConfiguration":{"alarmValue":true}}"`
*Issue:*
But our POST body DOES indeed match to only one schema listed in "oneOf" statement.
Only to the "BinaryPointAlarmConfiguration", as described in the above post.
*Question:*
Is that true, that the evaluation of http://pactflow.io does not perform as expected and that its a bug?
matt.fellows
2023-08-28 13:18
Thanks, and yes possibly. Any chance you could please provide a minimal version of your setup that induces the bug? We can then get it addressed
riley.marzka
2023-08-28 15:20
Following this thread as well. It would be good to be able to export that data to show on other dashboards and such. Especially for less technical audiences.
I am currently in the process of getting buy-in from the managers/product owners/directors across our engineering teams. Being able to show this data next to some of our other dashboards would help to get more buy-in.
It would also be more convenient, since then our engineers would not need to go to an additional page to see how their Pact workflows are doing ? I could pipe the data export into our centralized developer portal and show it alongside the other health, performance, and test metrics for our apps.
arpit.gawande
2023-08-28 15:30
@matt.fellows Hi Matt, attaching minimal version of the swagger and the payload cc/@alexander.friesen @rajat.chouhan
matt.fellows
2023-08-28 21:36
There are APIs, I'm just keen to understand the use cases and scenarios.
What Dev dashboard tools do you use? What else does it show?
matt.fellows
2023-08-28 22:57
I can?t reproduce the error you have with the files you have given. I have been able to produce a different error, but it would be ideal if you could please share a `payload.json` with the correct context (i.e. the pact you have generated, stripped down to the failing interaction only)
matt.fellows
2023-08-28 22:59
This being said, I did reproduce an `additionalProperties` failure:
``` AlarmConfigurationAttributesGET:
type: object
required:
- createdAt
- lastUpdatedAt
- message
- category
- enabled
- configuration
properties:
configuration:
# type: object # <- uncomment this and it fails
oneOf:
- $ref: "#/components/schemas/AnalogPointAlarmConfiguration"
- $ref: "#/components/schemas/BinaryPointAlarmConfiguration"
- $ref: "#/components/schemas/MultiStatePointAlarmConfiguration"
createdAt:
type: string
format: "date-time"
lastUpdatedAt:
type: string
format: "date-time"
message:
type: string
category:
type: string
enabled:
type: boolean```
This might be a bug, either in PactFlow, ajv (the schema validator we use) or just a misunderstanding of how `type: object` works.
matt.fellows
2023-08-28 22:59
You can test locally with this command:
`npx @pactflow/swagger-mock-validator@latest test-swagger.yaml payload.json`
matt.fellows
2023-08-28 23:21
Looks like we still have ticket #00560370 open with you. Can we please carry that there, otherwise we might double up :slightly_smiling_face:
matt.fellows
2023-08-28 23:22
_(I?ve asked for the ticket to go back to you to confirm the specific scenarios that are failing so I can have it investigated by a dev)_
tati.shep
2023-08-29 02:07
Thanks a lot for the example @matt.fellows!
> I?ve also realised you?re sending this on the _request_ side
yeah, that?s true. anyway it?s the part the response side is responsible for. so, it does make sense to update the provider?s contract.
abubics
2023-08-29 04:32
e.g. what kind of decision are you trying to make (using the extracted data)?
matt.fellows
2023-08-29 04:39
For further context, the matrix is a really large set of data, across a number of domain models. much of which is transient and probably not useful for decision making.
I'm guessing things like "which applications are currently deployable to production" might be useful.
matt.fellows
2023-08-29 05:58
yep. I?m avoiding stating the use cases I think are useful, because I want here from them. We get this kind of question a lot, but the specific information that is useful seems to be much more allusive.
I?m not saying it?s the case here, but people want to see ?data? but upon closer inspection, they can?t answer why
kedar.ghate
2023-08-29 10:30
all users do not log in to pactflow portal to see the matrix,
to understand how many failures in given week / certain duration
kedar.ghate
2023-08-29 10:33
is the export feature available or you will plan to add such feature by gathering the requirement?
matt.fellows
2023-08-29 11:48
Follow the public API discussion here: https://github.com/pactflow/roadmap/projects/1
matt.fellows
2023-08-29 11:49
> to understand how many failures in given week / certain duration
so, the matrix doesn?t have this data. Were you planning on taking the data, ingesting it, and then deriving this information from it?
yousafn
2023-08-29 14:13
I think the link should be https://github.com/pactflow/roadmap/issues/94
Please feel free to add any technical requirements or anything additional to there, as it will be useful for tracking, if you wish.
riley.marzka
2023-08-29 15:41
Just wanted to jump back in here to say that the API would fit the use cases that I was thinking of. I can't think of a scenario right now where an 'export' would better achieve the need over the API. The only _maybe_ would be exporting the data and then importing it to DataDog for analytics and to build dashboards ? but that can also be achieved using the API, just requires a little more processing before uploading to DataDog.
rajat.chouhan
2023-08-30 04:14
We are able to reproduce the error. Please use the attached point yml.
Request body is incompatible with the request body schema in the spec file: must match exactly one schema in oneOf
[root].interactions[7].request.body.data.attributes.configuration = {"binaryPointAlarmConfiguration":{"alarmValue":true}}
rajat.chouhan
2023-08-30 05:39
Unevaluated property is resolved but we are also facing `additionalProperties` failure and that existing 'oneOf' schema match issue. Should we raise separate ticket for additional property failure issue?
matt.fellows
2023-08-30 05:56
FYI I have created https://github.com/pact-foundation/pact-jvm/issues/1710 to track this issue. The HTTP client should be retrying, but because the request is not considered a safe one to retry (its idempotency strategy) it gives up.
I think we just need to not use the Apache client default retry strategy.
Given the number of `504`s should be (and were) very low, this is a good solution to provide resiliency in the system.
Please follow that ticket for progress
matt.fellows
2023-08-30 05:59
As per above
``` AlarmConfigurationAttributesUpdate:
type: object
properties:
configuration:
#type: object <--------- ? this needs to be removed
additionalProperties: true
oneOf:
- $ref: "#/components/schemas/AnalogPointAlarmConfiguration"
- $ref: "#/components/schemas/BinaryPointAlarmConfiguration"
- $ref: "#/components/schemas/MultiStatePointAlarmConfiguration"```
matt.fellows
2023-08-30 05:59
Using my pact file from yesterday (above) and your points file, if I comment out the `type: object` where the `oneOf`s exist, this passes
matt.fellows
2023-08-30 06:00
Unfortunately the key engineer I need to explain to me why this matters (and if it?s a bug/expected) had to head on leave due to a family emergency, but when they return I will follow up. For now, that workaround hopefully should address your issue
shayanadc
2023-08-30 06:09
After creating my account, I noticed a message indicating that I would receive my credentials via email. However, despite multiple attempts, I have not yet received this email.
Do I need to do something else??
matt.fellows
2023-08-30 06:17
Are you able to share any details about your account? Feel free to DM me your email or account name you created so I can look it up in our system
kedar.ghate
2023-08-30 07:07
okay, could you share documentation link where I can find technical steps to implement custom solution
kedar.ghate
2023-08-30 08:58
Also could you please confirm the max no. of records / statuses within one integration that pactflow can persist
rajat.chouhan
2023-08-30 09:39
By commenting those elements, the additional property issue is resolved. Then we are getting existing 'oneOf' matching issue described above.
matt.fellows
2023-08-30 10:41
Can you please share the failing test with the following command:
```npx @pactflow/swagger-mock-validator@latest <path to oas> <path to pact file>```
matt.fellows
2023-08-30 10:54
there are no ?limits? set in PactFlow, however we do clean data on occasion. See https://docs.pact.io/pact_broker/administration/maintenance for background on how the OSS pact broker cleaning works. At PactFlow, we manage this on your behalf automatically in order to optimise performance, so there are no specific numbers we can provide.
matt.fellows
2023-08-31 05:22
There is a `/matrix` endpoint that you can call to get the data. It?s not currently documented, but if you head to the `/can-i-deploy` page in PactFlow or the matrix tab of any integration, you can get a feel for the parameters.
matt.fellows
2023-08-31 05:22
you can also add `--verbose` to the `can-i-deploy` command calls and it will print it out
rajat.chouhan
2023-08-31 07:27
attached the error of 'oneOf' match schema which I got while running by the command you provided.
sharathkonda
2023-08-31 15:44
Unknown format integer is used in schema at path "#/properties/price"
https://github.com/pactflow/example-bi-directional-provider-postman/blob/master/oas/swagger.yml
In the above example swagger file, if I change the price property type from number to integer and introduce format as int32 based on the below documentation:
https://swagger.io/specification/v3/
I am getting the Unknown format integer is used in schema at path "#/properties/price"
Please note if I keep the type as number - all good but in our company, providers are using integer format.
jordan.nazemi
2023-08-31 18:47
Hey @matt.fellows could you share the `allOf` workaround your support team provided Rajat? Facing a similar issue in our implementation
matt.fellows
2023-08-31 21:13
If it?s the same issue, where there is a `oneOf` , drop the `type: object` bit. I?m still figuring out _why_ this works.
``` AlarmConfigurationAttributesUpdate:
type: object
properties:
configuration:
#type: object <--------- ? this needs to be removed
additionalProperties: true
oneOf:
- $ref: "#/components/schemas/AnalogPointAlarmConfiguration"
- $ref: "#/components/schemas/BinaryPointAlarmConfiguration"
- $ref: "#/components/schemas/MultiStatePointAlarmConfiguration"```
matt.fellows
2023-08-31 21:52
@rajat.chouhan I can reproduce the issue also. Please send that back to the original ticket. When the engineer working on it returns from leave, we will investigate. Thanks for putting that together
matt.fellows
2023-08-31 22:20
Hi!
It should work with something like this:
```price:
type: integer
format: int32
example: 99.99```
You folks are running on-prem right? Do you know which version you?re on?
matt.fellows
2023-08-31 22:33
oh, I think you?re saying:
```price:
type: number
format: integer
example: 99.99```
That doesn?t work on the latest on-prem, however in the next release it should, as it uses a newer validator version.
Is that an allowed format type though?
sharathkonda
2023-09-01 09:22
Thanks @matt.fellows
Noticed it is failing because the type is integer and the format is integer rather than int32/int64
We are letting the development team to correct the format type
mahesh.damavarapu
2023-09-02 10:28
Hello, In scope of bidirectional contract testing, when the body of a response is same in both consumer and provider contracts. But still I am getting incompatibility errors for all fields. Any help or fix here
bas
2023-09-02 11:01
Can you give us some more information here? Can you share your contracts? Example message payloads? The error messages you see?
My crystal ball is broken.
mahesh.damavarapu
2023-09-02 11:11
```Response.Body.Incompatible
Response body is incompatible with the response body schema in the spec file: must NOT have additional properties - limit
[root].interactions[0].response.body = {"limit":1,"products":[{"brand":"Apple","category":"smartphones","description":"An apple mobile which is nothing like apple","discountPercentage":12.96,"id":1,"images":["https://i.dummyjson.com/data/products/1/1.jpg","https://i.dummyjson.com/data/products/1/2.jpg","https://i.dummyjson.com/data/products/1/3.jpg","https://i.dummyjson.com/data/products/1/4.jpg","https://i.dummyjson.com/data/products/1/thumbnail.jpg"],"price":549,"rating":4.69,"stock":94,"thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg","title":"iPhone 9"}],"skip":0,"total":100}
Response.Body.Incompatible
Response body is incompatible with the response body schema in the spec file: must NOT have additional properties - products
[root].interactions[0].response.body = {"limit":1,"products":[{"brand":"Apple","category":"smartphones","description":"An apple mobile which is nothing like apple","discountPercentage":12.96,"id":1,"images":["https://i.dummyjson.com/data/products/1/1.jpg","https://i.dummyjson.com/data/products/1/2.jpg","https://i.dummyjson.com/data/products/1/3.jpg","https://i.dummyjson.com/data/products/1/4.jpg","https://i.dummyjson.com/data/products/1/thumbnail.jpg"],"price":549,"rating":4.69,"stock":94,"thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg","title":"iPhone 9"}],"skip":0,"total":100}
Response.Body.Incompatible
Response body is incompatible with the response body schema in the spec file: must NOT have additional properties - skip
[root].interactions[0].response.body = {"limit":1,"products":[{"brand":"Apple","category":"smartphones","description":"An apple mobile which is nothing like apple","discountPercentage":12.96,"id":1,"images":["https://i.dummyjson.com/data/products/1/1.jpg","https://i.dummyjson.com/data/products/1/2.jpg","https://i.dummyjson.com/data/products/1/3.jpg","https://i.dummyjson.com/data/products/1/4.jpg","https://i.dummyjson.com/data/products/1/thumbnail.jpg"],"price":549,"rating":4.69,"stock":94,"thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg","title":"iPhone 9"}],"skip":0,"total":100}
Response.Body.Incompatible
Response body is incompatible with the response body schema in the spec file: must NOT have additional properties - total
[root].interactions[0].response.body = {"limit":1,"products":[{"brand":"Apple","category":"smartphones","description":"An apple mobile which is nothing like apple","discountPercentage":12.96,"id":1,"images":["https://i.dummyjson.com/data/products/1/1.jpg","https://i.dummyjson.com/data/products/1/2.jpg","https://i.dummyjson.com/data/products/1/3.jpg","https://i.dummyjson.com/data/products/1/4.jpg","https://i.dummyjson.com/data/products/1/thumbnail.jpg"],"price":549,"rating":4.69,"stock":94,"thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg","title":"iPhone 9"}],"skip":0,"total":100}```
matt.fellows
2023-09-02 11:30
You haven?t defined a schema for the endpoint in your OAS file, just an example
matt.fellows
2023-09-02 11:34
if you put the OAS into https://editor.swagger.io/, and look at the schema for the endpoint response, you?ll note it?s empty (don?t look at the ?Example Value? section, because that uses what you have defined by default.
mahesh.damavarapu
2023-09-03 15:07
From this repo - https://github.com/pactflow/example-bi-directional-provider-postman/tree/master
When i manually converted the postman collection to OAS file, the file generated does not have schema related details.
Is it a manual effort to explicitly add schema details into oas file after conversion ?
bas
2023-09-03 18:16
The provider OpenAPI spec is in that project already, no need to convert it yourself, with all the things that can potentially go wrong there:
https://github.com/pactflow/example-bi-directional-provider-postman/tree/master/oas
matt.fellows
2023-09-03 22:10
The postman to API tool should do that, but perhaps for some reason it's not working. The OAS is the key artefact we need though, and without a schema it's useless
james_fraser
2023-09-04 19:25
Hey @matt.fellows :wave: we?ve got a workaround (removing that property from the OAS file before publishing it) but just to check has there been any update on resolving this issue?
sharathkonda
2023-09-04 21:49
#pactflow Response Body is incompatible with the response body schema in the spec file: should NOT have additional properties - itemA mismatched at: .........:
Support ticket case: #00565086
My provider response object contains in this format:
Body:
{
"listAddressCustomerResponse": {
"item":""
},
"*addressCustomerResponse*": {
"itemA": *[*
{ different objects },
{ different objects }
*],*
"itemB": *[*
{ different objects },
{ different objects }
*]*
}
}
The consumer contract is passing only if I don't mention addressCustomerResponse object which contains the array objects.
Can someone let me know whether the response can contain the array objects within an object.
matt.fellows
2023-09-04 23:23
Hi Venkat!
I?ve responded to the ticket just now. FYI no need to repost here after sending to support, it will get routed to the team for review.
You can ask your question in here first if you like and we can work through it, and if we need to create a ticket to track it, we can.
TL;DR - invalid OAS.
matt.fellows
2023-09-05 04:58
Oh, sorry for not coming back to this thread. I believe it should be fixed in PactFlow now (see also https://github.com/pactflow/swagger-mock-validator/pull/37)
james_fraser
2023-09-05 08:34
Awesome thanks Matt :smile: appreciate the response, I?ll take a look at removing our workaround and seeing if it still behaves
Cheers!
matt.fellows
2023-09-05 08:53
P.s. I noticed the ticket did take a while to get to us. I'm following up (and understand why you may have wanted to reach out)
prakhar.roy371
2023-09-06 10:51
Hi @matt.fellows, if my pactflow saas license allows 100 users, What happens when the 101th person logs in?
Do I get prompted with a rejection or extension options of my current plan?
Another question is, at a later stage if I want to change my domain name from http://abc.pactflow.io to http://xyz.pactflow.io is that possible?
matt.fellows
2023-09-06 12:11
You shouldn?t be allowed to exceed the number of users. As an enterprise customer, you would need to talk to your account manager. They would have access to see the growth of the account, and would reach out in advance of htat.
Yes, you can change the tenant URL. You would need to submit a ticket so that we can manage that for you
hope that helps!
prakhar.roy371
2023-09-06 13:05
Oh okay I see! That helps, thanks for the response :slightly_smiling_face:
nathan.deamer
2023-09-06 15:38
Hi,
Is there a way to create a secret in pactflow via an API or via the cli? I couldn?t see anything in the documentation.
nathan.deamer
2023-09-06 16:03
preference for curl. I had a look in the HAL browser
?No documentation exists for this relation yet.?
nathan.deamer
2023-09-06 16:10
works:
```curl 'https://xxx.pactflow.io/secrets' -H 'Authorization: Bearer xxx' -H 'Content-Type: application/json' -d '{
"name": "NathansTestSecret",
"teamUuid": "xxx",
"description": "Some Description",
"value": "NathsnTestValue"
}'```
yousafn
2023-09-06 16:12
nice. it should be part of the public api docs soon i believe. i always make calls in the browser and grab the curl req remove all the headers i don?t need and go from there
you can also plug a proxy in and spy on your traffic to get the calls for different actions
yousafn
2023-09-06 16:14
i have an alias for curl with the header value read form the token env var. i really like gh?s cli that let you do prescribed things, but then also get access to an api client, and also extend it with bash scripts/go scripts etc
prakhar.roy371
2023-09-07 12:56
Hey all,
I wanted to check one thing regarding "Contract Testing".
From what I understood, in Contract testing we take into consideration the STRUCTURE of the data as well as its datatypes.
If there are 10 use cases all of which return the same structure of data, we will write only ONE contract test for it. Correct?
I am trying to understand if say there are 100 tests that exist in a particular project, how do we understand how many contract tests need to be written?
bas
2023-09-07 18:37
As with every type of testing, there?s only one answer to the question of ?how many?: exactly enough to give you the amount of confidence you need to reasonably assume that you didn?t break an integration.
Without knowing more about your context, process, development and deployment practices and other things, it?s impossible to give you a number.
Also, this is more suited for #general I think.
prakhar.roy371
2023-09-08 04:11
Lets say there are two API
1. GetEmployeeByID
2. GetEmployeeByEmail
Both return:
{
Name: Prakhar,
EmpID : 1
Email :
Contact: 888-888-888
}
In the above scenario there would be 2 unit cases : one for testing each endpoint.
My question here is since we are getting same structure of data in return, do we write 1 contract test for this or 2?
bas
2023-09-08 05:46
It depends. Who are your consumers? Do they expect the same behaviour from both endpoints? If you?re going the consumer-driven route, your consumers decide on the number of tests (or scenarios, really). They might not even be aware that both endpoints have the same payload or the same implementation, and they shouldn?t have to.
thomas.koppensteiner
2023-09-08 06:12
Hello,
the pactflow Web UI allows to delete a contract.
Is there a way to delete contracts through the API as well. For example after a merge into the main branch?
What is the best way to clean up old contracts?
matt.fellows
2023-09-11 02:21
Yes, you can delete contracts in the same way. Issuing a `DELETE` to the contract URL will do that, so you could definitely do that
matt.fellows
2023-09-11 02:23
We have plans in PactFlow to auto-delete branches that are merged etc.
Note we also automatically clean up some old data (similar to https://docs.pact.io/pact_broker/administration/maintenance).
Feel free to vote on the SCM integration you use here (https://github.com/pactflow/roadmap/projects/1 - e.g. Github, gitlab, bitbucket) or create an issue if you have another one.
matt.fellows
2023-09-11 02:23
Out of interest, what are the use cases you have in mind that you would like to manage - is it just merging of PRs?
thomas.koppensteiner
2023-09-11 06:12
Thank you for the explanations.
Yes, merging of PRs is would be my main use case.
sharathkonda
2023-09-11 21:14
#pactflow Response Body is incompatible with the response body schema in the spec file: should NOT have additional properties - PhoneAddress
Mismatched Pact path:
[root].interactions[0].response.body.UpdatedContact.ContactPoint
mismatched Provider Path:
[root].paths........................schema.properties.UpdatedContact.properties.ContactPoint.additionalProperties
My provider response object contains in this format:
Body:
{
"UpdatedContact": {
"ContactPoint":{
"PhoneAddress": {
"code":"+77",
"phoneNumber":"1234567"
}
}
}
}
My spec for the following is below attachment in a truncated format:
matt.fellows
2023-09-11 22:42
Hi Venkat, I can already see at least 2 discrepancies:
1. `PhoneAddress` defined twice
2. `PhoneAddres.code` is used, but in the definition it?s `PhoneAddress.internationalCode`
Could you please ideally share a full working (or in your case, failing) example that we can review?
matt.fellows
2023-09-11 22:43
One guess (you could try):
``` EditCustDetailsContactPoint:
type: object # <- try commenting this out
properties:
ContactPoint:
anyOf:
- $ref: '#/components/schemas/VerifyEndpointPhoneData'
- $ref: '#/components/schemas/VerifyEndpointEmailData'```
sharathkonda
2023-09-12 08:37
Thanks Matt,
Will sort the first comment
For EditCustDetailsContactPoint, actual type was not there but later added. let me check again.
sharathkonda
2023-09-12 19:40
@matt.fellows
I think the spec is missing this discriminator keyword as we are using anyof openAPI schema
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/keyword-support/
matt.fellows
2023-09-13 01:37
> It turned out that the discriminator is for oneOf *not* for anyOf.
correct. It?s to _discriminate_ distinctly between 2 or more schemas. `anyOf` doesn?t care if it matches multiple
sharathkonda
2023-09-13 10:03
Thanks @matt.fellows
Now it brings back the question "What is making the error by PactFlow"
boer.k
2023-09-13 11:27
Hi, our main branch name is called 'release', not 'develop, main or master'. We can appearantly set this value in the config of the broker: https://docs.pact.io/pact_broker/configuration/settings#main_branch_candidates However this is the first time I am reading about this config file. At the top of the page it says that it needs to be "placed it in the location `config/pact_broker.yml`, relative to the working directory of the application". However since we use pactflow as a SaaS, how can we set this config?
matt.fellows
2023-09-13 11:32
You can set it on each application individually, but it might be possible for us to set it at the tenant level for you
boer.k
2023-09-13 11:35
I think eventually we could do it on tenant level since to my knowledge 'release' is the main branch for all our company's applications. But until I know for sure: how do we set it per application?
matt.fellows
2023-09-13 12:01
I was about to describe the PATCH request details and the https://github.com/pactflow/terraform-provider-pact/blob/4692b497838da3138648e5e8d40b24b24b793907/client/client_pact_test.go#L97 from the terraform client for it :laughing:
matt.fellows
2023-09-13 12:02
I believe it?s a known issue with the `type: object` and keywords.
Were you able to confirm the suggested fix above worked?
boer.k
2023-09-13 12:05
Thanks Yousaf, I am trying that now. But I am wondering: Is it only possible to view the config via terminal? Is there no UI for it? Is that worthy of a feature request?
yousafn
2023-09-13 12:09
Hmm, possibly via the can-i-deploy UI today, by searching using the main branch, although not intuitive.
You could do it through the HAL browser, I can get you the urls if you want?
Yeah you can raise on the feature req board for PactFlow.
We do have a refreshed UI that is in active development and you can jump in on the preview if you wanted
https://github.com/pactflow/roadmap/issues/103
or just add your comments to that
sharathkonda
2023-09-13 12:14
We removed type:object but it didn?t work.
If we remove anyOf , it is working but the provider is not going to change that.
matt.fellows
2023-09-13 12:29
the good news is that that?s actually fixed in the latest version of that code
sharathkonda
2023-09-13 12:33
Wonderful, thanks for your prompt attention. Atleast we know the issue and resolution.
prakhar.roy371
2023-09-14 04:46
Hi @matt.fellows @yousafn
I wanted to understand one feature of pactflow.
Say, in total there are 5 integrations.
If any person logs in, and is only assigned to Team A which enables him/her to just see 1 integration. And Default has no integrations associated with it.
Still when they log in, they are able to see all integrations. Only once they select a team (Team A), they see their respective integrations.
1. Is there a way for a user to avoid team selection if they are not admin?
2. We want any new user who logs in to see NO INTEGRATIONS until unless they are assigned to a team. How can we achieve that?
matt.fellows
2023-09-14 05:12
Hi Prakhar.
1. What do you mean by this, sorry?
2. This is currently not possible. The feature required to make this work is currently documented here (not yet prioritised): https://github.com/pactflow/roadmap/issues/37
matt.fellows
2023-09-14 05:12
_(also no need to @ people, we monitor the channels and will respond when we can)_
matt.fellows
2023-09-14 05:31
Just on the user:team association. Users may be associated with multiple teams, the selection is really about filtering what you see as needed. It doesn?t affect other actions once selected
nathan.deamer
2023-09-14 14:08
Hi All,
Looking at the webhook dynamic variable:
`${pactbroker.buildUrl}`
The URL of the build that published the resources that have triggered the webhook (currently only supported for contracts published using the ?all in one? endpoint).
What does ?currently only supported for contracts published using the ?all in one? endpoint? relate to?
nathan.deamer
2023-09-14 14:21
I?m publishing my pacts setting the `pact.publish.consumer.buildUrl`
And my assumption was that when the webhook gets triggered it would be sent with the `"${pactbroker.buildUrl}"` dynamic variable
matt.fellows
2023-09-14 22:17
That's a fair point and a confusing message. It just means that the pact publish must use a particular (a newer) API. If you're using the CLI it will, and I think most modern JVM versions do too. You'll find out pretty quick if yours doesn't support it
nathan.deamer
2023-09-15 07:52
Hey Matt,
Using the latest gradle plugin and not seeing the buildUrl come through on the webhook.
I can try publishing the pacts with the CLI to see if that makes a difference.
matt.fellows
2023-09-15 08:09
It also seems to be fetching the `buildUrl` from the same value: https://github.com/pact-foundation/pact-jvm/blob/master/core/pactbroker/src/main/kotlin/au/com/dius/pact/core/pactbroker/PactBrokerClient.kt#L718
nathan.deamer
2023-09-15 09:05
Thanks! I think I?m being an idiot and mistake in implementation. Will confirm
james_fraser
2023-09-18 15:39
Hey there :wave: I?ve been experimenting with using `application-instance` for record-deployment - is there any way for can-i-deploy to specify which instance it?s comparing? E.g.
```Computer says yes \o/
CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
---------|------------|------------------------|------------|----------|--------
raf-app | 7b1949c... | refer-a-friend-service | 69cf694... | true | 1
raf-app | 79301e0... | refer-a-friend-service | 69cf694... | true | 2
VERIFICATION RESULTS
--------------------
1. https://domain.pactflow.io/contracts/bi-directional/provider/refer-a-friend-service/version/69cf6942657ebb4bd91cee0a5bd6f22e4f20f2ee/consumer/raf-app/version/7b1949c72a39376ed66b4f3e9e770c0b62bb81c7/cross-contract-verification-results (success)
2. https://domain.pactflow.io/contracts/bi-directional/provider/refer-a-friend-service/version/69cf6942657ebb4bd91cee0a5bd6f22e4f20f2ee/consumer/raf-app/version/79301e0b38db501d7eac88387c9e8ac2bb38f4bd/cross-contract-verification-results (success)```
From this result (and from the Pactflow UI) it?s not clear which result maps to which application-instance (and if one were to have failed, which instance we would be breaking)
tanyaryzhova93
2023-09-18 18:23
Hi, I have the question regarding pre-verified contracts.
I have the following situation:
? My provider with version `P1` was verified with `C1` and the verification was successful. Provider with `P1` version was deployed to `prod`.
? Then a new contract with `C2` version was published on a consumer?s side. But the `C2` contract is identical to `C1` contract, no discrepancies there.
? I would expect that contract between `C2` and `P1` would be pre-verified automatically, since no changes in the contract were done. However, in my case this verification is missing (please see a screenshot of my matrix).
Do you have any ideas why contract between `C2` and `P1` wasn?t pre-verified?
matt.fellows
2023-09-19 00:36
If it?s not pre-verified, it could be because of dynamic variables changing the hash of the contract
matt.fellows
2023-09-19 00:36
You could try to see the diff this way: https://docs.pact.io/pact_broker/advanced_topics/see_changes_pact
tanyaryzhova93
2023-09-19 00:39
I am asking because I tried to see the difference and I don?t see the difference :thinking_face:
tanyaryzhova93
2023-09-19 00:39
however, the response body is not empty?I started to think maybe formatting changed :thinking_face:
matt.fellows
2023-09-19 00:41
hmmm it seems to think something has changed, but it certainly doesn?t look that way!
matt.fellows
2023-09-19 00:42
It would be interesting for you to generate the contract locally, and copy the file, and then generate it again and run a local `diff`
matt.fellows
2023-09-19 01:17
A diff between two pacts can, yes. But to rule out funkiness in the broker, I was suggesting it might be worth trying locally (on the two features/branches/commits) to see if it did indeed produce a different hash
eddie
2023-09-19 22:40
Deploying a (provider) component *for the first time*.
First bit of the deployment pipeline is a `can-i-deploy` check.
Because there are no contracts published for the component, the `can-i-deploy` check comes back with
> Error making request to https://mycompany.pactflow.io/matrix status=400 {"errors":["Pacticipant XYZ not found"]}
This seems like a chicken and egg :chicken: situation - how to resolve?
boer.k
2023-09-20 11:05
Hi, I did follow the instructions in this thread, and it worked last week. However, at the end of the day, the commands didnt seem to work anymore. We deleted everything, and came back a week later (now) in the hopes that it would magically restore itself. But it is still not working. Please have a look at this screenshot. I am trying to change the main branch of our "Upa verlof" consumer to "contract-testing-release-do-not-delete-or-merge" (funny name I know, we didnt want to test on our actual main branch called "release", so we created this dummy branch that we designated as main branch purely for learning contract testing.) So what happens in the screenshot:
1: We describe our freshly created consumer to see that there is no main branch set
2: We try to change the main branch
3: We describe again. We see that it has been updated, but no main branch has been set.
4: We try the same with the example app that was provide when we onboarded on pactflow. Here you can see it already has a main branch.
5: We try to change the main branch here too
6: Again, you can see that it has been updated, but the main branch is still the same.
Can someone help us?
matt.fellows
2023-09-20 12:03
I just tried this against a test account and it seems to work as expected:
```? pact-broker create-or-update-pacticipant --name XYZ --main-branch foo
Pacticipant "XYZ" created in Pactflow
? pact-broker describe-pacticipant --name XYZ
Name: XYZ
Display Name: XYZ
Main Branch: foo
Created At: '2023-09-20T12:02:19+00:00'
? pact-broker create-or-update-pacticipant --name XYZ --main-branch bar
Pacticipant "XYZ" updated in Pactflow
? pact-broker describe-pacticipant --name XYZ
Name: XYZ
Display Name: XYZ
Main Branch: bar
Updated At: '2023-09-20T12:02:32+00:00'
Created At: '2023-09-20T12:02:19+00:00'```
matt.fellows
2023-09-20 12:04
You could try adding the `--verbose` flag, to see if you can see what?s happening in the payload
matt.fellows
2023-09-20 12:05
You should see something like this in there:
```<- "PATCH /pacticipants/XYZ HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nContent-Type: application/json\r\nAuthorization: [redacted]\r\n"
<- "{\"name\":\"XYZ\",\"mainBranch\":\"bar\"}"```
Note the JSON payload
boer.k
2023-09-20 12:15
```Kevin@ip-172-23-17-171 ~ % pact-broker describe-pacticipant --name upa-verlof --broker-base-url=https://tkppensioen.pactflow.io -k <token>
Name: upa-verlof
Display Name: Upa Verlof
Updated At: '2023-09-20T11:12:04+00:00'
Created At: '2023-09-20T11:12:04+00:00'
Kevin@ip-172-23-17-171 ~ % pact-broker create-or-update-pacticipant --name=upa-verlof --broker-base-url=https://tkppensioen.pactflow.io -k <token> -?main-branch=contract-testing-release-do-not-delete-or-merge --verbose
opening connection to http://tkppensioen.pactflow.io:443...
opened
starting SSL for http://tkppensioen.pactflow.io:443...
SSL established, protocol: TLSv1.2, cipher: ECDHE-RSA-AES128-GCM-SHA256
<- "GET /? HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nAuthorization: [redacted]\r\n"
-> "HTTP/1.1 200 OK\r\n"
-> "Date: Wed, 20 Sep 2023 12:12:37 GMT\r\n"
-> "Content-Type: application/hal+json;charset=utf-8\r\n"
-> "Transfer-Encoding: chunked\r\n"
-> "Connection: keep-alive\r\n"
-> "Vary: Accept,Accept-Encoding\r\n"
-> "Server: Webmachine-Ruby/2.0.0 Rack/1.3\r\n"
-> "X-Pact-Broker-Version: 2.107.1\r\n"
-> "Content-Encoding: gzip\r\n"
-> "X-Content-Type-Options: nosniff\r\n"
-> "Strict-Transport-Security: max-age=63072000; includeSubDomains; preload\r\n"
-> "X-Pact-Broker-Git-Sha: 55bb62c6\r\n"
-> "X-Pactflow-Git-Sha: d2d1b5a88\r\n"
-> "X-Request-Id: e59bd06c62fe1fc744f715d8c57534b0\r\n"
-> "\r\n"
-> "43b\r\n"
reading 1083 bytes...
-> "\x1F\x8B\b\x00\xB5\xE1\ne\x00\x03\xBCXMo\xE36\x10\xFD+\x84\xCF\xD1j\xD1C\x0F\x06\x8CE\xDA\xA4\x8B\x05z\bv\xB7\xED\xA1-\x16\xB44\x8A\t\xD3\xA4@R\xC9\x06\x86\xFF{\x87\xA2(Q\xB2\xA4\x88^\xA7\x87\xC4\x12\xC5\x99\xF7\xE6\x93#\x1DW\xDF8\x13{\xBDZ\x1FW\x1Axa\x7Fw\n\xF0w\xB53\xA6\xD4\xEB45\xFB\xB2\x04\xA1\x99\x04\xF1\xAE\xA4\x99)\xB8|~\xC7\xE4\xEAfe\x98\xE1\x80;?\x89\x1C\xBE\xDB{8\x94\x9C\x1A\xC8W\xEB\x82r\r\xA7\x9BU\xB9]\x97\xD5\x963\xBDK\xAC\xECr\xF5\xA9\xBD\xD6i\xA9\xE4\x13\xCBA\xA5G\x7FuJ3)tu\xB0k\xFE\xEA\x94>\x81B\x15\xA2[\xBA-K\xCE2jp\xF1O\xF7\xEC\x140~p\x9C\b%5\xAB\x1Eu\xA3\xAA\x01s\xD4i\x94\xA5\x13A\xBF\x95I\e-#\xE8\x9D\xDE)\xDF\xD9\x15mj\xD7%\x8D\x891$\x9C\x0F\x9D\x92\x00\xFF\xF7z\xA16\x9D\xB4Z\xA7(\x18\xFA\xF8\b\xF9\x0FRX\x1AF\x04K\x8F\xF8/\x8C\xD5-\xE7-K\"\x8B&f\xA4\x90\n/\x1F\xD9\x13\b\xE2\x15\xDC\x10\xAF\x9EP\x91\xB7\xCB^\x9C\xA0\xE6\xE94E\xA5,c%\x15Qq\xEE\x89\x05!\x1E,\xBF\x06y!\"\xFA\xB3\xBB\v\x9D\xF6\e\x98lG\x82\x87d\xFBB\x04=\xC0T\xAE\xFBLk\xFCW\xC7\xFB:q\x9EK?mi\xF9\xADQ\xD4\x92gfv\x89\r\xE8\xF58\x9E\xA5^\x8F\xA9M\xB86\xBD,:1; \xBA\x84\x8C\x15\f\xF2\xF3\xE4\n\x1A\xC9\eR\x9F*\x99^u\xD7\xE4-\xDD9\x03^/\x97)\x8B\xAEa\xC8\"\xFA\x1D\xD9\x85I\xD3h\xB8Jy\xA5\x03\x9D\xA3y\xE2\xCB-\xD84C\xAFi\xAEo\xC8r.\xA7\a\\#\x93\xFA\x19\xB6;)\xF71\xC1oE::\x7F\x05K\xA3M\xB2\x91\xB9\x00%=V\x15\xCBO\xE7`S\x161a\xE0Q\xD53C\x8CU=\xB1p*\xEA/\xBFv\x04\xF8\x88]\xD0\x19&2\xC1\x97Ozl\xAE\xEAn\xA1\xCFR\xE2#\x98\e\x92)@v\x04\v-\a\x0ExEm\xF0\x9Bc6\"\xB3C\x8B\xB6\x8A\x8Alw\xDD\xF4v:\x01\x97\xDD\xD5\x98\x99}\xDB\xACM4\xCF\xD3\xC6\xC2Qk\x1A;\x9D\xCA%\xA6]\xD5\xA6\xD7,\x18\x8DNDL\x0E`\x14\xCBbr\xDAK\xF4\x1Di\xA7\x1A\xF2\x8B\x92{<$\xFC\x0E\x87\x90Q\x91\xB0$\x87\x92\xCB\x97d4\xA9ed^\x87\x1A?\x04\x1A7=\xD7\xFDS\xBD\x7F\xFF\xD3\xCF\r\xC8\xA6u\x9F[6r3L\xF5;\xF4\x9D:0\x01\x84\xE1$)\b\xED^\x13\xDAT@d\xB2\xC5&H\v\xE0/\xC41\xB0\xADPZ\x01\x10OLIq\x00\xF4:\x9E?\xC2\xB8>\x89c\x8Cm\x9Cn\x1C\x9Di\x9AK\xFC\x14@\xFC\xBF\xFE\n\x807\xC7\xE0\xE6\xAD\x1C\xB8p\\\xC2\xD2\xB4\xEEAG;\x9C\xAB\x8CMgJ\xFB\xB3{7z k\xB4\xC5mE#\"'\x91-\x18\xBA\xC4\x9Ez4_\xD7\xDBQ\xCF[Zww\xFF\xF0\xF9\xFE\xD7\xDB\xAF\xF7w$!%\a\xAA\x81T\xF8\xB7\xC0\xEFc\x06f\x95b\x80\x9D\xE5\xEF\xD6\x86r\xBB\xDC\x82\\f\xE9Q\x01?}\xB0\xEF\xC2\xF0\xDDl\xD8\xF9\xB7\x04\x87t\xA1\x8F\x96#\xFC[\xE7^\x90\x9Eu\xBF\xF4n\xBB\x0F\xD7\x17\xA3C_j|\x06\x18\xD4\xFB\b\xE2e\x80\xC1\xFCs^eE\xF7]\xC6G\xDD\x7F\x8E\x88\xA9\xAF\xA1\xECx:N\x7F\x02i\xDFGZ\xF0\x89\x9E@K\x86\xBDq\x0FQ\x93\x99\x06c\x98\xC0\x89\xA7\x91\f^1\x1E>\x91nq4,\xB4\xCAY\x8C/\xDC\xFE\x00\xC2\xDE\x134\x8A\xF1I\f\rx\xB0G\xBD?y\x89\x0E\xE7K\xB72\x82Q\xACi\x8E\xED:\xC1\x12W18\xB5T\xEA\xA4:\xAC?\xFC\xFD\f\x92\x01z\x88GrR\x1D\xD2W\x7F?\x83\xA4_4>Nh\x96\xC9*\xEE\xAB\x8D\xC3\x1C\xCA\a>\xAD\x9F\x90\xF0\xC9\f\x0F%9\xC4\xA3;\xA9\x0E\xF3\xB3\xBF\x9FA*\xED\xD9\xABc\xBF\xC19\xBCP6(\xC4\xFE\xEA\f6\xAD\xF0\xDC\xC3y\xC7\x1D\x06\x89\xAF\xAD\v\"-p\x12I\xA7\xD4\x85\x05\x14\xEE _\x82\x1D\xA34m\xB2&\x94s\x99E\xBF\xC25\x9C8;03\xCCwr\e\xA8\x1C\xC1>\x9D\xFE\x03\x00\x00\xFF\xFF\x03\x00\b\xB4\x84\xBAE\x17\x00\x00"
read 1083 bytes
reading 2 bytes...
-> "\r\n"
read 2 bytes
-> "0\r\n"
-> "\r\n"
Conn keep-alive
opening connection to http://tkppensioen.pactflow.io:443...
opened
starting SSL for http://tkppensioen.pactflow.io:443...
SSL established, protocol: TLSv1.2, cipher: ECDHE-RSA-AES128-GCM-SHA256
<- "GET /pacticipants/upa-verlof? HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nAuthorization: [redacted]\r\n"
-> "HTTP/1.1 200 OK\r\n"
-> "Date: Wed, 20 Sep 2023 12:12:38 GMT\r\n"
-> "Content-Type: application/hal+json;charset=utf-8\r\n"
-> "Transfer-Encoding: chunked\r\n"
-> "Connection: keep-alive\r\n"
-> "Vary: Accept,Accept-Encoding\r\n"
-> "Server: Webmachine-Ruby/2.0.0 Rack/1.3\r\n"
-> "X-Pact-Broker-Version: 2.107.1\r\n"
-> "Content-Encoding: gzip\r\n"
-> "X-Content-Type-Options: nosniff\r\n"
-> "Strict-Transport-Security: max-age=63072000; includeSubDomains; preload\r\n"
-> "X-Pact-Broker-Git-Sha: 55bb62c6\r\n"
-> "X-Pactflow-Git-Sha: d2d1b5a88\r\n"
-> "X-Request-Id: da13a8febf61dec729cdd7f38a1d3d0d\r\n"
-> "\r\n"
-> "251\r\n"
reading 593 bytes...
-> "\x1F\x8B\b\x00\xB6\xE1\ne\x00\x03\xE4U\xD1n\x9B0\x14\xFD\x15\x8B\xD7\xE1\x98\xA4\x9D\xD4\"MS\xD7D\xD3^\xA6i\xCA\xF6RU\x93\xC1\x97\x06\x15l\xCB\x98l\x15\xE2\xDFw\r!5m\xB62\x95\xB7=$\x18\xFB\xFA\xDCs\x8E\xEF5M y\tA\x1C\xD4\x9A\xD3=\x98BeA\x18\x88\xBC\xD2\x05\x7F\xF8\xDC/}\xD3\x9C|\x1F\x96J\x9E\xCB\x0F\x86\xCBt\x87+\xEE\x05\xE7j-\xB8\x05qeqj\x15\xAD\xCEhtIW\xD1v\xB9\x8C\x97\xAB8:\x7F\x13Eq\x14a\\j`R\xDC\x0F(\x13\x10\x02D\x107A\x81[*\x8B\xF9\xAB\\I7!k\\5\b\x11-\x96\xF4\xED\xE5\x92\xAE\xF8\xD9\xC5y\x1A]\\\xD2\f\xF1k\x03T\xF3\xD4:\x9C\"\x97\xF7\x95\xDBSA\x91\xB9\xA7\xCDm\xE1$\rp\xE1 \xFFE\xAC\x9D\x01D\bv\xD6\xEA*f\xCC\xDEk\r\x121@.\\DV\xA8\x9F\x8B\\17\xCE\xD3\\si+\xF6\xE8)\xDB\xF7\t+\xF6B\xA2\xB6mC\x94\x9C@\x81\xBCon\xC3\x83|\xBA\xFFo\xF4\x87G\x92\xEB\xCD\x97\xAF\x9B\xEB\xAB\xEDfM(\xD1\x05\xF0\nH\x8D\xBFqE\x1C%L\vw\t\x9E\xFB\xF2:u\x01b\xEA$\x1E4\xBE\x1E\xF0h\xD7\x18\xD9?\xC1\x8F`C\xD27\x14Q\x86\b(\x00G\x9Cx\x88d\x7F\xB4h\xAE\xD3k\x0E\xA3\x161-\x94\xDAy\x8B]jM\r#\xA6\xD4\xF2\xBB)l1\x8Cd8\xC1\a\xAEDedt\x15\xCD\xCE\x9CaN|\xC5\xFF?\x8AH\xBA\xFB\x8D\x9Ev\xDD\xF1\xE7B\xB0\x83\x9AG\xE6\xBD\x8E~\xEF\xDC2zT@\xDE\xFD\xA8\xFD\x87#\xE9n\x93)\x87\xD1\x05v2\xE6\xA4\xDE_f\xAC\xE9\x9E\xA7I\xFA\x8Dsl~\xD0\x06R\x17\x86\xDD\xEC\xDA\xD8o\xB0\xD9\xCA\xE2\xD0_)\x974\xA7\x02t\xA1\x1Eh\xC2\xC5\x1D\xF8T\xAE\xB9$\x9F\xC8\xBA[\xF5\xAC!}\xE0\f\x06\xF97|_\x99\xCCg\xC4\xACb\r\xC8}n\x94,A\xDA\xAD\v\x18\x92\x9F<\xF2\xB1\x9E\xBE\x9A\xAD\xA2\x1E\xC6t\x95\x99Q\xE5P\xD5V\x11\x0Fc.\x03\x9E\x17\xF7\x13K\x9E\x98\xE1\xEB\x18\x19\xF37W\xD2\xDA\xE4\xE0\xBE\xA9\xCD\xF0\xBD\xD0\xC9t\xEAB\xA5\xAC1X\xBF\xEFS%-\xFC\xB2\xEF<1'\xF2\xDD\xB6\xEDo\x00\x00\x00\xFF\xFF\x03\x00\xD5\x88j\xD5Z\t\x00\x00"
read 593 bytes
reading 2 bytes...
-> "\r\n"
read 2 bytes
-> "0\r\n"
-> "\r\n"
Conn keep-alive
opening connection to http://tkppensioen.pactflow.io:443...
opened
starting SSL for http://tkppensioen.pactflow.io:443...
SSL established, protocol: TLSv1.2, cipher: ECDHE-RSA-AES128-GCM-SHA256
<- "PATCH /pacticipants/upa-verlof HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nContent-Type: application/json\r\nAuthorization: [redacted]\r\n"
<- "{\"name\":\"upa-verlof\"}"
-> "HTTP/1.1 200 OK\r\n"
-> "Date: Wed, 20 Sep 2023 12:12:39 GMT\r\n"
-> "Content-Type: application/hal+json;charset=utf-8\r\n"
-> "Transfer-Encoding: chunked\r\n"
-> "Connection: keep-alive\r\n"
-> "Vary: Accept,Accept-Encoding\r\n"
-> "Server: Webmachine-Ruby/2.0.0 Rack/1.3\r\n"
-> "X-Pact-Broker-Version: 2.107.1\r\n"
-> "Content-Encoding: gzip\r\n"
-> "X-Content-Type-Options: nosniff\r\n"
-> "Strict-Transport-Security: max-age=63072000; includeSubDomains; preload\r\n"
-> "X-Pact-Broker-Git-Sha: 55bb62c6\r\n"
-> "X-Pactflow-Git-Sha: d2d1b5a88\r\n"
-> "X-Request-Id: 2079b27386bc4db8d90dd3df51cc246d\r\n"
-> "\r\n"
-> "256\r\n"
reading 598 bytes...
-> "\x1F\x8B\b\x00\xB7\xE1\ne\x00\x03\xE4U\xD1n\x9B0\x14\xFD\x15\x8B\xD7\xE1\x98$\x9D\xD4 MS\xD7D\xD3^\xA6i\xCA\xF6RU\x93\xC1\x97\x06\x15l\xCB\x98l\x15\xE2\xDFw\x1D\x025m\xB6f*o\x93B0\xF6\xE5\xDCs\x8E\xEF5M y\tA\x1C\xD4\x9A\xD3=\x98BeA\x18\x88\xBC\xD2\x05\x7F\xF8\xDC-}\xD3\x9C|\xEF\x97J\x9E\xCB\x0F\x86\xCBt\x87+\xEE\x01\xE7j-\xB8\x05qeqj\x11-\x964Z\xD1E\xB4\x9D/b\xFC-Wo\xA2(\x8E\"\x8CK\r\x9C\x8C\x9B\xBB\xB8\xE8b\x88\xFB\x01e\x02B\x80\b\xE2&(\xF0\x95\xCAb\xFE*W\xD2M\xC8\x1AW\rBD\xB39}\xBB\x9A\xD3\x05_^^\xA4\xD1\xE5\x8Af\x88_\e\xA0\x9A\xA7\xD6\xE1\x14\xB9\xBC\xAF\xDC;\x15\x14\x99\xBB\xDB\xDC\x16NR\x0F\x17\xF6\xF2_\xC4\xDA\x19@\x84`g\xAD\xAEb\xC6\xEC\xBD\xD6 \x11\x03\xE4\xCCEd\x85\xFA9\xCB\x15s\xE3<\xCD5\x97\xB6b\x8F\x9E\xB2}\x97\xB0b/$j\xDB6D\xC9\t\x14\xC8\xFB\xE66<\xCA\xA7\xFB\xFFF\x7F8\x90\\o\xBE|\xDD\\_m7kB\x89.\x80W@j\xBC\xC6\x151H8/\xDC%x\xEE\xCB\xEB\xD4\x05\x88\xA9\x93\xB8\xD7\xF8z\xC0\xC1\xAE1\xB2\xBF\x83\x1F\xC1\x86\xA4k(\xA2\f\x11P\x00\x8E8\xF1\x10\xC9~\xB0h\xAA\xDDk\x8E\xA3\x161-\x94\xDAy\x8B]jM\r#\xA6\xD4\xF2\xBBs\xD8b\x18\xC9p\x82\xF7\\\x89\xCA\xC8\xE8(\x9A\x9C9\xC3\x9C\xF8\x88\xFF\x7F\x14\x91\x1C\xCE7z\xDAu\xC7\x9F\v\xC1\x8Ej\x1E\x99w:\xBAw\xA7\x96\xD1\xA1\x02\xF2\xEEF\xED?l\xC9\xE149g3\x0E\x81\a\x19SR\xEF\x0E3\xD6\x1C\xEE\xA7I\xFA\x8D34?h\x03\xA9\v\xC3nvm\xEC7\xD8deq\xEC\xAF\x94K\x9AS\x01\xBAP\x0F4\xE1\xE2\x0E|*\xD7\\\x92Od}X\xF5\xAC!]\xE0\x04\x06\xF9'|W\x99\xCCg\xC4\xACb\r\xC8}n\x94,A\xDA\xAD\v\xE8\x93\x9F\xDC\xF2\xB1\x9E\xAE\x9A\xAD\xA2\x1E\xC6\xF9*3\xA3\xCA\xBE\xAA\xAD\"\x1E\xC6T\x06</\xEE'\x96<1\xC3\xD712\xE6o\xAE\xA4\xB5\xC9\xC1}S\x9B\xFE{\xA1\x93\xF3\xA9\v\x95\xB2\xC6`\xFD\xBEO\x95\xB4\xF0\xCB\xBE\xF3\xC4\x9C\xC8w\xDB\xB6\xBF\x01\x00\x00\xFF\xFF\x03\x00\xF6\r]\x89Z\t\x00\x00"
read 598 bytes
reading 2 bytes...
-> "\r\n"
read 2 bytes
-> "0\r\n"
-> "\r\n"
Conn keep-alive
Pacticipant "upa-verlof" updated in PactFlow```
Not sure where I should be looking in all this. Can you make sense of it?
yousafn
2023-09-20 12:38
This is where the patch call is emanating from - from your command
https://github.com/pact-foundation/pact_broker-client/blob/3a342dee5f90b957af4da7e09d6cb714b07afae5/lib/pact_broker/client/pacticipants/create.rb#L18-L23
yousafn
2023-09-20 12:39
`main_branch` param should be picked up here
https://github.com/pact-foundation/pact_broker-client/blob/3a342dee5f90b957af4da7e09d6cb714b07afae5/lib/pact_broker/client/pacticipants/create.rb#L42
yousafn
2023-09-20 12:45
just tracing through, looks like it should be passed, what version of the client are you using? `pact version`
1. PacticipantCommands.create_or_update_pacticipant
a. https://github.com/pact-foundation/pact_broker-client/blob/3a342dee5f90b957af4da7e09d6cb714b07afae5/lib/pact_broker/client/cli/pacticipant_commands.rb#L9
2. Opts should be parsed here
a. https://github.com/pact-foundation/pact_broker-client/blob/3a342dee5f90b957af4da7e09d6cb714b07afae5/lib/pact_broker/client/cli/pacticipant_commands.rb#L20
b. https://github.com/pact-foundation/pact_broker-client/blob/3a342dee5f90b957af4da7e09d6cb714b07afae5/lib/pact_broker/client/cli/custom_thor.rb#L114-L116
3. PactBroker::Client::Pacticipants.new
a. https://github.com/pact-foundation/pact_broker-client/blob/master/lib/pact_broker/client/pact_broker_client.rb#L24
yousafn
2023-09-20 12:46
:facepalm: no branch in that command :joy:
```pact-broker describe-pacticipant --name upa-verlof --broker-base-url=https://tkppensioen.pactflow.io -k <token>```
but looking for the `PATCH` command when you try and update the branch will help trace it a bit
yousafn
2023-09-20 12:47
oh wait, that was just the first command, the 2nd command is
```Kevin@ip-172-23-17-171 ~ % pact-broker create-or-update-pacticipant --name=upa-verlof --broker-base-url=https://tkppensioen.pactflow.io -k <token> -?main-branch=contract-testing-release-do-not-delete-or-merge --verbose```
definitely can?t see that branch being passed through
yousafn
2023-09-20 12:49
So that change should be in place for pact broker clients newer than
https://github.com/pact-foundation/pact_broker-client/releases/tag/v1.54.0
yousafn
2023-09-20 12:55
super weird - this is with a local broker.
Your version is new enough to include the change
yousafn
2023-09-20 13:01
You can try our test PactFlow instance out, I?ve just updated one successfully, I?ve added a screenshot of some names you can use
```?13:57:46 ? pact-broker describe-pacticipant --name SNSPactEventConsumer --broker-base-url=https://test.pactflow.io --broker-token 129cCdfCWhMzcC9pFwb4bw
Name: SNSPactEventConsumer
Display Name: SNS Pact Event Consumer
Main Branch: master
Updated At: '2023-09-05T00:42:55+00:00'
Created At: '2023-09-04T23:58:36+00:00'
~ via ? desktop-linux on ?? (eu-west-2) took 3s
?13:59:21 ? pact-broker create-or-update-pacticipant --name SNSPactEventConsumer --broker-base-url=https://test.pactflow.io --broker-token 129cCdfCWhMzcC9pFwb4bw --main-branch=main
Pacticipant "SNSPactEventConsumer" updated in PactFlow
~ via ? desktop-linux on ?? (eu-west-2) took 4s
?13:59:53 ? pact-broker describe-pacticipant --name SNSPactEventConsumer --broker-base-url=https://test.pactflow.io --broker-token 129cCdfCWhMzcC9pFwb4bw
Name: SNSPactEventConsumer
Display Name: SNS Pact Event Consumer
Main Branch: main
Updated At: '2023-09-20T12:59:52+00:00'
Created At: '2023-09-04T23:58:36+00:00'```
yousafn
2023-09-20 13:25
most peculiar - if you upload a fresh pact for a new consumer/provider does it exhibit the same behaviour? I assume you have a SaaS broker?
Raising a support ticket is probably best at this point
https://support.smartbear.com/pactflow/message
and someone from the team will be able to look into the specific details, you can link them to this thread for background information.
I?m not privy to that information (with good reason)!
boer.k
2023-09-20 13:26
Hmm we currently only have 1 consumer and provider to play around with. Would a rename of the consumer be enough?
yousafn
2023-09-20 13:27
Probably best just uploaded any pact file, or just renaming the consumer/provider keys in the json file
yousafn
2023-09-20 13:28
then upload that, just allows you to see if there is something up with the application pair, (if you can?t replicate on what would appear to the Broker, to be 2 new applications, and 1 new integration)
josh.steffensmeier
2023-09-20 14:46
Hello, I ran into an issue yesterday where we updated a contract with the same version as an existing contract. As a part of our CICD we are pushing contracts for each commit. We found that the broker would tell us that the contracts are now compatable but can-i-deploy would tell us that they were not. It looked like can-i-deploy was still using the older version of the contract instead of the one that had been updated.
Condensed timeline
1. Consumer version 2.1.62 was pushed - incompatible with provider 2.0.0
2. Meeting to discuss bringing these into alignment
3. Provider version 2.0.1 was pushed
4. Provider version 2.0.1 deployed because it is compatible with consumer version 2.1.61
5. Consumer version 2.1.62 pushed again to bring it into alignment with 2.0.1
a. Broker says everything is good
b. Can-i-deploy says it isn?t good, it looks like it is still using the contract from step 1
josh.steffensmeier
2023-09-20 14:48
My question is, are we using versions incorrectly or is there some unwanted caching going on with can-i-deploy?
matt.fellows
2023-09-20 22:56
Thanks Kevin and Yousaf. I?ve given customer care the heads up this will need to go straight to the dev team for review, peculiar!
matt.fellows
2023-09-20 23:00
hmmm may have been a UI/refresh bug. I have checked your account, and that contract is definitely red (consumer version `2.1.61` and provider version `2.0.1`)
The compatible consumer version is actually `2.1.62`.
matt.fellows
2023-09-21 11:00
> Hmm we currently only have 1 consumer and provider to play around with. Would a rename of the consumer be enough?
FYI you don?t need to upload a contract, you can simply run that same command with a new application name (e.g. `foo`) and see how it works on your broker
josh.steffensmeier
2023-09-21 14:56
100% agreed that those are the compatible versions. Yesterday I had to delete the `2.1.62` contract and reupload it because it couldn?t get past this issue
bethskurrie
2023-09-22 00:10
@boer.k I?ve got the support ticket, but it?s easier if I just chat to you here.
bethskurrie
2023-09-22 00:11
Can you try the command with the latest version of the pact broker client, because I?ve updated it to NOT use gzip when --verbose is turned on, because it makes it very hard to view the response!
bethskurrie
2023-09-22 00:17
This is the output I get when I run the create-or-update-pacticipant command.
bethskurrie
2023-09-22 00:19
I can?t reproduce the issue with this version of the client and my own test broker.
bethskurrie
2023-09-22 00:20
From the logs you?ve provided, I suspect there is a bug in the version of the client you?re using, because I could set the main branch via the API in the HAL browser no problem, and the latest CLI seems to be working fine too.
bethskurrie
2023-09-22 00:39
@josh.steffensmeier I can see that you?re using semantic versions, and that your broker is an ?old? one that allows consumer contracts to be modified after publication.
bethskurrie
2023-09-22 00:41
Newer tenants do not allow modification of the consumer contract after publication, for this reason.
bethskurrie
2023-09-22 00:41
I can disable contract modification, and it will then cause your build to error if it tries to modify, and this might highlight where your problem is.
ying.li
2023-09-22 01:37
Hello, PactFlow friends! My name is Ying, a UX researcher from PactFlow. I?m excited to announce that we are launching an exclusive beta program for the new PactFlow Onboarding flow, and we want you on board. :bananadance_colors:No matter you are new or advanced practitioner of contract testing and PactFlow, you are welcome to join.
:1000: Why Join:
*Early Access*: Be among the first to experience the revamped onboarding.
*Influence*: Your feedback shapes the final product.
*Incentive*: 50 Swag credits for a 30 min session :moneybag: You can redeem the credits at our online https://swag.smartbear.com/ :smartbear-bounce:
:wave: How to Join:
Directly *Slack me, put a thumbs up, or leave comment below.*
We will reach out to you and schedule a session.
:hourglass_flowing_sand: Dates: Sep 25-29, 2023, 30 min session
_Your expertise matters. Help us shape the future of contract testing and PactFlow!_
Have questions? Contact @ying.li or @shuying.lin
boer.k
2023-09-22 10:52
Hi, I will look into to this as soon as I can! Which will be next wednesday at the latest
josh.steffensmeier
2023-09-22 14:36
Ahh ok thanks for that link, that is what we were looking for. It should be an easy add for us to add the git sha to the version to fix this. Thank you!
boer.k
2023-09-25 09:26
Hi, I have downloaded version 2.0.7 of the standalone package (which include version 1.72.0 of the pact-broker cli), and I still get the same result:
boer.k
2023-09-25 09:30
It dont think the client is the issue, as it did work to change the main branch in Yousaf's instance. To me it looks like the problem is in our broker, as I cant change the main branch (havent tried other settings yet) of any pacticipants, existing or brand new ones. But I could with Yousaf's broker.
matt.fellows
2023-09-25 10:57
Thanks for letting us know. I?ll add these notes to the ticket for somebody to review :pray:
boer.k
2023-09-26 09:17
Heya, im setting up pactflow ci/cd in our company at the moment, and I am following this guide where possible (our pipelines are set up a little different): https://docs.pact.io/pact_nirvana. I have a question about this gold level in the setup guide. When you push the main branch consumer contract, you can set up a webhook to trigger the main pipeline of the provider, and with the right consumer version selectors, you test if the main branch of the provider is compatible with the main + deployed versions of the consumer.
However, what if the main branch of the provider is not yet deployed, and your consumer wants/needs to go to production first, then you lack the verification result for this pair right? (pair being: new consumer version + current provider prd version). As the only information you know is that your new consumer version works with the undeployed main version of the provider.
So in addition to testing main provider version against all deployed consumer version, wouldnt you also want to test your new consumer against all deployed provider versions? (somehow?)
matt.fellows
2023-09-27 00:17
Actually, it?s mentioned in the webhook setup on step 6: https://docs.pact.io/pact_nirvana/step_6 (specifically https://docs.pact.io/pact_nirvana/step_6#add-a-new-provider-verification-job)
When a pact changes, you use the `contract_requiring_verification_published` event to trigger a verification webhook. If verifications are required for deployed or release versions, it will fire one webhook for each of the required verifications (e.g. main branch, and each deployed version). If none are required, then no webhook is fired
> ? Create a separate provider verification CI job that will be called by a webhook.
> ? This job takes the following parameters (branch and commit allow multiple webhooks to be specified for deployed/released versions in Step 7)
> ?? the URL of the changed pact
> ?? the provider branch
> ? the provider commit
>
bethskurrie
2023-10-01 23:32
@boer.k can you please run the command with --verbose turned on and share the logs
bethskurrie
2023-10-01 23:42
We do want to work out what the issue is with the CLI, but in the meantime, you can just update the mainBranch via the HAL browser.
bethskurrie
2023-10-01 23:42
Open the HAL browser, and go to `https://tkppensioen.pactflow.io/pacticipants/upa-verlof`
bethskurrie
2023-10-01 23:43
Click the yellow ?NON-GET? button for the `self` relation. Change the HTTP method to `PATCH` and then set the body to `{"mainBranch": "your-new-value"}`
james_fraser
2023-10-03 09:24
Hey there :wave: is there an endpoint in the API to get a specific version of a published OAS?
Through the UI if I click the options then `Copy provider contract URL for <provider> version <version>` the link I get returns a 404 when I call it:
`https://<domain>.http://pactflow.io/contracts/bi-directional/provider/<provider>/version/<provider_version>/consumer/<consumer>/version/undefined/provider-contract`
matt.fellows
2023-10-03 10:17
That will be resolved with a new UI we?ll be releasing. I think it is possible, but you?d probably need to navigate to the HAL browser (i.e. via API) to get it
james_fraser
2023-10-03 10:18
Ah cool thanks, yeah I had a play about with the API and got something not quite what I was looking for but it works for now
Looking forward to the new UI then :smile:
prakhar.roy371
2023-10-06 07:31
Hi all!
I wanted to know if PactFlow supports Database as well.
Like if our PROVIDER is Java Based Kafka Service and it is calling SP on database CONSUMER SIDE for certain tasks.
Can we have contract testing implemented between the two?
michael.branders484
2023-10-06 08:14
Hello
I'm trying to set up a demo of pactflow for my current client and I'm trying to add a 3th user and I'm getting an error when i do so: An unexpected error has occurred. This error has been sent to the PactFlow team and will be investigated ASAP. Is there some way i can add 2 more users so i can let 1 team try pact out with making and validating 1 contract?
bas
2023-10-06 13:19
Same here, see screenshot. Also, the email invite for user no. 2 doesn't seem to make it to my email inbox. I've also checked the spam folder.
Temporary issue, I guess?
matt.fellows
2023-10-06 22:14
You're right, there is an issue. It's shown in our monitoring reports but the volumes weren't high enough to raise to the level of an on call incident. We'll investigate and fix ASAP. A notice should go up shortly on our status page.
Apologies folks, i know this is a pain!
matt.fellows
2023-10-06 22:20
As in, can you contract test between an application and a database? Probably technically yes, but it's not common
matt.fellows
2023-10-06 23:43
Please let us know if you see any further issues and our apologies, I hope it gets your demo back on track Michael!
michael.branders484
2023-10-09 06:20
Should be ok @matt.fellows, its only a demo of pact to show the teams what i can bring to help there development
boer.k
2023-10-09 07:48
Hi!, Apologies, I hadnt gotten to it last week. I tried it just now, and it seemed to work! I guess we can rule out that our broker is broken then. Still weird that it doesnt seem to work from the cli, while the cli did work with the broker from Yousaf :thinking_face:
matt.fellows
2023-10-09 08:42
No worries and thanks! Can you please try using the CLI again (the latest version) with the verbose flag set?
matt.fellows
2023-10-09 08:42
It makes no sense the API call works but the CLI doesn't which calls the same API!
matt.fellows
2023-10-09 09:32
I think it?s actually a bug!
I?ve installed that exact version and it fails for me. The previous version I had (1.66.0) works, but that?s obviously a few versions old
matt.fellows
2023-10-09 10:19
OK, you won?t believe what the bug is - please re-type the command above, beieng careful to type `--main-branch` correctly (note the hyphens - I think you?ve copy/pasted from somewhere that has converted _hyphens_ to another character)
matt.fellows
2023-10-09 10:33
```? /tmp printf %x "'?" # <- incorrect one used in your failing command
2014%
? /tmp printf %x "'-" # <- correct one
2d%```
boer.k
2023-10-09 11:06
I blame Slack for this :see_no_evil: Ive been sending commands to colleagues via Slack. We've already noticed it was changing double quotes " to something else, but appearantly hyphens too :unamused:
boer.k
2023-10-09 11:18
I feel like I owe you a drink or something :sweat_smile: Thanks a lot for digging so deep!
yousafn
2023-10-09 13:12
Yes! I found slack uses opening double quote and closing double quote, which cli tools don?t tend to like
https://practicaltypography.com/straight-and-curly-quotes.html
matt.fellows
2023-10-09 22:28
> I feel like I owe you a drink or something :sweat_smile: Thanks a lot for digging so deep!
haha well if you ever make it to Melbourne (Aus) feel free to buy me one :slightly_smiling_face:
matt.fellows
2023-10-09 22:29
Thor is the library used in the CLI. It?s widely used/popular, but has a bunch of quirks (see https://mattbrictson.com/blog/fixing-thor-cli-behavior).
I?m currently googling to see if we can make it error if it gets an argument it didn?t expect - in this case, it should (I mean, that argument would probably still be hard to parse in _this_ instance, but it is a problem more generally)
matt.fellows
2023-10-09 22:49
We have created https://github.com/pact-foundation/pact_broker-client/issues/146 to track this
matt.fellows
2023-10-10 10:40
sorry, it?s in a private chat I think. @yousafn did you have anything public on this one?
prakhar.roy371
2023-10-10 11:19
Hey all,
Consider this scenario :
Provider is C# based
Consumer is Java based.
On both sides, there is a property call SellingPrice which is a Decimal on Provider side while it is BigDecimal on consumer side.
Will this cause any issues while performing verification on Provider side?
prakhar.roy371
2023-10-10 11:26
Can you elaborate what you mean by "sent over the wire", please?
Do you mean the format, like json or text?
matt.fellows
2023-10-10 11:33
You can?t send a `Decimal` over JSON, so how are you representing it when it is sent from consumer to provider over the internet. Assuming JSON, is it a `number` or a `string` ?
prakhar.roy371
2023-10-10 11:50
This cleared my initial doubt as well. Thanks for the clarification. :sweat_smile:
yousafn
2023-10-11 10:27
hmmm, if you are just testing the message, then fact its a database shouldn?t matter.
I made some noddy example of using postgres and logical replication, to ensure that the correct json message is being emitted from the database (which would be consumed into a kinesis stream, transformed and sent to s3 - as the full flow)
https://github.com/YOU54F/pact-logical-replication/tree/main
? consumer https://github.com/YOU54F/pact-logical-replication/blob/main/message_pact_js/message.consumer.test.ts
? provider https://github.com/YOU54F/pact-logical-replication/blob/main/message_pact_js/message.provider.test.ts
yousafn
2023-10-11 10:28
I don?t think we get asked about it very often, especially as changes to our stored procs in databases, were covered with a large suite of unit tests for psql which covered migration paths, but it would still be possibly to say remove a field from a SP that a consumer relied on
matt.fellows
2023-10-11 12:26
ah yes, the logical replication - that?s what I couldn?t remember. thanks
thomas.koppensteiner
2023-10-12 06:06
Hello,
I'm testing a service (`users-service`) which is both an API provider and a API consumer.
The provider test works
```CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
-------------------|--------------------------------------------|---------------|-----------------------------------------------|----------|--------
gdprexport-service | C-5cf6a5f1b2e171382a3f0dee4e8696d8a0c34685 | users-service | P-v2-f1e31679d8f7a7c144ca63d80a5ebf69ac92719d | true | 1 ```
But for some reason an the `gdprexport-service` , which is the API consumer, is also tested against the `users-service` consumer pact :exploding_head:
```CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
-------------------|--------------------------------------------|---------------|-----------|----------|--------
gdprexport-service | C-5cf6a5f1b2e171382a3f0dee4e8696d8a0c34685 | users-service | ??? | ??? |
users-service | C-f1e31679d8f7a7c144ca63d80a5ebf69ac92719d | accounts-api | 6.5.2 | true | 1 ```
```There is no verified pact between the latest version of gdprexport-service (C-5cf6a5f1b2e171382a3f0dee4e8696d8a0c34685) and version C-f1e31679d8f7a7c144ca63d80a5ebf69ac92719d of users-service```
We prefix the consumer version with `C-` and the provider version with `P-`.
Actually pactflow should only test
```gdprexport-service | C-5cf6a5f1b2e171382a3f0dee4e8696d8a0c34685 | users-service | P-v2-f1e31679d8f7a7c144ca63d80a5ebf69ac92719d
users-service | C-f1e31679d8f7a7c144ca63d80a5ebf69ac92719d | accounts-api | 6.5.2```
and not
```gdprexport-service | C-5cf6a5f1b2e171382a3f0dee4e8696d8a0c34685 | users-service | ???```
How can I fix this?
bas
2023-10-12 06:11
Hi @thomas.koppensteiner, could you try identifying the pacticipants as `users-service-consumer` when it is a consumer and `users-service-provider` when it is a provider?
thomas.koppensteiner
2023-10-12 06:15
Didn't thought about this option so far, but I can give it a try.
thomas.koppensteiner
2023-10-12 06:16
On the other hand, shouldn't it be clear for pactflow, that a consumer should only be tested against a provider and vice versa?
bas
2023-10-12 06:39
I don't know the exact logic that Pactflow users to determine whether some pacticipant is a consumer or a provider, maybe it's all in one big list / table :slightly_smiling_face: The core maintainers can probably tell you more about that.
The different `c-` and `p-` versions should not be the problem, you can use pretty much whatever you want as a provider or consumer version identifier (although there are some good practices)
abubics
2023-10-12 06:51
The logic should only be "does this pacticipant send requests or receive requests?" (i.e. just the directionality of requests, which is typically one-way per contract)
abubics
2023-10-12 06:53
You shouldn't need to identify different parts of the same app as consumer vs provider. That would make some things hard to reconcile (like the pacticipants dependency graph, and maybe preverification).
abubics
2023-10-12 06:54
I'm not sure why the verifications are getting confused, unless you've specified the wrong names in the contract setup, or something :think_up: hopefully a maintainer can jump in and answer soon :slightly_smiling_face:
matt.fellows
2023-10-12 09:02
> You shouldn?t need to identify different parts of the same app as consumer vs provider. That would make some things hard to reconcile (like the pacticipants dependency graph, and maybe preverification). (edited)
yes, why are you doing this? It doesn?t make sense (at least without further explanation) for a _version_ of the software to be a consumer and another _version_ of the software to be a provider. Either it is both, or it isn?t, and the version is orthogonal to that
thomas.koppensteiner
2023-10-12 09:05
Thank you for your answers.
So far I didn't change the pacticipant-name.
Any other suggestions?
matt.fellows
2023-10-12 09:06
It would help if you explain your setup in a diagram. In fact, the network diagram in PactFlow would help to see what?s happening
matt.fellows
2023-10-12 09:07
these also look like `can-i-deploy` outputs, it would be helpful to see how you are invoking that command
matt.fellows
2023-10-12 09:07
> So far I didn?t change the pacticipant-name.
what do you mean by this?
matt.fellows
2023-10-12 09:09
This diagram has `rt-` prexfies on the apps - did you manually modify the output from above to drop that?
thomas.koppensteiner
2023-10-12 09:09
>what do you mean by this?
@matt.fellows I referred to the first suggestion from @bas
thomas.koppensteiner
2023-10-12 09:10
> did you manually modify the output from above to drop that?
Yes, because the prefix doesn't matter for the example.
thomas.koppensteiner
2023-10-12 09:14
```if [ "$(params.publish-consumer-contract)" = "true" ] ; then
if [ -n "$(params.pact_deploy_to_environment)" ] ; then
pact-broker can-i-deploy\
--broker-base-url=$(params.pact_broker_base_url)\
--broker-token=$PACT_BROKER_TOKEN\
--pacticipant=$(params.pact_pacticipant)\
--version=C-$(params.git_sha)\
--to-environment=$(params.pact_deploy_to_environment)\
--retry-while-unknown=2\
--retry-interval=10
else
pact-broker can-i-deploy\
--broker-base-url=$(params.pact_broker_base_url)\
--broker-token=$PACT_BROKER_TOKEN\
--pacticipant=$(params.pact_pacticipant)\
--version=C-$(params.git_sha)\
--retry-while-unknown=2\
--retry-interval=10
fi
else```
Currently we are not setting the `to-environment` (we are on it).
matt.fellows
2023-10-12 09:15
This row says `gdprexport-service` can?t be deployed (to wherever the target you specified is) because the `users-service` is required for it to operate.
```CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
-------------------|--------------------------------------------|---------------|-----------|----------|--------
gdprexport-service | C-5cf6a5f1b2e171382a3f0dee4e8696d8a0c34685 | users-service | ??? | ??? | ```
It goes on to say:
```There is no verified pact between the latest version of gdprexport-service (C-5cf6a5f1b2e171382a3f0dee4e8696d8a0c34685) and version C-f1e31679d8f7a7c144ca63d80a5ebf69ac92719d of users-service```
I think you are confused by versioning. The version is the piece of software you?re releasing (e.g. git sha a1b2c3). If `users-service` is both a consumer and a provider, then it should only have one version. Because when you deploy it, you only deploy one version of it.
I suspect this is why it?s picking up the `C-` version. The last time `users-service` is deployed to <target>, it was the version with `C-`. I think that is causing some confusion about how all of this works.
matt.fellows
2023-10-12 09:16
OK, so in lieu of specifying an environment, I think this will just verify against the _latest_ provider. If the latest version of `users-service` is `C-` then that would also explain it
thomas.koppensteiner
2023-10-12 09:24
We came up with this, because the services offers different (legacy, but still active) API versions.
Currently we are testing the provider against API version 2 -> `P-V2-` .
I think using only the `` as version lead pactflow test the consumer against all APIs, where of course only one test succeeded, but I can't remember exactly.
Is your recommendation to use the `` without prefix for both consumer and provider and for all API versions?
matt.fellows
2023-10-12 09:27
Yes, that would be my recommendation. If the API is versioned and logically different, you could:
1. Use the `releases` to model that two concurrent released versions in the wild (see https://docs.pact.io/pact_broker/recording_deployments_and_releases#releases)
2. Create a new name of the API to indicate it?s a separately deployed thing (e.g. `users-service-v2` )
It sounds like it?s the same deployable unit that supports multiple interfaces (API versions), so I don?t think these actually make sense for you.
joseph
2023-10-12 22:50
Hi team. I?m new to Pact and contract testing in general. I?m working on a POC with Pactflow and I seem to have a chicken and egg dilemma with `can-i-deploy` in CI when a change is made in the contract to an existing field?s data type/expected value. For example:
? Verified contract between consumer (`main` branch) & provider (`main` branch) has JSON response body field `age` as an integer
? Consumer team publishes revised contract (`feature-age` branch) to update `age` to be a string. Contract for `feature-age` branch has status ?Unverified? so `can-i-deploy` check in CI fails for no verified contract w/ provider. Pull request is blocked from merging to `main` due to failing check.
? Provider team implements change to `age` field changing it from integer to string in a feature branch and creates a pull request. Pull request `can-i-deploy` CI check fails verification because latest version of `main` branch in the consumer still has `age` as an integer (consumer PR is blocked from merging).
In this situation, neither the provider nor the consumer can merge PRs to implement the data type change _to an existing field_. I?d like to prevent a consumer from merging to `main` if a contract is unverified or failing when comparing to its provider(s) `main` branches. And vice versa for providers. `can-i-deploy` seems to fit the bill for that as a CI check when adding or removing fields, but when I try to change the data type/expected value for a field, I hit this wall. Any guidance would be appreciated!
matt.fellows
2023-10-12 23:46
Of course. You?ll have a deployment timing issue if you do it this way.
The ?correct? way to do this sort of thing is via ?expand and contract? strategy here.
matt.fellows
2023-10-12 23:46
(the consumer can try first of course, but the build will fail until the provider supports it)
matt.fellows
2023-10-13 11:21
I'd be happy to have an Easter egg where given a certain combination of parameters the series of issues I've documented about office are presented as a warning to the user about the dangers of that product
joseph
2023-10-13 14:54
Ahh, I like that solution as it allows us to add a deprecation notice on the old integer field and continue to support it for a while until all consumers have migrated to the new field. Thanks a lot Matt, I appreciate the help!
prakhar.roy371
2023-10-16 10:08
Apologies for the late reply on this. @yousafn In my scenario,
we have a java based kafka messaging system which calls an SP from the database team and the SP helps store the message into DB.
I wanted to know HOW the database would act as a Consumer/Provider in this scenario?
How would be publish/verify results from Database side to the pact broker?
matt.fellows
2023-10-16 12:21
See the examples above for Kafka. Same principle. I don't think it's a good use case tho tbh
prakhar.roy371
2023-10-17 06:10
Can you elaborate on why its not a good use case?
I am trying to understand that myself so that I can make my leads understand it as well.
matt.fellows
2023-10-17 06:17
There are a few reasons, but the main one is that, usually, a database only has a relationship to one system. It?s an anti-pattern to have multiple systems talk to the same database (because that?s what leads to breaking changes - a schema change). So usually, only one system ?manages? the database. It?s an implementation detail abstracted by an API (one over a network boundary or otherwise) In that case, you can test using other means (unit tests and integration tests of the system)
Maybe you could describe the scenario in a bit more detail.
> we have a java based kafka messaging system which calls an SP from the database team and the SP helps store the message into DB.
Is the database is implementation detail in this scenario? The contract sounds like it?s between the SP and kafka producer(s).
Who?s reading from the db, if any? Are they reading via a public API or reaching into the database? What kind of database is it?
a better question - what problems are you facing or what problems are you looking to prevent?
prakhar.roy371
2023-10-17 09:45
Well, the Java Kafka project sends minimal information in the payload... and this SP in the database project creates new payload using the minimal information from the Kafka Project and puts in to a AQ.
There is a job which is scheduled BU wise. Whenever there is READY message in the AQ it consumes the data and stores the data in DB tables.
matt.fellows
2023-10-17 11:47
Perhaps a diagram would help, I think a few concepts are unclear here. What is SP and BU? These aren't familiar to me
yousafn
2023-10-17 12:04
Kafka Service (Publisher) publishes events to a (AQ) active queue, in this case a READY message
DB Service has a SP (stored procedure) which is
? triggered by the READY message on the AQ
? takes the payload and performs a database action
You want to assert that
Kafka -> Ready Message -> AQ
AQ -> SP -> DB
You would want to replay the message to the AQ (probably being agnostic of the queuing mechanism, so having something that can construct the message for the DB, without knowing about kafka, Pact can then provide this the event to pass to your SP handler)
Your DB does it?s thing and successfully returns, so Pact is happy
You can assert on your DB state after executing the verification. Having a setup and teardown based on the state would be valuable.
It?s a bit messy though - one there is alot of moving parts and this would probably be better tested closer to the DB.
I feel its actually more complicated because the consumer of these messages (the SP would be the one generating the Pact files) - and the producer - the kafka service would verify it would send them. That?s the inverse of consumer/provider paradigm for HTTP. The SP as a consumer would need to generate pacts for all its possibly many producers.
yousafn
2023-10-17 12:06
> It?s an anti-pattern to have multiple systems talk to the same database (because that?s what leads to breaking changes - a schema change). So usually, only one system ?manages? the database. It?s an implementation detail abstracted by an API (one over a network boundary or otherwise) In that case, you can test using other means (unit tests and integration tests of the system)
I?ve seen this a few times, where business logic is applications is directly tied to database schemas, and it gets painful, quickly
yousafn
2023-10-17 12:13
So in a previous org, we have stored procs in a postgres database, we used PGTap to test all of our migrations would apply, and that our stored procs would work as expected. These were tested in isolation.
We would use logical replication, which would cause json messages to be spat out, whenever the DB was touched (app updates, inserts etc)
These messages would be picked up by Kinesis -> Lamda (Transforms) -> S3 (Store)
They were later ingested by other systems.
For the PG -> Kinesis, we would use a test database, with the logical replication setup, and trigger inserts/updates into the db, and check the kinesis handler could process them, in component integration tests.
We then separately tested our transform layer, that it could handle the expected kinesis payloads, and convert to the expected format consumers required, who would pick up the transformed data.
That is why I wrote the logical replication stuff with Pact, as I could see an opportunity to capture those contracts between each hop, just to help increase the confidence, which retaining the speed of testing these in insolation.
We still had integration tests that tested both subcutaneously (tested via API calls) and via the UI, that exercised all the systems moving parts. (We had some external deps, both in the org and third party to deal with as well)
yousafn
2023-10-17 12:17
Some of the effort we went through to test locally, didn?t pay off, as some of the for example serverless-offline and localstack, were close but not quite compared to the real infra, so there were some compromises in the confidence, which increased as the project grew in scale. We then switched to considering more ephemeral environments on PR?s so that we could test our changed bit of an application, with all the real infra (things like aurora, serverless and configurable/modular apps meant we could point them to different apps/services )
Separately there was a-lot of work done to consider atomic transactions, and journals of system activities, especially with EDA, as there was alot of failure points and we didn?t want our system transactions in a half enacted state
pavikrish
2023-10-18 12:53
Hi,
We are onboarding pact consumer driven contract testing in our micro-services architecture. Since we have just started with the process, I have a basic question on integrating pact with our CI/CD pipelines.
I have a consumer C1 having a contract with provider P1, publishing pact for the first time to the pact broker. If I?m configuring contract_requiring_verification_published web hook, this will trigger the pact verification and publish the results to pact broker. I haven?t yet recorded the provider deployment in pact broker though. Hence in the consumer pipeline, if I configure can-i-deploy after publishing pact this will fail.
Unless I trigger a deployment for my provider, record deployment which is configured after deployment is not going to record the provider version and environment in pact broker. How to automate this process for the consumer build to work.
Can you help me with this?
matt.fellows
2023-10-18 22:41
> How to automate this process for the consumer build to work.
there is a CLI call you would make when you deploy
2023-10-18 22:41
Here are a number of useful hands-on labs that teach all of the key concepts: https://docs.pactflow.io/docs/workshops and https://docs.pact.io/implementation_guides/workshops
thomas.koppensteiner
2023-10-19 05:57
Short update:
Using only the git-sha as `version` in combination with `releases` and `deployments` worked so far.
thomas.koppensteiner
2023-10-19 06:01
We use a shared model for parts of the request body and response body.
In there the `updated_at` attribute uses the `read-only` behaviour.
Which indicates it is only part of the response body (where it is required), but not from the request body.
For me it looks like, that the validation only checks for the `required` attribute, but not for the `behavior` attribute.
Is there a quick fix for that or do I need to use two different models for the request and the response?
matt.fellows
2023-10-19 06:10
Interesting, I?m not sure we support this scenario. Could you please raise a support ticket?
2023-10-19 06:11
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
thomas.koppensteiner
2023-10-19 06:24
OK, will do.
Here the swagger doc for https://swagger.io/specification/v3/?sbsearch=readOnly.
thomas.koppensteiner
2023-10-19 06:34
OK, will use separate models for the request and response for now.
pavikrish
2023-10-19 07:36
Thanks for the response @matt.fellows. As per the CI/CD workshop, can-i-deploy from the consumer side will fail and we need to manually trigger the provider's workflow (test, can-i-deploy, record-deployment). If the provider is already deployed in an environment, and contract is published later by the consumer, we need to manually run the record deployment command for the provider to let the consumer's workflow to get through?
matt.fellows
2023-10-19 10:53
Thanks for raising, much appreciated! We'll get that onto the backlog for review. But glad you have a plan B for now.
matt.fellows
2023-10-19 10:55
If you're up for it, you could consider adding a PR to : https://github.com/pactflow/swagger-mock-validator
Which is the core matching engine for this feature
thomas.koppensteiner
2023-10-19 12:15
... until now.
I reproduced the original issue, which lead to add a prefix to the `version`.
We have a micro service with 2 API versions (`v1` and `v2`).
Those two API versions are documented in 2 separate OAS files.
```Error making request to https://adidascdc.pactflow.io/provider-contracts/provider/rt-consents-service/publish status=409 {"title":"Conflict","type":"https://adidascdc.pactflow.io/problems/conflict","detail":"A provider contract for rt-consents-service version 64d9021bfcc4df13014b52384290c270da92b089 already exists with different content and may not be modified. You will need to delete this resource and re-publish it. You can delete the resource by opening https://adidascdc.pactflow.io/hal-browser/browser.html#https://adidascdc.pactflow.io/contracts/provider/rt-consents-service/version/64d9021bfcc4df13014b52384290c270da92b089 in your browser, clicking on the `NON-GET` button for the `self` relation, and sending a `DELETE` request to it.","status":409,"instance":"/","errors":[{"type":"https://adidascdc.pactflow.io/problems/invalid-body-property-value","title":"Invalid body parameter","detail":"conflicts with existing value","pointer":"/contract/content"}]}```
thomas.koppensteiner
2023-10-19 12:18
Can't pactflow take the filename also into consideration? You check the content of the OAS file, but not the filename. :disappointed:
How ist this supposed to work?
yousafn
2023-10-19 13:03
You would need to merge them into a single document, PactFlow doesn?t currently support multiple documents (as you only upload one, so references aren?t involved)
In your instance, its trying to overwrite the contents of the file, which it?s saying is dangerous (quite rightly) as that contract content, against that version would be non deterministic.
Also on can-i-deploy, using just the version, is essentially checking against the latest published contracts for related parties, which is really underministic too. Without envs/releases, I would use can I deploy to the main branch.
yousafn
2023-10-19 13:03
> You would need to merge them into a single document
Not sure if this would actually be possible, based on your content but just spit balling
yousafn
2023-10-19 13:04
A bit of documentation on the merging reqs
https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#document-limitations
thomas.koppensteiner
2023-10-19 13:42
> You would need to merge them into a single document
Oh, boy.
Didn't know that only one file is supported.
I was aware, that it should not include references etc, but I thought we can upload multiple files.
allen.ayala
2023-10-19 15:16
*Allen Ayala* [11:13 AM]
Good morning,
I?m trying to pull some reports from the PactFlow DB, and I wanted to figure out whether a participant is doing CDC or BDC. Is there a way to check for this on the DB?
yousafn
2023-10-20 12:51
Hey,
I would ask if you are using on-prem or the cloud, as SaaS doesn?t afford users access to the databse.
You could string some stuff together with API?s.
If a pacticipants has provider contracts, then they are doing, however they can also still perform a full fat verification with a Pact Verifier, against the consumer contracts, and therefore would be doing both
The UI is aware, as we show ?view contracts? if the integration as a provider contract uploaded in its list
allen.ayala
2023-10-20 13:26
We?re running the PactFlow image in a Fargate cluster that we manage (so cloud?)
matt.fellows
2023-10-23 01:07
we would prefer not to give you an SQL query, because it?s likely to be unstable over time
matt.fellows
2023-10-23 01:07
do you want to know the general numbers, or do you need to know for each application, if they are doing CDC vs BDC?
rebecca.fulton
2023-10-23 09:31
Hi,
As of Friday morning we're getting a large amount of failures in contract comparison for APIs that we haven't changed. The main types we are seeing are
? Request body is incompatible with the request body schema in the spec file: must be object
? Request content type header is not defined but spec specifies mime-types to consume
? Response Content-Type header is defined but the spec does not specify any mime-types to produce
Would anyone be able to help with this please? It really doesn't look to me like we've actually broken our contracts, so I'm wondering if it's a Pact bug.
matt.fellows
2023-10-23 12:08
Hi Becky, are you a cloud or on prem customer?
I'll follow up tomorrow to check our changlog.
Have you published recently? We have released some changes a few weeks ago that I'm aware of but they may not have impacted you until you've published.
If you like, you could raise a support ticket. Please ensure you include the OAS and Pact file causing the issue so we can investigate..
(See howtosmartbearsupport)
2023-10-23 12:08
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
rebecca.fulton
2023-10-23 13:49
Thanks for the quick response. We've managed to narrow down the cause after a lot of digging. Turns out that node 18.18.2 changed the behaviour of `JSON.stringify` so it was writing all our request bodies as strings of JSON instead of as JSON objects!
allen.ayala
2023-10-23 14:00
We would like to know for each application if they?re doing CDC or BDC from the consumer or provider perspective
gunarajs
2023-10-23 21:08
Hi,
I am new to Pactflow and doing a POC on contract based testing. I want to understand a little more about how pact verification with provider works for certain use cases. Lets say, there is an api like `GET /v1/tickets?user_id=123` which returns list of tickets a user had purchased. Sample response as follows
```{
"tickets": [
{ upcoming_ticket1 },
{ upcoming_ticket2 },
{ upcoming_ticket3 },
{ expired_ticket1 },
{ expired_ticket2 }
],
page: 1,
page_size: 5
}```
Now, I want to have following 3 UI test cases.
1. TestUserWithOnlyExpiredTickets
2. TestUserWithBothUpcomingAndTickets
3. TestUserWithOnlyUpcomingTickets
While I can create the request/response pairs for these 3 cases that makes the api consumers to test with those contracts, how would the provider validation work for these 3 cases. As I understand from some of the demo videos, the pact broker validates the contract with actual provider. In these 3 examples of contracts, how does the pact broker knows it has to create user with specific state such as the number of tickets with expired/active flag or how do I write the pact to instruct the pact broker to create the users with a given state ? Is there an example that I could refer ?
matt.fellows
2023-10-23 23:54
If you?re asking about how state handlers work, all of our examples have them
matt.fellows
2023-10-23 23:55
> As I understand from some of the demo videos, the pact broker validates the contract with actual provider
for clarity, for *Pact tests* the Pact Broker _does not_ do the validation, it brokers the contracts. Pact does the verification
for *bi-directional contract testing* it is where the validation happens.
matt.fellows
2023-10-23 23:57
> how does the pact broker knows it has to create user with specific state such as the number of tickets with expired/active flag or how do I write the pact to instruct the pact broker to create the users with a given state
You would use provider states. These are how the consumer states the pre-conditions required for the test to pass. They are written as strings with optional parameters.
The provider during verification can optionally use them to setup state on a per-interaction level, to ensure they pass.
The consumer shouldn?t know _how_ the provider sets up the state, that is the provider?s implementation detail.
matt.fellows
2023-10-24 12:30
:mega: *New webinar alert:* Accelerating Deployment Velocity: The Power of Bi-Directional Contract Testing
:red_circle: Recording available here :point_right: https://smartbear.wistia.com/medias/xtnv0qroo3
*Description*
From the beginning, PactFlow was closely and solely aligned to the popular Open-Source Pact framework facilitating consumer driven contract testing. With no signs of slowing, this practice is a proven way to increase developer confidence, deploy faster, reduce issues in production and decrease testing costs. However, like most development practices, there?s no silver bullet and some users experience barriers to scaling up for success.
As a team who enjoy drinking our own champagne?and being as close to the Pact community as we are?we were aware of some of the pains and sought a methodology and framework to solve some challenges. So, we expanded the breadth of contract testing capabilities beyond the popular Pact framework, to enable new modes of contract capture and compliance, a broader set of use cases, and a simplified developer experience.
Join us for this informative webinar to:
? Understand when to choose consumer-driven or bi-directional contract testing
? See the value and ROI of contract testing faster
? Learn how bi-directional contract testing lowers the barriers
? Ask any questions you have
? See a demo
thibault.alix
2023-10-25 02:03
I just started playing with pactflow cli but while trying to publish a contract to our broker, I get the following error:
```./pact/bin/pactflow publish-provider-contract ./<contract>.json --broker-base-url https://<url>.http://pactflow.io/ --provider "<provider>" --provider-app-version 0.0.1
Error making request to https://<url>.http://pactflow.io/provider-contracts/provider/<provider>/publish status=400 {"errors":{"_body.contract.contentType":["is missing"]}}```
``` provider
.given("a test")
.uponReceiving("a request")
.withRequest({
method: "GET",
path: "/endpoint",
headers: { 'Content-Type': 'application/json' }
})
.willRespondWith({
body: like(jsonResponse),
status: 200,
headers: { 'Content-Type': 'application/json', 'Accept': 'application/json' }
});```
any idea what am I missing?
thibault.alix
2023-10-25 02:13
```./pact/bin/pact-broker publish ~./<contract>.json --broker-base-url https://<url>.http://pactflow.io/ --consumer-app-version 0.0.1```
prakhar.roy371
2023-10-26 07:12
On top of contract testing where we see if consumer and provider are in sync in terms of the structure of data exchanged, can we also have data level validations?
Let's consider this example of a kafka message :
{
Name : ABC
Age : 34
Address : XYZ
}
Now, structure wise -> all is good on both consumer and provider. Contract is Valid. PACT SUCCESS.
But can we also incorporate certain validations in our pact test cases where in if AGE < 18 is sent, this accounts for breach of contract and fail the pact.
(Here we would be considering data sent as well, not just the structure)
matt.fellows
2023-10-26 07:29
this is probably bordering on a functional test. I don?t think you can currently set that sort of restriction in Pact
cyrus.devnomad
2023-10-27 11:55
Hi All,
In our pact scenario we have multiple services and one of them (ServiceC) is both provider and consumer.
We keep running into the following error when trying to run the pipeline for ServiceC:
The cross contract comparison between the pact for the version of ServiceA currently in dev (610554d ******************) and the oas for version b461002***************** of ServiceC is currently being executed. Enable polling for the can-i-deploy command to retry until all results are available. See https://docs.pactflow.io/docs/cid-polling-bdc for details.
##[error]PowerShell exited with code '1'.
Usually the error resolves itself by re-running the pipeline for ServiceC without running any other pipeline.
For example last time this error happened, we had the following situation in Pact logs:
Computer says no ¯\_(?)_/¯
CONSUMER |C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
-----------------------|----------|-----------------------|------------|----------|--------
ServiceA | 610554d... | ServiceC | b461002... | ??? |
ServiceC | b461002... | ServiceB | 3bce4a0... | true | 1
ServiceC | b461002... | ServiceD | 3bce4a0... | true | 2
After the re-run of ServiceC pipeline can-i-deploy succeeded and we had the following situation in Pact logs:
Computer says yes \o/
CONSUMER |C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT#
-----------------------|----------|-----------------------|------------|----------|--------
Admin | 610554d... | ServiceC | b461002... | true | 1
ServiceC | b461002... | ServiceB | 3bce4a0... | true | 2
ServiceC | b461002... | ServiceD | 3bce4a0... | true | 3
Any hints what causes this or how we can debug it?
likekeys
2023-10-27 14:02
Hi,
We have a contract with another team as follows:
{
"TypeOfPayload": "A",
"Details": [
{
"ID": "000-AAZ",
"Quantity": "1",
"HashTags": [
"aa-000"
],
"SubIds": [
"aaa-zzzz-700000000"
]
}
]
}
Because the message provider breaks the contracts too often, which breaks our system as well, so we really want to use Pactflow to verify the following contracts:
1. The value for "TypeOfPayload" should either be "A", "B", or "D".
2. The "Quantity" value should match the count of SubIds array elements.
3. If the "TypeOfPayload" value is "B", then "HashTags" should not be null or empty.
Can we use Pactflow to create the contract on the broker for the message provider to verify?
yousafn
2023-10-27 15:10
Yes, see
https://docs.pact.io/getting_started/how_pact_works#non-http-testing-message-pact
1. you can marshal the message type via enum matchers
2. you can setup a state based scenario for this type of scenario
3. same as 2
Just remember with Message Pact you aren?t testing the transport mechanism (kafka) but the message handler, so its important to separate them in your code
matt.fellows
2023-10-28 11:46
Have you read the output above? It literally has the solution printed in it
cyrus.devnomad
2023-10-30 12:11
Yes and we also have that pulling option activated but I've been asked by other team members why is this required and what causes this. Which results that can-i-deploy requires are not ready? The results of the other pacticipant? In our case when this happens the pipeline for the other pacticipant was executed long time before that.
matt.fellows
2023-10-31 00:10
so basically, when you use bi-directional contract testing, PactFlow generates comparisons for the provider contract against common versions (`mainBranch`, `deployedOrReleased` etc.) of the consumer. This happens when a new consumer or provider contract is pushed.
These have to happen asynchronously and within PactFlow, otherwise the publish command would be held up. In these cases, you would get this result.
This is not a problem for pure Pact testing, because the verification is done when the provider runs a build, so the result is either not published or published - never in between
allen.ayala
2023-10-31 15:03
Is there any follow up to this? Basically I?m trying to generate a matrice of consumer and provider interactions and trying to see whether that pact is BDC or CDC
matt.fellows
2023-10-31 23:00
Hi Allen,
In case you weren?t aware a colleague of yours has created ticket #00575650 with the same request. It?s with the team to review, please allow us some time to respond
cyrus.devnomad
2023-11-01 09:08
Does it mean that the more branches and environments we have (those that can be seen in the matrix view), the longer it takes for the results to become ready?
We do cleanup our environments ourselves but we don't care about the branches. Do we have to cleanup the branches that no longer exist? Could this be the reason why we experience a lot of retrials during can-i-deploy?
Our polling is currently set like this:
--retry-while-unknown 3 --retry-interval 10
But the polling does not seem to be enough because often times we have to re-run our deploy pipeline due to failed can-i-deploy because the results are not ready even after the polling attempts.
matt.fellows
2023-11-01 09:54
I don't think all branches are included, no. The size of the pact files and OAS will be the main contributor. If you have many released versions that could be a part of it.
30s is a short window though given some of these comparisons might be queued. Some comparisons may take multiple or 10s of seconds each
ravi_shankar_lakkims
2023-11-01 11:17
Hello there!
Consumer Messaging Pact Test failing with below error
"Expected property Date with a length of 24, but has a length of 20, differs near Z"
what's observed is... failing only when the date milliseconds are all zeros such as... "2023-10-31T06:34:24.000Z"
cyrus.devnomad
2023-11-01 17:12
In our case we have pact configured for 4 services only. And the most complex of them has an OAS with like 20 methods. Some of our team members express the concern, that if we run into performance issues already, with the few services and methods we have at this early stage of our project, how will those performance issues grow if we have like dozens of services and more complex OAS files. Right now, one of our services requires around 30 or more seconds to have its results ready for can-i-deploy. The concern is how this time might increase as our system becomes more complex, will we be faced with delays of minutes or tens of minutes? Also, you mentioned that the many released versions might contribute to our performance issues. Is there any strategy or approach to reduce the number of released versions inside Pactflow to counter the performance costs caused by them?
matt.fellows
2023-11-01 20:57
So if you are using `record-deployment` then you are _not_ using the releases - best to check if you are. Releases are for things like mobile, where you might have multiple versions live in the wild. Most of the time, you wouldn?t need this. See https://docs.pact.io/pact_broker/recording_deployments_and_releases
> The concern is how this time might increase as our system becomes more complex, will we be faced with delays of minutes or tens of minutes?
I understand, I?d need to check with the dev team to fully understand if this is something to be worried about.
How is your pipeline currently configured? At what point does the OAS get published? Perhaps a simple fix is to move it earlier into the pipeline, it sounds like you publish the OAS and immediately check `can-i-deploy` .
ravi_shankar_lakkims
2023-11-02 10:36
Hi Matt,
Thanks for the reply, please find the attached screenshot for complete detailed code which is erroring out...
likekeys
2023-11-02 15:19
Hi
We are trying to run this test with PactNet, as the following screenshot. When we put breakpoints and run the codes in the debugging mode, we can see that both the matcher and deserialized json have the proper value for the "Id" property, which is not null. But the result of the test keeps giving us this error message:?
PactNet.Exceptions.PactMessageConsumerVerificationException : The message could not be verified by the consumer handler
---- Expected property events[0].Items[0].locations[0].Ids not to be <null>.
likekeys
2023-11-02 15:42
We find that it's because the property is "Id" in our json text, but it's "Ids" in our model. We use [JsonProperty("Id")] to deserialize it.
likekeys
2023-11-02 15:42
But we cannot change this logic, can we make it work by adding any pactflow setting?
bas
2023-11-02 15:56
This isn't a Pactflow problem, so this belongs in #pact-net I guess. You'll probably have better luck getting an answer there..
matt.fellows
2023-11-02 23:06
In fact, it?s not a Pact problem as far as I can tell. If your JSON is one shape and your model expects another, that is a serialisation/deserialisation problem.
matt.fellows
2023-11-02 23:09
This is a problem with your .NET code. Clearly, you are passing in the same value on line 46 and line 53, so that implies that `TestOrder` is truncating what you are giving to it
cyrus.devnomad
2023-11-03 11:27
> So if you are using `record-deployment` then you are _not_ using the releases -
Yes, we are using record-deployment, so we shouldn?t have those released versions.
> How is your pipeline currently configured? At what point does the OAS get published?
We have two pipelines for each service. A build and a deployment pipeline. The publishing of the provider and/or consumer OAS/pact file happens in the build pipeline shortly after the actual build. The can-i-deploy step happens in the deployment pipeline that is triggered immediately after the build pipeline is done. So can-i-deploy is executed, depending on the service, around maybe 30..40 seconds after the publishing command.
> Perhaps a simple fix is to move it earlier into the pipeline, it sounds like you publish the OAS and immediately check `can-i-deploy` .
The publishing command is executed around 15..20 seconds after the actual build, so maybe there is indeed some potential to place it a bit earlier and closer to the actual build. I will try to bring up this idea in the team.
> I don't think all branches are included, no.
What I see is that whenever we publish a new OAS version, there are new entries in the matrix view for many branches, many of them no longer existing in our DevOps repository. So, it seems that we have some old branches around in the Pactflow that cause unnecessary compatibility checks? The question is how are old branches supposed to disappear from Pactflow? Or are they supposed to remain there for recording reasons? And if they remain there won?t they cause more and more compatibility checks whenever new pact or OAS files are published?
jowhalen13
2023-11-03 13:22
The link to `Publishing` under Pact JS here is broken https://docs.pact.io/getting_started/sharing_pacts#language-support
Where is the documentation for publishing to pactflow in Pact JS?
jowhalen13
2023-11-03 13:24
or I assume we are to just use the broker CLI here? https://docs.pact.io/pact_broker/client_cli/readme
yousafn
2023-11-03 13:33
Apologies about that!
Yes its generally recommend to use the standalone cli tools :+1:
If you are using pact-js, it contains pact-js-core, which exposes the cli tools as npm bin stubs
https://github.com/pact-foundation/pact-js-core?tab=readme-ov-file#cli-tools
there is also an api exposed for programatic use, but I wouldn?t recommend that approach now, as we are more than likely due to deprecate that in the future.
The bin stubs will be exposed in https://github.com/pact-foundation/pact-js-cli which is still due to be published (just a wrapper over the cli tools) - not sure if we should expose the programmatic api here as well (I think we would rather avoid it, to avoid maintainence efforts)
Will look to sort those links so they point to the relevant bits
jowhalen13
2023-11-03 13:52
Since we use nodejs in our backend along with docker compose, the docker cli solution works perfectly actually
jowhalen13
2023-11-03 14:38
Another broker question - I successfully published my first pact, but I found a mistake. I've since fixed the test, and it is correctly making the new contract, but when I go to publish, the older version is overwriting the new version. I've updated the consumer app version and the tag, and am getting this message:
```Pact successfully published for flow-controller version v7 and provider normalizer.
View the published pact at https://my-company.pactflow.io/pacts/provider/normalizer/consumer/flow-controller/version/v7
Events detected: contract_published, contract_content_changed (first time any pact published for this consumer with consumer version tagged v7)```
yousafn
2023-11-03 15:01
> it is correctly making the new contract, but when I go to publish, the older version is overwriting the new version
the Pact Broker doesn?t allow you to overwrite published contract contents, which is why you need to publish against a new version. It isn?t _overwriting_ anything
> I?ve updated the consumer app version and the tag, and am getting this message:
The recommendation is to publish with `branch` not `tag` . or if you wish to use tags, use `branch` as well (so the Pact Broker knows if your pacts relate to the configured `main` branch)
The message is expected, what are you expecting?
The Pact Nirvana setup, would now have a webhook setup which would be triggered by the contract being published, this would trigger a provider verification task, where it would be provided the url of the contract to verify.
If you haven?t done the ci/cd workshop, it would be highly recommended as it teaches you these concepts :+1:
jowhalen13
2023-11-03 15:08
I've not only done the ci/cd workshop, but I've actually successfully implemented the full webhook verification flow across multiple contracts at a previous company. Right now I'm starting on a new project.
> publish against a new version
I thought I was doing this by updating the `--consumer-app-version` tag. So you're saying I need to update the `--branch` version to get the new pact published?
yousafn
2023-11-03 15:10
awesome! the message you?ve posted shows the pact has successfully been published, so I?m not sure what the issue is :elmo_shrug:
yousafn
2023-11-03 15:11
ie
1. you published first on one tag, updated contract and tried to publish with same version = :not-vegan:
2. you updated tag and tried to publish = :white_check_mark:
jowhalen13
2023-11-03 15:13
1. run consumer tests, which writes a new pact file with the correct fields
2. run publish command with updated version tag
3. that same pact file is now missing those fields
yousafn
2023-11-03 15:15
> 1. that same pact file is now missing those fields
same pact by version or tag? is this when viewing in the pact broker?
It would be useful if you can create a simple repro steps by providing the commands you are using to publish, that way we can replicate it and see if we get the same behaviour
prasant.iragavarapu
2023-11-03 15:16
Hi
We have experienced a problem while implementing a pact test from consumer side.
Although our C#.Net Object has all properties in PascalCase, And when our pact test execution is complete, it converts all the properties into camel case if the *CamelCasePropertyNamesContractResolver* is used. However, this conversion does not apply to C# Anonymous Types.
For instance:
We have a property
_public object BusinessRules { get; set; }_
Matcher code
_BusinessRules = Match.Type(new_
_{_
_Comments = Match.Type("This is a comment"),_
_})_
_For some reason only businessRules is properly converted into camelcase. But Comments remain in Pascal case._
_Any Specific reason why the *CamelCasePropertyNamesContractResolver* is not applied to *Anonymous Types*_
#pact-net
yousafn
2023-11-03 15:26
1. Create new Pact
2. Publish pact `pact-broker publish pact.json --consumer-app-version 1 --tag 1`
3. Update contract with a new field `foo`
4. Publish pact again `pact-broker publish pact.json --consumer-app-version 1 --tag 1`
a. This published fine, overwriting the contents, and warning about content versioning.
jowhalen13
2023-11-03 15:28
> is this when viewing in the pact broker?
This is when viewing the pact json file in my editor locally
> same pact by version or tag?
By branch name. I've updated both the tag and the version.
This is my publish command (running in docker compose)
```pact-broker publish pacts --consumer-app-version=v7 --branch=my-branch-name --tag=v7 --verbose```
Running in the `pactfoundation/pact-cli:latest` container
yousafn
2023-11-03 15:29
Hey @prasant.iragavarapu
That question is probably best asked on the pact-net repository either via an issue or discussion, where the author would see that as it is an implementation question of the library ( He doesn?t view this channel much if at all :+1: )
https://github.com/pact-foundation/pact-net
jowhalen13
2023-11-03 15:31
then I run that publish command, and the file reverts back to its old state
jowhalen13
2023-11-03 15:33
even when updating the `--branch` param to something else, it's still reverting back to the old version
yousafn
2023-11-03 15:40
So tested against another PactFlow broker, as the test broker is setup for (https://docs.pact.io/pact_broker/configuration/settings#allow_dangerous_contract_modification) which allows me to overwrite Pact?s.
I can replicate the message of the contract not allowing modification, so forcing me to publish with a new version. I can see both versions in the pact broker, with the changes pre and post change, against each respective version
jowhalen13
2023-11-03 15:44
Oh yeah I'm getting the different version URL's and everything, but see how yours actually has "foo":"bar" in one but not the other? Yeah my "foo bar" is missing from both
jowhalen13
2023-11-03 15:45
if it helps, this is in the request not the response if that changes anything
yousafn
2023-11-03 15:45
most bizarre! could you try using a diff name for consumer/provider to use a new integration to see if its a glitch in the matrix
i?ll try a req change :+1:
yousafn
2023-11-03 15:47
what was the req change? path, param, header? just so I can try and mirror the same
prakhar.roy371
2023-11-06 13:03
@matt.fellows
Basically after Json serialization, the trailing insignificant 0's are removed.
In the mocked data, we have mentioned those 0s. Hence there is a length difference causing pact test cases to fail with the error :
*"Expected property Date with a length of 24, but has a length of 20, differs near Z"*
Can you suggest some insights to this issue without removing those 0's from our mock data?
matt.fellows
2023-11-08 10:15
> What I see is that whenever we publish a new OAS version, there are new entries in the matrix view for many branches, many of them no longer existing in our DevOps repository. So, it seems that we have some old branches around in the Pactflow that cause unnecessary compatibility checks? The question is how are old branches supposed to disappear from Pactflow? Or are they supposed to remain there for recording reasons? And if they remain there won?t they cause more and more compatibility checks whenever new pact or OAS files are published? (edited)
I?ll follow this up. There is a new endpoint that can be used to delete old branches, perhaps that could be something to look into. Let me get back to you in a few days.
matt.fellows
2023-11-08 10:17
This is really a unit testing question with .NET, and not related to Pact
matt.fellows
2023-11-08 10:18
Your code / tooling / frameworks used for serialising and deserialising data (line 53) is comparing to a standard JSON blob (line 46). The line 46 code doesn?t know to zero out the data.
matt.fellows
2023-11-08 10:18
i.e. your test assertion is a bad one. This has nothing to do with Pact. The problem is a useless assertion, probably because you?re just trying to setup a demo to show a basic workflow?
boer.k
2023-11-08 13:46
Hi, curious question. We have created a consumer test to test how we handle error 500 by the provider. However, a http 500 is not defined in the openapi spec (see pic). Thinking about it, I guess that makes sense? a 500 is usually an unexpected exception. So defining a http 500 response in your openapi implies that you expect the unexpected? We are testing this using bi-directional (hence the #pactflow channel), but I think the problem persists in consumer-driven too. Given we give the provider a contract which expects a 500 response, how would the provider even simulate an exception for that test? Are there any thoughts on covering http 500 cases?
yousafn
2023-11-08 13:59
:blobwave:
> Are there any thoughts on covering http 500 cases?
Does your consumer code do anything different if it gets a 5xx error, over a 4xx error.
If not, then I don?t think its worth covering that granular of a case on the consumer side and encoding that into a pact file.
yousafn
2023-11-08 14:01
> However, a http 500 is not defined in the openapi spec (see pic). Thinking about it, I guess that makes sense? a 500 is usually an unexpected exception. So defining a http 500 response in your openapi implies that you expect the unexpected?
Ignoring the philosophical aspect, it is valid as part of the OpenAPI specification to define 5xx responses.
Personally I do expect the unexpected, always xD
yousafn
2023-11-08 14:02
see
https://swagger.io/docs/specification/describing-responses/
under HTTP Status Codes
yousafn
2023-11-08 14:02
> Given we give the provider a contract which expects a 500 response, how would the provider even simulate an exception for that test?
Probably an implementation specific question.
boer.k
2023-11-08 14:10
"Does your consumer code do anything different if it gets a 5xx error, over a 4xx error."
In this case no, we are just looking to add a few more testcases to our test suite to see how it grows. But _what if it did_? :smile: Based on your response, I think in this case we would have to open discussions with the provider team. I am sure they would like to know how internal server errors get exposed to consumers too.
yousafn
2023-11-08 14:13
> In this case no, we are just looking to add a few more testcases to our test suite to see how it grows
Yeah, Pact provides a powerful mock server, that would allow you to test additional cases, which you may not want to send over to provider, and it would remove the need to use another mocking tool, for these corner cases.
There has been some discussing in the past about having pending pacts, or allow pacts not to be serialised for particular tests. In theory it would be doable but would involve some work in each client library as its a composition of calls from the FFI, or we could compose them in the FFI, and then just have a way of exposing that to the client libs.
> But _what if it did_? :smile: Based on your response, I think in this case we would have to open discussions with the provider team. I am sure they would like to know how internal server errors get exposed to consumers too.
Yep, I think I would certainly be having this conversation with the provider team, and maybe a pairing session where you can do some exploratory poking of each others systems/apis. If often does highlight things you may not have considered on both sides.
boer.k
2023-11-08 14:49
Another question: pactflow provides webhooks primarily for triggering provider pipelines when using consumer-driven testing. In the case of bidirectional, are there any webhooks available that we could use for example posting verification results to slack?
daniel.phillips
2023-11-08 17:21
Hi, I am trying to use the pactflow stubs, however when making a GET request to my stub URL, I receive the following error:
`Internal Server Error. An error has occurred. The details have been logged with the reference XXXXXXXXX`
Does anyone know why this could be, or how I find further information linked to the reference number the response gave me?
matt.fellows
2023-11-08 22:28
> However, a http 500 is not defined in the openapi spec (see pic)
You should still define this in your OAS as an expected-unexpected response type. The point of an OAS is to try to be explicit about these things, to the degree possible with OAS
matt.fellows
2023-11-08 22:29
> but I think the problem persists in consumer-driven too. Given we give the provider a contract which expects a 500 response, how would the provider even simulate an exception for that test? Are there any thoughts on covering http 500 cases?
In Pact, that?s what provider states are for. A provider could take that state, and modify state internally to cause a `500`. I?m not entirely convinced of the value of doing so in a contract test, but it?s supported / possible
matt.fellows
2023-11-09 22:08
Good question, we don?t have those yet but it?s on our backlog. The best public place to ask that would be here: https://github.com/pactflow/roadmap/issues
matt.fellows
2023-11-10 01:14
OK back on this - you?re right, we do actually check branches. This could explain the behaviour
matt.fellows
2023-11-10 01:14
Keep an eye on this PR: https://github.com/pact-foundation/pact_broker-client/pull/151
matt.fellows
2023-11-10 01:15
If you used Github (or an SCM with hooks), something like this will work to clean up old branches from PactFlow:
```name: Clean up application branch in Pactflow
on:
delete:
branches:
- "*"
jobs:
delete-branch-in-pactflow:
name: delete
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ruby/setup-ruby@v1
- name: Clean up branch for application pactflow-application-saas in Pactflow
run: script/ci/clean-up-pactflow-branch.sh
env:
PACTFLOW_TOKEN: ${{ secrets.PACTFLOW_TOKEN }}
GIT_BRANCH: ${{ github.event.ref }}```
The script (needs better error handling!):
```#/bin/bash
set -Eeuxo pipefail
APPLICATION="your-app"
PACTFLOW_URL="https://YOUR_TENANT.pactflow.io"
echo "Deleting branch ${GIT_BRANCH} for ${APPLICATION} in Pactflow"
ENCODED_GIT_BRANCH=$(echo "$GIT_BRANCH" | ruby -e "require 'erb'; puts ERB::Util.url_encode(ARGF.read.chomp)")
BRANCH_URL="${PACTFLOW_URL}/pacticipants/${APPLICATION}/branches/${ENCODED_GIT_BRANCH}"
curl -X DELETE $BRANCH_URL -H "Authorization: Bearer ${PACTFLOW_TOKEN}"```
prakhar.roy371
2023-11-10 11:54
Hi all,
PactFlow provides version control for contracts.
CURRENTLY :
Our CICD jobs have a dynamic variable for version that changes every time the job is run.
But if we don't have any new changes in our contract, it is still showing as a new version in Pact Broker history since the version gets changed every time.
The issue we are facing is :
We do not want to publish new versions of contract when there are no changes.
How can we publish the pact only when there is actual changes in the contract?
FYI @john.joel.thetla
yousafn
2023-11-10 12:35
if a pact is published, and the contents have not changed, and the contents have already been verified an a previous version, the published pact with the new version will automatically be verified.
> Our CICD jobs have a dynamic variable for version that changes every time the job is run.
It should only change, when your code changes. Are you suggesting you could retrigger the job, and it would publish with a new version number?
> How can we publish the pact only when there is actual changes in the contract?
You would have to build tooling to pull down the contents of all verified contracts, check the contents of the pact file you have, against the contents of the remote pact files, and if there is a delta publish it.
It doesn?t sound like you are using Pact in the intended way, and therefore seeing issues you are trying to solve problems for, that if you follow the recommended flow, you wouldn?t have
matt.fellows
2023-11-10 21:46
Just because your build number changes each time doesn't mean the version of your apps you publish to Pactflow should. Use the git sha and this problem will go away
prakhar.roy371
2023-11-14 05:25
Building a tool for that purpose was suggested by one of our team members as well but we thought to take a step back and ask here if there would be another way.
We know that pact CICD jobs have arguments for versioning just wanted to know how it is supposed to be used dynamically in the correct manner.
Will try git sha.
matt.fellows
2023-11-14 06:38
This article is going to be your best source to understand versioning in detail: https://docs.pact.io/getting_started/versioning_in_the_pact_broker
prakhar.roy371
2023-11-14 07:20
If you see the top 2 in matrix image below, we used $`CI_COMMIT_SHA` in our job.
The contracts for the two versions published here are same. There was a commit made that did not change the contract itself. Still we see 2 versions here in matrix which is what I want to avoid every time we make a small change.
Is $`CI_COMMIT_SHA` what you were going with @matt.fellows when you suggested git sha?
prakhar.roy371
2023-11-14 10:04
Every single change done by the team is causing a new pact to get published at Consumer side.
Even when there is no change in the contract.
We want to avoid that.
prakhar.roy371
2023-11-14 10:42
Thinking about it, it isn't really a problem when this statement by Yousaf is true.
"if a pact is published, and the contents have not changed, and the contents have already been verified an a previous version, the published pact with the new version will automatically be verified."
If code changes, a new pact WILL be published.
We will convey the same further. Thank you!
matt.fellows
2023-11-14 10:51
More importantly, we need to ensure each application version is modelled in PactFlow to allow other tooling to work (e.g. can I deploy)
jrede
2023-11-18 23:31
Hi all, Our org is running into an issue with CICD implementation. Specifically when a dev updates a provider and breaks consumers. How is the dev supposed to get the pipeline into a green state in order to merge into main? Currently we have the can-i-merge check happening which fails as the current dev work has broken the pact. So dev finds consumers and updates them, but CICD can-i-merge check still fails since provider work is on a feature branch and consumer update is on a feature branch, not main. We?re now soft-locked from introducing any change. I read a bit about WIP pacts and pendingpacts, However im not seeing where those settings are to be set or really any example using them. Were utilizing all our pact commands with Cypress & pact-broker-cli. Any help is appreciated, Thanks!
matt.fellows
2023-11-19 00:57
Have you read / done the CI/CD workshop? It both explains those concepts and shows a demo app (howtolearn)
2023-11-19 00:57
Here are a number of useful hands-on labs that teach all of the key concepts: https://docs.pactflow.io/docs/workshops and https://docs.pact.io/implementation_guides/workshops
jrede
2023-11-21 21:28
Appreciate that. Thanks for the links, I went ahead and completed the CICD tutorial but am still struggling. In the https://docs.pactflow.io/docs/workshops/ci-cd/workshop/verifying-feature-pacts#commit-our-changes-for-the-provider, it goes over a consumer driven approach however part of the steps require updating the provider directly on the `main/master` branch, which is obviously not feasible. Mix in that our org usually drives changes from the provider and im not really seeing anything on a `provider` driven approach. Do you happen to have more info on `provider` driven cicd approaches?
jrede
2023-11-21 21:50
Lets say the provider makes a change on a feature branch that removes a property that the consumer uses. So the dev creates a branch to update consumers that fail on this new change. Now we have two branches that successfully work with one another. How would the provider get merged into main when can-i-deploy only checks the deployed version on an env
matt.fellows
2023-11-21 21:51
> How would the provider get merged into main when can-i-deploy only checks the deployed version on an env
There is now a `can-i-merge` which would allow you to select a target branch, instead of environment (which I think you said you were using)?
jrede
2023-11-21 21:52
the dev would add integration tests in the same pr with their breaking change and also update ?consumers? that it breaks
jrede
2023-11-21 21:52
ah ok thats awesome then, I mistyped as were using can-i-deploy and not can-i-merge
matt.fellows
2023-11-21 21:55
The point I was trying to make, is that after you make the breaking change to the provider, you would need to get it into production. In the way you described, I think you?d still have a problem unless you also deployed the consumer(s) at the same time - there will be a small window where either the consumer will have been deployed relying on behaviour not yet in the provider (the breaking change) or the other way around.
matt.fellows
2023-11-21 21:56
usually, you would release a change in an expand-and-contract method. Add a new field/behaviour, get all consumers to use that, and then remove the now unused behaviour (field or whatever)
jrede
2023-11-21 21:59
appreciate the insight and yeah your right. Ill see if can-i-merge solves one of our issues
jowhalen13
2023-11-22 16:36
For once I don't have any issues to report, just wanted to say that, as of now, we have both publication and verification fully automated in our CI system for our first two services! Special thanks to both @matt.fellows and @yousafn, who were instrumental in working through the myriad of issues I faced while implementing this.
matt.fellows
2023-11-22 21:17
Haha really appreciate that - thanks for letting us know!
Perhaps we should review the challenges you had and see how we could improve?
jrede
2023-11-22 21:18
Hey Matt, unfortunately It doesn?t seem like can-i-merge does a cross-check against the same branch name. Also it seems like the --help for this command doesn?t list all the options as its missing --branch from the list, while thats a valid option. Maybe im missing something else with how can-i-merge is supposed to work?
matt.fellows
2023-11-22 21:19
Might be worth taking a quick look at this also: https://github.com/pact-foundation/pact_broker/discussions
jowhalen13
2023-11-22 21:20
Sure, it was mostly getting the right combination of arguments for the broker on both publishing and verification; and then wrangling docker compose, both locally and in ci (since the consumer side writes to file you have to deal with docker volumes, which are suuuuuper fun in ci)
jrede
2023-11-22 21:23
For context I have something like this that contains updates for both provider and consumer but can-i-deploy & can-i-merge both fail
matt.fellows
2023-11-22 21:26
Yep, that makes sense. We?re looking at ways to be able to be more intelligent/infer the best arguments. The more information we get about your workflow e.g. branches the better. This is the direction we?re heading in generally (see the discussion link above)
yousafn
2023-11-22 21:56
if they are valid on the branch, but not against main, and you can?t make either party tolerant of the backwards and forwards versions, you have the information to progress the provider and consumer into your main branch.
They won?t be compatible with the deployed consumers, but will be compatible with the main branch.
You?ll then need to co-ordinate deployments with potentially some outage whilst they both deploy.
Pact is simply informing you of your situation and it is up to your team to resolve it.
> We?re now soft-locked from introducing any change.
That is of your own making, not by Pact, Pact is simply informing you of it.
yousafn
2023-11-22 21:58
> Currently we have the can-i-merge check happening which fails as the current dev work has broken the pact. So dev finds consumers and updates them, but CICD can-i-merge check still fails since provider work is on a feature branch and consumer update is on a feature branch, not main.
This is good that the insight from Pact, is marking the relevant consumer services which need updating :+1:
If you can either the consumer tolerant of the deployed/main branch provider and the soon to be deployed provider, you can perform no downtime deployments.
same on the provider side, if you can make it tolerant of consumers using the old and the new format (context dependant on your provider change)
yousafn
2023-11-22 22:00
You can also set `PACT_BROKER_CAN_I_DEPLOY_DRY_RUN` to `true` as a break-glass to ignore the verification result, if you do want to deploy based on a failing result. This could be where you are okay with introducing that breaking change (co-ordinated change window for example)
jrede
2023-11-22 22:02
got it, so can-i-merge and can-i-deploy are correctly failing since this is a breaking change against main. Its now up to the dev to either:
1. make the provider backwards compatible or
2. release the breaking change with follow-up work to update consumers
yousafn
2023-11-22 22:03
I hear a blog post brewing! :memo: I?m sure it would help a fair few
Absolute pleasure dude. glad you are on the path to Pact Nirvana and thanks for taking the time to pass back some props :raised_hands:
PS. This is a neat post Nathan from GoCity wrote on his journey
https://medium.com/go-city/how-we-reached-pact-nirvana-diamond-level-5d55e5c388d0
Might be some other goodies you can add in (badges or notifications) :chefkiss:
yousafn
2023-11-22 22:04
> release the breaking change with follow-up work to update consumers
I wouldn?t release it without the work having already been done to the consumers, and then its a case of just deploying them all within a certain change window. I.e you know the new consumers will be compatible with the new provider.
jrede
2023-11-22 22:05
right the consumer update work should be ready to merge as soon as the provider update is merged
yousafn
2023-11-22 22:05
but either of those options are feasible, usually option 2, with the work passing on a feature branch and then deploying at the same time (ish) will work for some teams depending on how they release (last client has change windows, so all the ci/cd in the world didn?t help us get straight to prod :sweat_smile: )
yousafn
2023-11-22 22:09
pleasure dude, it?s kinda a head-scratcher at first, but becomes more natural. I had everything flipped when we went cloud / microservices based and rolling out changes which were relatively trivial when everything all lived in one code base, because a bit of a juggling act when using microservices.
Feel free to pop up more questions as you get them :+1:
prakhar.roy371
2023-11-27 06:06
Hi all,
We observed a peculiar behavior in PactFlow and want to understand it.
In the screenshot given below, all 4 contracts are the same just the version is different as we are using $*CI_COMMIT_SHA* in our pipelines.
Our expectation here is :
1. Change in contract -> A new contract is shown in integration.
2. Change in version only -> Version of the existing contract gets updated.
*ISSUE :* We are seeing 4 contracts in the integration even though the contracts are exactly the same.
matt.fellows
2023-11-27 06:17
The version is the version of your _application_ and not the contract. You don?t deploy a contract, you deploy an application. PactFlow internally manages the contract version (because for the 4 above, if one of them is verified by a provider and the contents of the contract is the same, then they are _all_ valid f or that specific verification.
matt.fellows
2023-11-27 06:17
So that?s why you have 4 versions, they represent the contract associated with each _version_ of your consumer application
jaspal.puri
2023-11-27 10:26
Hello. A question about stubs.
If I have an interaction that requires an auth header. How do I call the stub generated from this where the auth header I provide to access my Pactflow tenant doesn't conflict with the stub.
For example, I'm calling for a stub on my PactFlow server but I get an error response like:
```{
"message": "No interaction found for POST some/request/requiring/auth",
"interaction_diffs": [
{
"description": "some description",
"provider_state": "some provider stage",
"headers": {
"Authorization": {
"EXPECTED": "Bearer 1234",
"ACTUAL": "Bearer {{PACTFLOW_API_KEY}}"
}
}
}
]
}```
matt.fellows
2023-11-27 10:56
It is a current limitation that you can't have two different types of authentication headers. In your case, If you add flexible matchers to the auth header it would pass (because a bearer token header has a standard format so the Pactflow token would match)
shaun.carmody
2023-12-04 10:09
Hey team, our pactflow webhooks are currently being blocked because our platform has strict access requirements, the platform team have requested all IP's that they can whitelist to allow your webhooks to trigger our Jenkins pipelines. Can you please advise the best way forward here? Thanks!
antal.tettinger
2023-12-05 15:51
Hello,
We are using http://pactflow.io in our company, and I was wondering how could we implement a good consumer driven testing flow into our CI. My idea is that when we change or implement a route then we create and publish the Pact test (we have implemented this already) that is also verifies with the backend. And we could add PR checks on GitHub for the backend that wouldn't let code merge when the contract testing is not green. But you might have a better idea of how to integrate PactFlow into our CI and deployment process, it would be nice to hear. Hope its the right channel to ask?
yousafn
2023-12-05 16:06
Hey,
Working through this workshop should help with the core concepts and recommended setup
https://docs.pactflow.io/docs/workshops/ci-cd/
srinu.innovatus
2023-12-07 08:53
Hello team - I'm getting the below error while trying to publish swagger openapi json file --- Error making request to <https:makeit.pactflow.io/provider-contracts/provider/nexus/publish|https://makeit.pactflow.io/provider-contracts/provider/nexus/publish> status=400 {"title":"Validation errors","type":"https://problems-registry.smartbear.com/validation-error","status":400,"instance":"/","errors":[{"detail":"is missing","pointer":"#/contract/selfVerificationResults/content"},{"detail":"is missing","pointer":"#/contract/selfVerificationResults/contentType"},{"detail":"is missing","pointer":"#/contract/selfVerificationResults/verifier"}]}
srinu.innovatus
2023-12-07 09:28
this is my command - pactflow publish-provider-contract C:/Users/srinivasraghupatruni/Downloads/personDetailsQueryOpenApi.json --content-type=application/json --provider=plexus --branch=test-pactflow-command --broker-base-url=https://makeit.pactflow.io/ --provider-app-version=0.0.1 --broker-token={broker token} --verification-success --verification-results=C:/Users/srinivasraghupatruni/Downloads/results.yml --verification-results-content-type=application/yaml --verifier=handmatig
bas
2023-12-07 09:47
Can you capture the call that is being made to your broker using Fiddler or something similar and post it here?
srinu.innovatus
2023-12-07 10:32
Now I'm able to publish provider contract without any error. This error has come due to an empty restults.yml file. I've added junit results file to resolve this error.
jaspal.puri
2023-12-07 10:43
Hello. Is my reading of the docs (https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas/features/) correct that in BDCT there's no way of writing a consumer contract which specifies an oauth2 scope used in a theoretical token as part of it's `given` which is validated against the security schema of a providers oas contract?
matt.fellows
2023-12-07 10:46
It won?t check the token/scopes, but you can write tests that have OAuth2 on them. Here is an example repo if it helps: https://github.com/pactflow/bdct-oas-examples/tree/main/examples/security
jaspal.puri
2023-12-07 10:59
Thanks, @matt.fellows. Are there plans or desire to expand on OAuth checking? Since scopes are often defined in the oas, it might be a nice extra layer to be able to check the correctness of requested scopes in a hypothetical oauth token.
matt.fellows
2023-12-07 11:00
I remember thinking about it a while back and it either being difficult or not possible. How would you check the scopes?
matt.fellows
2023-12-07 11:01
Specifically, how would you know what scopes the consumer has on their token from a Pact file? You would presumably need to encode the JWT in the pact file and then unpack it?
matt.fellows
2023-12-07 11:02
If you had thoughts on how to do it, you could have a crack at adding it here: https://github.com/pactflow/swagger-mock-validator
jaspal.puri
2023-12-07 11:28
I had not gotten even nearly that far :slightly_smiling_face:. I'll have a look in the repo - my instinct was not to encode a JWT but some other oauth specific custom matching rule on the request which provides instruction on what scopes would be in the JWT.
antal.tettinger
2023-12-07 15:06
is this the correct route described in the docs?
In GitHub docs I found this:
yousafn
2023-12-07 15:08
The route is correct, its this one in the docs
https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#create-a-repository-dispatch-event
yousafn
2023-12-07 15:10
The workflow in the example-provider repo, is configured for the repository_dispatch event
https://github.com/pactflow/example-provider/blob/594529a11b188f55d6dc86d95355f8a947d357b3/.github/workflows/contract_requiring_verification_published.yml#L8
You could use the one in your docs, for a `workflow_dispatch` event, but would need to update your `.github/workflows/contract_requiring_verification_published.yml` to match
antal.tettinger
2023-12-07 15:25
Thanks for the quick reply! I'd rather stick to the guide then with your dispatch but it doesn't seem to work
antal.tettinger
2023-12-07 15:32
I have tried to remove the duplicate Access: access also from the header, not sure if its a typo or on purose, but same result
yousafn
2023-12-07 15:37
yeah that looks like a typo in the headers
```Content-Type: application/json
Accept: application/vnd.github.everest-preview+json
Authorization: Bearer ${user.githubToken}```
yousafn
2023-12-07 15:38
duplicate Accept, will update the docs, not sure if that will solve your problem. You have got the gh token added in your secrets?
antal.tettinger
2023-12-07 15:39
Thanks! Unfortunately fixing that doesn't solve the issue. I get the same error message.
Yes I have added it there tried multiple times
yousafn
2023-12-07 15:42
sweet! wonder what that was, well anyway, thanks for noticing an issue in the docs, and I?ve pushed a change up for that now which is just deploying now so thank you
antal.tettinger
2023-12-07 15:43
Cool! Thank you! I am not sure, because I did exactly the same just navigated away from the Webhook tab and went back and clicked on create a new one
antal.tettinger
2023-12-07 15:44
The only difference I didn't add a description. Maybe some value was not updated for the test when I've changed it or something. Thank you! I will keep doing the workshop
antal.tettinger
2023-12-07 20:25
OK so I have finished.. The WIP part was a bit heavy.
It's nice, but a more realistic workflow for me would be to replace deployments with merging to a staging branch.
Because at the last steps you have to notify the consumer maintainers that at that point in time its OK to merge their feature branch.
I think in your example code the provider master and consumer master is always matching, but I can imagine a scenario where the feature branch of consumer matches the main branch of the provider but then the main branch of the provider doesn't match with the main branch of the consumer.
antal.tettinger
2023-12-07 20:25
So ideally all this process that is related to the feature branch waiting for the provider to comply with the contract should be done on a staging branch
antal.tettinger
2023-12-07 20:26
And the actual deployment should be a check between master-master Pact check?
antal.tettinger
2023-12-07 20:41
It'd be great if when the main of provider passes the test with the feature branch of the consumer the consumer can-i-deploy would be re-run by a webhook or something like that
antal.tettinger
2023-12-07 20:53
Does WIP means that the main branch of the consumer can be mismatched while the main branch of provider matches the feature branch of the consumer?
parmin.rock
2023-12-12 06:25
hi all, was having some errors in re-publishing the same pact to Pactflow. If we publish pact with some name for the first time then it gets published.
But if we again publish the pact with same name then it gives error with a 403 Forbidden Error. We have checked and are using the Read/Write Token
matt.fellows
2023-12-12 12:16
Can you please clarify the calls you are making to Pactflow? Ideally, the arguments (with sensitive info redacted) so we can help?
If you can try it with the verbose flag and share the request ID we can lookup the details for you
ilie.cotiuga
2023-12-12 14:53
Hello,
I've started to lean pact and pactflow and got to the point to mock a github runner on a repo that contains both and I'm having just one question. You know that pactflow shows the tags, environment and branch of consumer and provider but they are missing or wrong.
Where exactly does them supposed to change? Or maybe I haven't completely done the setup for it?
The verify and can i deply commands works but it doesnt show the branch, env and tags.
antal.tettinger
2023-12-12 15:33
Hello! how do you Pact test an endpoint that returns a file download to say
matt.fellows
2023-12-13 02:39
*Consumer side*
Tags/branches are adding as part of the `pact-broker publish` command. See https://docs.pact.io/implementation_guides/cli#pact-broker-cli
*Provider side*
Tags/branches are added during the verification step (you would usually use a language SDK for this)
*Environments*
Deployment to an environment is what records this information. You would use the `pact-broker record-deployment` (or `record-release`) command for this. See https://docs.pact.io/pact_broker/recording_deployments_and_releases
jowhalen13
2023-12-18 16:20
Hey all, any ideas on how to get github API status check webhooks to work with a monorepo? Our issue is we want to conditionally fail a PR when pact verification fails, but only for the service that is being changed. We have well over a dozen different services, so we can't put a blanket "you can't merge this PR until the pact status comes back successful" block. Thoughts?
jowhalen13
2023-12-18 16:47
actually the monorepo isn't the issue, I'm just overcomplicating how webhooks work in my head. I just need to implement `can-i-deploy` tagging environments and checking in the PR and that should work.
My only question now is how does `can-i-deploy` work in CI when it needs to wait for the separate pact verification job to run, which takes 5 to 8 minutes?
jowhalen13
2023-12-18 16:48
since the original consumer CI job has no knowledge of the separate provider verification CI job
ulises.cervino
2023-12-18 19:20
there?s a flag in can-i-deploy to keep polling + retries for however long while waiting for results IIRC
jowhalen13
2023-12-18 19:47
or would I use can-i-merge here since it's for a PR? I don't see any docs for the can i merge api here https://github.com/pact-foundation/pact-ruby-standalone/releases/tag/v2.0.4
ulises.cervino
2023-12-18 19:47
can-i-merge? never seen that one. Only familiar with can-i-deploy, which for our purposes is what we want since 1 merge = 1 rollout
jowhalen13
2023-12-18 19:50
doesn't use environments since I want to check verification results before merging instead of deploying to an environment
ulises.cervino
2023-12-18 19:50
right, so it?s equivalent to 1 run of can-i-deploy / environment? almost as if you could do something like `--environment *`?
ulises.cervino
2023-12-18 19:51
we just have N pipelines, each checking each environment (we have a handful) and we?re done with it
jowhalen13
2023-12-18 19:54
I would be able to say more after playing with `can-i-merge`, but I can't read the entire help doc in the cli and it's not on the link I sent either. How do you expand this message?
mike.key
2023-12-19 21:51
Having issues logging into http://pactflow.io with SSO connection....have a ticket open but haven't heard anything back since this morning....is this a better place to get some assistance? Case # 00587732
matt.fellows
2023-12-19 22:58
I?ve re-enabled GitHub auth for your tenant now, I?ll follow up the ticket to find out _why_ it was disabled
matt.fellows
2023-12-19 23:20
Thanks Tyler, my sincere apologies for this. It looks like an issue applying updates when your account was updated on the 18th December.
rebecca.fulton
2023-12-21 16:00
I'm having issue with our Pactflow comparisons suddenly being incredibly slow. I'm currently at 16 mins and counting for the latest one. Anyone else having this issue? Any suggestions of how to resolve it?
matt.fellows
2023-12-21 22:06
When you say comparisons, are you referring to BDCT or provider verification?
rebecca.fulton
2023-12-22 09:22
Sorry I'm not really sure what either of those terms refers to. It's the Pact broker can-i-deploy check that compares the consumer and provider specs to see if they're compatible. We've had a lot of failures of this because it says the cross contract comparison is still being executed.
matt.fellows
2023-12-22 10:04
Ok then you are using bi directional contract testing (or BDCT for short, sorry)
matt.fellows
2023-12-22 10:11
It can be due to a few things. The complexity of the comparison (although usually each comparison is sub minute). If you have a lot of branches that can also contribute. One option is to delete old branches (e.g. after merging)
rebecca.fulton
2023-12-22 10:23
Hi @matt.fellows when I try the request from that API I get this error `You need to send the SwaggerHub Portal cookie with this request`. How do I resolve this?
bgrgincic
2023-12-29 08:49
Hi, there is a bit contradicting information whether Pactflow supports bidirectional contract testing for GraphQL. On this page https://docs.pactflow.io/docs/bi-directional-contract-testing/#comparison-to-pact GraphQL is mentioned under Bi-Directional column, on the other hand there is an open Github Issue on the PactFlow roadmap which suggests this support does not yet exists https://github.com/pactflow/roadmap/issues/53.
Could you please provide some insights into the current status of bi-directional GraphQL contract testing in Pactflow?
matt.fellows
2023-12-29 21:34
I can see why that's confusing, but it does not support graphql at this stage
kai.moseley
2024-01-04 13:45
Hi y'all! I'm currently encountering the errror `The cross contract comparison between the pact for version ... is currently being executed ...` error for Can I deploy (we have a whole bunch of retries over the span of 10 minutes), but it seems like the comparison is just never happening even after an hour or two (i.e. it's not a failure, just hasn't ran at all and has an unverified status). Does anyone have any tips on how to investigate why the comparisons aren't running?
If this is the wrong channel, just let me know :smile:
prakhar.roy371
2024-01-05 12:35
Hi all,
Wanted to check one scenario.
Let's say we have a consumer and a provider.
Now provider sends this payload
```{
"A": "XYZ1",
"B": "XYZ2",
"C": "XYZ3",
}```
The consumer however consumes only this :
```{
"A": "XYZ1",
"B": "XYZ2"
}```
The pact still passes as consumer's needs are met. (It got A and B)
In my project. Both team were initially using A and B in their payload.
A new element C needs to be added.
I am from the provider side. When I add element C in the provider side, tests still pass. And the CONTRACT on pact-broker is not showing the updated contract (with C element).
This has created a dependency for consumer to complete its implementation for Provider side story to be marked "COMPLETED" (after adding green test cases).
Can this be avoided? Or is this the intended behavior?
ratnajenisha
2024-01-05 18:03
Hey there, I want to try the trial version of packflow. Yet on the sign up page, the page keeps loading and doesnt create a pactflow account. Could somebody help?
matt.fellows
2024-01-07 22:42
I reviewed what I believe to be your registration, it looks like a bunch of the form fields are incomplete. If you have an ad blocker or other browser extension it may interfere with the signup form
matt.fellows
2024-01-07 23:06
> This has created a dependency for consumer to complete its implementation for Provider side story to be marked ?COMPLETED? (after adding green test cases).
why does the provider side story include waiting for the consumer contract to be updated?
matt.fellows
2024-01-07 23:07
> This has created a dependency for consumer to complete its implementation for Provider side story to be marked ?COMPLETED? (after adding green test cases).
>
> Can this be avoided? Or is this the intended behavior?
this is the intended behaviour. Consider that some consumers may not need the additional field, by forcing it into the contract now you don?t know what consumers actually use/need from the provider
matt.fellows
2024-01-08 01:59
After investigation, some of the comparisons are queueing up much longer than anticipated. We will be pushing out a change in the next iteration to
1. Increase the number of workers processing the comparisons
2. Reducing the number of (recent) branches we ?pre-calculate? results for
This just means any branches not in the pre-calculated list will be determined on-demand in the `can-i-deploy` call. In most cases, it shouldn?t be an issue
matt.fellows
2024-01-08 02:07
After investigation, some of the comparisons are queueing up much longer than anticipated. We will be pushing out a change in the next iteration to
1. Increase the number of workers processing the comparisons
2. Reducing the number of (recent) branches we ?pre-calculate? results for
This just means any branches not in the pre-calculated list will be determined on-demand in the `can-i-deploy` call. In most cases, it shouldn?t be an issue
ratnajenisha
2024-01-08 10:14
I made sure I filled all required fields and I tried in incognito mode as well. Still the same.
@matt.fellows
vinicius.gabriel
2024-01-08 20:31
Hi there, whats is the key benefits regarding using Pactflow Bi-directional test versus using something like Specmatic (which do simple Swagger validations into provider and validate swagger diffs)? I'm a bit confused about what really benefit me when I write code-based consumer tests to have a pact file if I can just use a provider Stub based on a OAS to test my consumer?
matt.fellows
2024-01-08 22:14
Specmatic is about checking your API provider doesn?t drift from the API specification. That is helpful, but it doesn?t tell you what your consumers are doing with your API. PactFlow?s tools are aimed at giving you that information also
ankush.hasani
2024-01-10 12:13
@matt.fellows Is there any update on this? We`ve bought the paid version since months but unable to achieve anything fruitful until now. Fast actions would be appreciated!
christopher.tonog
2024-01-10 14:58
Hi, we're evaluating Pact (and Pactflow) to see if it can fit into our org's workflow. One thing that might hinder adoption is the need to write the consumer tests using Pact tooling/DSL. I was going through the workshop https://docs.pactflow.io/docs/bi-directional-contract-testing/consumer/#writing-consumer-contracts and it looks like as long as the pact json is properly generated/serialized and pushed to the broker, it doesn't really matter how that file is created (but it's mostly created from mock files). Is this a fair statement? I'm part of a platform team and don't have much context into service's unit/functional tests, so it's been a little hard to wrap my mind around how we'll go from just unit/functional tests to a generated Pact json. Thank you!
ulises.cervino
2024-01-10 15:24
the framework is in charge of exactly that: you configure your mock + write a unit test -> the json is generated
matt.fellows
2024-01-10 21:55
As Ulises said, if you use Pact (e.g. a client language framework like Pact JS), it is used in unit tests and the contract file is generated automatically.
The idea of the BDCT flow you linked above, is that you can bring your own mocks, so long as you can convert those mocks to a pact format. That would enable you to use mocks you?re comfortable with, but you would need to create a serialiser. For example, customers that use MSW may use https://github.com/pactflow/pact-msw-adapter to do that.
matt.fellows
2024-01-11 05:24
Surry Ankush, it looks like some of the history has been removed here. do you recall the customer care ticket number associated with this? I can then check to see. I believe it was fixed
cyrus.devnomad
2024-01-11 09:06
Thank you for your additional input. Our system has been stable after adjusting the retry parameters so far. If we encounter those errors again, I will try your latest suggestions.
matt.fellows
2024-01-11 09:11
We are also releasing changes this week that should speed up this process
christopher.tonog
2024-01-11 14:06
Thank you, I must've missed the answer. The idea would be to not have to use the client language framework, and just be able to generate the pact file (but use the cli tools to push it to the broker). The services are in JS, and I'm still working my head around what those services use for mocking data, or even if stubs are used instead. I think we want to avoid having to subject those teams to having to write/maintain new tests in addition to the unit/integration tests they already have, so if there was a way to say to teams "use your existing data/process, our team will provide the tooling to get that data into a pact format and handle pushing it to the broker" then I think it'll go a long way in adoption.
ulises.cervino
2024-01-11 14:13
I can fully understand the sentiment, we have considered this ourselves and sort of went the other direction: these are ?different? tests, and so they should be written accordingly. Let me give you an example, we have apps which use a client to talk to a service. This service has 2 API versions, V1 and V2. This app has a thin wrapper around the client to make the app code nicer to read, maintain, all of the good stuff that encapsulation brings. At the unit/stub/mock/semi-integ testing this thin wrapper is a mock. At the CT level this thin wrapper actually makes requests to the pact mock server. These two kinds of tests serve two very different kinds of purposes. And so they exist (both kinds) and are maintained.
milda.abromaviciute
2024-01-16 12:38
hey, I'm making a custom tool to convert wiremock json to pact contracts. When the generated contract is uploaded, I have stumbled onto an error message in the contract comparison tab: `requestQuery[queryName].join is not a function` . The consumer contract tab is showing the expected requests/responses with matchers, there is also an existing open api spec to compare against. any help much appreciated
yousafn
2024-01-16 17:15
Is it a valid contract? Can you provide a sample?
There is a Pact specification schema that has been developed, which you can use to validate against
https://github.com/pactflow/pact-schemas
matt.fellows
2024-01-16 21:05
There are two existing wiremock Pact serialisers:
? https://docs.pactflow.io/docs/bi-directional-contract-testing/tools/wiremock/
? https://docs.pactflow.io/docs/bi-directional-contract-testing/tools/wiremock-net
I?m assuming you?re not using the Java/.NET versions of wiremock?
matt.fellows
2024-01-16 21:06
or is it more that you want to convert the wiremock JSON (rather than from execution)?
milda.abromaviciute
2024-01-17 09:44
hi Matt, thanks for the responses. Yes we have multiple languages used in the company so can't rely on Java (don't use .NET at all). Also the Java implementation is quite outdated and does not allow for easy support for multiple providers per consumer, without spinning up multiple wiremock servers.
We are also using the output from wiremock post execution as you mentioned, where we'll have all the data / metadata present that we need to generate multiple contract files.
I have used pact v4 to create those contracts. Here's the query part of the request:
``` "query": {
"amount": "100",
"buy_currency": "USD",
"sell_currency": "GBP",
"fixed_side": "sell",
"delivery_date": "2023-11-29"
},
"body": {
"content": "",
"contentType": "application/json",
"encoded": false,
"contentTypeHint": "TEXT"
},
"matchingRules": {
"query": {
"amount": {
"matchers": [
{
"match": "type"
}
]
},
"buy_currency": {
"matchers": [
{
"match": "type"
}
]
},
"sell_currency": {
"matchers": [
{
"match": "type"
}
]
},
"fixed_side": {
"matchers": [
{
"match": "type"
}
]
},
"delivery_date": {
"matchers": [
{
"match": "type"
}
]
}
},
"body": {
"$": {
"combine": "AND",
"matchers": [
{
"match": "type"
}
]
}
}```
milda.abromaviciute
2024-01-17 10:00
using a schema validator, the full file validates against v4 without errors :thinking:
boer.k
2024-01-17 13:21
Hi! We are trying out writing contract tests for our asynchronous message (kafka in our case) following guide: https://docs.pact.io/recipes/kafka. The provider in this case is an existing pacticipant for other services, which are being tested bi-directional. So the provider has uploaded an openapi spec. However, this spec is only defines http request/response definitions, not asynchronous kafka messages. So when I uploaded the asynchronous consumer contract, I expected it to ignore the openapi spec completely, and wait for a provider test to provide the verification result. However, the broker immediatly says our consumer contract is verified as success. ...But how? Based on what? When I delete both the consumer and provider, and then upload the consumer contract, then it will correctly say unverified. But when I upload the provider contract again, which doesnt contain any information regarding asynchronous interfaces, the broker says the consumer contract is verified. Can someone help me understand this behavior? Do I need to provide additional information?
matt.fellows
2024-01-17 21:28
Interesting use case. Any chance you could share the whole file? I can then try to determine the problem
matt.fellows
2024-01-17 21:30
This is one of those edge cases, unfortunately. We have an internal tracking issue for it. Basically, the comparison between the OpenAPI and the pact file ignores non-HTTP requests. So when the consumer contract has only async interactions in it, it is effectively an ?empty? pact and so is validated when compared to the OpenAPI file.
juliemarierosok
2024-01-18 11:12
Hi #pactflow. My project has newly introduced pactflow, and we are trying to figure out naming conventions etc, to make the integration work for all the different products and teams (it is a large project with many different products and teams within those products, as well as different service providers - so not always an easy task). A suggestion for the naming convention for the provider name in the pacts, was to attatch the stage (dev/int/prod etc) as part of the name, to make it possible to have different versions of a pact on different stages. I have a feeling that this is possible to do without adding it to the provider name, it is also available as a tag, and I think it must somehow be a part of your versioning concept. So my question: does pactflow offer a way to handle different versions of pacts on different stages?
matt.fellows
2024-01-18 11:18
> have a feeling that this is possible to do without adding it to the provider name,
Yep! I?d suggest not using stage names in the provider name, because that?s the whole point of https://docs.pact.io/pact_broker/recording_deployments_and_releases
juliemarierosok
2024-01-18 12:23
Thank you Matt. I will suggest we use this approach :slightly_smiling_face:
juliemarierosok
2024-01-18 14:06
One follow up question, is it only possible to record deployments using the pact broker CLI? We are currently using the maven plugin...
matt.fellows
2024-01-18 22:18
https://smartbear.portal.swaggerhub.com/pactflow/default/getting-started (It?s the PactFlow API, but deployments are the same as the open source)
jaspal.puri
2024-01-22 09:16
Is BDCT for GraphQL realistically on the roadmap at all (I know there's an open issue for it but it's a number of years old now)?
anna.nava
2024-01-23 11:02
:blobwave: How can I adjust in my `Provider Service` 's gitlab job, or pactVerify selectors (or other alternatives?),
to run the pactVerify for all the past consumer versions (released) and publish results.... but allow it to fail on the released....
PLUS ONLY really prevent my MR to be merged (ie: not allowed to fail) if and only if the `pactVerify`failed for `latest_from_main_branch`or deployed. (but ignore if it failed a past released version of the consumer)
*CONTEXT*
I have this scenario
? a `Consumer Service` publishing a *pact* when it's merged to it's main branch, and this pact is ready in my pact broker to be verified with `Provider Service`
? then I have the `Provider Service` gitlab pipeline, to verify against that *pact* from `Consumer Service` with selectors `latest_from_main_branch` + `released_or_deloyed`
? Also, I am marking each and all versions of the consumer as released (and only one as deployed)
If my `Provider Service` 's `pactVerify` finds 10 released versions (with of them been the latest + deployed) it would run 10 times it's verification and publish those 10 results...
(I am still trying to test if the `pactVerify` fails only 1/10 (eg the very first version of the consumer) would the total result be `PASSED` or `FAILED )`
yousafn
2024-01-23 11:46
you only need to record a deployment or a release. not both.
a deployment, removes existing records to deployed contracts, and replaces with the version now deployed
a release, will mark that version as active, and all other versions remain active.
do you actually multiple versions of your consumer deployed at one time?
anna.nava
2024-01-23 11:59
yes, you are right and I understand the difference between both (release or deployed) but my question remains the same:
I want my `pactVerify` task to run against all the consumer's released versions (and publish the results so they are available later on for me to use `can I deploy`
but my provider's pipeline ONLY to prevent to be merged if only the latest fails (but not to prevent the merge if an old version of the consumer failed)
anna.nava
2024-01-23 12:10
... I guess my question boils down to what I am still trying to test:
if the `pactVerify` from the provider's gitlab's pipeline fails 1/10 released consumer versions (eg the very first version of the consumer, but passes all the other 9 ones, including the latest) ... would the total result be `PASSED` or `FAILED` :question: :thinking:
anna.nava
2024-01-23 12:45
I think my best option is to simply run pactVerify against all release consumer versions... and on my open MR's pipeline allow this task to fail (so the purpose is only to publish the results of ALL verifications)
and then on a further step, run a `can I merge` which only checks against the latest version, which would fail only if the verification against the latest consumer fails, and passes otherwise
(and then of course, later on I will run `can I deploy` on each of the pipelines that actually deploy any given version of the producer and the provider)
marek19922209
2024-01-23 14:08
@jowhalen13 did you have any problem with mounting volume using docker cli approach ?
marek19922209
2024-01-23 14:10
docker run --rm \
-w $(Build.SourcesDirectory)/$(bffFolder)/src/pacts \
-v $(Build.SourcesDirectory)/$(bffFolder)/src/pacts:$(Build.SourcesDirectory)/$(bffFolder)/src/pacts \
-e PACT_BROKER_BASE_URL="https://orbus.pactflow.iorl" \
-e PACT_BROKER_TOKEN="token" \
pactfoundation/pact-cli:latest \
publish \
$(Build.SourcesDirectory)/$(bffFolder)/src/pacts/pact.json --consumer-app-version=$(version.packageSemVer)
this command is nor working for me, saying that file does not have permissions..
jowhalen13
2024-01-23 14:33
If CI doesn't have write permissions, you need to give it permission to write to that file or folder. I don't know how to configure that in Azure, but I'm sure they have documentation for that.
As for docker volumes, I was using docker compose and put my API as one service, and the pact cli image as another service. Then in the highest level of the docker compose file, you want to define a shared volume between the two like so:
volumes:
my-shared-volume:
Then define 'my-shared-volume' in each services volumes key
matt.fellows
2024-01-24 06:19
> if the `pactVerify` from the provider?s gitlab?s pipeline fails 1/10 released consumer versions (eg the very first version of the consumer, but passes all the other 9 ones, including the latest) ... would the total result be `PASSED` or `FAILED`
the result would be `FAILED` , unless the pacts are pending (http://docs.pact.io/pending).
matt.fellows
2024-01-24 06:20
To achieve what you want there, I would just run multiple verification jobs with the various selectors, and then fail/not-fail the step based on the behaviour you are after
rajat.chouhan
2024-01-25 04:57
@matt.fellows
Case #00560370
Still getting the same issue. Raised concern to the tool team. They are verifying.
matt.fellows
2024-01-29 00:10
Looks like there is still a bug there somewhere, i?ve passed it on to the dev team for review. Apologies for the hassle!
christopher.tonog
2024-01-30 03:04
Sorry for the delay in response, but good points to consider. Thank you!
jyothi.kaspa
2024-02-01 16:14
Hi Team,
We have the contract tests integrated in PR pipelines and lately we noticed that even though our contract tests are failing, the pipeline shows the status as Green.
Can you please suggest what could be going on here?
matt.fellows
2024-02-02 00:31
see http://docs.pact.io/pending (you obviously have that feature enabled)
sonal3062000
2024-02-05 09:56
Trying to publish pact verification result , Using this doc: https://docs.pact.io/pact_broker/advanced_topics/api_docs/publish_verification_result. We wanted to know which API we have to use from HAL browser.
danut.turta
2024-02-05 12:03
hello! A small question about `can-i-deploy`: I have a provider with 3 consumers, which regularly deploy new versions to an environment called "production". Since we are releasing a few times per hour, we just use as versions the commit sha, but this creates a problem with `can-i-deploy` for the provider. We would like to tell it to verify always with the latest version of the consumers on that environment, but based on the documentation it doesn't seem to be possible. The only thing I can thing of is to use a tag and tell the provider to always check with the latest of that tag, however, I don't want us to specify the consumers, in case others will appear and we will miss adding them there. So in short, is there a way to specify to `can-i-deploy` to check with the latest versions of the consumers from an environment?
matt.fellows
2024-02-05 12:04
Yes, you use the `record-deployment` tool for your consumers, and specify `--to-environment` in your `can-i-deploy` command calls (on both consumer and provider CI pipelines)
danut.turta
2024-02-05 12:05
Yes, that's how we are doing the current verification, but it still checks with all of the releases ever recorded there rather than the latest one
matt.fellows
2024-02-05 12:06
a deployment will override the previous deployment. If you use `record-release` that will verify all consumers currently ?released? (e.g. multiple versions of a mobile app). When that version is decommissioned, you need to unrelease it
matt.fellows
2024-02-05 12:07
Can you please share the command you are running and the can-i-deploy output? Perhaps you can redact / change the names
danut.turta
2024-02-05 12:13
yes, the command for can-i-deploy is:
```pact pact-broker can-i-deploy
--pacticipant xhr-api \
--version ${CI_COMMIT_SHA} \
--to-environment "production"```
and the output looks like this:
danut.turta
2024-02-05 12:14
I think we need to unrelease them indeed, but since the version is based on the commit sha, it's not easy to target them for unreleasing
matt.fellows
2024-02-05 12:16
Do you actually run multiple applications in production? Or are you inadvertently using `record-release` instead of `record-deployment`?
danut.turta
2024-02-05 12:19
hmm yes, we used record-release instead of deployment. I mistakenly thought that record-release should be used as the final "release" step of a contract, but now I am re-reading and seeing my mistake
danut.turta
2024-02-05 12:20
I will switch to record-deployment instead, since that one looks to be the one we actually need
filip.olszewski
2024-02-06 12:43
Hello! :wave:
My team is working on a proof of concept of BiDirectional Contract Testing using PactFlow. Our aim is to assess its possibilities / limitations and we?ve stumbled across a pretty big limitation for us. https://github.com/pactflow/bdct-oas-examples/tree/main/examples/inheritance#use-of-discriminator for allowed usage of the `discriminator` keyword in the provider?s OpenAPI, one of which is that the use of `discriminatorMapping` is not supported. Using such mapping is our preferred approach to polymorphism, based not only on the default behaviour of our openAPI generation library, but also on some internal tools working with OpenAPIs. If I understood the docs correctly, https://github.com/pactflow/bdct-oas-examples/blob/main/examples/inheritance/inheritance.oas.yml, in all subschemas, the discriminator field as required const of given value.
? Is the support for `discriminatorMapping` ?on the roadmap?, or should I interpret it as a by-design limitation?
? In the PactFlow?s suggested approach, can the const values be different than the schema names? https://github.com/pactflow/bdct-oas-examples/blob/main/examples/inheritance/inheritance.oas.yml the value is `Dog`, just like the schema - could it be `ANIMAL_DOG` for example? Or maybe in general - does PactFlow support discriminator values that are different from the schema names?
matt.fellows
2024-02-06 22:54
There is a public https://github.com/pactflow/swagger-mock-validator/issues/51 to go with our internal feature request for this problem.
_The TLDR of it: `avj` is the library we use to parse the JSON schema. It currently does not support the https://swagger.io/docs/specification/data-models/inheritance-and-polymorphism/ keyword, making it cumbersome for users who follow this style._
I understand the frustration. When we implemented the OAS 3.1 upgrade earlier this year, the scenario came up and we had investigated several paths to address it. The biggest blocker we have is lack of support in a validation library we use (`ajv` ) which narrows the possible solutions to fixing this to dodgey workarounds.
There are a few issues and this https://github.com/ajv-validator/ajv/pull/2262/files#diff-dfb4dfa85df2a197a315b3a399455b51978dafc7077850f10690d851368fe08e to attempt it. There was a response from the maintainer of ajv the recently essentially brushing aside https://github.com/ajv-validator/ajv/issues/2349#issuecomment-1899540208, from what I can see, and ignoring the bit about us happy to create a PR if they are open to it - presumably they are not. I have https://github.com/ajv-validator/ajv/issues/2349#issuecomment-1899540208 because I wasn?t entirely satisfied with the reasoning. This being said, there may be some light at the end of the tunnel.
It looks like an uphill battle though getting a change into ajv, so I think plan B (or beyond) is needed.
I?ve done a small spike locally, there are a couple of options that _might_ work for you.
1. Remove the `mapping` (or indeed, the discriminator) prior to uploading to PactFlow
2. Remove the `mapping` but modify the components/references in the `oneOf` clauses to match the _implicit_ discriminator (converts an explicit mapping to an implicit mapping)
*Option 1*
This is fairly straightforward, and involves just walking the OAS tree and deleting any `mapping` you find, and then writing the file back out.
I need to review why this works, but my guess is that where there is _no ambiguity_ in the types, the validator is clever enough to tease apart the differences. And even with the discriminator, mapping is not needed because the validator can see the types.
_My ask to you - if you could see if this works in your case, we could look to add that onto our backlog to update the core library here: https://github.com/pactflow/swagger-mock-validator/ (you could also submit a PR, as there is already a visitor pattern in that library that walks the OAS tree and manipulates it)_
*Option 2*
i.e. given the following response schema:
``` schema:
oneOf:
- $ref: '#/components/schemas/CatObject'
- $ref: '#/components/schemas/DogObject'
discriminator:
propertyName: petType
mapping:
Cat: '#/components/schemas/CatObject'
Dog: '#/components/schemas/DogObject'```
This translation would be converted to:
``` schema:
oneOf:
- $ref: '#/components/schemas/Cat'
- $ref: '#/components/schemas/Dog'
discriminator:
propertyName: petType```
(and the `Cat` and `Dog` schemes would be moved to `components`).
The `mapping` is no longer needed, because the implicit schema aligns.
I started testing (2) first actually, but realised that in many cases the mapping isn?t actually needed, so didn?t complete this. But I think you get the idea. The library used below would be easy enough to do this manipulation, I think
matt.fellows
2024-02-06 22:57
Here is the JS spike I whipped together:
```const SwaggerParser = require("@apidevtools/swagger-parser");
const fs = require("fs");
(async () => {
try {
// 1. Parse the document
const api = await SwaggerParser.validate("./inheritance.oas.yml");
console.log("Rewriting API name: %s, Version: %s", api.info.title, api.info.version);
// 2. Iterate the top level endpoints in the API
Object.keys(api.paths).forEach((path) => {
const methods = [ "get", "post", "put", "patch", "delete", "head", "options" ];
methods.forEach((method) => {
// TODO: look at request bodies
// TODO: handle $ref and not just inline schemas
// 3. iterate the combinations of resources
Object.keys(api.paths[path]?.[method]?.responses || {}).forEach(
(status) => {
const response = api.paths[path]?.[method]?.responses[status];
Object.keys(response.content).forEach((mediaType) => {
if (mediaType) {
delete response.content[mediaType]?.schema?.discriminator?.mapping;
}
});
}
);
});
});
// 4. Write out the new file
const output = await SwaggerParser.bundle(api);
fs.writeFileSync("inheritance.oas.json", JSON.stringify(output));
} catch (err) {
console.error(err);
}
})();```
And some test files:
```npm init -f
npm i @apidevtools/swagger-parser
node index.js // this should write out the inheritance.oas.json file```
To test it out:
```# original OAS with mapping, should fail
npx @pactflow/swagger-mock-validator@latest inheritance.oas.yml inheritance.pact.json
# updated file, should pass
npx @pactflow/swagger-mock-validator@latest inheritance.oas.json inheritance.pact.json ```
filip.olszewski
2024-02-07 10:32
Hi Matt, thanks a lot for detailed answer. I?ll check all the info / links / code you?ve provided and get back to you.
ryan.quinn
2024-02-07 19:13
I'm working on using the providerVersionBranch and providerVersionNumber for webhooks, so we can validate all deployed provider environments when a consumer updates a contract. The issue I'm having is that in gitlab, I can't check out a commit, I need to have either a branch or tag name. When recording deployments, it seems that we are only recording the version number (which I think is a commit). It is unclear to me how to associate a branch with that version number.
matt.fellows
2024-02-08 00:03
> in gitlab, I can?t check out a commit
really? is that a constraint of GitLab itself, or just the way it?s configured? Is that because GitLab doesn?t do a clone of the repository?
Presumably your CI build itself could do a `git checkout <commit>` even if GitLab won?t do it automatically?
matt.fellows
2024-02-08 00:04
I don?t think we retain a mapping of versions to branches you can query, but I?ll check
ryan.quinn
2024-02-08 00:10
I tried `git checkout <commit>`, and `git fetch` and then `git checkout <commit>` and got an error (only in some cases, such as the commit not being on the main branch), even though the same thing worked locally. I will investigate further. I saw https://stackoverflow.com/questions/63454536/build-manual-gitlabci-pipeline-job-using-specific-commit-id on stackoverflow which referenced this open https://gitlab.com/gitlab-org/gitlab/-/issues/22886.
ryan.quinn
2024-02-08 03:04
I saw this comment on https://gitlab.com/gitlab-org/gitlab/-/issues/22886:
> This feature would be very useful for integrating with Pact, where for full Pact integration, you need to be able to re-run a GitLab CI/CD job for a specific commit hash that Pact provides, via a webhook. I'm wondering if a reasonable workaround would be to create a tag with the commit hash as part of the pipeline when it first runs, then Pact could re-trigger the job using that tag as the ref?
matt.fellows
2024-02-08 04:35
Another approach, if it?s useful, is to not just use _only_ the commit SHA as the version identifier, but include other bits - such as the tag/branch (e.g. a https://semver.org/ compatible thing. This https://www.npmjs.com/package/absolute-version was written by an ex Pact Maintainer you might find helpful). Then in the build job, you could destructure the version into its parts?
anna.nava
2024-02-08 09:45
what I did in the end was run the pactVerify against all versions in branch main (but allow this step to fail the pipeline, as the aim is simply historical breaking changes)
and setup the blocker for merging with `can-i-merge` checking against the latest version of pacts from all consumers
and setup the blocker for deploying with `can-i-deploy` which would check the deployed version of all consumers of this producer in that given environment
matt.fellows
2024-02-08 09:58
I mean what's the use case? What are you trying to do that you can't achieve with pact client libraries now?
sonal3062000
2024-02-08 10:04
o inside Java test we use the "http://au.com.dius.pact.provider.junit5" library and use annotations:_@Provider("TAX")@PactBroker(url = "url")_
This leads to "403 Forbidden" during publishing verification results using our token.
Trying manualy to publish the result via Pactflow API we tried using this url as POST:API
and this body:
_{ "success": true, "providerApplicationVersion": "0.0.1" }_
We set our Pactflow read-write Token as Bearer header. This results in the response:
_Forbidden. Either you are using a read only token for a request that requires a write token (the most likely cause), or you do not have the required permissions._
filip.olszewski
2024-02-08 10:23
Hi Matt. The complexity I have not mentioned on purpose makes this situation a bit more complicated for us. The root of all evil lays here:
https://bitbucket.org/atlassian/swagger-request-validator/issues/771/discriminator-is-not-working-with-oneof
We use this library and to my big surprise, I?ve found this issue. basically they are not supporting `oneOf` + `discriminator` - which is the default approach to polymorphism according to OpenAPI spec.. But hope is there - a PR was suggested 4 days ago that could introduce this much needed support.
As a workaround, we are currently using a weird approach, where we $ref to the base class schema. Anonymised, simplified example:
```"items": {
"type": "array",
"items": {
"$ref": "#/components/schemas/BaseItem"
}
},```
And we define discriminator inside BaseItem schema instead:
```"BaseItem": {
"required": [
"itemType"
],
"itemType": "object",
"properties": {
"itemType": {
"type": "string",
"enum": [
"T1",
"T2",
"T3"
]
}
},
"discriminator": {
"propertyName": "itemType",
"mapping": {
"T1": "#/components/schemas/Item1",
"T2": "#/components/schemas/Item2",
"T3": "#/components/schemas/Item3"
}
}
}```
This approach proven to work with the validator, but is also not ?liked? by other tools - and I have no idea how PactFlow would react to this if we removed the `discriminatorMapping` .
For now, the validator issue is our main blocker. When it?s resolved (luckily looks like it gained traction), I will come back to the topic, go back to `oneOf` approach and try your suggestion to run a script and preprocess the document before sending to PactFlow - it looks promising :crossed_fingers:
anna.nava
2024-02-08 10:36
btw thanks a lot for replying to all these threads, very appreciated to have help from someone who knows pact.... when it's your first time and no one in your company knows pact it can get pretty lonely not having someone to bounce ideas / challenge you back with good practices, thanks a lot!
yousafn
2024-02-08 10:39
That is lovely feedback! I can empathise with how you feel, and glad that the Slack community is working for empowering you to get to Pact Nirvana. The more we can empower champions in orgs to love Pact and rave about it, the better.
Matt is incredibly knowledgable and a seal of a approval from him is like :chefkiss:
anna.nava
2024-02-08 10:42
`Pact Nirvana` I am going to quote this in the blogpost that I'll write at the end of our implementation, hope you dodn't mind?
yousafn
2024-02-08 10:48
No of course, it?s not my original quote. I do :heart: Nirvana and am a grunge kid, but the quote came from Beth
https://docs.pact.io/pact_nirvana
Awesome to hear you are writing a blog. You will have to let us know when its written and we can cross promote it via our Pact blog and get it listed on our community articles page
matt.fellows
2024-02-08 22:54
You?re most welcome Anna, we love hearing from the community and more than happy to provide that support.
Smells like teen spirit was one of the first songs I learned on my guitar too (well, just the riff :stuck_out_tongue: ) - so it was a good choice. Only later in life did I come to appreciate meditation and realise what the real ?nirvana? meant - one day we?ll all get there!
matt.fellows
2024-02-08 22:57
So we currently use a forked version of that same validator under the hood in BDCT. We took a few different design paths and decisions (e.g. proper support for oneOf and anyOf, as well as discriminator etc.).
You can actually try it directly here: https://github.com/pactflow/swagger-mock-validator/
We?ve noticed some significant performance issues as the OAS grows, and a team member has spiked improving it - it?s orders of magnitudes faster.
matt.fellows
2024-02-08 22:59
Perhaps you can share how you are setting the token for the java test so we can look? You could also set logs to debug and share the response (it should have a request id in the header). We can look it up to tell you what?s wrong. But the message is pretty clear, so I?m guessing either the wrong token was given to the Java setup, or the setup is incorrect and it?s not flowing into the tool
ryan.quinn
2024-02-09 03:39
I am not sure what the issue was, but as long as our commits are relatively recent from mainline, checking them out inside our gitlab pipeline seems to work fine.
matt.fellows
2024-02-09 04:27
Nice! A lot of CI tools do a shallow check-out, or don?t pull new commits. Sometimes a clean CI agent is all you need
rajat.chouhan
2024-02-12 09:16
Hello, I am getting below error 'no such library file found' while running my pact provider pipeline in gitlab. it may be from pact_broker_client related error as the version is upgraded in December last year. Before that it was working fine. I have updated packages and also done pod install but did not work out. Can anyone please help me here.
pipeline - https://code.siemens.com/horizon/api-manager/api-services-service/-/jobs/163219196
yousafn
2024-02-12 10:05
youve got a missing var $Env you can see the missin g value between openapi//points-v3.yml
matt.fellows
2024-02-12 10:14
It's not complaining about a missing library, it can't find a missing file
rajat.chouhan
2024-02-13 05:18
yes, there is env issue with this. I will setup this. Thanks for finding.
eran.bergman560
2024-02-13 21:35
Hi @matt.fellows I've looked at the https://youtu.be/a9K43CHSRM0?t=942 demo you ran and I'm trying to do a similar thing with our own code (mostly Spring Boot + Maven based) as a POC.
I noticed that you were using https://github.com/kevinswiber/postman2openapi (also mentioned in some https://docs.pactflow.io/docs/examples/bi-directional/provider/postman/) to convert Postman collections to a OAS that you published into PactFlow as a provider contract and I was wondering if it is equivalent to https://blog.postman.com/creating-an-openapi-definition-from-a-collection-with-the-postman-api/ for that purpose or are there any differences/tradeoffs to using either of those.
matt.fellows
2024-02-13 22:02
All of the usual caveats still apply of course - you need to be sure the postman collection was executed and passed, for example.
r_connolly
2024-02-15 20:30
:clap: Hi Everyone, we have several stubs that are working correctly, but one that is giving us trouble I suspect because of some character encoding.
Here's our Interaction:
Method:GET
Path:/api/OrderDetail/GetOrderEligibility/orderNumber/1000089929/buid/1401/IssueType/Have%20Defective%20Product/Issuesubtype/Dead%20On%20Arrival%20%2F%20No%20Boot%20%2F%20No%20Post%20%2F%20No%20Power/creType/Return/environment/STUB
But when we call:
https://dell.pactflow.io/pacts/provider/1003307_VNR_Provider_GetPolicyInfo/consumer/1003307_CNE_Consumer_GetPolicyInfo/latest/stub/api/OrderDetail/GetOrderEligibility/orderNumber/1000089929/buid/1401/IssueType/Have%20Defective%20Product/Issuesubtype/Dead%20On%20Arrival%20%2F%20No%20Boot%20%2F%20No%20Post%20%2F%20No%20Power/creType/Return/environment/STUB
We get this error:
HTTP/1.1 500 Internal Server Error
Date: Thu, 15 Feb 2024 20:19:56 GMT
Content-Type: application/json
Content-Length: 261
Connection: close
Vary: Origin
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
X-Pact-Broker-Git-Sha: 2bf2a16e
X-Pactflow-Git-Sha: 6cb1c822e
X-Pact-Broker-Version: 2.108.0
X-Request-Id: a0390735fcb1b442f6ad67d5037a60ca
{
"message": "No interaction found for GET /api/OrderDetail/GetOrderEligibility/orderNumber/1000089929/buid/1401/IssueType/Have Defective Product/Issuesubtype/Dead On Arrival / No Boot / No Post / No Power/creType/Return/environment/STUB",
"interaction_diffs": []
}
Any ideas?
matt.fellows
2024-02-16 05:44
Hi Ryan,
Mind sending a support ticket for this please? It will require us to check your account, which we can?t do without a ticket
r_connolly
2024-02-16 13:47
Sure. I'll need to connect to another dev here in Dell that has access to the ticket system.
2024-02-19 04:34
Please create a ticket here with the issue details so that we can properly track the resolution and the PactFlow product support team will continue to work through it with you: https://support.smartbear.com/pactflow/message
marek19922209
2024-02-19 15:02
Hello, im trying to create a webhook which will trigger my azure devops pipeline. I created a PAT token and encoded it.
I added it to secrets in pactflow and i created a system user (which i hope should be able to trigger it :slightly_smiling_face: )
In my webhook configuration im passing it like in documentation :
Authorization: "Basic ${devopsUser.AzureDevopsServicesPAT}" - where devopsUser is a name which i gave to system user, i was trying also with only user.
For some reasons when im looking into loggs there is no partcilar error and anythiung special beside this :
```[2024-02-19T15:01:08Z] INFO: x-tfs-serviceerror: TF400813%3A%20The%20user%20%27aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27%20is%20not%20authorized%20to%20access%20this%20resource.
[2024-02-19T15:01:08Z] INFO: x-vss-s2stargetservice: 00000002-0000-8888-8000-000000000000/visualstudio.com
[2024-02-19T15:01:08Z] INFO: request-context: appId=
[2024-02-19T15:01:08Z] INFO: access-control-expose-headers: Request-Context
[2024-02-19T15:01:08Z] INFO: x-content-type-options: nosniff
[2024-02-19T15:01:08Z] INFO: x-cache: CONFIG_NOCACHE
[2024-02-19T15:01:08Z] INFO: x-msedge-ref: Ref A: Ref B: SYD03EDGE1719 Ref C: 2024-02-19T15:01:07Z
[2024-02-19T15:01:08Z] INFO: date: Mon, 19 Feb 2024 15:01:08 GMT
[2024-02-19T15:01:08Z] INFO: <html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="https://spsproduks1.vssps.visualstudio.com/_signin?realm=dev.azure.com&reply_to=">here</a>.</h2>
</body></html>```
I was checking PAT token permissions and they should allow to access this resource.
Is there any working example of such webhook for azure or im missing something more ?
marek19922209
2024-02-19 15:13
now i have like this
Authorization: Basic ${devopsUser.AzureDevopsServicesPAT}
marek19922209
2024-02-19 15:14
now i see less meaningful
```-02-19T15:13:37Z] INFO: HTTP/1.1 401 Unauthorized```
marek19922209
2024-02-19 15:17
```[2024-02-19T15:16:30Z] INFO: HTTP/1.1 POST https://dev.azure.com/[]
[2024-02-19T15:16:30Z] INFO: accept: "application/json",
[2024-02-19T15:16:30Z] INFO: user-agent: Pact Broker v2.108.0
[2024-02-19T15:16:30Z] INFO: content-type: "application/json",
[2024-02-19T15:16:30Z] INFO: authorization: [REDACTED]
[2024-02-19T15:16:30Z] INFO: host: http://dev.azure.com
[2024-02-19T15:16:30Z] INFO: content-length: 0
[2024-02-19T15:16:30Z] INFO:
[2024-02-19T15:16:30Z] INFO: HTTP/1.1 401 Unauthorized```
marek19922209
2024-02-19 15:18
and btw can i use in secret this syntax :
${myUsername????.AzureDevopsServicesPAT}
dpunna
2024-02-19 18:38
hello team :wave:,
I was trying to configure the `consumer_version_selector` for the provider verification to run only for each consumer on `main` branch. I tried like below
`consumer_version_selectors [{tag: "message", latest: true}, {mainBranch: true}]` , however it doesnt like it. And I also noticed in the documentation you have mentioned https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors, `As of October 2021, this is not yet supported in all Pact client libraries.` what do you currently recommend for this case? Please suggest, documentation is not clear what to use correctly :pray:
yousafn
2024-02-19 19:17
> `consumer_version_selectors [{tag: "message", latest: true}, {mainBranch: true}]` , however it doesnt like it.
What error are your receiving?
> `As of October 2021, this is not yet supported in all Pact client libraries.` what do you currently recommend for this case? Please suggest, documentation is not clear what to use correctly
What client library, and what version are you using?
dpunna
2024-02-19 20:26
this is i found from `Gemfile.lock`
`pact (1.63.0), pact_broker-client (1.51.1)`
marek19922209
2024-02-20 09:38
Tbh worth to mention somewhere in documentation how to setup this from GUI
matt.fellows
2024-02-20 09:44
We're redoing this UI and I'll make sure it's covered.
Commas are valid in headers, but not in this use case
marek19922209
2024-02-20 09:50
sure, thanks, maybe also worth to mention that for azure PAT token needs to be base64 encoded first
dpunna
2024-02-20 10:19
We have added this line for the pact helper file `consumer_version_selectors [{tag: "message", latest: true}, {mainBranch: true}]` only difference was added `mainBranch:true` since we want to test against consumer main branch, right now it is verifying all the consumer opened branches instead `main` branch
dpunna
2024-02-20 10:21
I would like if the syntax is correct to use `consumer_version_selectors [{tag: "message", latest: true}, {mainBranch: true}]`
dpunna
2024-02-20 10:25
Trying to run the consumer verification for the `tag:message` against consumer `main` branch only. Looks like `tag:message` is not working anymore and now all the tests running against `main` including `tag:REST` after i added the `{mainBranch:true}` . Wondering if the syntax was wrong?
yousafn
2024-02-20 11:14
The syntax looks okay to me
https://github.com/pact-foundation/pact-ruby/blob/fc35041f55e887a2a6d040af31efe869ccb1d650/spec/lib/pact/pact_broker/pact_selection_description_spec.rb#L35
You are using pact-ruby 1.63.0 which is one version from the latest and supports consumer version selectors
You can test them directly against your browser, via the HAL browser, or API passing in the consumer version selectors, to see if that returns what you expect
dpunna
2024-02-20 11:57
Looks like it is somehow not using the tag: message when I gave the main branch:true
dpunna
2024-02-20 12:41
Some of our main branch also called ?master? , wondering if we give main branch:true, it will look the branch name value as main only? Some of our consumer main branch value is master. Wondering how can we separate them to run one for master and others are from main?