mcon
2018-12-12 21:39
Hi Matt, thanks for creating the channel for everyone to discuss protobufs in pact.
For those who haven't got any context, I've put together a quick issue and gist explaining my ideas of how we might be able to add this functionality.
https://github.com/pact-foundation/pact-specification/issues/62
mcon
2018-12-12 21:40
@rahul.neotech it sounds like you've been working on this idea too: do you have anything you're able to share at this point?
matt.fellows
2018-12-13 11:18
@mcon that gist is really great and thanks for the detail. It?s really positive to hear you?ve got a working prototype!
matt.fellows
2018-12-13 11:20
One concern/comment is that we?ve been trying to move away from having to implement protocol + transport specific concerns into the system, instead opting for finding a way to get the message to be exchanged in a consistent way. e.g. for message queues, rather than actually put messages onto, and retrieve messages from a queue, we simply have the function that generates the message (provider) invoked and capture it?s output, and on the consumer side we pass the message directly to the function. The benefit is that it bypasses all of these hassles, the downside is that we may lose specific aspects of a technology that may still be a ?contract? (e.g. message headers)
matt.fellows
2018-12-13 11:22
My second comment is around the approach to overload the existing http (or synchronous) contract file, rather than opt for a new contract file structure (see `messages` for the message pact type). I haven?t thought too much on it, but it could be a sticking point.
matt.fellows
2018-12-13 11:23
I can certainly understanad the decisions, and it might provide some form of basis to do other types of protocols.
mcon
2018-12-13 13:39
I'm not sure we're on the same page, based on your last messages: you're talking a lot about async communication, i.e. message queues and the like. I'm not sure how that's relevant here - likely you're making a generalization that I don't have the context to understand. Could you clarify what you're trying to get across there?
mcon
2018-12-13 13:57
Ultimately we're looking to use the same sync transport/protocol as is encoded in the Pact Specification (i.e. HTTP), but just with a different serialization format.
If you're saying that the blessed future approach will be to define `messages` along the lines of what's included in the JS version of pact, is there a reason that's not in the V4 specification? (I wouldn't be keen to implement in another language without a spec)
Ultimately we could use the new `messages` pattern, and replicate the request / response nature of HTTP, by mocking out our HTTP client, however I'm not keen on that because proto over HTTP is still a sync thing. One of the big advantages that exists with Pact, is that you can also test your code that calls out to HTTP without the concern you've not mocked it correctly.
https://github.com/pact-foundation/pact-js#asynchronous-api-testing
mcon
2018-12-13 13:59
Do you have any suggestions as to how best to go about adding this testing for protobuf?
matt.fellows
2018-12-17 22:55
Yes, making somewhat of a generalisation. I understand that Protobufs uses a synchronous pattern, however when we started to consider how to deal with messages, we realised that it would be an arms race to try and support every transport, wire protocol, every queue?
matt.fellows
2018-12-17 22:57
So we what we said was ?hey, all we _really_ care about, is the message that moves between the parties and if the consumer can handle it and the provider can produce it?. At this point, we removed the transport and had the framework invoke the functions directly, and asserted the result.
What this enables us to do, is test any payload, without having to implement the wire protocol.
As you note, what we gain in simplicity in the engine, we lose in fidelity of the overall test. It?s a tradeoff, but we felt that each protocol/framework usually has good ways of testing itself and that Pact could then focus in on the bit we do well
matt.fellows
2018-12-17 23:01
This being said, there is already significant req/res infrastructure that we can leverage, so adding a thin layer over it than enables us to encode new types of payloads could make a lot of sense
mcon
2019-01-28 20:56
Hi Matt (sorry for the delay between responses), I've taken a look at the pending protobuf-net support for Messages, and some of the support for the JS client - it looks like that approach works well for async operations, but I'm not completely sure how it would apply to sync operations without adding additional abstractions.
mcon
2019-01-28 21:12
I've been thinking a bit recently about trying the following approach for sync operations that are non-JSON.
I was thinking that it would be possible to add an intermediate layer between the language-specific client and the (currently Ruby) mock server which could translate from the given transport/encoding under test to HTTP/JSON which could be understood by the mock server.
I envisage this working something like:
- Client creates test which specifies the transport / data object is expected for a particular request/response of an API.
- That data is sent as part of the interaction POST, which is sent to an intermediate (language specific) server, which would then perform the appropriate translation to JSON, which is then passed to the standard pact mock server.
*client test* ---transport/encoding---> *Intermediate server* ---HTTP/JSON---> *standard Pact mock server*
- Any requests that would normally go to the standard mock server in the form HTTP/JSON, are sent to the intermediate server, which forwards them on to the standard mock server.
- When the standard mock server writes the standard contract JSON, this is picked up by the language-specific server, which adds a `metadata` section for each route, including details of the transport encoding expected to be used with the corresponding verification.
Given the approach outlined above doesn't require any changes to the mock server (only language-specific implementations which would like to implement this), it remains agnostic of transport/encoding, allowing others to plug in their own encodings/transports at will for sync communications.
Does this sound like an approach which you might be amenable to? I'm happy to knock up a demonstration in pact-net if this is something you could get behind.
mcon
2019-04-15 15:46
Hi @matt.fellows @bethskurrie, I've been doing work where I can find time on protobufs with pact: I've got something that works with proto over http, and I've also got some encouraging results on adding proper grpc support to pact (with only one obvious caveat at this point). I'll get some code into shareable states - mainly just letting you guys know I'm still on the case.
nboyd
2019-05-06 20:59
Hey @mcon as an FYI I'm _very_ interested in the work you're doing :slightly_smiling_face: I've been beating the "a tightly coupled contract between services != infallibility in said contract" drumfor some time to little success; nothing really to add other than if you think anything your doing is in vain, you're wrong.
bethskurrie
2019-07-01 09:56
What do you need help with? There isn't a working implementation yet, however, it has been discussed.
bethskurrie
2019-07-01 10:20
We think it's good in principle, but the implementation was going to require some fineness. I'm not a protobuf user, so I didn't fully understand the design, but it was going to require some dynamic conversions between the json and the protobuf format.
matt.fellows
2019-07-01 10:58
Design proposal for Protobufs with Pact https://docs.google.com/presentation/d/13rTmXp7Gdcd_hHC_0YP5FR0fB1KIjZf9p5yrucjTlSc/edit#slide=id.gc6f9e470d_0_0
mcon
2019-07-02 20:40
Hi folks, so I've got a vaguely working implementation of a pact encoding proxy which I have working with the C# Pact library - I've not spent much time on it as of late, and I was hoping to add some proper testing before showing where I'm up to. I have some time towards the end of the week, in that time I'll try to:
- Point to an end-to-end example of testing protobufs over HTTP
- Create a list of TODOs for the encoding proxy (mainly around testing - given the shape of the app was quite uncertain, a TDD-like approach didn't make much sense to me)
- Create a list of TODOs for the C# Pact library implementation
@dorons it sounds like there could be some scope for collaboration here - is your usecase protobuf over HTTP, or are you in GRPC-land?
matt.fellows
2019-07-02 20:53
Tests are great, but progress is better so others can look and provide feedback (or be inspired).
mcon
2019-07-09 20:15
Okay, so I've pushed my pact-net code and serialization-proxy code to GitHub:
https://github.com/mcon/pact-net/blob/protobuf-serialization/TODO.md
https://github.com/mcon/pact-serialization-proxy/blob/master/README.md
There are a few more things on the pact-net side I need to fix up before it's easily demo-able.
My main problem at the moment is making sure the right headers are propagated through the serialization proxy, and the only way to stabilise that is by refactoring and adding tests.
Once I've refactored the go code some more, I'd be keen to get some help writing tests from anyone who's interested and has the time.
mcon
2019-07-09 20:17
In terms of "working example", I've got a new pact JSON here which can be verified successfully with the below tests - there are a few things on the Consumer side I need to tidy up, but that json was generated programatically and not hand-constructed (you'll have to take my word for that one)
https://github.com/mcon/pact-net/blob/protobuf-serialization/Samples/EventApi/Consumer.Tests/pacts/event_api_consumer-event_api.json
https://github.com/mcon/pact-net/blob/protobuf-serialization/Samples/EventApi/Provider.Api.Web.Tests/EventAPITests.cs
matt.fellows
2019-07-09 21:12
Oh right, so the proxy has been written in Go. That means I won?t be completely useless to you
mcon
2019-07-10 19:33
Haha, yeah I ended up taking your suggestion and doing it in go, it was either c++, Java or go - for this sort of thing I think go was an obvious choice!
johnny.nilsson83
2019-11-27 09:58
Hi all.
I am new to the pact-slack since yesterday. I joined for one specific reason and that is to get an update on how the work on the protobuf-support is moving along. Does anyone know?
matt.fellows
2019-11-27 10:01
Given the recent support on our issues page and the fact that this channel is the 3rd largest in the group, we're thinking about it more but don't have solid time frames. It will take quite a coordinated effort to get it across all languages so one thing we'll need to do is set a language order. What are you using?
johnny.nilsson83
2019-11-27 11:13
Thank you for the response.
We are using java as our main language. Our frontend is built in javascript. Most/all of our data is accessed through SOAP-endpoints to access the mainframe systems, or a stored procedure
matt.fellows
2019-12-09 20:10
@christian.johann.eder we can explore non http protobufs etc here for now
christian.johann.eder
2019-12-10 10:49
@matt.fellows sure - but before we do this, I've got a question related to the planned protobug support. In the slides, I saw that you plan to include the protobuf definition in a binary format in the PACT files. What is the intention behind this? I am asking, because in the DIY contract testing approch I implemented in my last project, we just went with writing the base64 encoded binary protobuf messages as interaction bodies, instead of serializing them as JSON into the PACT. Being able to deserialize the interaction bodies was already part of the verification process. Usually, consumer and provider already have a way of sharing their protobuf message format, i.e. they both generate code from the same .proto files. So, why add another source of truth for protobuf message formats in the PACT file?
christian.johann.eder
2019-12-10 10:52
And regarding the non-HTTP question - I think we should separate between transport (e.g. HTTP, TCP...) and Serialization (e.g. JSON, Protobuf)...
For the question of transport, I couldn't find up to now any samples (or ideas in the slides) that indicate any current or future support for non-HTTP Apis. Did I miss anything?
cbergstrom
2019-12-13 17:55
Regarding the source of truth I could see this being useful for distributing the protos if you haven't already got a separate solution for that. Although it might be slightly awkward as this might be mixing test and production contexts. I just wonder if this would be abusing the PACT framework or if it makes sense to cross those feature boundaries
matt.fellows
2019-12-15 21:22
Please take that slide pack as an FYI-only and not how we plan to do the strategic implementation. It was created by @mcon in the community as a practical way to move forward for a use case he had. More broadly, we?re going to need to find a way to represent other contract types as first class citizens in the overall ecosystem
matt.fellows
2019-12-15 21:24
> For the question of transport, I couldn?t find up to now any samples (or ideas in the slides) that indicate any current or future support for non-HTTP Apis. Did I miss anything?
No you are quite correct. We?ve mostly taken the view that transport is less important to test, and it?s more about the contract that is shuttled over that transport. My view on this is shifting on this position a little, because whilst it sounds nice in theory, it makes for some awkward interfaces/experiences for the test framework
bethskurrie
2020-01-20 10:26
Protobuf people, your feedback helps us plan our roadmap. If you're interested in getting protobuf pact happening, place your votes here https://pact.canny.io/admin/board/feature-requests/p/support-protobufs
mcon
2020-01-25 21:22
Hi folks (@matt.fellows and @bethskurrie in particular),
Despite saying I wouldn't have time to continue working on protobufs for Pact (which I still don't), I *have* found time to finish the proof of concept to a level which is runnable by others (I've tried my instructions from scratch, and I seem to be able to get them working).
I've put together a gist detailing how one might go about running the proof of concept, and linking to the interesting bits of code which have changed to be protobuf-related. I don't know what architecture you guys are planning on longer term, but hopefully at the very least you might find this interesting. https://gist.github.com/mcon/2e3ae3eff0b9712d02e06ac1275b7f65
For now at work we're using other methods to validate protobuf-over-HTTP interactions, and pursuing Pact for this isn't a priority at the moment.
I'm happy to answer any questions: I'll stick around in the Pact Foundation slack for at least a few weeks.
Thanks Matt and Beth for your support in getting this far, and for all the work you do on Pact more generally.
matt.fellows
2020-01-26 01:18
I wonder how the modern idea of a"schema registry" afits in or competes with Pact and the broker
E.g. https://github.com/spotify/proto-registry
matt.fellows
2020-01-26 01:19
https://github.com/anemos-io/metastore
AWS registry and the confluent schema registry
matt.fellows
2020-01-26 01:20
It feels more like an anemic broker (i.e. without the CI/CD tools) and obviously no test framework around it
mcon
2020-01-26 09:22
Interesting thought Matt, I guess it would be possible to, in stead of storing the `FileDescriptorSet` in the pact contract itself (as I'm doing in the PoC), in stead store it in a schama registry (e.g. metastore), and have whichever component that does the serialization conversion fetch the schema from there.
The resulting pact contracts would be more concise, but there would then be a dependency on the registry to get the protobuf definition - ultimately there would still need to be something to do the serialization conversion (e.g. the serialization proxy, or some functionality integrated into the ruby core)
matt.fellows
2020-02-18 20:14
want to see protobufs implemented? Vote to help us prioritise it at https://pact.canny.io/feature-requests/p/support-protobufs
ngrigoriev
2020-02-18 20:24
Thanks for sharing! I was thinking about using https://github.com/jnewmano/grpc-json-proxy to deal with gRPC for Pact.
mcon
2020-02-18 21:31
I've started thinking recently whether Pact is the right way to test protobuf contracts - the `.proto` files which are used to describe a gRPC service are *almost* usable as contracts in their own right and, for me, the thing that's missing is some layer to prevent breaking changes being made to those `.proto` files. I've recently discovered Uber's Buf tool, which provides exactly that: a layer on top of `.proto` files to ensure breaking changes aren't made:
https://github.com/bufbuild/buf
This is obviously quite a different approach to Pact, and they may complement each other - it's not for me to say really, it depends what fits people's individual needs. Buf was a huge discovery for me, as it achieves most of the benefits I was looking to gain with protobufs and Pact, and I thought I should mention it here as folks may find it useful.
christian.johann.eder
2020-03-27 08:47
Just wanted to share that I finally managed to release my custom pact implementation with support for protobuf serialization.
It?s compatible with the pact spec v2 (I.e. makes the specs tests green), and is available for .NET Standard 2.0
https://github.com/ChristianEder/impact
The path I took allows to add custom serialization formats - such as protobuf - in addition to Json, and also to exchange the transport layer (out of the box, I support HTTP) at will.
Pleased let me know what you think!
matt.fellows
2020-03-27 08:54
That's awesome! Well probably be in touch when there time comes to get this into pact core
christian.johann.eder
2020-03-27 08:55
It?s a .net implementation- I?m not sure how this will work out?
christian.johann.eder
2020-03-27 08:57
In terms of pact compatibility, my focus was on the generated pact file - not the api / syntax to define them. The path I took for that is somewhat similar to the one in pact-net, but still different.
But I?m happy to exchange experiences on that one!
rbenbolton
2020-04-22 18:18
Hey everyone, I?m going to be jumping into protobufs and (with some help) try to see if it?s feasible to get Pact Go working with gRPC. I?m starting with reading the backlog here, and some of the github repos I?ve seen linked. Our CTO seems a bit skeptical and wants to look at other options as well. Anyone familiar with https://buf.build/?
matt.fellows
2020-04-22 20:23
Also read https://pact.canny.io/feature-requests/p/support-grpc if you haven?t already
matt.fellows
2020-04-22 20:26
I?ve seen buf as Matt C mentioned it above. Would definitely be interested to hear where you get to
matt.fellows
2020-04-22 20:27
For what it?s worth, I think there is still merit in bringing it into the Pact ecosystem because of the workflows we can provide - I believe a consistent approach to polyglot microservices (languages, protocols, transports) will simplify things for a lot of folks. But it is great to see other tools tackling it from other angles (perhaps we can find a way for them to work together)
matt.fellows
2020-04-22 20:28
There is a new version of the Golang protobuf library, which again has come out since Matt C?s awesome early work on the topic.
matt.fellows
2020-04-22 20:33
At this stage, this is what I,m thinking at a high level, but happy to jump on a call if you want to share/bounce ideas around:
1. We?ve made the strategic decision to use Rust as the ?engine? going forward, to power other Pact supported languages - it implements all versions of our specification. This is obviously going to be a barrier to entry for some
2. Create a Golang library that will be embedded into our Rust implementation. It will spin up a transport (gRPC, or HTTP) to serve a dynamic payload (the protobuf contents). The interface for this will need some serious thinking.
3. Serialising the Pact file is still something to be agreed on. It?s possible we embed/encode the proto file in the Pact itself, update the specification to support other types, or even _externalise_ the protobuf itself (this may have implications for other types of contracts also - e.g. kafka schema registry, Swagger definitions etc.). As a starting point, it will probably be encoded within the Pact file itself
If you?re looking to help with a strategic contribution, this will be a lot of work and a steep learning curve - I?ll do my best to support you! If not, that?s cool with us. Let us know how we can help
rbenbolton
2020-04-22 22:00
Also, pulling in @jon.stevens to the conversation as he?ll be working with me.
Dang, you?re right. I should have read the channel backlog _before_ posting and I would have seen Matt C?s post! oops. As it turns out, our CTO contacted the co-founder of buf and set up a meeting on Friday. I also just learned that Jason Newman who created https://github.com/jnewmano/grpc-json-proxy (https://pact-foundation.slack.com/archives/CEQBDD5U4/p1582057444018600) is the chief architect at our company.
matt.fellows
2020-04-22 22:02
I don?t quite get what the proxy?s purpose is at a glance - is the idea that it would translaten a HTTP / JSON request to a gRPC/protobufs request and back again?
rbenbolton
2020-04-22 22:04
Essentially yeah, specifically for use with Postman, which, at the time, was the go to for API testing within the company. He wrote the tool and then open sourced it.
matt.fellows
2020-04-22 22:06
Cool. And if I understand buf.build, it?s really about static analysis of protobuf files, and provides backwards compatibility detection, faster compile times and an improved ?image? for better reflection and analysis type tools. It may be of use to Pact actually given it?s richer API.
matt.fellows
2020-04-22 22:07
But we?ll have to see. One of the issues with protobufs (and gRPC also if protobufs are the payload) is that reflection and dynamic dispatch isn?t well supported acronss languages, so being able to dynamically ?mock? out an endpoint reliably can be hard. Avoiding the use of `protoc` or compile steps in tests is ideal.
matt.fellows
2020-04-22 22:07
It looks like Go now supports this reflection use case in its latest library. (NodeJS seems to support it the most flexibly/easily, but that can?t really be properly bundled into a general library distribution unlike exporting a c interface in Go)
rbenbolton
2020-04-22 22:12
The backwards compatibility detection I think is what caught the CTO?s eye right away, which is a big plus. I don?t think it gives the same protection as a pact contract that prevents breakage by running tests created by the consumer, but it is a step in the right direction. The determining factor I think will be whether or not the effort and benefit to get Pact fully working with gRPC will be greater than the risk of going with something already built like buf. That?s part of what I need to determine in the next week or so.
matt.fellows
2020-04-22 22:22
I see but as a tool like spectral or swagger linters. Complimentary tech
matt.fellows
2020-04-22 22:24
Have you looked at that canny feature request? Your commentary around the issues and approaches would be appreciated
nhodaly
2020-05-07 18:04
Hey everyone, I am very new to contract testing and pact. Would I be able to use pact as-is to conduct testing between a consumer / provider that are both using protobufs and gRPC?
ivangsa
2020-06-01 06:12
Hi
we are succesfully doing contract testing with gRPC + Pact in java and spring-boot
we have implemented an easy solution that involves no extra software, dependencies or extra open ports
our solution is tied to java and spring-boot but can inspire others in their stack
we have published all details here
https://medium.com/@ivangsa/consumer-driven-contract-testing-for-grpc-pact-io-d60155d21c4c?sk=9693d32e83ff72a467f4c15df80c3a70
*TL;DR;*
- on consumer test we are writing a custom io.grpc.Channel that marshall/unmarshall - protobuf/json and connects via http to mocking server (very simple class)
- on provider verification, we listen to spring-boot 'grpc server started event' to collect all metadata and write a very simple SpringMVC controller (only on tests context) that does the reverse proccess as Channel did on consumer side
- we map grpc to http request via POST to url "/grpc/<grpc method fullname>"
nice and easy..
matt.fellows
2020-06-01 06:26
awesome! You prototype is perfect, because i?m doing the same with JS right now. Thanks for sharing, let me look at how you?ve done it too
matt.fellows
2020-06-01 06:27
so basically, you?ve added a proxy/facade on other side. Nice, that?s what Matt C did with his .NET example
matt.fellows
2020-06-01 06:28
For core Pact to work, we?d prefer that indirection to not have to be there, but i?ve certainly been considering creating a universal binary that can be used as a shim on either side
matt.fellows
2020-06-01 06:28
I?m assuming you are just doing req/res style? Not bi-directional streaming with gRPC or anything?
ivangsa
2020-06-01 14:45
for us the io.grpc.Channel on the consumer side is a very clean solution, and an _http-proxy_ on the provider side is good enough...
ivangsa
2020-06-01 14:45
I'm not sure how can a universal binary can serialize/deserialize protobuf without knowing the specific api, as serialization depends on knowing default values and field order.. :thinking_face:
matt.fellows
2020-06-02 00:11
We?d need to pass more information into the Pact itself - e.g. the FileDescriptorSet
ivangsa
2020-06-02 08:13
what is that extra information you want to add to the pact itself? (what are the FileDescriptorSet?)
matt.fellows
2020-06-02 23:24
See ?self describing messages? at https://developers.google.com/protocol-buffers/docs/techniques#self-description
matt.fellows
2020-06-02 23:24
Alternatively, we pack the protocol buffer definition into the contract (so it can be read on framework side itself)
ivangsa
2020-06-03 07:17
But *you don't need to store any extra information* on the pact itself to un/marshall messages, *you have every thing you need at runtime*
- on the consumer tests side you have the gprc MethodDescriptor you are about to call (with all info you need), just need to write the 'generic' grpc Channel that connects to the http mocking server
- on the provider verification tests, it's true you don't have the information on a "per request" basis but you can collect all MethodDescriptors at service startup and store them, mapping them to urls (url <-> method.fullname)
so you already have all information you need to do the un/marshalling and with zero modifications to pacts or pact core...
anyone can start doing contract testing with pact+grpc right now... no extra binary, no extra FileDescriptorSet, no .proto definition on pacts...
I was the first supprised to find how easy and generic this was to roll out...
We are still thinking how we are going to manage/map exceptions to http status codes, but no a technical problem, each project may want to do this in a different way..
matt.fellows
2020-06-03 08:04
Thanks Ivan - I still haven?t fully digested your post yet so it?s more than possible you know more than I do!!
matt.fellows
2020-06-03 08:45
> so you already have all information you need to do the un/marshalling and with zero modifications to pacts or pact core...
It?s a pretty neat approach, similar to Matt C. I?d need to study other language APIs, but I?m aware that Java is one of those that supports better runtime reflection. Go now supports it also, but most don?t (so this might make that hard).
Regardless of those potential issues that may be completely misguided, my main concerns are that there may be some side-effects of reducing the fidelity using this strategy (i.e. downcasting gRPC+Protobufs to JSON/HTTP). Some downsides that come to mind:
? Protobuf specific contract checks (whether things for now or later)
? Sharing contracts through the Pact Broker / Pactflow: you?ll see an HTTP representation rather than the gRPC / protobuf
? Any tools (e.g. insights or quality tools like linters, coverage maps) may not appreciate that the messages aren?t actually HTTP and return false positives or bad advice.
? etc.
matt.fellows
2020-06-03 08:46
So that was a quick response, and I owe you a proper one. Also, what you have done is AWESOME and we?ll no doubt be stealing a lot of that
matt.fellows
2020-06-03 08:46
In fact, i?ll include it in our next Pactflow POST because people are really keen on this.
matt.fellows
2020-06-03 08:46
Ron may also have thoughts about stealing it for the JVM implementation, so if you?re keen to contribute your work, that would be something we?d be totally up for discussing!
ivangsa
2020-06-04 09:28
we are going to use Pact+REST and Pact+gRPC so the best support we get "out of the box" the better for us... you can count on me to contribute and collaborate as needed..
heytaco
2020-08-25 03:57
Hi there! My name is HeyTaco!, and you can use me to give people tacos to show your appreciation. My tacos will spread joy through Slack!
rcrodrigues39
2020-10-02 17:23
Hi everyone. I've been trying to get PACT to work with gRPC on NodeJs environment and I think I now have a satisfactory workaround based on the Java approach described by@ivangsa on https://medium.com/@ivangsa/consumer-driven-contract-testing-for-grpc-pact-io-d60155d21c4c.
The only difference tho is that on Consumer side, instead of overriding the Channel I decided to create an Interceptor that can translate gRPC calls to Rest (http+json) following a predefined standard.
On the provider side, I basically do the opposite (Rest to gRPC).
There's still some things to wrap up, but If anyone is interested, please follow the github repository https://github.com/ZupIT/pact-framework-examples/tree/node_grpc_example.
I'd love to hear your thoughts and feedback on this.
johnny.nilsson83
2020-10-15 12:39
This is really great news! Have you gotten it up and running in a larger context?
matt.fellows
2021-04-22 00:48
You?ve probaly seen https://github.com/pact-foundation/pact-specification/issues/83
matt.fellows
2021-04-22 00:50
It?s on the Pactflow roadmap, after we?ve made inroads into the initial bi-directional feature set.
matt.fellows
2021-04-22 00:50
The dependency is creating a pact plugin famework, and we?ll implement gRPC/protobufs as the first one
matt.fellows
2021-04-22 01:57
FYI this looks realy promising: https://github.com/ZupIT/pact-framework-examples/tree/master/examples/node/gRPC
matt.fellows
2021-04-22 02:00
TL;DR - convert protobuf into an intermediate JSON structure, pop in pact file, reverse the other side
matt.fellows
2021-04-22 02:01
I?ve seen it before (maybe it was these guys too) - but a good intermediate solution whilst we get our plugin sh*t sorted
matt.fellows
2021-04-27 10:35
This might be interesting to you: https://medium.com/@ivangsa/consumer-driven-contract-testing-for-grpc-pact-io-d60155d21c4c
jarmy
2021-04-27 17:15
I've seen that and discussed with my team. Our initial thought was to wait for full support in pact but we may have to dabble with this approach
matt.fellows
2021-04-28 01:18
I?d start here, because even when we add to Pact, there is going to be bedding in time etc.
matt.fellows
2021-04-28 01:18
The strategy there I?ve actually seen before, this is just an example where it was published
matt.fellows
2021-05-12 02:54
@aparnachaudhari :point_up: here is a link to an example that might be helpful
aparnachaudhari
2021-05-12 03:41
@matt.fellows thx.. will take a look shortly... I am adding async tests tho... 2 services that communicate using async messages. These messages are marshalled (consumer) and unmarshalled(provider) to protobuf as they are put on the cluster and read from.
matt.fellows
2021-05-12 04:04
The principle I think is the same though. The key insight is hiding the actual payload within a non protobuf message structure (i.e. JSON)
dmitry.korolev
2021-07-02 12:27
Hey guys,
We use custom 'application/protobuf' type for our protobufs.
In pact there is apparently an automatic validation of the body type, which fails as Pact recognizes a binary and assigns 'application/octet-stream' type. Then contract verification fails as it expects 'application/protobuf'.
Did anyone by any chance experience such an issue with protobuf?
matt.fellows
2021-07-02 12:59
Most people seem to build a translation layer that converts to JSON (so Pact can serialise/deserialise nicely)
matt.fellows
2021-07-02 13:00
also https://github.com/ZupIT/pact-framework-examples (if you speak what I believe is Portugese)
dmitry.korolev
2021-07-02 13:42
Thanks Matt, I will give it a read. I hope google translate will not fail me :sweat_smile:
matt.fellows
2021-07-27 00:27
FYI progress has started on the plugin infra to support gRPC/Protobufs: https://pact.canny.io/admin/board/feature-requests/p/support-grpc
uglyog
2021-08-30 06:36
we have a working prototype of a Pact consumer test with Protobufs. The provider verification is next, but you can see the test here: https://github.com/pact-foundation/pact-plugins/blob/main/examples/protobuf/protobuf-consumer/src/test/java/io/pact/example/protobuf/provider/PactConsumerTest.java
uglyog
2021-08-30 06:36
This is the message for that test
```message InitPluginRequest {
string implementation = 1;
string version = 2;
}```
uglyog
2021-08-30 06:40
If you look carefully, you may notice a new matcher as well as a language for describing matching rules
uglyog
2021-08-30 06:41
This is the important bit (refer to message above):
``` "implementation", "notEmpty('pact-jvm-driver')",
"version", "matching(semver, '0.0.0')"```
tjones
2021-08-30 06:42
Right. Could the rest of the boilerplate be inferred from `usingPlugin("protobuf")`?
tjones
2021-08-30 06:43
I really like the `usingPlugin(...)` pattern. I was thinking that if we can refactor the existing http pact so that it is *also* a plugin, then we'll get a lot of the rest of pact's functionality for free
tjones
2021-08-30 06:43
`matching(semver, '0.0.0')` <-- this is great. At first I didn't even notice that it was a custom matcher
uglyog
2021-08-30 06:44
No, that is because with plugins we have to have a generic interface because we don't know what the structure the plugin defines. But for major plugins, I'm hoping we can define DSLs to hide the boilerplate
tjones
2021-08-30 06:45
right, but couldn't the `usingPlugin` define extra properties to put on the object?
uglyog
2021-08-30 06:45
Yes, I imagine that the plugin could return that data. Haven't explored that yet
tjones
2021-08-30 06:47
Unless I'm missing something, this example could be shortened to:
``` return builder
.usingPlugin("protobuf")
.expectsToReceive("init plugin message", "core/interaction/message")
.with(Map.of(
"implementation", "notEmpty('pact-jvm-driver')",
"version", "matching(semver, '0.0.0')"
))```
tjones
2021-08-30 06:48
(if it is valid to specify things that aren't `message.contents` for a `protobuf` pact, then you'd still need that key)
uglyog
2021-08-30 06:50
For instance, you could have a plugin that matches a particular format (like CSV), and another that provides an interaction type (say HTTP/3).
tjones
2021-08-30 06:52
I reckon it's possible to infer a lot of the boilerplate if we are clever about it
uglyog
2021-08-30 06:55
I chose Java for the initial test, because that is a common language used with Protobufs. A dynamic language would be more fluent.
matt.fellows
2021-09-02 07:43
:wave: Further to Ron?s awesome prototype above, here is a post that captures what I believe are some of the deficiencies of protobufs (and similar technology): https://pactflow.io/blog/is-grpc-and-protobufs-truly-backwards-compatible/
Very keen for feedback so that we can iterate on any solution!
zhaopeng
2021-10-14 14:59
Hey @uglyog, that's super excited to see the protobus become true, I would like to ask :
1. maybe do you know when we can have a provider verification example?
2. can I also use it with pact-broker?
Thanks in advance
uglyog
2021-10-14 22:01
Do you need the provider verification in a particular language? I've been using the stand alone verifier to verify the protobuf message against the provider.
Pact broker is supported in the same way as any other pact test.
zhaopeng
2021-10-15 07:12
Hey @uglyog thanks for your quick reply, I think if possible we can start from jvm<->jvm also(I'm not sure what I could help there, but maybe if there are some PR, that would be cool). PS: I try to apply it for gRPC (jvm <-> jvm / jvm <-> nodjes / nodejs <-> nodejs)
matt.fellows
2021-10-18 02:51
Hi @zhaopeng great to hear you?re excited about this too! Perhaps we can chat offline, but we would definitely be keen on your feedback on this as we shape the plugin and ecosystem around it.
zhaopeng
2021-10-20 19:21
Hey @matt.fellows , thanks a lot I think we already talked offline.
ps: I noticed we already have verifier for protobuf @uglyog :tada:
zhaopeng
2021-10-20 19:24
I found an interesting part in https://github.com/pact-foundation/pact-plugins/tree/main/examples/helloworld-grpc
I noticed no matter consumer or provider, they both need to load "proto" folder.
so in real practice, what we need to do? in most of cases, the proto are stored inside of "producer"'s repo. it will be always updated together with producer.
Ref:
``` .withPluginConfig("grpc", Map.of("proto", "examples/helloworld/proto"))```
May I ask what's the purpose that Consumer need to load whole `proto` if everything are mocked?
it maybe could be an overkill which maybe indicate that we need to somehow maintain and versioning of "proto" file in the middle (not only the contract, we have different files to maintain outside of the box)
uglyog
2021-10-20 21:57
You only need to specify the proto file in the consumer test. Look at the protobuf example in https://github.com/pact-foundation/pact-plugins/blob/main/drivers/jvm/core/src/test/groovy/io/pact/plugins/jvm/core/DriverPactTest.groovy
uglyog
2021-10-20 22:00
Once that test is run, the proto file and compiled descriptor is persisted into the Pact file
zhaopeng
2021-10-21 07:35
hmm, seems we still need to link proto file in consumer side? (then maybe where we gonna store this proto file for development purpose? in producer side? and also in consumer side?)
it seems developer think proto file should be saved in Producer git repo.
but now it seems the proto file also need to be saved in Consumer git repo?
``` 'pact:proto': filePath("../../../proto/plugin.proto"), ```
maybe I'm wrong, pls feel free to correct me
uglyog
2021-10-21 08:28
You need it on the consumer side. The consumer test will persist it to the pact file, so it is available in the pact file for the provider side. This is because the proto file can change, so you need the version at the time the consumer test runs.
uglyog
2021-10-21 08:28
It doesn't need to be saved in the repo, it just needs to be available. You need it to build your consumer anyway.
matt.fellows
2021-10-22 09:25
apologies, this one: https://pactflow.io/blog/the-case-for-contract-testing-protobufs-grpc-avro/
jarmy
2021-11-30 21:09
Hi. I'm starting a POC with the protobuf plugin and had a question about this:
```"pact:proto", filePath("../../../proto/plugin.proto"), // Need to provide the proto file```
Does that file need to come from the provider?
uglyog
2021-11-30 22:22
No, the proto file is what was used to generate the code used on both the provider and consumer sides.
jarmy
2021-11-30 22:35
where does that file originate from? is that something we must generate ourselves?
uglyog
2021-11-30 23:00
The proto file is the IDL (interface definition language) used by Protobufs and hence gRPC. It is what is passed to the protoc compiler to generate the consumer and provider interface code. See https://developers.google.com/protocol-buffers/docs/proto3
How are you using Protobufs? Is it via a shared library?
jarmy
2021-11-30 23:07
I'm not entirely sure (I sit outside of the group that develops the service). I was given an example of an endpoint they consume via gRPC:
``` public List<ListenerFeedbackWrapper> getFeedbackList(long listenerId, long stationId) {
GetContextFeedbackListRequest request =
GetContextFeedbackListRequest.newBuilder()
.setListenerId(listenerId)
.setContextType(com.savagebeast.remoting.api.pithos.grpc.ContextType.Station)
.setPrimaryContextId(PandoraId.fromStationId(stationId).toString())
.setFeedbackType(GetContextFeedbackListRequest.FeedbackType.All)
.build();
GetContextFeedbackListResponse response = getStub().getContextFeedbackList(request);
return filterOutNonTracksProto(response.getFeedbackList());
}```
which appears to map to this proto file generated by the provider service
jarmy
2021-11-30 23:17
ok, so the file _does_ come from the provider. i'm just learning about protobufs so bear with my dumb questions
jarmy
2021-11-30 23:18
does that file need to be checked into source control? I imagine it does in order for the tests to run on CI
matt.fellows
2021-11-30 23:22
I guess ultimately it will be from the provider. I?me keen to understand the various use-cases and workflows so this is interesting. Is the proto file readily available in the code base you?re testing? I?m assuming it must be for the consumer endpoints to work?
uglyog
2021-11-30 23:24
Matt, I'm thinking that we can drop the need of the proto file if we encode the types in the pact test
jarmy
2021-11-30 23:25
I searched for any proto files in the consumer codebase and don't see it but they must be accessing it somehow. I'll have to touch base with the developer
uglyog
2021-11-30 23:25
I assume the data access classes would have been generated by the provider team
jarmy
2021-11-30 23:32
ok, the developer tells me the proto file is just a definition file that gets bundled into the .jar file produced by the provider. it's then pulled in as a dependency by the consumer service
jarmy
2021-12-03 20:20
How would I design a contract for this RPC protobuf call?
``` rpc GetContextFeedbackList (GetContextFeedbackListRequest) returns (GetContextFeedbackListResponse) {
} ```
When I try to use `"pact:message-type", "GetContextFeedbackList",` in my test, I get the following exception:
```Caused by: au.com.dius.pact.consumer.dsl.InteractionConfigurationError: Failed to set the interaction: Message 'GetContextFeedbackList' was not found in proto file '/Users/jarmy/src/git/pandora/listeners/stations/stations-service/src/test/resources/protos/pithos/FeedbackApi.proto'```
I'm probably using the wrong pact _type_ above. If clients are using RPCs then I'd think pact should support them, no?
jarmy
2021-12-07 00:59
How can I verify my protobuf contract in my provider? The https://github.com/pact-foundation/pact-plugins/blob/main/plugins/protobuf/README.md#verifying-the-provider says to use `pact_verifier_cli` but I'm not finding an installer anywhere. If it matters, I'm using macOS
matt.fellows
2021-12-07 01:14
Sorry Jason, I saw this yesterday too when I was doing a demo. I?ve got a few notes to update the docs
matt.fellows
2021-12-07 01:14
the verifier is here: https://github.com/pact-foundation/pact-reference/releases/tag/pact_verifier_cli-v0.9.2
matt.fellows
2021-12-07 01:15
(see also the list of our https://docs.pact.io/implementation_guides/cli#provider-verifier)
uglyog
2021-12-07 01:21
You can download it from https://github.com/pact-foundation/pact-reference/releases/tag/pact_verifier_cli-v0.9.2
jarmy
2021-12-07 17:32
I downloaded it and gzip'd it but it's not an executable
```jarmy@jarmy-m01 Downloads % ls -ltr pact_verifier*
-rw-r--r--@ 1 jarmy staff 5762709 Dec 7 09:28 pact_verifier_cli-osx-x86_64.gz
jarmy@jarmy-m01 Downloads % gzip -d pact_verifier_cli-osx-x86_64.gz
jarmy@jarmy-m01 Downloads % ls -ltr pact_verifier*
-rw-r--r-- 1 jarmy staff 16169864 Dec 7 09:28 pact_verifier_cli-osx-x86_64
jarmy@jarmy-m01 Downloads % pact_verifier_cli
zsh: command not found: pact_verifier_cli
jarmy@jarmy-m01 Downloads % pact_verifier_cli-osx-x86_64
zsh: command not found: pact_verifier_cli-osx-x86_64```
jarmy
2021-12-07 17:42
@uglyog @matt.fellows?? If I use gRPCurl to execute against a protobuf, I can only call the RPC methods. If I understand protobufs correctly, clients can only access the RPC methods.
jarmy
2021-12-07 19:26
I figured it out: `cargo install pact_verifier_cli`
```pact_verifier_cli --version
pact_verifier_cli 0.9.2
pact verifier version : v0.9.2
pact specification : v4.0
models version : v0.2.2 ```
jarmy
2021-12-07 19:26
I figured it out: `cargo install pact_verifier_cli`
```pact_verifier_cli --version
pact_verifier_cli 0.9.2
pact verifier version : v0.9.2
pact specification : v4.0
models version : v0.2.2 ```
matt.fellows
2021-12-07 21:55
It?s like saying JSON can only be accessed over HTTP - you could definitely transport JSON in other ways (e.g. TCP)
uglyog
2021-12-07 22:09
`GetContextFeedbackList` is a service method, not a message. With a service method call, there is an input message and an output message. You can use `pact:proto-service` set to `<service_name>/GetContextFeedbackList` where `service_name` is the name of the service that method belongs to. You then need to provide a request message and a response message.
It gets a bit tricky to test, depending on the language you are using. With Java, I had to mock some gRPC final classes.
jarmy
2021-12-09 00:34
I'm getting a contract produced when using this code (I have not fully implemented the `MockPlugin` class due to groovy > java syntax issues:
``` @Pact(consumer = "stations")
V4Pact getContextFeedbackList(PactBuilder builder) {
return builder
.usingPlugin("protobuf")
.expectsToReceive("GetContextFeedbackList", "core/interaction/synchronous-message")
.with(Map.of(
"pact:proto", filePath("src/test/resources/protos/pithos/FeedbackApi.proto"),
"pact:proto-service", "FeedbackService/GetContextFeedbackList",
"pact:content-type", "application/protobuf",
"request", Map.of(
"listenerId", "notEmpty('2')",
"contextType", "matching(equalTo, 'Station')",
"primaryContextId", "notEmpty('ST:0:4294967298')",
"feedbackType", "matching(equalTo, 'All')"
),
"response", Map.of(
"feedback", Map.of(
"listenerId", "notEmpty('2')",
"contextType", "matching(equalTo, 'Station')",
"primaryContextId", "notEmpty('ST:0:4294967298')",
"targetId", "notEmpty('TR:91149')",
"positive", "notEmpty('true')",
"feedbackId", "notEmpty('2147483649')",
"secondaryContextId", "notEmpty('AR:876')",
"dateModified", "notEmpty('1400084172141')",
"dateCreated", "notEmpty('1400084172141')"
)
)
)
)
.toPact();
}
@Test
@PactTestFor(pactMethod = "getContextFeedbackList")
void consumeGetContextFeedbackList(V4Interaction.SynchronousMessages message) throws InvalidProtocolBufferException {
Plugin.InitPluginRequest requestMessage = Plugin.InitPluginRequest.parseFrom(message.getRequest().getContents().getValue());
assertThat(requestMessage.getImplementation(), is(""));
}```
But the contract that is produced flat out fails with a 404 error when I run the `pact_verifier_cli`
uglyog
2021-12-09 00:41
Side note: In your test, you are using `Plugin.InitPluginRequest`, that will not be able to parse the message (it will just have all the default field values)
Are you specifying the HTTP server/proxy when running the verifier? What it will do is send the request message to the proxy, and expect the response message back to verify. It it is getting a 404, which means that whatever it is sending the request to is not recognizing the path of the request.
jarmy
2021-12-09 00:48
My provider service is running at `http://localhost:9009/pithos` and then I'm running the following command:
```pact_verifier_cli --disable-ssl-verification --enable-pending --broker-url https://pact-broker.docker.savagebeast.com --provider-name pithos -h localhost -p 9009 --filter-consumer stations --consumer-version-selectors '{"branch": "DEVTOOLS-1353-pact-gRPC-POC"}' --loglevel debug```
uglyog
2021-12-09 00:48
You should be using the messages from the service. I think that is `GetContextFeedbackListRequest`
uglyog
2021-12-09 00:57
Ah, we haven't implemented gRPC support yet (it's on the roadmap after Protobufs is working). Your provider won't understand the paths that are sent because they will not be the gRPC ones.
You will need to setup either a proxy HTTP server that can get the message, and then call your provider. The format of the request is documented here: https://github.com/pact-foundation/pact-reference/tree/master/rust/pact_verifier_cli#verifying-message-pacts, except in this case it needs to handle the request message.
jarmy
2021-12-09 00:57
ah, this works
```GetContextFeedbackListRequest requestMessage = GetContextFeedbackListRequest.parseFrom(message.getRequest().getContents().getValue());```
jarmy
2021-12-10 18:35
@uglyog what's the point of the mocking in https://github.com/pact-foundation/pact-plugins/blob/main/drivers/jvm/core/src/test/groovy/io/pact/plugins/jvm/core/DriverPactTest.groovy#L150-L156? We're having a bit of trouble getting `withGrpcStub` to work in Java
uglyog
2021-12-16 00:06
That test is testing the interaction between the plugin driver and the plugins. The plugin driver uses a Plugin class to encapsulate the login to call the plugin. If we don't mock out the plugin class, when we call `DefaultPluginManager.INSTANCE.initPlugin` which is the method we want to test, it will try make a real gRPC call.
uglyog
2021-12-16 00:07
Using a mock class, we can get it pretend to make the gRPC call, by validating the request and if it is ok return the response from the Pact instead.
elliot.weiser
2021-12-16 20:34
@uglyog I'm a little new to pact, so please bear with me. Can you expand on the need to setup an HTTP proxy server? I think @srinagasai.krishnasan is still a bit stuck. Sri, are you also having trouble setting up `withGrpcStub`?
uglyog
2021-12-16 22:14
A HTTP proxy server is not needed if you are using Java. because you can mock things out
srinagasai.krishnasan
2021-12-17 19:08
We have done these things.
? Setup the consumer contract test for grpc and publish to pact-broker
? On the verifier side, we were able to get the service running and get the contract published in pact-broker.
? During verification, it's giving a 404.
srinagasai.krishnasan
2021-12-20 16:42
> Using a mock class, we can get it pretend to make the gRPC call, by validating the request and if it is ok return the response from the Pact instead.
On another note, What you suggest is
? Create a class that extends PactPlugin class. Since it's an interface we will have to override the unimplemented methods. Out of which we had to deal with `getStub()` and `withGrpcStub`
? In `getStub()` we are doing this
```@Override
public AbstractBlockingStub<PactPluginGrpc.PactPluginBlockingStub> getStub() {
return PowerMockito.mock(PactPluginGrpc.PactPluginBlockingStub.class);
}```
As the `PactPluginBlockingStub.class` is static to start with.
? Also for request and response we have our own req and resp's
```// Plugin.InitPluginRequest request;
// Plugin.InitPluginResponse response;
GetContextFeedbackListRequest request;
GetContextFeedbackListResponse response;```
While doing this we are having problems with `withGrpcStub` . So should we use our own req and resp. Or Must we go with `Plugin.InitPluginRequest` and `Plugin.InitPluginResponse`
srinagasai.krishnasan
2021-12-20 21:20
> Using a mock class, we can get it pretend to make the gRPC call, by validating the request and if it is ok return the response from the Pact instead.
@uglyog Can you please elaborate on this?
uglyog
2021-12-20 21:58
> Create a class that extends PactPlugin class
You shouldn't do this, you should create a class that extends your own gRPC server class. What are you calling to make the gRPC service call?
elliot.weiser
2021-12-20 22:34
@uglyog We're having some difficulty getting our footing here (or at least I am).
My current (limited) understanding is that, since gRPC is not fully supported yet, that in order to write a semi-meaningful test, the underlying gRPC interactions must be mocked out completely. The level at which that needs to happen is currently unclear to us.
> create a class that extends your own gRPC server class
To clarify, are you suggesting that it is the `GetContextFeedbackListRequest` class which must be extended/mocked? Is the child class supposed to be providing a stubbed instance of `GetContextFeedbackListResponse` when the mocked request is called?
uglyog
2021-12-20 22:45
You need to be calling something, which you give the request class to (`GetContextFeedbackListRequest`), and you get the response from. In my example project, this is the PactPlugin classes which are generated from the plugin.proto file. In your project, there must be similar type of classes. With Java, the protoc compiler generates a class named <name>Stub, which I found was the easyest to mock out as that is what makes the actual gRPC call.
uglyog
2021-12-20 22:49
In my example project, my code uses the PactPlugin class, which in turn uses the PactPluginStub class. So I was able to create a MockPactPlugin so that the actual gRPC call is not made.
srinagasai.krishnasan
2021-12-20 23:24
> You need to be calling something, which you give the request class to (`GetContextFeedbackListRequest`), and you get the response from.
You mean the implementation classes on the side where you send the request to?
srinagasai.krishnasan
2021-12-20 23:44
A service implementation class which is implemented through an interface. In this case `FeedbackService` . Cz our proto looks like this..
```service FeedbackService {
rpc GetContextFeedbackList (GetContextFeedbackListRequest) returns (GetContextFeedbackListResponse) {
}```
srinagasai.krishnasan
2021-12-21 18:57
Let's say we have two services. A and B.
Service A consumes endpoints provided by Service B.
Now we write a consumer contract test For Service B within Service A.
Since it's a gRPC with Protobufs. All the mocking needs to be done on the consumer contract side?
And then for verification, we need to start the server in one terminal and then verify it in another terminal using pact_verifier_cli?
I am not able to make sense of why we need to mock on consumer side.
Please clarify this.
uglyog
2021-12-22 00:14
Ok, why not try run the consumer tests without mocking anything, and see what happens. Try first stating what you expect to happen, and then what actually happens.
We can look at the provider side once you have the consumer side working.
srinagasai.krishnasan
2021-12-22 02:31
My colleague(@jarmy) has managed to get the contract ready and push it to pact-broker. While running the verification on the provider side, we are getting a 404.
The way we were verifying is, locally, we spin up the server and then in another terminal using `pact_verifier_cli` we are verifying with required params/options. During that phase, we are getting 404.
uglyog
2021-12-22 02:38
Because I assume your provider is a gRPC server, and expecting to be communicated with via the gRPC protocol
srinagasai.krishnasan
2021-12-22 02:56
Right. For that reason. You are saying we need to make the mock related changes on the consumer side?
srinagasai.krishnasan
2021-12-22 02:58
Cz on the provider side. We are using `gretty` and are also have a workaround to have pact task run after the server runs and then stops after pact task is run. We end it using `terminateProviderTask`
matt.fellows
2021-12-22 03:00
Sri - remember, at the moment we have only implemented the protobufs plugin (the payload) and have not built a gRPC plugin (the transport)
matt.fellows
2021-12-22 03:01
The `pact_verifier_cli` only knows how to communicate to your provider over HTTP, not gRPC
matt.fellows
2021-12-22 03:02
So you can?t just start your gRPC server and point the verifier at it, they won?t be able to communicate correctly.
srinagasai.krishnasan
2021-12-22 03:08
ah that's why uglyog has suggested to use a proxy http server which in turn makes a grpc call to real provider service?
srinagasai.krishnasan
2022-01-10 18:11
@uglyog @matt.fellows I am trying to do this with nginx running on port 80. However I am getting this..
``` pact_verifier_cli --disable-ssl-verification --enable-pending --broker-url https://pact-broker.docker.savagebeast.com --provider-name pithos-grpc -h localhost -p 55871 --filter-consumer stations-grpc --consumer-version-selectors '{"branch": "DEVTOOLS-1353-pact-gRPC-POC"}'```
---
```thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Builder, source: Os { code: 2, kind: NotFound, message: "No such file or directory" } }', /Users/skrishnasanka/.cargo/registry/src/github.com-1ecc6299db9ec823/pact_verifier-0.12.3/src/pact_broker.rs:534:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace```
srinagasai.krishnasan
2022-01-10 22:12
It was a problem with openssl. Was able to resolve it. You can disregard this.
srinagasai.krishnasan
2022-01-10 22:30
Like @uglyog said. I have created a nginx forward proxy running on port 80 that forwards the requests to grpc port of my local application server. Now in other terminal I am running the `pact_verfier_cli` it's giving me this
``` GetContextFeedbackList
Test Name: com.pandora.remoting.api.stations.contract.consumer.grpc.PithosConsumerContractTest.consumeGetContextFeedbackList(SynchronousMessages)
This pact is still in pending state for pithos-grpc as the successful verification results have not yet been published.
Pending Failures:
1) Verifying a pact between stations-grpc and pithos-grpc - GetContextFeedbackList - error sending request for url (http://localhost/): invalid HTTP version parsed```
My nginx.conf looks like this
``` server {
listen 80 http2;
location / {
grpc_pass ;
}```
matt.fellows
2022-01-10 23:37
Nginx (via `grpc_pass` is not taking an HTTTP request and then converting it to a gRPC request to your gRPC service - it?s actually communicating gRPC over the HTTP/2 protocol. The verifier must be able to communicate to an HTTP server. So I don?t think this approach is going to work.
srinagasai.krishnasan
2022-01-11 20:02
Hello @uglyog @matt.fellows Would you be able to recommend a proxy server technology that does the translation of http 1.1 to grpc or http2? We weren't able to find any.
Or When can we expect http2 support for `pact_verfier_cli`
matt.fellows
2022-01-12 00:04
The simplest approach would be to write a tiny little HTTP server in whatever language you?re using (Java, I think?) and wrap your gRPC service.
@uglyog I thought you had an example of that, but can?t find it on the plugins github. Am I misremembering?
uglyog
2022-01-12 00:08
I don't have an example of HTTP -> gRPC. I have an HTTP server that returns the Protobuf message. I.e. if a service is defined:
```rpc InitPlugin(InitPluginRequest) returns (InitPluginResponse);```
I have an HTTP server that will return the encoded form of `InitPluginResponse`
jarmy
2022-01-12 00:14
do we think envoy would work? https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_http1_bridge_filter
matt.fellows
2022-01-12 00:17
could be PoC?d pretty quickly I suppose. I think nginx doesn?t work, because all it is doing is transport and (presumably) load balancing
uglyog
2022-01-12 00:17
It is going to have 5 bytes in front of the actual message, so you will need to be able to remove those
> The body should be the serialized grpc body which is:
> ? 1 byte of zero (not compressed).
> ? network order 4 bytes of proto message length.
> ? serialized proto message.
>
jarmy
2022-01-25 17:52
We're struggling to get the envoy proxy working with `pact_verifier_cli`. We believe we've set up the `grpc_http1_bridge` correctly. When the verifier issues the request to the proxy server, this happens:
```17:50:08 [INFO] Running provider verification for 'GetContextFeedbackList'
17:50:08 [INFO] Sending request to provider at http://localhost:10000/
17:50:08 [DEBUG] (1) pact_verifier::provider_client: Provider details = ProviderInfo { name: "provider", protocol: "http", host: "localhost", port: Some(10000), path: "/" }
17:50:08 [DEBUG] (1) pact_verifier::provider_client: Sending request HTTP Request ( method: POST, path: /, query: None, headers: Some({"Content-Type": ["application/json"]}), body: Present(535 bytes, application/json) )
17:50:08 [DEBUG] (1) reqwest::connect: starting new connection: http://localhost:10000/
17:50:08 [DEBUG] (18) hyper::client::connect::dns: resolving host="localhost"
17:50:13 [DEBUG] (1) hyper::client::connect::http: connecting to 127.0.0.1:10000
17:50:13 [DEBUG] (1) hyper::client::connect::http: connected to 127.0.0.1:10000
17:50:13 [DEBUG] (5) hyper::proto::h1::io: flushed 675 bytes
17:50:13 [DEBUG] (14) hyper::proto::h1::io: parsed 7 headers
17:50:13 [DEBUG] (14) hyper::proto::h1::conn: incoming body is chunked encoding
17:50:13 [DEBUG] (1) reqwest::async_impl::client: response '415 Unsupported Media Type' for http://localhost:10000/
17:50:13 [DEBUG] (1) pact_verifier::provider_client: Received native response: Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(10000), path: "/", query: None, fragment: None }, status: 415, headers: {"content-type": "text/plain; encoding=utf-8", "grpc-status": "13", "grpc-message": "Content-Type 'application/json' is not supported", "x-envoy-upstream-service-time": "5", "date": "Tue, 25 Jan 2022 17:50:13 GMT", "server": "envoy", "transfer-encoding": "chunked"} }
17:50:14 [DEBUG] (14) hyper::proto::h1::conn: incoming body decode error: unexpected EOF during chunk size line
GetContextFeedbackList
Test Name: com.pandora.remoting.api.stations.contract.consumer.grpc.PithosConsumerContractTest.consumeGetContextFeedbackList(SynchronousMessages)
Failures:
1) Verifying a pact between stations-grpc and pithos-grpc - GetContextFeedbackList - request or response body error: error reading a body from connection: unexpected EOF during chunk size line```
? And here's the request coming into Envoy (notice it's content-type is `application/json` which doesn't satisfy the filter requirements: _When a request is sent, the filter sees if the connection is HTTP/1.1 and the request content type is application/grpc._
```[2022-01-25 09:50:13.213][1580445][debug][http] [source/common/http/conn_manager_impl.cc:283] [C15] new stream
[2022-01-25 09:50:13.213][1580445][debug][http] [source/common/http/conn_manager_impl.cc:873] [C15][S12515614025377970192] request headers complete (end_stream=false):
':authority', 'localhost:10000'
':path', '/'
':method', 'POST'
'content-type', 'application/json'
'accept', '*/*'
'accept-encoding', 'gzip, deflate'
'content-length', '535'
[2022-01-25 09:50:13.214][1580445][debug][router] [source/common/router/router.cc:486] [C15][S12515614025377970192] cluster 'pithos' match for URL '/'
[2022-01-25 09:50:13.215][1580445][debug][router] [source/common/router/router.cc:702] [C15][S12515614025377970192] router decoding headers:
':authority', 'localhost:10000'
':path', '/'
':method', 'POST'
':scheme', 'http'
'content-type', 'application/json'
'accept', '*/*'
'accept-encoding', 'gzip, deflate'
'content-length', '535'
'x-forwarded-proto', 'http'
'x-request-id', '0748ec38-0a40-4ce7-bcf1-cc562328a837'
'x-envoy-expected-rq-timeout-ms', '15000'
[2022-01-25 09:50:13.215][1580445][debug][pool] [source/common/conn_pool/conn_pool_base.cc:250] [C5] using existing connection
[2022-01-25 09:50:13.215][1580445][debug][pool] [source/common/conn_pool/conn_pool_base.cc:176] [C5] creating stream
[2022-01-25 09:50:13.215][1580445][debug][router] [source/common/router/upstream_request.cc:418] [C15][S12515614025377970192] pool ready
[2022-01-25 09:50:13.215][1580445][debug][http] [source/common/http/filter_manager.cc:835] [C15][S12515614025377970192] request end stream
[2022-01-25 09:50:13.220][1580445][debug][router] [source/common/router/router.cc:1320] [C15][S12515614025377970192] upstream headers complete: end_stream=false
[2022-01-25 09:50:13.220][1580445][debug][http] [source/common/http/conn_manager_impl.cc:1472] [C15][S12515614025377970192] encoding headers via codec (end_stream=false):
':status', '415'
'content-type', 'text/plain; encoding=utf-8'
'grpc-status', '13'
'grpc-message', 'Content-Type 'application/json' is not supported'
'x-envoy-upstream-service-time', '5'
'date', 'Tue, 25 Jan 2022 17:50:13 GMT'
'server', 'envoy'
[2022-01-25 09:50:13.220][1580445][debug][http2] [source/common/http/http2/codec_impl.cc:1241] [C5] stream closed: 2```
We're unfortunately unable to make any further progress.
matt.fellows
2022-01-27 02:45
We discussed this the other day, and feel like the next best course of action is for us to create the gRPC plugin, rather than burning your time (and more importantly, eating away at your frustrations bank balance - we all have a limit!)
jarmy
2022-01-27 19:57
I did a little experiment by modifying this https://github.com/pact-foundation/pact-reference/blob/master/rust/pact_verifier/src/messages.rs#L189-L192 code locally to use `application/grpc`, built a new version and fired up the pact_verfier_cli, which allowed it to talk to the Envoy proxy and hit the gRPC service. It resulted in this:
```20:00:42 [DEBUG] (1) pact_matching: expected content type = 'application/protobuf;message=GetContextFeedbackListResponse', actual content type = 'application/grpc'
20:00:42 [DEBUG] (1) pact_matching: Matching message metadata
20:00:42 [DEBUG] (1) pact_matching: Matching message metadata. Expected '{}', Actual '{"contentType": String("application/grpc")}'
GetContextFeedbackList
Test Name: com.pandora.remoting.api.stations.contract.consumer.grpc.PithosConsumerContractTest.consumeGetContextFeedbackList(SynchronousMessages)
Failures:
1) Verifying a pact between stations-grpc and pithos-grpc - GetContextFeedbackList
1.1) has a matching body
expected 'application/protobuf;message=GetContextFeedbackListResponse' body but was 'application/grpc'
There were 1 pact failures```
nithyag.ganesan
2022-03-08 16:02
:wave: Hello, team! I am looking at using PACT to build contract tests for gRPC APIs in my organisation. Are the plugins (protobuf and grpc plugins) close to completion for me to try out a proof of concept?
matt.fellows
2022-03-08 23:28
there is a protobufs plugin but not a gRPC one yet (currently being worked on)
matt.fellows
2022-03-09 10:24
the protobufs plugin exists, and @uglyog is making some progress on the gRPC plugin, but I?ve yet to implement the plugin interface into the new v2.x.x branch
matt.fellows
2022-03-09 10:25
If you?re interested in contributing, we would want to create a new interface in https://github.com/pact-foundation/pact-go/tree/2.x.x/internal/native for supporting plugin based interactions.
matt.fellows
2022-03-09 10:26
see the pact.h header from the latest core (e.g. https://github.com/pact-foundation/pact-reference/releases/tag/libpact_ffi-v0.2.3)
nithyag.ganesan
2022-03-09 10:28
Sure Matt, I can give it a go, I am new to Golang, can do it with guidance. I will start looking the links you have shared. Is this channel the right place to ask questions as I try understanding the internals?
jarmy
2022-04-07 22:53
Hi everyone. My colleague @srinagasai.krishnasan and I have found a way to support gRPC and with pact-jvm. We are leveraging Envoy's https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_json_transcoder_filter: _This is a filter which allows a RESTful JSON API client to send requests to Envoy over HTTP and get proxied to a gRPC service_. This allows for consumer contract tests to be written for gRPC endpoints using traditional Pact consumer methods (JSON) and verified by traditional Pact verification routes.
mathias.duesterhoeft
2022-04-19 11:00
Hi there :wave: - while waiting for the pact js plugin driver to use the protobuf plugin I started wondering if one could also get things to work without the language support. I saw that on the provider side I could just let the CLI fire request against a provider server which returns messages in response. Is something similar also possible on the consumer side. It would be perfect if you could point me to a sample for this.
matt.fellows
2022-04-20 02:01
I don?t think there is a consumer side for it, but I did spike the new plugins in Pact JS recently.
matt.fellows
2022-04-20 03:40
FYI plugins are support in the node js core: https://github.com/pact-foundation/pact-js-core/blob/master/test/consumer.integration.spec.ts#L256-L327 (example test using protobufs, using one of the core Plugin protobuf message types to test it out :stuck_out_tongue: )
mathias.duesterhoeft
2022-04-20 06:54
Interesting - thanks a lot for the hint. Is it advisable to just start with pact-js-core for now?
matt.fellows
2022-04-20 06:59
It?s not intended for use outside of Pact JS, but no reason you couldn?t at least try it
matt.fellows
2022-04-20 07:00
I?m in the process of thinking about the API that we would expose to Pact JS, so perhaps after having a play with the core you?ll have some ideas too
rholshausen
2022-05-09 00:23
*Announcement: Pact gRPC/Protobuf Developer Preview 2*
The second developer preview of our gRPC/Protobuf support for Pact is now available. This update brings support for
using gRPC directly with Pact tests, using a gRPC mock server in consumer tests and the Pact verifier supporting gRPC
for provider verification tests.
Documentation can be found at https://github.com/pactflow/pact-protobuf-plugin#testing-a-grpc-service-method-interaction
gRPC is supported with:
* Pact-JVM 4.4.0-beta.1 and JUnit 5 (https://github.com/pact-foundation/pact-jvm/tree/master/consumer/junit5 and https://github.com/pact-foundation/pact-jvm/tree/master/provider/junit5)
* Pact-Rust consumer crate 0.9.2 (https://crates.io/crates/pact_consumer)
* Pact Verifier CLI 0.9.10 (https://github.com/pact-foundation/pact-reference/releases/tag/pact_verifier_cli-v0.9.10)
The Pactflow gRPC/Protobuf plugin can be downloaded from: https://github.com/pactflow/pact-protobuf-plugin/releases
For examples of Pact tests using gRPC:
* https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/consumer-jvm
* https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/consumer-rust
* https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/provider-jvm
*What is coming next:*
We want to bring gRPC support to more languages (see https://github.com/pact-foundation/pact-plugins/projects/1).
We will be updating the Pact Rust FFI with support for plugins, and then updating Pact-Go and Pact-JS with gRPC support.
james.weng
2022-05-16 02:15
Is the consumer-jvm tests supposed to be working? I installed the plugin following the instructions but the test is still failing
james.weng
2022-05-16 02:28
i used this 3 commands as per read me (downloading json and then the ziped plugin and then extracting it) looks ok?
wget https://github.com/pactflow/pact-protobuf-plugin/releases/download/v-0.1.5/pact-plugin.json -O ~/.pact/plugins/protobuf-0.1.5/pact-plugin.json
wget https://github.com/pactflow/pact-protobuf-plugin/releases/download/v-0.1.5/https://github.com/pactflow/pact-protobuf-plugin/releases/download/v-0.1.5/pact-protobuf-plugin-osx-x86_64.gz -O ~/.pact/plugins/protobuf-0.1.5/https://github.com/pactflow/pact-protobuf-plugin/releases/download/v-0.1.5/pact-protobuf-plugin-osx-x86_64.gz
gunzip -N ~/.pact/plugins/protobuf-0.1.5/https://github.com/pactflow/pact-protobuf-plugin/releases/download/v-0.1.5/pact-protobuf-plugin-osx-x86_64.gz
rholshausen
2022-05-16 02:31
The gunzip does not make the binary for the plugin executable, leading to https://github.com/pact-foundation/pact-plugins/issues/3
rholshausen
2022-05-16 02:32
check with
```? ls -la ~/.pact/plugins/protobuf-0.1.5/
total 311140
drwxr-xr-x 5 ronald ronald 4096 May 10 11:43 .
drwxr-xr-x 4 ronald ronald 4096 May 11 14:12 ..
drwxr-xr-x 2 ronald ronald 4096 May 12 10:58 log
-rw-r--r-- 1 ronald ronald 309 Apr 29 10:08 pact-plugin.json
-rwxr-xr-x 1 ronald ronald 316877496 May 10 11:43 pact-protobuf-plugin
drwxr-xr-x 4 ronald ronald 4096 May 5 13:10 protoc
-rw-r--r-- 1 ronald ronald 1699854 May 5 13:10 protoc-3.19.1-linux-x86_64.zip
drwxr-xr-x 2 ronald ronald 4096 May 12 10:58 tmp```
rholshausen
2022-05-16 02:36
`chmod +x ~/.pact/plugins/protobuf-0.1.5/pact-protobuf-plugin` will make it executable
james.weng
2022-05-16 02:39
sorry im very new to PACT im just wondering why this plugin cant be built it part of a library which you can import just like the PACT core module?
rholshausen
2022-05-16 02:41
It's new. It will improve.
We can't add direct support for every new protocol because there are so many languages we need to add the support to (there are 10+ Pact supported languages).
james.weng
2022-05-16 02:42
for provider-jvm also just run the test? do i need to explicitly copy the pact?
rholshausen
2022-05-16 02:45
You will need to either copy the pact file, or change the test to point to the file in the consumer build
matt.fellows
2022-05-16 03:04
> sorry im very new to PACT im just wondering why this plugin cant be built it part of a library which you can import just like the PACT core module?
As Ron said, we need a language independent way to create extension points into the system.
For example, you could (if you wanted) write a Pact plugin in whatever language you were comfortable in to support some new feature. Because you don?t need to add it to Pact JS or Pact JVM, you could also choose not to make it a public plugin as well, if that were desired
matt.fellows
2022-05-16 03:04
see also the design objectives here: https://github.com/pact-foundation/pact-plugins
james.weng
2022-05-16 03:05
Thanks for the background. :thumbsup: so if we need it running in the remote server[for CI/CD purposes] we need one of the steps to install the plugin before running the build?
rholshausen
2022-05-16 03:21
Yes, in the future we will have a CLI command to install the plugins, but for now you can just use a script
james.weng
2022-05-20 06:54
hi @rholshausen i was able to publish my consumer for grpc but when i tried publishing my provider it didnt give error but i didnt see the corresponding verification in pactflow.
I added
```import au.com.dius.pact.provider.junitsupport.loader.PactBroker;
import au.com.dius.pact.provider.junitsupport.loader.PactBrokerAuth;
......
......
@PactBroker(host = "http://airwallex.pactflow.io", scheme = "https",
authentication = @PactBrokerAuth( token = "xxxxx"))```
in the pact verification for the areacalculator example you all provided
and was using this gradle command
./gradlew -DpactPublishResults=true provider:test --tests *Pact*Test
Could you please kindly advise
matt.fellows
2022-05-20 10:59
according to https://docs.pact.io/implementation_guides/jvm/provider/gradle should it be `-P pact.verifier.publishResults=true` ?
james.weng
2022-05-23 03:01
Hey Matt still no joy. I was initially trying to integrate w Junit. Then i tried with gradle it still did not seem to publish
I added these lines in the gradle files as per docu. Am i missing something?
```plugins {
id "au.com.dius.pact" version '4.1.0'
}```
```
pact {
broker {
pactBrokerUrl = 'https://airwallex.pactflow.io'
pactBrokerToken = 'xxxx-xxxx"'
}
}```
james.weng
2022-05-23 03:02
What i see in the console when i run the test with publish results for provider ^^
james.weng
2022-05-23 03:22
sorry do u mind elaborating which property and whether i have an example to refer to what i should add in gradle
rholshausen
2022-05-23 03:25
Ok, when you run tests with Gradle, Gradle starts a new JVM environment to runs the tests. By passing the system property on the command line, you are setting it in the Gradle environment, but not in the test one. You need to tell Gradle to set it on the test environment.
rholshausen
2022-05-23 03:25
See the example at https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html
rholshausen
2022-05-23 03:25
You need to add a test block in your Gradle build file, and set the property there using `systemProperty`
james.weng
2022-05-23 03:28
something like this?
```test {
useJUnitPlatform()
systemProperty 'pact.verifier.publishResults', 'true'
}```
james.weng
2022-05-23 03:32
eg during CI/CD during a pr i want to tag it as a dev environment run , and then down the track staging , demo environment
rholshausen
2022-05-23 03:34
All the properties can be found at https://github.com/pact-foundation/pact-jvm/blob/master/docs/system-properties.md
rholshausen
2022-05-23 03:34
Tags are just generic labels. Environments are label specific for an environment
rholshausen
2022-05-23 03:36
Pact JVM doesn't support environments yet, but you can set the tags or git branch
james.weng
2022-05-23 03:47
is it necessary to put provider version in consumer files or it should be dynamically generated by PACT?
matt.fellows
2022-05-24 22:18
Right now the plan is to:
1. Finish gRPC / protobufs plugins (work in progress now)
2. Improve the documentation and visibility of the plugin ecosystem - make examples, tutorials, workshops etc. so that maintainers and contributors can use and create them
3. Get the rust core into all key languages (JVM, JS, Golang, .NET, Python)
4. Add plugin support to all key languages
Ruby is not on that list, but after (2) it could be done with community support
matt.fellows
2022-05-24 22:26
If you?re interested in picking up (3) for Ruby, there?s nothing stopping you now. See #libpact_ffi-users if you?d like to get involved! (and also probably pop your hand up in #pact-ruby)
matt.fellows
2022-06-07 05:36
To answer your https://pact-foundation.slack.com/archives/C5F4KFKR8/p1654563602737729 Tim:
The current protobuf (and gRPC) plugin is here: https://github.com/pactflow/pact-protobuf-plugin/#unsupported-features and is only supported in Pact JVM and Rust at the moment.
We will be adding support into Golang in the next little while, and then seeking feedback from the early adopters before rolling out widely
priyank.shah217
2022-06-08 10:18
@matt.fellows have one question, we have client(Android/IOS) basically sent protobuf over HTTP. does pact support protobuf + http along with json?
matt.fellows
2022-06-08 10:30
No swift package support that I?m aware of yet, but you could plausibly test it out for your android (JVM) project
matt.fellows
2022-06-15 03:36
Hey Cody! It?s in a branch, let me get it merged and you can have a play with the next beta release.
matt.fellows
2022-06-15 03:37
What do you plan on testing, btw? If you can give us some idea of your use case that would be great to know
matt.fellows
2022-06-15 03:37
also see the supported/unsupported features here: https://github.com/pactflow/pact-protobuf-plugin/#supported-features
matt.fellows
2022-06-15 03:37
When do you think you?d have the headspace/capacity to play? Right away?
cody
2022-06-15 03:40
Yeah, we have an intern (Runlin) digging into all of this right now. I think day 2 of his internship he had a pact broker running (locally) already and has been writing tests on the HTTP side of the house.
Happy to setup a zoom at some point to chat through everything. It definitely feels like right now we?re seeing what all ways we can abuse your tools :joy:
We have a bunch of weird requirements we?re trying to balance that leads use quite a few of your tools together.
? Ex: bidirectional contract testing looks pretty exciting, and I assume it?ll work better than the swagger-mock-validator we use today ? unless that?s what is used under the hood. But we?re using the OSS broker, so this isn?t supported (yet?).
? Using the wiremock-pact-generator to try to reduce the work on consumer teams. Runlin already has a POC using this too. But then we expect we?ll want BDCT to validate these much quicker, since my understanding is that a bunch of duplicative tests like we?d generate would be too slow to validate normally. But trying to see if this theory is true
? And now we get to add GRPC into the mix, because a lot of our API calls are over GRPC and ?faking? our contracts using grpcgateway seems? fallible.
matt.fellows
2022-06-15 03:46
> Ex: bidirectional contract testing looks pretty exciting, and I assume it?ll work better than the swagger-mock-validator we use today ? unless that?s what is used under the hood. But we?re using the OSS broker, so this isn?t supported (yet?).
It is, but we have customised it to address a few things.
cody
2022-06-15 03:47
Yeah, I heard from a couple of our teams that tested that it has? issues. :see_no_evil:
matt.fellows
2022-06-15 03:47
> Using the wiremock-pact-generator to try to reduce the work on consumer teams. Runlin already has a POC using this too. But then we expect we?ll want BDCT to validate these much quicker, since my understanding is that a bunch of duplicative tests like we?d generate would be too slow to validate normally. But trying to see if this theory is true
:100: do not use the standard pact verification here. It won?t just be duplicative, but likely very flakey and hard for the provider team to verify reliably.
matt.fellows
2022-06-15 03:48
> And now we get to add GRPC into the mix, because a lot of our API calls are over GRPC and ?faking? our contracts using grpcgateway seems? fallible.
got it. The plugin stuff is still very new and the authoring interface is still a level below where we?ll want to be (you?ll see what I mean), so I expect us to get through the beta phase and get to DX improvements on that. But conceptually we want to get that working first
cody
2022-06-15 03:48
Interesting. What would make them flakey? I assumed the main challenge on the providers (besides speed) would be handling a million different provider states.
matt.fellows
2022-06-15 03:50
> Interesting. What would make them flakey? I assumed the main challenge on the providers (besides speed) would be handling a million different provider states.
yes, that?s one. Because the wiremock pact generator doesn?t care for states (unless I?m mistaken), the provider won?t know how to deal with all of the scenarios.
cody
2022-06-15 03:50
wiremock itself has pretty solid state/scenario support, but yeah no idea how that translates into the pact.json
matt.fellows
2022-06-15 03:51
to my knowledge it doesn?t. But that would be an obvious addition to the tool
matt.fellows
2022-06-15 04:09
let me know how the research goes on the supported features. I?m keen to know if the protobufs/gRPC features we support initially will cover the majority of use cases
ringo
2022-06-27 12:18
@matt.fellows hello, any progress on Pact-Go + GRPC we could leverage at this very moment?
matt.fellows
2022-06-27 13:14
Yes, albeit it?s very much alpha and I?d expect the API to change. Given the introduction of the new V4 spec and plugins, it has (positively) impacted the vision for the API, which might take a few goes/feedback to get right
matt.fellows
2022-06-27 13:14
https://github.com/pact-foundation/pact-go/compare/2.x.x...feat/v3-type-system is the branch, which will merge back into 2.x.x once it?s clean and then a new beta version released
matt.fellows
2022-06-27 13:16
I haven?t yet put together a proper running gRPC example using the new interface, but this is an internal test that uses the public API: https://github.com/pact-foundation/pact-go/compare/2.x.x...feat/v3-type-system
matt.fellows
2022-06-27 13:18
https://github.com/pact-foundation/pact-go/blob/feat/v3-type-system/internal/native/message_server_test.go#L159is a working gRPC example using the FFI methods. Those methods aren?t exposed publicly but you can probably glean what it enables, and how you?d translate that into the actual API above
matt.fellows
2022-06-27 13:18
I plan to spend a bit more time on it tomorrow, if I?m up to it (have been unwell with The Rona??.)
matt.fellows
2022-06-28 02:42
matt.fellows
2022-06-28 12:43
New Golang release out with plugins support: https://pact-foundation.slack.com/archives/C9UTHTFFB/p1656420217055019
cc: @cody @ringo
andrevpuc
2022-07-04 15:06
Hello there,
I am totally new to Pact (and contract testing) but I am trying to implement a POC using Pact to make contract tests using the protobuf plugin. I already managed to right tests on the consumer side using the example https://github.com/pact-foundation/pact-plugins/blob/main/examples/gRPC/area_calculator/consumer-jvm/src/test/java/io/pact/example/grpc/consumer/PactConsumerTest.java as a guide. I am now trying to figure out how to make the tests on the provider side, but without much success so far. I have a few questions about it:
- I am trying to run the tests as a gradle task, in a similar way to what is done https://github.com/uglyog/pact-maven-amqp-test/tree/master/provider/src/test/java/com/github/uglyog/mavenamqp (I am, of course, using the V4 pact spec, not the one used in that example), without a mock server or anything. All I am doing is annotating a method with `@PactVerifyProvider` which then returns an instance of the protobuf message that I want to test against the one in the pact (generated by the consumer). Is that possible or even a correct approach?
- My provider sets the target to a MessageTestTarget like so
```@BeforeEach
void before(PactVerificationContext context) {
context.setTarget(new MessageTestTarget());
}```
Is that the right way to do it? Or should I used the PluginTestTarget instead? When I tried to use it, I got an exception about not providing a transport (which, as far as I could understand, can only be gRPC, which I am not using).
andrevpuc
2022-07-07 15:02
Sorry for bumping this :point_up: but I cannot find an example in Java that explains how a test on the provider side, using the protobuf plugin, is used.
andrevpuc
2022-07-08 07:24
Thanks for the reply @uglyog. I am going to give it a try, but does it mean that I must use gRPC in order to use the protobuf plugin? In my POC I am trying to use an asynchronous message without gRPC. Is it possible?
jkaur
2022-07-20 12:23
Cross Posting
https://pact-foundation.slack.com/archives/C5F4KFKR8/p1658299715045689
Do we have some good documentation around using Pact for writing contract tests for gRPC services ? Wondering if there are explained examples , before I try out things myself
yousafn
2022-07-20 12:34
Hey,
See https://pact-foundation.slack.com/archives/CEQBDD5U4/p1652055827960579 for current details and links to the relevant docs/examples
matt.fellows
2022-07-20 21:08
What language support do you need? Pact JVM and Go have support as of today
jkaur
2022-07-21 06:13
Pact-Go is what I am looking for , I am glad that the support exists :slightly_smiling_face:
I see that the current support is kind of a tech preview , when do we plan to take this GA
matt.fellows
2022-07-21 08:00
It?s pretty close to GA. Mostly I?m actively seeking user feedback on the API from real world usage. So it?s a bit of a chicken-and-egg situation. I don?t expect the API to change significantly, but there might be a few name changes to cleanup
matt.fellows
2022-07-21 08:00
Also, I think the verifier will unify into a single one - currently we have different ones for HTTP, Message and Plugins - but in V4 you can have multiple - so it makes sense to combine into a single one.
But that can potentially be managed in other ways
jkaur
2022-07-21 08:55
Sure , thanks @matt.fellows for the insights.
I am still trying to convince myself if we really need contract tests for grpc interactions . I did go through this nice article https://pactflow.io/blog/the-case-for-contract-testing-protobufs-grpc-avro/
Hence looking forward to atleast give this a try , it will be hard time convincing dev if I am not convinced myself
yousafn
2022-08-03 19:53
Posting for others benefits :-
This was cross-posted in pact-go
https://pact-foundation.slack.com/archives/C9UTHTFFB/p1659556214160889?thread_ts=1659548848.877339&cid=C9UTHTFFB
TL;DR - Alpine containers are not supported for language implementations using the Rust Core. :slightly_smiling_face:
robert.henzel
2022-08-04 09:10
Is there a way to resolve imports in proto. When I define grpcInteraction a put a path to protofile with service. When this protofile imports another file the test fails.
robert.henzel
2022-08-04 09:11
However, if I remove all imports from the service protofile. The test passes. It looks like the protoc binary does not have acces to the imports defined inside the service protofile.
robert.henzel
2022-08-04 09:19
My try to put multiple protofiles paths into interaction. path1 - service file, path2 - message definitions.
robert.henzel
2022-08-04 10:33
Maybe with `pact-plugin.json`? Can I define a paths where `protoc` will find imports?
matt.fellows
2022-08-04 13:34
at the moment I believe the full inclined proto must be provided. Is that right @uglyog?
matt.fellows
2022-08-04 13:36
It looks like you're running in docker on alpine. We don't support Alpine, or at least it's likely to be fraught with incompatibilities. I'd suggest using debian/Ubuntu
robert.henzel
2022-08-04 13:56
I see that it includes a parent_dir I can copy the imports there but don't know where it is.
robert.henzel
2022-08-04 14:56
if the import path is
```import "http://gitlab.com/elon/musk/hela/module/secure/pb/definition_modulerule.proto";```
what will be parent_dir?
robert.henzel
2022-08-04 18:41
Ok, I succeeded. If any1 have similar problem, ping me. Basically you have to copy all the imported proto files into `~/.pact/plugins/protobuf-0.1.8/protoc` or into directory where the service.proto is (
```"pact:proto": "` + service.proto + `", ```
Remeber when the import look like this
```import "http://gitlab.com/group/project/definitions.proto";```
then you have to copy it under
`~/.pact/plugins/protobuf-0.1.8/protoc/gitlab.com/group/project/definitions.proto` or
`./gitlab.com/group/project/definitions.proto`
robert.henzel
2022-08-05 15:02
Do you know what can be the cause of this error? It is inside a docker.
robert.henzel
2022-08-05 15:03
It looks like it cannot spawn a child process? protoc compiler is not downloaded to ~/.pact/plugins/protobuf-0.1.8 so I copied it from my local machine
```root@e3116ee8672e:~/.pact/plugins/protobuf-0.1.8# ls -la
total 29040
drwxr-xr-x 1 root root 4096 Aug 5 15:12 .
drwxr-xr-x 1 root root 4096 Aug 5 14:11 ..
drwxr-xr-x 2 root root 4096 Aug 5 14:14 log
-rw-r--r-- 1 root root 309 Jul 15 14:13 pact-plugin.json
-rwxr-xr-x 1 root root 28004704 Jul 15 14:13 pact-protobuf-plugin
drwxr-xr-x 4 root root 4096 Aug 5 15:12 protoc
-rw-r--r-- 1 root root 1699854 Aug 5 15:12 protoc-3.19.1-linux-x86_64.zip
root@e3116ee8672e:~/.pact/plugins/protobuf-0.1.8# ^C
root@e3116ee8672e:~/.pact/plugins/protobuf-0.1.8# ```
uglyog
2022-08-08 00:35
Ok, that is surprising, but then it is just shelling out to protoc, so I guess that is how that behaves.
uglyog
2022-08-08 00:37
That looks like the test can't start the plugin. Can you try start the same docker container and try run `pact-protobuf-plugin`
uglyog
2022-08-08 01:22
Ideally, it should behave without people needing to know that protoc is being used
robert.henzel
2022-08-08 08:42
Ideally it would be great if we could just provide a custom import path config file : `/home/robert/.pact/plugins/protobuf-0.1.8/pact_plugin.json`
Something like that:
```{
"pluginInterfaceVersion": 1,
"name": "protobuf",
"version": "0.1.8",
"executableType": "exec",
"minimumRequiredVersion": null,
"entryPoint": "pact-protobuf-plugin",
"entryPoints": {},
"dependencies": null,
"pluginConfig": {
"protocVersion": "3.19.1",
"downloadUrl": "https://github.com/protocolbuffers/protobuf/releases/download",
"imports": [
"/home/robert/go/src"
]
}
}```
Then the protoc binary would execute with
```protoc -I /home/robert/go/src ... service.proto```
That would simplify a lot the process cause right now I have to copy all the imports in this one path where service.proto is.
@uglyog
prapurna.manda3
2022-08-08 08:46
Hello All,
I am trying to write tests for Grpc api's (Java, Maven) using pact protobuf plugin. I installed plugin and trying to write tests using the example and I am getting this error while using Mockserver annotation.
`java.lang.NoSuchFieldError: TRANSPORT`
any help please?
robert.henzel
2022-08-08 09:08
I'm sorry but there was no problem. I just forgot to add
```"hostToBindTo": "127.0.0.1"```
So the `protoc` binary could not be invoked inside docker.
But the import parts still holds true.
robert.henzel
2022-08-08 09:09
I forgot to add
```"hostToBindTo": "127.0.0.1"```
into 'pact-plugin.json`
{
"manifestVersion": 1,
"pluginInterfaceVersion": 1,
"name": "protobuf",
"version": "0.1.8",
"executableType": "exec",
"entryPoint": "pact-protobuf-plugin",
"pluginConfig": {
"protocVersion": "3.19.1",
"downloadUrl": "https://github.com/protocolbuffers/protobuf/releases/download",
"hostToBindTo": "127.0.0.1"
}
}
prapurna.manda3
2022-08-08 10:43
```import au.com.dius.pact.consumer.MockServer;
import au.com.dius.pact.consumer.dsl.PactBuilder;
import au.com.dius.pact.consumer.junit.MockServerConfig;
import au.com.dius.pact.consumer.junit5.PactConsumerTestExt;
import au.com.dius.pact.consumer.junit5.PactTestFor;
import au.com.dius.pact.consumer.junit5.ProviderType;
import au.com.dius.pact.consumer.model.MockServerImplementation;
import au.com.dius.pact.core.model.PactSpecVersion;
import au.com.dius.pact.core.model.V4Interaction;
import au.com.dius.pact.core.model.V4Pact;
import au.com.dius.pact.core.model.annotations.Pact;
import com.atoss.template.servicetemplate.adapter.handler.grpc.employee.v1.EmployeeServiceGrpc;
import com.atoss.template.servicetemplate.adapter.handler.grpc.employee.v1.Message;
import com.atoss.template.servicetemplate.adapter.handler.grpc.employee.v1.MessageRequest;
import com.google.protobuf.InvalidProtocolBufferException;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import java.util.List;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static http://org.hamcrest.Matchers.is;
@ExtendWith(PactConsumerTestExt.class)
@PactTestFor(providerName = "adapter-handler-grpc", providerType = ProviderType.SYNCH_MESSAGE, pactVersion = PactSpecVersion.V4)
public class PactConsumerTest {
@Pact(consumer = "adapter-client-grpc")
V4Pact testMsg(PactBuilder builder) {
return builder
// Tell Pact we need the Protobuf plugin
.usingPlugin("protobuf", "0.1.8")
// We will use a V4 synchronous message interaction for the test
.expectsToReceive("message", "core/interaction/synchronous-message")
// We need to pass all the details for the interaction over to the plugin
.with(Map.of(
// Configure the proto file, the content type and the service we expect to invoke
"pact:proto", builder.filePath("../adapter-handler-grpc-proto/src/main/resources/proto/EmployeeService.proto"),
"pact:content-type", "application/grpc",
"pact:proto-service", "EmployeeService/getEmployee",
// Details on the request message (ShapeMessage) we will send
"request", Map.of("id", 1),
// Details on the response message we expect to get back (AreaResponse)
"response", List.of(
Map.of(
"msg", "Hello Grpc"
)
)
))
.toPact();
}
@Test
@PactTestFor(pactMethod = "testMsg")
@MockServerConfig(implementation = MockServerImplementation.Plugin, registryEntry = "protobuf/transport/grpc")
void readMessage(MockServer mockServer, V4Interaction.SynchronousMessages interaction) throws InvalidProtocolBufferException {
ManagedChannel channel = ManagedChannelBuilder.forTarget("[::1]:" + mockServer.getPort())
.usePlaintext()
.build();
EmployeeServiceGrpc.EmployeeServiceBlockingStub clientStub = EmployeeServiceGrpc
.newBlockingStub(channel);
Message response = clientStub.getMessage(MessageRequest.newBuilder().setId(1).build());
System.out.println("after response");
assertThat(response.getMsg(), is("{\"msg\":\"Hello Grpc\"}"));
}
}```
ssomepalli
2022-08-08 17:38
Hi, Copied the protofiles into directory where the service.proto is (
```"pact:proto": "` + service.proto + `", ```
But, faced the below error
uglyog
2022-08-09 07:08
`Pact file does not contain any Protobuf descriptors` can you check the generated Pact file. That means there are no descriptors stored in the file.
uglyog
2022-08-09 07:11
I have just released 0.1.10 which allows additional imports. See https://github.com/pactflow/pact-protobuf-plugin#additionalincludes-string-or-liststring
prapurna.manda3
2022-08-09 07:18
Hello sure,
`[INFO] Running com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest`
09:15:46.174 [main] DEBUG http://au.com.dius.pact.consumer.junit5.PactConsumerTestExt - Found @PactTestFor annotation on test method void com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest.readMessage() throws com.google.protobuf.InvalidProtocolBufferException
09:15:46.175 [main] DEBUG http://au.com.dius.pact.consumer.junit5.PactConsumerTestExt - Found @PactTestFor annotation on test class com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest
09:15:46.324 [main] DEBUG http://au.com.dius.pact.consumer.junit5.PactConsumerTestExt - providerInfo = ProviderInfo(providerName=adapter-handler-grpc, hostInterface=, port=, pactVersion=V4, providerType=SYNCH_MESSAGE, https=false, mockServerImplementation=Default, keyStorePath=, keyStoreAlias=, keyStorePassword=, privateKeyPassword=)
09:15:46.333 [main] DEBUG http://au.com.dius.pact.consumer.junit5.PactConsumerTestExt - Looking for @Pact method named 'testMsg' for provider 'adapter-handler-grpc'
09:15:46.345 [main] DEBUG http://au.com.dius.pact.consumer.junit5.PactConsumerTestExt - Invoking method 'testMsg' to get Pact for the test 'readMessage'
09:15:46.911 [Thread-1] WARN http://au.com.dius.pact.core.support.Metrics -
Please note: we are tracking events anonymously to gather important usage statistics like JVM version
and operating system. To disable tracking, set the 'pact_do_not_track' system property or environment
variable to 'true'.
09:15:46.949 [main] INFO http://au.com.dius.pact.consumer.junit5.PactConsumerTestExt - Writing pacts out to default directory
[ERROR] Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 0.801 s <<< FAILURE! - in com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest
[ERROR] readMessage Time elapsed: 0.772 s <<< ERROR!
java.lang.NoSuchFieldError: TRANSPORT
[ERROR] com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest Time elapsed: 0.799 s <<< FAILURE!
java.lang.AssertionError:
The following methods annotated with @Pact were not executed during the test: PactConsumerTest.testMsg
If these are currently a work in progress, add a @Disabled annotation to the method
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] PactConsumerTest The following methods annotated with @Pact were not executed during the test: PactConsumerTest.testMsg
If these are currently a work in progress, add a @Disabled annotation to the method
[ERROR] Errors:
[ERROR] PactConsumerTest.readMessage » NoSuchField TRANSPORT
[INFO]
[ERROR] Tests run: 16, Failures: 1, Errors: 1, Skipped: 0
[INFO]
jaroslavburi
2022-08-09 09:46
Hello there :lightsaber:
I am trying to run the *grpc example* and my code fails here
```return builder
.usingPlugin("protobuf", "0.1.10")```
with
```java.lang.NoSuchMethodError: 'java.lang.String au.com.dius.pact.core.support.Utils.lookupEnvironmentValue$default(au.com.dius.pact.core.support.Utils, java.lang.String, kotlin.jvm.functions.Function1, kotlin.jvm.functions.Function1, int, java.lang.Object)'
at io.pact.plugins.jvm.core.DefaultPluginManager.pluginInstallDirectory(PluginManager.kt:854)
at io.pact.plugins.jvm.core.DefaultPluginManager.loadPluginManifest(PluginManager.kt:812)
at io.pact.plugins.jvm.core.DefaultPluginManager.loadPlugin(PluginManager.kt:341)
at au.com.dius.pact.consumer.dsl.PactBuilder.usingPlugin(PactBuilder.kt:96)
at io.pact.example.grpc.consumer.PactConsumerTest.calculateRectangleArea(PactConsumerTest.java:45)```
I have the plugin binary and in `/.pact/plugins/protobuf-0.1.10` with both *pact-plugin.json* and *pact-protobuf-plugin*
I am not very good with java - its complaining it doesn't have the `default` method for some reason?
```cat pact-plugin.json
{
"manifestVersion": 1,
"pluginInterfaceVersion": 1,
"name": "protobuf",
"version": "0.1.10",
"executableType": "exec",
"entryPoint": "pact-protobuf-plugin",
"pluginConfig": {
"protocVersion": "3.19.1",
"downloadUrl": "https://github.com/protocolbuffers/protobuf/releases/download",
"hostToBindTo": "127.0.0.1"
}
}```
ssomepalli
2022-08-09 17:47
Hi Team,
I followed pact plugin example and tried to implement the consumer test case Chronos project.
Initially I faced the following issues.
1)Issue with wrapper types
->It is not able to load the google.protobuf.StringValue
To fix this, I changed to ?string?.
2)Issue with import statements
->It is not able to import ?delivery/dispatch/common.proto?
To fix this merged the whole common.proto file content into current Proto file(ad_break_api.proto)
With the above two fixes I am able execute the consumer test case properly.
Now, I am trying to execute the consumer test case without merging the common.proto file content
->To resolve the import statement (?delivery/dispatch/common.proto?). I copied all import files into Chronos project.
Hi, Copied the protofiles into directory where the service.proto is (
"pact:proto": "` + service.proto + `",
But, faced the below error:
uglyog
2022-08-09 23:50
Can you check that you are using the same version for all the Pact-JVM libraries
uglyog
2022-08-09 23:52
The latest version of the gRPC plugin now allows you to configure additional import directories.
prapurna.manda3
2022-08-10 07:33
```<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java-util</artifactId>
<version>3.21.4</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>io.pact.plugin.driver</groupId>
<artifactId>core</artifactId>
<version>0.0.7</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>au.com.dius.pact.consumer</groupId>
<artifactId>junit5</artifactId>
<version>4.4.0-beta.3</version>
</dependency>```
prapurna.manda3
2022-08-10 07:34
and protobuf plugin 0.1.10 , do I need any other dependencies? Is there an example with maven java?
prapurna.manda3
2022-08-10 07:36
`[main] INFO http://au.com.dius.pact.consumer.junit5.PactConsumerTestExt - Writing pacts out to default directory`
[ERROR] Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 0.564 s <<< FAILURE! - in com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest
[ERROR] readMessage{MockServer, SynchronousMessages} Time elapsed: 0.541 s <<< ERROR!
java.lang.NoSuchMethodError: http://au.com.dius.pact.core.support.Utils.lookupEnvironmentValue$default(Lau/com/dius/pact/core/support/Utils;Ljava/lang/String;Lkotlin/jvm/functions/Function1;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)Ljava/lang/String;
at com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest.MessageTest(PactConsumerTest.java:36)
[ERROR] com.atoss.template.servicetemplate.adapter.client.grpc.employee.pact.PactConsumerTest Time elapsed: 0.562 s <<< FAILURE!
java.lang.AssertionError:
The following methods annotated with @Pact were not executed during the test: PactConsumerTest.MessageTest
If these are currently a work in progress, add a @Disabled annotation to the method
09:35:24.510 [Thread-1] DEBUG io.pact.plugins.jvm.core.DefaultPluginManager - SHUTDOWN - shutting down all plugins
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] PactConsumerTest The following methods annotated with @Pact were not executed during the test: PactConsumerTest.MessageTest
If these are currently a work in progress, add a @Disabled annotation to the method
[ERROR] Errors:
[ERROR] PactConsumerTest.MessageTest:36 » NoSuchMethod http://au.com.dius.pact.core.support.U...
[INFO]
[ERROR] Tests run: 16, Failures: 1, Errors: 1, Skipped: 0
[INFO]
uglyog
2022-08-10 07:39
There is something wrong with your dependencies, you shouldn't need to add that library, it should be automatically added from the Pact-JVM ones
prapurna.manda3
2022-08-10 07:41
0.1.3 for which dependency? I only have one Pact dependency consumer.
I am using latest from maven
```<dependency>
<groupId>au.com.dius.pact.consumer</groupId>
<artifactId>junit5</artifactId>
<version>4.4.0-beta.3</version>
</dependency>```
prapurna.manda3
2022-08-10 08:15
Thank you ,It was solved after upgrading dependency, I have installed plugin as per readme in $Home/.pact/plugins/protobuf-0.1.10
but I am getting PluginNotFoundException , are there any additional steps required?
prapurna.manda3
2022-08-10 20:04
Hi Team,
I am trying protobuf plugin for Grpc tests with Java, maven.
I am getting this error while startup
`ERROR http://au.com.dius.pact.consumer.dsl.PactBuilder - Init call to plugin protobuf failed: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception`
`io.pact.plugins.jvm.core.DefaultPluginManager - Init call to plugin protobuf failed`
robert.henzel
2022-08-10 20:22
How do you define a `google.protobuf.Timestamp` struct in grpcInteraction, which is
```/ In JSON format, the Timestamp type is encoded as a string in the
// [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt) format. That is, the
// format is "{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z"
// where {year} is always expressed using four digits while {month}, {day},
// {hour}, {min}, and {sec} are zero-padded to two digits each. The fractional
// seconds, which can go up to 9 digits (i.e. up to 1 nanosecond resolution),
// are optional. The "Z" suffix indicates the timezone ("UTC"); the timezone
// is required. A proto3 JSON serializer should always use UTC (as indicated by
// "Z") when printing the Timestamp type and a proto3 JSON parser should be
// able to accept both UTC and other timezones (as indicated by an offset).
//
// For example, "2017-01-15T01:30:15.01Z" encodes 15.01 seconds past
// 01:30 UTC on January 15, 2017.
//
// In JavaScript, one can convert a Date object to this format using the
// standard
// [toISOString()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString)
// method. In Python, a standard `datetime.datetime` object can be converted
// to this format using
// [`strftime`](https://docs.python.org/2/library/time.html#time.strftime) with
// the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in Java, one can use
// the Joda Time's [`ISODateTimeFormat.dateTime()`](
// http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime%2D%2D
// ) to obtain a formatter capable of generating timestamps in this format.
//
//
message Timestamp {
// Represents seconds of UTC time since Unix epoch
// 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
// 9999-12-31T23:59:59Z inclusive.
int64 seconds = 1;
// Non-negative fractions of a second at nanosecond resolution. Negative
// second values with fractions must still have non-negative nanos values
// that count forward in time. Must be from 0 to 999,999,999
// inclusive.
int32 nanos = 2;
}```
This approach does not work
```grpcInteraction := `{
"pact:proto": "` + path1 + `",
"pact:proto-service": "Service/Method",
"pact:content-type": "application/protobuf",
"request": {
"id": "matching(type, '00000000-0000-0000-0000-000000000001')"
},
"response": {
"id": "matching(type, '00000000-0000-0000-0000-000000000001')",
"card_manufacture_last_send" : {
"seconds":"matching(number, 1)",
"nanos":"matching(number, 1)"
}
}
}````
This too, does not work
```grpcInteraction := `{
"pact:proto": "` + path1 + `",
"pact:proto-service": "Service/Method",
"pact:content-type": "application/protobuf",
"request": {
"id": "matching(type, '00000000-0000-0000-0000-000000000001')"
},
"response": {
"id": "matching(type, '00000000-0000-0000-0000-000000000001')",
"card_manufacture_last_send": "matching(type,'2022-08-10T20:25:04.835478326Z')"
}
}````
The rest of nested fields work.
robert.henzel
2022-08-10 21:10
message is defined like that:
```message GetCardConfigManufacturerResponse {
string id = 1;
google.protobuf.Timestamp card_manufacture_last_send = 2;
}```
uglyog
2022-08-11 07:52
Timestamp is just a message, there is nothing special about it. What doesn't work?
robert.henzel
2022-08-11 07:54
It' very hard to tell cause I don't get any error feedback. The test just fails. It goes throught buy the response is empty.
uglyog
2022-08-11 07:58
Maybe try a number bigger than 1. Current epoc timestamp is 1660204683. Also, set the nanos to 0, you don't need that precision.
uglyog
2022-08-11 08:07
```Caused by: io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /[0:0:0:0:0:0:0:1]:50052
Caused by: java.net.ConnectException: Connection refused```
robert.henzel
2022-08-11 08:08
Can you point where in pact-protobuf-plugin source code should I set log print so I can debugg it. Some functions?
prapurna.manda3
2022-08-11 08:10
I am running using maven install and It runs tests as well. It is from local ide not from container
uglyog
2022-08-11 08:12
This would probably be the functions: https://github.com/pactflow/pact-protobuf-plugin/blob/main/src/message_builder.rs#L119
uglyog
2022-08-11 08:17
I'll need to chat to @matt.fellows because the log level is not being passed through to the plugin
matt.fellows
2022-08-11 08:19
Interesting. I thought the plugin logs went to a separate log file? How do I get them into the main one?
uglyog
2022-08-11 08:34
They go to both, but they are staying as DEBUG so the trace level is not being passed through
matt.fellows
2022-08-11 08:37
Ok cool. I think the issue is that setloglevel needs to panic if used like the above. As you know, the log level can't be changed after the runtime is established
matt.fellows
2022-08-11 08:38
When I'm back at my desk Robert I'll show you how to address that. I think it's just a matter of setting the log level before creating the NewSyncPact...
robert.henzel
2022-08-11 08:41
NewSyncPact :confused: I've set it like in your example.
https://github.com/pact-foundation/pact-go/blob/feat/v3-type-system/examples/grpc/grpc_consumer_test.go
jaroslavburi
2022-08-11 14:29
*FYI*
by trial and error I checked that the example project
https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/consumer-jvm
works only up until `pact-protobuf-plugin` version *0.1.8*
*0.1.9 and 0.1.10* are failing to run
uglyog
2022-08-11 23:07
I think I know what it is. See https://pact-foundation.slack.com/archives/CEQBDD5U4/p1660228191109989
uglyog
2022-08-11 23:08
In the manifest file, there is a `"hostToBindTo"` field. Try change that from `"127.0.0.1"` to `"::1"`
uglyog
2022-08-11 23:30
Oh, we now default the plugin to IP4 addresses to be able to run from within docker containers. In the manifest file, there is a `"hostToBindTo"` field. Try change that from `"127.0.0.1"` to `"::1"`
jaroslavburi
2022-08-12 06:05
there is a change in behaviour for sure, now it fails with
```08:03:26.959 [Test worker] ERROR a.c.d.pact.consumer.dsl.PactBuilder - Plugin process did not output the correct startup message in 10000 ms - got null```
jaroslavburi
2022-08-12 06:07
with `127.0.0.1`
it fails with
```Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /0:0:0:0:0:0:0:1:58408 ```
I saw it in some other thread here as well
uglyog
2022-08-12 06:07
Ok, maybe just remove that field. I'm rolling out a fix to Pact-JVM now as well
jaroslavburi
2022-08-12 08:24
on plugin *0.1.10*
having
```"request", StringValue.of("aaa"),
"response", Map.of("id", "bbb")```
Maybe I am not calling it correct but
```interaction.getRequest().getContents().getValue()```
*is empty*
(with `Map.of("email", "aaa")` it works fine)
robert.henzel
2022-08-12 08:43
When I set it before NewSyncPact. and it switches between DEBUG and INFO. But TRACE has no impacto on logs in terminal.
robert.henzel
2022-08-12 08:54
Something is wrong with parsing a table in grpcInteraction. The provider test always passes.
Example grpcInteraction:
```"bulk_addresses": [
{
"id": "matching(type, '00000000-0000-0000-0000-000000000004')",
"title": "matching(type, 'resTitle')",
"first_name": "matching(type, 'resFirstName')",
"middle_name": "matching(type, 'resMiddleName')",
"address_line1": "matching(type, 'resAddressLie1')"
}
]```
When a field is set as `repeated` in proto file, the changes in consumer grpcInteraction does not have a impact on provider test. As soon as I remove the `[]` the test behaves as it should.
uglyog
2022-08-15 02:42
How do you define a service that just has a string as the message? I get an error from protoc that it requires a message type.
uglyog
2022-08-15 02:47
What is your expectation on the provider side? That the provider is not returning any examples, and it should fail?
jaroslavburi
2022-08-15 06:12
```syntax = "proto3";
import "google/protobuf/wrappers.proto";
package area_calculator;
option php_generic_services = true;
option go_package = "io.pact/area_calculator";
service Calculator {
rpc getUserByEmail(UserEmail) returns (User) {}
rpc getUserByEmailGoogle(google.protobuf.StringValue) returns (User) {}
rpc getUserByEmailGoogleMessage(UserEmailGoogle) returns (User) {}
}
message UserEmailGoogle {
google.protobuf.StringValue email = 1;
}
message UserEmail {
string email = 1;
}
message User {
string id = 1;
}```
jaroslavburi
2022-08-15 06:15
ah right its not visible from the first example as its imported `StringValue.of` is from `google.protobuf`
I am also planning to ask my devs why do they use it instead of `string` today :thinking_face:
uglyog
2022-08-15 06:16
`string` won't work (that is what I tried), it has to be a message type and that is a primitive
robert.henzel
2022-08-15 08:05
I expect the provider to fail.
Let me explain further. When I deliberately make the provider response with a data that is not matching data in the grpcInteraction then the provider fails only when in grpcInteraction the repeated field is defined without the the square brackets. e.g.
```"bulk_addresses":
{
"id": "matching(type, '00000000-0000-0000-0000-000000000004')",
"title": "matching(type, 'resTitle')",
"first_name": "matching(type, 'resFirstName')",
"middle_name": "matching(type, 'resMiddleName')",
"address_line1": "matching(type, 'resAddressLie1')"
}```
But if I add the brackets, no matter how is the provider response data defined the test always passes.
jaroslavburi
2022-08-15 13:27
you probably already know, but just in case, this is how the `StringValue` looks like
```message StringValue {
// The string value.
string value = 1;
}```
jaroslavburi
2022-08-15 13:31
when trying to use
```import "google/api/annotations.proto";```
I am getting
`pact_protobuf_plugin::protoc: Protoc stderr: google/api/annotations.proto: File not found.`
The *annotations.proto* is in
``` consumer-jvm/build/extracted-include-protos/main/google/api/annotations.proto```
*on the other hand*
```import "google/protobuf/wrappers.proto";```
gets found at
```consumer-jvm/build/extracted-include-protos/main/google/protobuf/wrappers.proto```
uglyog
2022-08-16 00:53
The protoc bundle does not include those files. You can see what is there in `~/.pact/plugins/protobuf-0.1.10/protoc/include/`
uglyog
2022-08-16 00:54
If you have the files somewhere, you can configure an additional import directory for the plugin
uglyog
2022-08-16 01:03
Just an aside, wouldn't `google/api` be internal files? They are not art of the released bundle
jaroslavburi
2022-08-16 07:16
Thanks, it works fine now :slightly_smiling_face: should have searched the slack channel better
The google/api are here https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto
jaroslavburi
2022-08-16 10:47
is there a way to include *Authorization header in provider verification?*
I found example which is not using this plugin
```@TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider.class)
void pactVerificationTestTemplate(PactVerificationContext context, HttpRequest request) {
request.addHeader("Authorization", "Bearer ...");
context.verifyInteraction();
}```
Unfortunately it fails with
`No ParameterResolver registered for parameter [org.apache.http.HttpRequest arg1] in method [void io.pact.example.grpc.consumer.PactProviderTest.pactVerificationTestTemplate(http://au.com.dius.pact.provider.junit5.PactVerificationContext,org.apache.http.HttpRequest)].`
uglyog
2022-08-17 03:56
I've updated the plugin (version 0.1.11) which supports StringValue now. You can use a normal string in the setup for it. I.e.
```"request", "aaa",
"response", Map.of("id", "bbb")```
uglyog
2022-08-17 03:57
Can you try plugin version 0.1.11, I put a fix in for using arrays to configure repeated fields.
uglyog
2022-08-17 03:59
Because it's using the plugin, there is no HTTP request to inject into the test.
jaroslavburi
2022-08-17 06:01
Thanks a lot :slightly_smiling_face: This is last step missing to be able to demo this plugin to our engineering here at Pure Storage :wink:
robert.henzel
2022-08-17 11:52
Still the same.
When I remove the brackets the test does return an error:
robert.henzel
2022-08-17 11:54
After adding the brackets:
The test passes. There is only one instance of bulkaddress. And I know it is different than pact.json expect. Nevertheless the test passes.
uglyog
2022-08-21 23:30
You mentioned it does not work. Can you share the updated manifest file and debug logs?
robert.henzel
2022-08-22 16:09
FYI
When I run the tests like
```LOG_LEVEL=debug go test ./...```
It does show the debugging informaton. I wonder if there can be some conflict between `logrus` and `hashicorp/logutils` and this variable `LOG_LEVEL` sets it for both of them.
matt.fellows
2022-08-22 22:03
`LOG_LEVEL` is a pretty standard environment variable. Are you suggesting we default to a different one for Pact to avoid conflict with logrus?
uglyog
2022-08-24 03:54
I'm having trouble replicating this. I always get an error if something is not correct.
```grpcInteraction := `{
"pact:proto": "` + path + `",
"pact:proto-service": "Calculator/calculateMulti",
"pact:content-type": "application/protobuf",
"request": {
"shapes": [
{
"rectangle": {
"length": "matching(number, 3)",
"width": "matching(number, 4)"
}
},
{
"square": {
"edge_length": "matching(number, 3)"
}
}
]
},
"response": {
"value": [
{
"id": "matching(regex, '\\d+', '1234')",
"shape": "matching(type, 'rectangle')",
"value": "matching(number, 12)"
},
{
"id": "matching(type, '4')",
"shape": "matching(type, 'square')",
"value": "matching(number, 9)"
}
]
}
}````
uglyog
2022-08-24 03:54
On the provider side:
```Verifying a pact between grpc-consumer-go and area-calculator-provider
calculate rectangle area request
Given a Calculator/calculateMulti request
with an input .area_calculator.AreaRequest message
will return an output .area_calculator.AreaResponse message [FAILED]
generates a message which
has a matching body (FAILED)
Failures:
1) Verifying a pact between grpc-consumer-go and area-calculator-provider - calculate rectangle area request
1.1) has a matching body
$.value[0].id -> Expected '0abc' to match '\d+'
There were 1 pact failures```
uglyog
2022-08-24 03:55
If I remove the square brackets, then I did notice that the paths for the matchers are not correct, but then I still get an error on the provider, just a different one
uglyog
2022-08-24 03:57
I.e.
```Failures:
1) Verifying a pact between grpc-consumer-go and area-calculator-provider - calculate rectangle area request
1.1) has a matching body
$.value[0].id -> Expected '1234' to be equal to '0abc'
$.value -> Expected repeated field 'value' to have 1 values but received 2 values
$.value[0].shape -> Expected 'rectangle' to be equal to 'box'```
uglyog
2022-08-24 04:10
However, if I use a each value matcher, I do get the issue you have mentioned:
```"value": {
"pact:match": "eachValue(matching($'area'))",
"area": {
"id": "matching(regex, '\\d+', '1234')",
"shape": "matching(type, 'rectangle')",
"value": "matching(number, 12)"
}
}```
robert.henzel
2022-08-25 06:12
D you know why I get this 2 value error. Also this error is not consistent, one time it is password another user.
```2022/08/24 17:38:41 [DEBUG] initialising rust mock server interface
2022/08/24 17:38:41 [DEBUG] log_to_stdout res 0
2022/08/24 17:38:41 [DEBUG] waiting for port 8222 to become available
2022/08/24 17:38:41 [DEBUG] executing verifier FFI with args [--hostname localhost --port 8222 --user admin --password 3@Gqd4Z7MkcLqD --broker-url https://pact.internal.apps-broker.io/ --provider-version dev --provider-name module-producer --loglevel info]
2022-08-24T15:38:41.195073Z TRACE ThreadId(01) mio::poll: registering event source with poller: token=Token(0), interests=READABLE | WRITABLE
2022-08-24T15:38:41.195726Z ERROR ThreadId(01) pact_ffi::verifier::verifier: error verifying Pact: "error: The argument '--user <user>' requires 1 values, but 2 were provided\n\nUSAGE:\n pact_verifier_cli --broker-url <broker-url> --dir <dir>... --file <file>... --hostname <hostname> --loglevel <loglevel> --password <password> --port <port> --provider-name <provider-name> --provider-version <provider-version> --scheme <scheme> --url <url>... --user <user>\n\nFor more information try --help" Error { message: "error: The argument '--user <user>' requires 1 values, but 2 were provided\n\nUSAGE:\n pact_verifier_cli --broker-url <broker-url> --dir <dir>... --file <file>... --hostname <hostname> --loglevel <loglevel> --password <password> --port <port> --provider-name <provider-name> --provider-version <provider-version> --scheme <scheme> --url <url>... --user <user>\n\nFor more information try --help", kind: WrongNumberOfValues, info: Some(["user"]) }
=== RUN TestGrpcProvider/Provider_pact_verification
plugin_verifier.go:56: an unknown error ocurred when verifying the provider (this indicates a defect in the framework
=== CONT TestGrpcProvider
provider_test.go:166:
Error Trace: provider_test.go:166
Error: Received unexpected error:
an unknown error ocurred when verifying the provider (this indicates a defect in the framework
Test: TestGrpcProvider
--- FAIL: TestGrpcProvider (0.06s)
--- FAIL: TestGrpcProvider/Provider_pact_verification (0.00s)
FAIL
FAIL command-line-arguments 2.891s
FAIL```
I think it has something to do with parsing the variables and calling ffi from library.
I use *Pact-go*
matt.fellows
2022-08-25 06:19
Can you please check if you have any `PACT_BROKER_` variables in your shell? I think it might be that
matt.fellows
2022-08-25 06:20
I need to migrate the verifier code to use the newer handles interface, which would improve this (just created a quick https://github.com/pact-foundation/pact-go/issues/237 for this)
robert.henzel
2022-08-25 06:20
```? producer git:(ROSI-145) ? echo $PACT_BROKER_PASSWORD
3GX6u4sa7MdsacLqD```
matt.fellows
2022-08-25 06:21
OK thanks. If you set the environment variables, you don?t need to set them in code (and vice versa). That?s why it thinks it?s getting 2 values instead of one
robert.henzel
2022-08-25 06:35
For any1 concerned:
SOLUTION:
```In ~/.zshrc or ~/.bashrc:
export PACT_BROKER_BASE_URL=https://pact.internal.apps-apps.com/
export PACT_BROKER_USERNAME=admin
export PACT_BROKER_PASSWORD=3GX6udsadsacLqD```
In code
```err = verifier.VerifyProvider(t, provider.VerifyPluginRequest{
ProviderAddress: "http://localhost:8222",
Provider: "module-producer",
BrokerURL: os.Getenv("PACT_BROKER_BASE_URL"),
ProviderVersion: "dev",
})
assert.NoError(t, err)```
*Don't fill the username and password VerifyPluginRequest fields. They will be taken from enviroment vars.*
matt.fellows
2022-08-25 06:40
I?m going to fix this Robert. The implementation under the hood essentially wraps the CLI verifier. There is a newer FFI set of methods that I can use in Pact Go which will improve this experience. I?ll aim to get this added in the next week or so
robert.henzel
2022-08-25 06:54
Ok no problem. I just wonder if I should put my solutinons in form of github issue so more people can find the answer? Plus for me it is a github activity :slightly_smiling_face:.
matt.fellows
2022-08-25 07:04
haha you can raise it, and i?ll close it when I get the above issue done (and you?ll get the kudos!)
uglyog
2022-08-26 01:30
@robert.henzel Ok, I think I have have fixed this issue. Updated plugin released 0.1.12
robert.henzel
2022-08-26 07:08
Omg, Thank your work, I did not notice you were writing here. Please feel free to mention me with @. Cause I was talking with Matt but totally missed your messages. I'll update to v 0.1.12 and will return with feedback.
ssomepalli
2022-08-30 18:39
```Plugin protobuf failed to start a mock server: Failed to start gRPC mock server: Pact file does not contain any Protobuf descriptors
io.pact.plugins.jvm.core.PactPluginMockServerErrorException: Plugin protobuf failed to start a mock server: Failed to start gRPC mock server: Pact file does not contain any Protobuf descriptors
at app//io.pact.plugins.jvm.core.DefaultPluginManager.startMockServer(PluginManager.kt:604)
at app//au.com.dius.pact.consumer.PluginMockServer.start(PluginMockServer.kt:45)
at app//au.com.dius.pact.consumer.junit5.PactConsumerTestExt.beforeTestExecution(PactConsumerTestExt.kt:220)
at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeTestExecutionCallbacks$5(TestMethodTestDescriptor.java:188)
at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeMethodsOrCallbacksUntilExceptionOccurs$6(TestMethodTestDescriptor.java:199)
at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:199)
at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeTestExecutionCallbacks(TestMethodTestDescriptor.java:187)
at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:133)
at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base@11.0.16/java.util.ArrayList.forEach(ArrayList.java:1541)
at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base@11.0.16/java.util.ArrayList.forEach(ArrayList.java:1541)
at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy2.stop(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Suppressed: java.lang.RuntimeException: Mock server is not running
at au.com.dius.pact.consumer.PluginMockServer.stop(PluginMockServer.kt:59)
at au.com.dius.pact.consumer.junit5.JUnit5MockServerSupport.close(JUnit5MockServerSupport.kt:14)
at au.com.dius.pact.consumer.junit5.PactConsumerTestExt.afterTestExecution(PactConsumerTestExt.kt:489)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAfterTestExecutionCallbacks$9(TestMethodTestDescriptor.java:233)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$13(TestMethodTestDescriptor.java:273)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeAllAfterMethodsOrCallbacks$14(TestMethodTestDescriptor.java:273)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeAllAfterMethodsOrCallbacks(TestMethodTestDescriptor.java:272)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeAfterTestExecutionCallbacks(TestMethodTestDescriptor.java:232)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
... 61 more
Suppressed: java.lang.RuntimeException: Mock server is not running
at au.com.dius.pact.consumer.PluginMockServer.stop(PluginMockServer.kt:59)
at au.com.dius.pact.consumer.junit5.JUnit5MockServerSupport.close(JUnit5MockServerSupport.kt:14)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$closeAllStoredCloseableValues$3(ExtensionValuesStore.java:68)```
ssomepalli
2022-08-31 18:34
Here is the Manifest:
```{
"manifestVersion": 1,
"pluginInterfaceVersion": 1,
"name": "protobuf",
"version": "0.1.10",
"executableType": "exec",
"entryPoint": "pact-protobuf-plugin",
"pluginConfig": {
"protocVersion": "3.19.1",
"downloadUrl": "https://github.com/protocolbuffers/protobuf/releases/download",
"hostToBindTo": "127.0.0.1",
"additionalIncludes":["//Users/ssomepalli/Documents/chronos/ad-break-chronos/src/test/resources/grpc"]
}
}```
jarmy
2022-08-31 19:04
@uglyog I don't believe the `additionalIncludes` is working. Here's my manifest:
```{
"manifestVersion": 1,
"pluginInterfaceVersion": 1,
"name": "protobuf",
"version": "0.1.12",
"executableType": "exec",
"entryPoint": "pact-protobuf-plugin",
"pluginConfig": {
"protocVersion": "3.19.1",
"downloadUrl": "https://github.com/protocolbuffers/protobuf/releases/download",
"hostToBindTo": "127.0.0.1",
"additionalIncludes":["/Users/jarmy/src/git/pandora/delivery/ad-break-chronos/src/test/resources/delivery/dispatch/common.proto", "/Users/jarmy/src/git/pandora/delivery/ad-break-chronos/src/test/resources/delivery/dispatch/display_ad_api.proto"]
}
}```
As you can see from the attached log, the `common.proto` is not found:
```11:58:58.612 [Thread-6] DEBUG io.pact.plugins.jvm.core.ChildProcess - Plugin protobuf [20011] || 2022-08-31T18:58:58.612323Z DEBUG tokio-runtime-worker request{method=POST uri=http://127.0.0.1:52364/io.pact.plugin.PactPlugin/ConfigureInteraction version=HTTP/2.0 headers={"content-type": "application/grpc", "te": "trailers", "user-agent": "grpc-java-netty/1.43.2", "grpc-accept-encoding": "gzip", "authorization": Sensitive}}: pact_protobuf_plugin::protoc: Protoc stderr: delivery/dispatch/common.proto: File not found.
11:58:58.612 [Thread-6] DEBUG io.pact.plugins.jvm.core.ChildProcess - Plugin protobuf [20011] || ad_break_api.proto:6:1: Import "delivery/dispatch/common.proto" was not found or had errors.```
uglyog
2022-09-01 02:58
@jarmy I have just checked the protoc docs, and the additional includes should only be directories
> `IMPORT_PATH` specifies a directory in which to look for `.proto` files when resolving `import` directives. If omitted, the current directory is used. Multiple import directories can be specified by passing the `--proto_path` option multiple times; they will be searched in order. `-I=_IMPORT_PATH_` can be used as a short form of `--proto_path`
uglyog
2022-09-01 03:29
@ssomepalli I can't see the problem in those logs. Can you delete any Pact files you have and try rerun it, but with only directories configured in the additional includes.
ssomepalli
2022-09-01 16:43
When I tried with the directories it is not displaying any error log. But, the test case is not passing.
patrickbadley
2022-09-06 13:31
Hi @marti92! I am also interested in ruby grpc support and am just joining the pact slack party. Do you know of any updates or community members working on this support? Or have you found some other solution?
james.grubb
2022-09-06 21:27
Hello, team! I am working in an enterprise environment with many teams, communicating over Kafka, using extensive and deeply nested protobufs and am trying to evaluate how difficult it would be to get Pact working with the existing stack. So I installed the https://github.com/pactflow/pact-protobuf-plugin using the pact-plugin-cli, and can confirm that the plugin `protobuf-0.1.12` was added to my `.pact/plugins` folder. However, running the https://github.com/pact-foundation/pact-plugins/tree/main/examples/protobuf/protobuf-consumer-jvm from the `pact-plugins` repo results in the following error (for the test consumeInitPluginMessage):
```Failed to resolve parameter [au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage arg0] in method [void io.pact.example.protobuf.provider.PactConsumerTest.consumeInitPluginMessage(au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage) throws com.google.protobuf.InvalidProtocolBufferException]: Plugin protobuf with version any was not found```
Is the protobuf plugin from pactflow incompatible with the tests in the pact-foundation pact-plugins examples? The Readme in pact-protobuf-plugin lists it as its JVM example.
Apologies in advance if I've missed some critical setup step. Any help is appreciated.
uglyog
2022-09-06 23:15
Ah, those are the older Protobuf examples, not the gRPC ones. Let me fix them for you.
jarmy
2022-09-07 18:24
@uglyog We got the additional includes to work via the following
```"additionalIncludes":["/Users/jarmy/src/git/pandora/delivery/ad-break-chronos/src/test/resources"]```
proto file is set in the test as such:
```"pact:proto", filePath("src/test/resources/delivery/dispatch/ad_break_api.proto")```
uglyog
2022-09-08 23:31
That example works fine for me with 0.1.12. What OS are you running? And can you run the plugin executable from a terminal?
jarmy
2022-09-16 15:37
Hello. My colleagues @ssomepalli, @srinagasai.krishnasan, and I have been looking into the gRPC plugin but cannot find a way to get the pact libraries to build requests from imported .proto files. Basically, any request that contains portions included in another proto file don't end up in the contract. Here's how I illustrated this in my https://github.com/pact-foundation/pact-plugins/pull/7/files of `pact-plugins`:
1. Add `common.proto` into `examples/gRPC/area_calculator/proto`
2. Add `import "common.proto"` into `area_calculator.proto` and add `DeviceContext` and `ListenerContext`
3. Modify the `request` in `PactConsumerTest.calculateRectangleArea` as follows:
``` "request", Map.of(
"device_context", Map.of(
"device_id", "matching(number, 4)",
"carrier_name", "matching(type, 'foo')"
)),
"listener_context", Map.of(
"listener_id", "matching(number, 1)"
),```
Now when you run the `PactConsumerTest` the generated `grpc-consumer-jvm-area-calculator-provider.json` does _not_ contain the `listener_context.listener_id`. I'll attach the generated pact along with _plugin.log_ in the thread
patrickbadley
2022-09-19 19:42
Hello! I was wondering if there is any updated guidance on using Pact with gRPC services (and their consumers) written in Ruby?
uglyog
2022-09-20 00:10
Pact Ruby is still using the old implementation, so it doesn't support using the gRPC plugin. However, if you can write your tests in another language that does, and use the verifier CLI, you could us it, but it will require a fair amount of work to get going.
matt.fellows
2022-09-20 06:36
oh, does this mean the protobuf plugin can resolve relative file imports?
patrickbadley
2022-09-20 13:25
We are looking into taking an approach similar to this one that was used for java, do you see any challenges to this approach? https://medium.com/@ivangsa/consumer-driven-contract-testing-for-grpc-pact-io-d60155d21c4c
srinagasai.krishnasan
2022-09-20 15:26
We tried the 0.1.14, it is currently working as desired. However, we are having stub generation problems, if we comment it out, it?s working out fine. Will let you know more.
Thank you!
uglyog
2022-09-20 23:08
No, I think people managed to get that to work. I haven't heard any bad things about that approach
srinagasai.krishnasan
2022-09-21 15:43
Hello guys, we are currently experiencing `InvalidProtocolBufferException: Protobuf message had a invalid wire type` .
We are not very big on protobufs. Upon trying the example you?ve provided, it?s going through. With our example, it?s not.
The error is specifically happening at `parseFrom` line in the pact test. Any help is appreciated. TIA
jarmy
2022-09-21 22:19
We've solved the `invalid wire type` issue. The .proto we were using in the test did not match the version in the proto library so they were encoded differently
jarmy
2022-09-21 23:30
I'm facing an issue with the latest `pact-protobuf-plugin` and `junit5:4.4.0-beta.4` library and enum field. The proto contains the following:
```message AdBreakRequest {
delivery.dispatch.DeviceContext device_context = 1;
delivery.dispatch.ListenerContext listener_context = 2;
delivery.dispatch.StationContext station_context = 3;
AdBreakAdType ad_break_type = 5;
enum AdBreakAdType {
MISSING_AD_BREAK_AD_TYPE = 0;
AUDIO_AD_BREAK = 1;
VIDEO_AD_BREAK = 2;
}```
When I try to use this in the request `"ad_break_type", "AUDIO_AD_BREAK"` I get the following error:
```Failed to set the interaction: Failed to process protobuf: Protobuf enum value .delivery.dispatch.AdBreakAdType has no value AUDIO_AD_BREAK
au.com.dius.pact.consumer.dsl.InteractionConfigurationError: Failed to set the interaction: Failed to process protobuf: Protobuf enum value .delivery.dispatch.AdBreakAdType has no value AUDIO_AD_BREAK
at app//au.com.dius.pact.consumer.dsl.PactBuilder.setupMessageContents(PactBuilder.kt:266)
at app//au.com.dius.pact.consumer.dsl.PactBuilder.with(PactBuilder.kt:190)```
I've tried a bunch of combinations using matchers to no avail.
jarmy
2022-09-21 23:31
Another issue I'm facing is pact doesn't seem to support message fields _within_ message fields. Here's the proto snippet:
```message AdBreakRequest {
repeated AdBreakContext ad_break_context = 1;
message AdBreakContext {
google.protobuf.StringValue forced_line_id = 1;
google.protobuf.StringValue forced_creative_id = 2;
google.protobuf.StringValue ad_break_id = 3;
google.protobuf.StringValue sessionId = 4;
AdBreakAdType ad_break_type = 5;
google.protobuf.StringValue cached_source_id = 6;
google.protobuf.Int32Value number_of_ads = 7;
google.protobuf.Int32Value pod_duration_seconds = 8;
}
}```
When I try to build the request with this code:
``` "request", Map.of(
"ad_break_context", List.of(Map.of(
"ad_break_id", Map.of("value", "matching(type, 'VldOSV9W')"),
"sessionId", Map.of("value", "matching(type, 'wJwZIZNX')"),
"number_of_ads", Map.of("value", "matching(number, 2)")
))),```
I get the following exception:
```Failed to set the interaction: Failed to process protobuf: Did not find a message type 'AdBreakContext' in any of file descriptors
au.com.dius.pact.consumer.dsl.InteractionConfigurationError: Failed to set the interaction: Failed to process protobuf: Did not find a message type 'AdBreakContext' in any of file descriptors
at app//au.com.dius.pact.consumer.dsl.PactBuilder.setupMessageContents(PactBuilder.kt:266)
at app//au.com.dius.pact.consumer.dsl.PactBuilder.with(PactBuilder.kt:190```
uglyog
2022-09-22 01:25
Ok, I'll need to look into these. I haven't tried messages within messages, so the descriptors may be encoded differently.
edouard.lopez
2022-09-28 13:35
Hello, is protobuff/gRPC plugin working in the open-source ecosystem or solely on pactflow?
yousafn
2022-09-28 14:14
It forms part of the open source ecosystem
https://github.com/pact-foundation/pact-plugins
so you can work with an up to date Pact broker or Pactflow as the plugin mechanism is agnostic of your broker (you can create, share and consumer pacts without a Broker )
uglyog
2022-09-30 05:23
Sorry, didn't get a chance to look into this, I'll see if I can do it next week
uglyog
2022-10-05 03:28
`0.1.15` has been released with fixes for the enums and embedded messages types
jarmy
2022-10-05 21:34
embedded message types is working. having a bit of difficulty getting enums working. I'll need to consult with @srinagasai.krishnasan
varnit.garg2424
2022-10-06 07:54
Hello, does anyone have experience with using protobuf plugin in case of http endpoint takes request in protobuf format instead of json?
matt.fellows
2022-10-06 10:15
Unfortunately Ruby doesn?t have support for plugins. It might be a while before it does, to be honest. We need to update it to use the newer Rust core, as per JS, Golang and .NET.
varnit.garg2424
2022-10-06 10:21
I see.
Do you see any limitation using the workaround suggested by you? I mean would it work out of the box using the custom marshaller as mentioned in this blog ? :slightly_smiling_face:
https://medium.com/@ivangsa/consumer-driven-contract-testing-for-grpc-pact-io-d60155d21c4c
jarmy
2022-10-06 19:19
Ok, we figured out the proper syntax for enums is to use the _actual_ enum: `"ad_break_type", AdBreakAdType.AUDIO_AD_BREAK`
jarmy
2022-10-06 20:29
However, the contract that is produced doesn't have anything in the request body for `ad_break_type` and when I run the `pact_verifier_cli`, the provider complains with this error: `com.pandora.delivery.dispatch.api.exception.InvalidRequestParameterException: Invalid Request Parameter: AdBreakAdType is missing or invalid`
I've attached the generated contract
uglyog
2022-10-06 22:01
I didn't have to use the actual enum, just the string value, but that was in Java, so maybe Go is different
jarmy
2022-10-07 15:36
If I try the string value `"ad_break_type", "matching(type, 'AUDIO_AD_BREAK')"` I get this error: `Failed to process protobuf: Protobuf enum value .delivery.dispatch.AdBreakAdType has no value AUDIO_AD_BREAK`
srinagasai.krishnasan
2022-10-09 22:37
yep the upgraded plugin version is being used. You can verify that in the `.json` file @jarmy has provided
uglyog
2022-10-10 02:51
Ok, let me look into that, when you are using a matcher. Maybe I missed something.
matt.fellows
2022-10-10 10:33
If you could please add additional details to https://github.com/pactflow/pact-protobuf-plugin/issues/14 that would help :slightly_smiling_face:
uglyog
2022-10-11 01:17
Can you make sure you don't have multiple versions of the plugin installed? There is an issue where it will load the first plugin it finds.
jason.taylor2
2022-10-31 16:57
@jarmy did the enum issue get sorted out? also having trouble getting enum to work in go
jason.taylor2
2022-11-01 14:11
@uglyog any thoughts? trying to get an enum working based off the go gRPC example and having similar issue trying to sort out a valid enum value. on 0.1.17
https://github.com/pact-foundation/pact-go/blob/3bb31bcc34a19e984bea58117cadb72b9856a8df/examples/grpc/grpc_consumer_test.go#L44
```Protobuf enum value .authz.v2.EnforceEffect has no value ENFORCE_EFFECT_ALLOW```
yousafn
2022-11-01 21:55
*Feature Preview: Pact Plugins*
Hey all, we have an exclusive early briefing of our new Pact Plugin Framework
Plugins enable you to extend the capabilities of Pact, without having to make changes to the Pact framework itself. For example, you could create a plugin to handle the Avro or GraphQL content types, or create a websockets transport.
We know you live on the bleeding edge and want to have a chance to create a plugin for a key technology before it is unveiled to the wider community - this is your chance to get ahead!
*Agenda*
? An overview of the framework - the problem it solves and how it works
? Demonstration of the capability in a gRPC and Protobufs use case
? How to author your own plugin
*Q&A*
After the session, if you have indicated you would like to create your own plugin, we will follow up with you to ensure you have the support you need.
*Register*
There will be two 60 minute briefing sessions scheduled:
1. 9 PM GMT / 5 PM ET / 2 PM PT Wednesday 2nd November (https://smartbear.zoom.us/meeting/register/tJwpcuGsrzkrG9EoxnV2G00xN5xPgKblZZ_k?utm_source=hs_email&utm_medium=email&utm_content=2&_hsenc=p2ANqtz-_EE0admLqgn_a3QSiyHkedGcqvvs69wiPiNIGYzO5V59AGjWOIQAsQ9ejFSIsHRtikJ9v3XniB1TYQgn9H2IbFZ3E2Bw&_hsmi=2)
2. 10 AM GMT / 11 AM CET Thursday 3rd November (https://smartbear.zoom.us/meeting/register/tJApcO-prz8rH9FumdJZ3XhcB1kdnwNa7wNT?utm_source=hs_email&utm_medium=email&utm_content=2&_hsenc=p2ANqtz-_EE0admLqgn_a3QSiyHkedGcqvvs69wiPiNIGYzO5V59AGjWOIQAsQ9ejFSIsHRtikJ9v3XniB1TYQgn9H2IbFZ3E2Bw&_hsmi=2)
jaroslavburi
2022-11-02 08:59
wow this is super last minute invite to be honest :smile: did I miss this in some other channel?
yousafn
2022-11-02 11:43
Hey hey,
The above message was sent to our developer preview programme members for Pactflow, as it?s a smaller session, with the bigger launch event in 2 weeks
https://docs.pact.io/events/plugins-framework-launch
We popped a message on twitter and in slack too
https://pact-foundation.slack.com/archives/C5F4KFKR8/p1666870369011319
We don?t have huge numbers for today and tomorrows early preview session, so thought I would throw it out on here!
Fine edged sword between trying to get people hyped and itching to use the tool and spamming people to death :)
uglyog
2022-11-03 06:26
I can't replicate this issue. With the latest plugin (0.1.15), enums work fine for me. I tried using direct values, matching them by type and also include the enum from another proto file. They all seem to work.
Could someone kindly create an example project that displays the issue you are having?
patrickbadley
2022-11-03 14:05
Hello! We are just getting started with contact tests and Pact and primarily use gRPC apis. I was wondering if anyone has guidance on what the development workflow would look like for consumer driven contract testing with pact for gRPC/protobuf defined apis since the provider would typically (I assume) own the protobuf definition? For example:
1. Consumer engineer creates a protobuf file describing the api they would like to be implemented
2. Consumer engineer builds any tooling to make requests to said protobuf file
3. Consumer engineer writes pact unit tests against the protobuf file
4. Consumer engineer publishes pactfile generated from said unit tests
5. Provider engineer is notified of new pactfile (I'm curious about how this would work for a new api, but I assume an existing API this could be automatically discovered during the provider's verify step)
6. Provider engineer copies/creates matching protobuf file (and potentially publishes it)
7. Provider engineer implements api that satisfies contract defined in pactfile
8. Provider publishes new version of api and notifies consumer
There are a couple of potential issues I see with this process:
1. The consumer may have to make some assumptions about the proto file like namespacing/endpoint name or other organizational concerns
2. We currently have our provider publish ruby and python grpc libraries to consume our grpc apis, fi the consumer is starting the process, presumably they are responsible for generating this on their own until the provider publishes an official library
Please let me know your thoughts from anyone with experience in this area so we can get off on the right foot. Thank you in advance!
jason.taylor2
2022-11-03 21:00
am using this for match:
```"result": "TEST_TWO"```
it works if i put the enum def inside the message. but if enum is outside it fails
```enum TestEnum {
TEST_ONE = 0;
TEST_TWO = 1;
TEST_THREE = 2;
}
message TestResponse {
TestEnum result = 1;
}```
uglyog
2022-11-03 22:14
I used both `"result": "TEST_TWO"` and `"result": "matching(type, 'TEST_TWO')"`
jason.taylor2
2022-11-03 22:52
ok - both of those work if i have the enum in the message but not if enum is outside
jason.taylor2
2022-11-04 15:18
A modified route_guide.proto to exhibit the behavior. Seemed tied to the package name having `.v2`, but then was also seeing additional odd behavior when I transferred the enum to route_guide. The file below is not hitting the enum issue with`"pact:proto-service": "RouteGuide/GetFeature"` but fails with `"pact:proto-service": "Test/GetFeature2"`. Dropping the `.v2` or moving the enum into the message makes the second case work.
```// Copyright 2015 gRPC authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package routeguide.v2;
enum TestEnum {
VALUE_ZERO = 0;
VALUE_ONE = 1;
VALUE_TWO = 2;
}
// Points are represented as latitude-longitude pairs in the E7 representation
// (degrees multiplied by 10**7 and rounded to the nearest integer).
// Latitudes should be in the range +/- 90 degrees and longitude should be in
// the range +/- 180 degrees (inclusive).
message Point {
int32 latitude = 1;
int32 longitude = 2;
}
// A latitude-longitude rectangle, represented as two diagonally opposite
// points "lo" and "hi".
message Rectangle {
// One corner of the rectangle.
Point lo = 1;
// The other corner of the rectangle.
Point hi = 2;
}
// A feature names something at a given point.
//
// If a feature could not be named, the name is empty.
message Feature {
TestEnum result = 1;
}
// A RouteNote is a message sent while at a given point.
message RouteNote {
// The location from which the message is sent.
Point location = 1;
// The message to be sent.
string message = 2;
}
// A RouteSummary is received in response to a RecordRoute rpc.
//
// It contains the number of individual points received, the number of
// detected features, and the total distance covered as the cumulative sum of
// the distance between each point.
message RouteSummary {
// The number of points received.
int32 point_count = 1;
// The number of known features passed while traversing the route.
int32 feature_count = 2;
// The distance covered in metres.
int32 distance = 3;
// The duration of the traversal in seconds.
int32 elapsed_time = 4;
}
// Interface exported by the server.
service Test {
// A simple RPC.
//
// Obtains the feature at a given position.
//
// A feature with an empty name is returned if there's no feature at the given
// position.
rpc GetFeature2(Point) returns (Feature) {}
// A server-to-client streaming RPC.
//
// Obtains the Features available within the given Rectangle. Results are
// streamed rather than returned at once (e.g. in a response message with a
// repeated field), as the rectangle may cover a large area and contain a
// huge number of features.
rpc ListFeatures(Rectangle) returns (stream Feature) {}
// A client-to-server streaming RPC.
//
// Accepts a stream of Points on a route being traversed, returning a
// RouteSummary when traversal is completed.
rpc RecordRoute(stream Point) returns (RouteSummary) {}
// A Bidirectional streaming RPC.
//
// Accepts a stream of RouteNotes sent while a route is being traversed,
// while receiving other RouteNotes (e.g. from other users).
rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}
}```
uglyog
2022-11-09 23:46
I would assume the provider team would first create the protobuf file, or maybe, a solution architect. And distribute this to all parties.
jason.taylor2
2022-11-15 20:35
@uglyog trying to verify the protobuf enum fix in 0.1.16 and hitting a non-obvious transport issue when i update to that version of the plugin. any tips on where to look or additional debug params to get more info on this?
```2022-11-15T19:24:42.769373Z ERROR ThreadId(01) pact_plugin_driver::content: Call to plugin failed - status: Unknown, message: "transport error", details: [], metadata: MetadataMap { headers: {} }
2022-11-15T19:24:42.769568Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Call to plugin failed - status: Unknown, message: "transport error", details: [], metadata: MetadataMap { headers: {} }```
uglyog
2022-11-15 21:47
Oh, damn. Someone else is also having some issues with that version. There may have been some updates to dependent libraries, let me investigate that.
yousafn
2022-11-15 23:28
Hey @jason.taylor2 are you seeing this via Pact-Go? otherwise which client library are you using?
I've been seeing the same issue in Go, and got the same issue in spike Ruby FFI application, if I have my logger enabled,
https://github.com/pact-foundation/pact-reference/issues/230
yousafn
2022-11-18 12:54
:wave: It has been awesome seeing so many of you try out our Protobuf plugin.
As we gear up for an official launch of the Pact Plugin framework, I wondered if anyone would be happy to spare a few comments, either about what the new framework means to you, the potential possibilities you see going forward?
elluru.kri.mohan
2022-11-23 16:46
Hello, team! I am working in an enterprise environment with many teams, communicating over Kafka, using extensive and deeply nested protobufs and am trying to evaluate how difficult it would be to get Pact working with the existing stack. So I installed the https://github.com/pactflow/pact-protobuf-plugin using the pact-plugin-cli, and can confirm that the plugin `protobuf-0.1.17` was added to my `.pact/plugins` folder. However, running the https://github.com/pact-foundation/pact-plugins/tree/main/examples/protobuf/protobuf-consumer-jvm from the `pact-plugins` repo results in the following error (for the test consumeInitPluginMessage):
```Failed to resolve parameter [au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage arg0] in method [void io.pact.example.protobuf.provider.PactConsumerTest.consumeInitPluginMessage(au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage) throws com.google.protobuf.InvalidProtocolBufferException]: Plugin protobuf with version any was not found```
Is the protobuf plugin from pactflow incompatible with the tests in the pact-foundation pact-plugins examples? The Readme in pact-protobuf-plugin lists it as its JVM example.
Apologies in advance if I?ve missed some critical setup step. Any help is appreciated.
yousafn
2022-11-23 17:29
Hey @elluru.kri.mohan,
It should work fine with the examples in the pact-plugin repo, there are some tests there, which are tested cross platform.
Which OS are you running?
is there any more logs from your consumer test, if you set it to debug level, as to the directory the plugins are expected to be loaded from.
I assume they were installed to your home folder, when installed with the pact-plugin-cli
yousafn
2022-11-23 17:30
It is probably something simple, or something we need to fix, probably the former, so no need to apologise :slightly_smiling_face:
elluru.kri.mohan
2022-11-23 17:32
I am just using the example script given in pact-protobuf-plugin to test if I can integrate my tests yet Yousaf.
elluru.kri.mohan
2022-11-23 17:33
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] MessageTransformerProducerPactProtoCTest The following methods annotated with @Pact were not executed during the test: MessageTransformerProducerPactProtoCTest.initPluginMessage
If these are currently a work in progress, add a @Disabled annotation to the method
[ERROR] Errors:
[ERROR] MessageTransformerProducerPactProtoCTest.consumeConfigureInteractionResponseMessage:101 » InvalidWireType Protocol message tag had invalid wire type.
[ERROR] MessageTransformerProducerPactProtoCTest.consumeInitPluginMessage(V4Interaction$AsynchronousMessage) » ParameterResolution Failed to resolve parameter [http://au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage arg0] in method [void http://com.hpe.gravity.cx.ctests.MessageTransformerProducerPactProtoCTest.consumeInitPluginMessage(http://au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage) throws com.google.protobuf.InvalidProtocolBufferException]: Plugin protobuf with version any was not found
yousafn
2022-11-23 17:33
np, I am running on a mac m1 too, I didn't use an editor to run the tests, just ran the example with `./gradle check`
yousafn
2022-11-23 17:34
are you able to run this example on your machine ?
https://github.com/pact-foundation/pact-plugins/tree/main/examples/protobuf
it uses gradle, just to confirm that the plugin can load, and run the example on your machine. If that does, then we narrow down to maven and look at the setup
elluru.kri.mohan
2022-11-23 17:34
have added
```<plugin>
<groupId>au.com.dius.pact.provider</groupId>
<artifactId>maven</artifactId>
<version>4.1.11</version>
</plugin>```
as per the steps mentioned in the link and using it
yousafn
2022-11-23 17:41
no worries buddy! I am not sure how to convert from a gradle to a maven project, been so long since I have created java apps from scratch, but happy to help get a maven example rocking for you and others benefit.
At least this will hopefully rule out communicating with the plugin as per the example with gradle, and then we can look at creating a maven example in the repo to test out.
If you could share a fork with the changes to the project too, I can happily pull it down and take a look :slightly_smiling_face:
elluru.kri.mohan
2022-11-23 17:47
Thanks for considering the same Yousaf. I am quickly checking with gradle compilation now. Will upload changes tomorrow as it is already 11PM for me :slightly_smiling_face:.
elluru.kri.mohan
2022-11-23 17:58
`krishnamohanelluru@Krishnas-MacBook-Pro protobuf-consumer-jvm % ./gradlew check`
FAILURE: Build failed with an exception.
* Where:
Settings file ?/Users/krishnamohanelluru/pact/pact-plugins/examples/protobuf/protobuf-consumer-jvm/settings.gradle?
* What went wrong:
Could not compile settings file ?/Users/krishnamohanelluru/pact/pact-plugins/examples/protobuf/protobuf-consumer-jvm/settings.gradle?.
> startup failed:
General error during semantic analysis: Unsupported class file major version 62
java.lang.IllegalArgumentException: Unsupported class file major version 62
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:196)
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:177)
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:163)
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:284)
at org.codehaus.groovy.ast.decompiled.AsmDecompiler.parseClass(AsmDecompiler.java:81)
at org.codehaus.groovy.control.ClassNodeResolver.findDecompiled(ClassNodeResolver.java:251)
elluru.kri.mohan
2022-11-23 18:04
gradle --version
------------------------------------------------------------
Gradle 7.5.1
------------------------------------------------------------
Build time: 2022-08-05 21:17:56 UTC
Revision: d1daa0cbf1a0103000b71484e1dbfe096e095918
Kotlin: 1.6.21
Groovy: 3.0.10
Ant: Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM: 17.0.5 (Homebrew 17.0.5+0)
OS: Mac OS X 12.6.1 x86_64
krishnamohanelluru@Krishnas-MacBook-Pro protobuf-consumer-jvm %
elluru.kri.mohan
2022-11-23 18:14
@yousafn are these settings okay? Am I missing any element here to compile?
yousafn
2022-11-23 18:44
I get that if I am on the wrong java version, believe that example runs with java 11, which I am running on my machine, and it pulls down in CI
https://github.com/pact-foundation/pact-plugins/blob/main/.github/workflows/examples.yml#L82
yousafn
2022-11-23 22:28
No problem, has a great night Krishna. I think setting Java 11 will cure your gradle issue.
We will see if we can sort you out a maven example. And check if we are hard restricted to Java 11 or if you can run later versions. What version is the app you wish to target using?
uglyog
2022-11-24 01:21
I have added Maven examples for both gRPC and plain Protobufs. See https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/consumer-maven and https://github.com/pact-foundation/pact-plugins/tree/main/examples/protobuf/protobuf-consumer-maven
elluru.kri.mohan
2022-11-24 04:42
Thank you Uglyog., Will there be any hard dependency on java version? I will check anyways.
elluru.kri.mohan
2022-11-24 10:22
```Getting the same error like before Uglyog.
trace_id= span_id= 2022-11-24 02:20:00.744 GMT-08:00 [main] [ERROR] i.p.p.jvm.core.DefaultPluginManager [PluginManager.kt:525] [] - Init call to plugin protobuf failed
java.lang.RuntimeException: java.lang.NoClassDefFoundError: io/netty/channel/ChannelFactory
at io.pact.plugins.jvm.core.Utils.handleWith(Util.kt:46)
at io.pact.plugins.jvm.core.DefaultPluginManager.initialisePlugin(PluginManager.kt:513)
at io.pact.plugins.jvm.core.DefaultPluginManager.loadPlugin(PluginManager.kt:290)
at au.com.dius.pact.consumer.dsl.PactBuilder.usingPlugin(PactBuilder.kt:96)
at au.com.dius.pact.consumer.dsl.PactBuilder.usingPlugin$default(PactBuilder.kt:93)
at au.com.dius.pact.consumer.dsl.PactBuilder.usingPlugin(PactBuilder.kt)
at com.hpe.gravity.cx.ctests.MessageTransformerConsumerPactProtoCTest.initPluginMessage(MessageTransformerConsumerPactProtoCTest.java:30)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
trace_id= span_id= 2022-11-24 02:20:00.747 GMT-08:00 [main] [ERROR] a.c.d.pact.consumer.dsl.PactBuilder [PactBuilder.kt:99] [] - Init call to plugin protobuf failed: java.lang.RuntimeException: java.lang.NoClassDefFoundError: io/netty/channel/ChannelFactory
trace_id= span_id= 2022-11-24 02:20:00.753 GMT-08:00 [Thread-4] [WARN] a.com.dius.pact.core.support.Metrics [Metrics.kt:88] [] -
Please note: we are tracking events anonymously to gather important usage statistics like JVM version
and operating system. To disable tracking, set the 'pact_do_not_track' system property or environment
variable to 'true'.
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at org.junit.platform.commons.support.ReflectionSupport.invokeMethod(ReflectionSupport.java:198)
at au.com.dius.pact.consumer.junit5.PactConsumerTestExt.lookupPact(PactConsumerTestExt.kt:387)```
elluru.kri.mohan
2022-11-24 16:46
if I run using mvn test is passing. I think the issue is with intellij app from my MAC..
elluru.kri.mohan
2022-11-25 03:46
HI @uglyog @yousafn tried creating a fresh project with the settings given in maven example, facing the error with java17..
consumeConfigureInteractionResponseMessage{AsynchronousMessage} Time elapsed: 0.124 s <<< ERROR!
org.junit.jupiter.api.extension.ParameterResolutionException: Failed to resolve parameter [http://au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage message] in method [void com.pact.maven.MavenContractTests.consumeConfigureInteractionResponseMessage(http://au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage) throws com.google.protobuf.InvalidProtocolBufferException]: Failed to set the interaction: Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1
at com.pact.maven.MavenContractTests.configureInteractionResponseMessage(MavenContractTests.java:57)
uglyog
2022-11-25 03:50
`Failed to invoke protoc binary: exit code exit status: 1` means it can't run the Protobuf compiler. What OS are you running with? Is it running within docker with Alpine Linux?
uglyog
2022-11-25 03:54
What do you get if you run `~/.pact/plugins/protobuf-0.1.17/protoc/bin/protoc --version`
elluru.kri.mohan
2022-11-25 03:54
~/.pact/plugins/protobuf-0.1.17/protoc/bin/protoc --version
libprotoc 3.19.1
krishnamohanelluru@Krishnas-MacBook-Pro protobuf-provider %
uglyog
2022-11-25 03:55
can you provide the plugin logs? You may need to enable debug logs in the test
elluru.kri.mohan
2022-11-25 04:18
2022-11-25T09:47:04.997+05:30 WARN 9816 --- [ main] i.p.p.j.core.DefaultPactPluginManifest : args field in plugin manifest is invalid
this error rings any bell Uglyog?
elluru.kri.mohan
2022-11-25 04:28
09:54:32.431 [grpc-nio-worker-ELG-1-5] DEBUG io.grpc.netty.NettyClientHandler - [id: 0xfc6f1942, L:/127.0.0.1:51654 - R:/127.0.0.1:51651] OUTBOUND PING: ack=false bytes=1234
09:54:32.431 [grpc-nio-worker-ELG-1-5] DEBUG io.grpc.netty.NettyClientHandler - [id: 0xfc6f1942, L:/127.0.0.1:51654 - R:/127.0.0.1:51651] INBOUND HEADERS: streamId=9 headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0 endStream=true
09:54:32.432 [main] DEBUG io.pact.plugins.jvm.core.DefaultPluginManager - Got response: error: ?Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1?
elluru.kri.mohan
2022-11-25 04:32
{?pluginInterfaceVersion?:1,?name?:?protobuf?,?version?:?0.1.17?,?executableType?:?exec?,?minimumRequiredVersion?:null,?entryPoint?:?pact-protobuf-plugin?,?entryPoints?:{},?args?:null,?dependencies?:null,?pluginConfig?:{?protocVersion?:?3.19.1",?hostToBindTo?:?127.0.0.1",?downloadUrl?:?https://github.com/protocolbuffers/protobuf/releases/download?}}*%*
uglyog
2022-11-25 04:33
That has extra fields in it, compare it to https://github.com/pactflow/pact-protobuf-plugin/releases/download/v-0.1.17/pact-plugin.json
elluru.kri.mohan
2022-11-25 04:36
I think, I found the problem for the error. I had made incorrect path reference to proto, found it from logs and fixed it. now able to compile. Sorry for the false Alarm Uglyog..
uglyog
2022-11-25 05:14
This one? `Got response: error: "Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1"`
ringo
2022-12-06 19:36
Can I find a working example of Go `testing` with `pact-go` and the `pact-protobuf-plugin` somewhere?
yousafn
2022-12-06 20:46
https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator
You can run it in a killercoda here
Along with the other examples in that repo
https://killercoda.com/safdotdev/course/safacoda/grpc_quick_start
yousafn
2022-12-06 20:47
Demos are all listed on the site in the new Pact plugins directory page
https://docs.pact.io/plugins/directory#demos
ringo
2022-12-06 21:33
The current state of my POC can be found here: https://github.com/ringods/pulumi-pact-poc
But when running `go test -v .`, I get this output:
```$ go test -v .
# http://github.com/ringods/pulumi-pact-poc.test
/usr/local/Cellar/go/1.19.3/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: warning: -no_pie is deprecated when targeting new OS versions
Undefined symbols for architecture x86_64:
"_pactffi_verifier_add_provider_transport", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_verifier_add_provider_transport in 000004.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_verifier_add_provider_transport)
"_pactffi_verifier_set_no_pacts_is_error", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_verifier_set_no_pacts_is_error in 000004.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_verifier_set_no_pacts_is_error)
"_pactffi_with_header_v2", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_with_header_v2 in 000003.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_with_header_v2)
"_pactffi_with_query_parameter_v2", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_with_query_parameter_v2 in 000003.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_with_query_parameter_v2)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL http://github.com/ringods/pulumi-pact-poc [build failed]
FAIL```
I don?t really have a test yet. I?m trying to get our language host started. (FYI, language host is a Pulumi concept, see the readme of my repo)
But does this error tell you something whether I?m missing something in my setup?
yousafn
2022-12-06 22:12
Have you ran the go install command for pact to install the pre req libs? I should be in the readme on the v2 branch
Assume you are on an m1 mac? are you in rosetta mode? You prob need to be in darwin m1 mode `uname -sm` should be `arm64` i think.
You get different results using the x86_64 libs on an m1 machine
yousafn
2022-12-06 22:13
ty for sharing the link to the POC, look forward to getting the chance to glance over it
https://github.com/pact-foundation/pact-go/tree/2.x.x#installation
matt.fellows
2022-12-06 22:30
it should work in rosetta mode and in pure M1 mode.
But that does look like maybe you have an old library
ringo
2022-12-07 14:07
Doesn?t help. I ran these 2 commands directly one after the other:
```$ pact-go -l DEBUG install -f
2022/12/07 15:04:09 [INFO] downloading library from https://github.com/pact-foundation/pact-reference/releases/download/libpact_ffi-v0.3.2/libpact_ffi-osx-x86_64.dylib.gz to /usr/local/lib/libpact_ffi.dylib
&{}
2022/12/07 15:04:11 [DEBUG] obtaining hash for file /usr/local/lib/libpact_ffi.dylib
2022/12/07 15:04:11 [DEBUG] writing config {map[libpact_ffi:{libpact_ffi 0.3.2 d6503769896eecbc027815d20aff19c3}]}
2022/12/07 15:04:11 [DEBUG] writing yaml config to file libraries:
libpact_ffi:
libname: libpact_ffi
version: 0.3.2
hash: d6503769896eecbc027815d20aff19c3
2022/12/07 15:04:11 [INFO] setting install_name on library libpact_ffi for osx
2022/12/07 15:04:16 [DEBUG] output from command []
2022/12/07 15:04:16 [INFO] package libpact_ffi found
2022/12/07 15:04:16 [INFO] checking version 0.3.2 of libpact_ffi against semver constraint >= 0.3.0, < 1.0.0
2022/12/07 15:04:16 [DEBUG] 0.3.2 satisfies constraints 0.3.2 >= 0.3.0, < 1.0.0
2022/12/07 15:04:16 [INFO] package libpact_ffi is correctly installed
2022/12/07 15:04:16 [DEBUG] skip checking ffi version() call because FFI not loaded. This is expected when running the 'pact-go' command.
Time: 0h:00m:07s
$ go test -v .
# http://github.com/ringods/pulumi-pact-poc.test
/usr/local/Cellar/go/1.19.3/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: warning: -no_pie is deprecated when targeting new OS versions
Undefined symbols for architecture x86_64:
"_pactffi_verifier_add_provider_transport", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_verifier_add_provider_transport in 000004.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_verifier_add_provider_transport)
"_pactffi_verifier_set_no_pacts_is_error", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_verifier_set_no_pacts_is_error in 000004.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_verifier_set_no_pacts_is_error)
"_pactffi_with_header_v2", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_with_header_v2 in 000003.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_with_header_v2)
"_pactffi_with_query_parameter_v2", referenced from:
__cgo_6fba9656f5d3_Cfunc_pactffi_with_query_parameter_v2 in 000003.o
(maybe you meant: __cgo_6fba9656f5d3_Cfunc_pactffi_with_query_parameter_v2)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL http://github.com/ringods/pulumi-pact-poc [build failed]
FAIL
Time: 0h:00m:05s ```
For your information:
```$ ls -lsa /usr/local/lib/libpact_ffi.dylib
43392 -rw------- 1 ringods staff 22214200 7 Dec 15:04 /usr/local/lib/libpact_ffi.dylib```
yousafn
2022-12-07 14:47
:thinking_face: hmmm weird! It has downloaded the correct x86_64 binary for the intel platform, but this line here
`/usr/local/Cellar/go/1.19.3/libexec/pkg/tool/darwin_amd64/link` shows `darwin_amd64`
I've had it running on my m1 mac, I've got an intel mac (although it's a hackintosh) will test on there today at some point (might take a little while, as I haven't used the machine in quite a while)
Which version of `pact-go` are you using?
The pact ffi version is `0.3.2` - Pact-go on v2.0.0-beta.16 https://github.com/pact-foundation/pact-go/releases/tag/v2.0.0-beta.16 is using `0.3.15`
yousafn
2022-12-07 14:52
```? go version
go version go1.19.3 darwin/arm64```
Just pulled the latest pact-go to install the latest binary on my m1, had to sudo as install doesn't have perms to write to `/usr/local/lib`
yousafn
2022-12-07 15:02
So I think the issue is your FFI lib is too old
https://github.com/pact-foundation/pact-reference/releases/tag/libpact_ffi-v0.3.2
latest is https://github.com/pact-foundation/pact-reference/releases?q=%22Pact+FFI+Library%22&expanded=true
quite a few changes, not sure these should have all gone under patch tho :thinking_face:
ringo
2022-12-07 15:28
with pact-go on beta17, reinstalling the ffi lib, it indeed gives a better outcome! Tnx for the help!
ringo
2022-12-07 21:14
If I get this error, where do I have to look for that control character?
```2022-12-07T21:11:27.659360Z ERROR ThreadId(01) pact_ffi::plugins: Contents is not a valid JSON - control character (\u0000-\u001F) found while parsing a string at line 7 column 0
2022-12-07T21:11:27.659383Z ERROR ThreadId(01) pact_ffi::plugins: contents='{
"pact:proto": "/Users/ringods/Projects/pulumi/pulumi-pact-poc/proto/pulumi/resource.proto",
"pact:proto-service": "ResourceMonitor/RegisterResource",
"pact:content-type": "application/protobuf",
"request": {
"type": "matching(string),
"name": "matching(string),
"parent": "matching(string),
"custom": "matching(boolean),
},
"response": {
"urn": "matching(string)",
}
}'```
ringo
2022-12-07 21:15
Ouch? after posting I see I didn?t close the double quotes on the request matchers.
ringo
2022-12-07 21:19
Any idea on this?
```2022-12-07T21:16:20.081180Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Request to configure interaction failed: Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1```
It is available:
```$ which protoc
/usr/local/bin/protoc```
ringo
2022-12-07 21:21
Could this be because I made an error in my grpc interaction and it might not comply to the spec?
uglyog
2022-12-07 21:33
The protoc binary exited with an error code. It should have printed out an error message, which will be in the logs
ringo
2022-12-07 21:35
Here is the full output I have on my console:
```$ go test -v .
=== RUN TestUserAPIClient
2022/12/07 22:32:54 [DEBUG] pact synchronous message validate config
2022/12/07 22:32:54 [DEBUG] initialising native interface
2022/12/07 22:32:54 [DEBUG] log_to_stdout res 0
2022/12/07 22:32:54 [DEBUG] add sync message
2022-12-07T21:32:54.911626Z WARN tokio-runtime-worker pact_plugin_driver::metrics:
Please note:
We are tracking this plugin load anonymously to gather important usage statistics.
To disable tracking, set the 'pact_do_not_track' environment variable to 'true'.
2022-12-07T21:32:55.485108Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Request to configure interaction failed: Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1
2022/12/07 22:32:55 [DEBUG] mock server starting on address: 127.0.0.1 0
2022-12-07T21:32:55.487658Z ERROR ThreadId(01) pactffi_create_mock_server_for_transport{pact=PactHandle { pact_ref: 1 } addr=0x101708190 port=0 transport=0x1017043c0 transport_config=0x101706330}: pact_ffi::mock_server: Failed to start mock server - Mock server failed to start: Failed to start gRPC mock server: Pact file does not contain any Protobuf descriptors
2022/12/07 22:32:55 unable to start plugin transport: unable to start the mock server
FAIL http://github.com/ringods/pulumi-pact-poc 0.934s
FAIL
Time: 0h:00m:03s ```
matt.fellows
2022-12-07 23:41
Yes, I need to improve the validation on this Ringo, thanks for reminding me
matt.fellows
2022-12-07 23:42
Oh, Ron did actually create the issue for me: https://github.com/pact-foundation/pact-go/issues/250.
matt.fellows
2022-12-07 23:43
Ringo - could please check the logs of the protobuf plugin to see if anything else is in there?
It should be `~/.pact/plugins/protobuf-0.1.xxx/log/plugin.log.*`
matt.fellows
2022-12-07 23:45
Thanks for your help Yousaf, and sorry for the pain Ringo - when in doubt, upgrade :stuck_out_tongue:
ringo
2022-12-08 07:20
@matt.fellows only two lines in the log file:
```2022-12-08T07:19:42.612162Z ERROR tokio-runtime-worker pact_protobuf_plugin::server: Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1
2022-12-08T07:19:42.623750Z ERROR tokio-runtime-worker pact_protobuf_plugin::server: Failed to start gRPC mock server: Pact file does not contain any Protobuf descriptors```
uglyog
2022-12-08 21:40
If you enable TRACE level logging, it will print out the command it is trying to execute
matt.fellows
2022-12-09 05:58
I?ve noticed a bug in the way logging is handled in Go - having to set an env var and use `SetLogLevel` is not the intention, I?ll fix that shortly.
ringo
2022-12-09 07:44
Where/how should I configure the TRACE level logging? Any pointer to a doc page describing that?
matt.fellows
2022-12-09 09:03
Just on the go but I'll push up a fix tonight. Exporting the following should work though: `export LOG_LEVEL=trace`
(I think it's currently mentioned in the troubleshooting guide)
ringo
2022-12-09 10:12
Found the problem:
```2022-12-09T10:06:12.991143Z TRACE tokio-runtime-worker request{method=POST uri=http://127.0.0.1:52887/io.pact.plugin.PactPlugin/ConfigureInteraction version=HTTP/2.0 headers={"te": "trailers", "content-type": "application/grpc", "authorization": Sensitive, "user-agent": "tonic/0.8.2"}}: pact_protobuf_plugin::protoc: Invoking protoc: Command { std: "./protoc/bin/protoc" "-o/Users/ringods/.pact/plugins/protobuf-0.2.0/tmp/.tmpN76QN0" "-I/Users/ringods/Projects/pulumi/pulumi-pact-poc/proto/pulumi" "--include_imports" "/Users/ringods/Projects/pulumi/pulumi-pact-poc/proto/pulumi/language.proto" "-Iprotoc/include", kill_on_drop: false }
2022-12-09T10:06:13.024890Z DEBUG tokio-runtime-worker request{method=POST uri=http://127.0.0.1:52887/io.pact.plugin.PactPlugin/ConfigureInteraction version=HTTP/2.0 headers={"te": "trailers", "content-type": "application/grpc", "authorization": Sensitive, "user-agent": "tonic/0.8.2"}}: pact_protobuf_plugin::protoc: Protoc output:
2022-12-09T10:06:13.024927Z DEBUG tokio-runtime-worker request{method=POST uri=http://127.0.0.1:52887/io.pact.plugin.PactPlugin/ConfigureInteraction version=HTTP/2.0 headers={"te": "trailers", "content-type": "application/grpc", "authorization": Sensitive, "user-agent": "tonic/0.8.2"}}: pact_protobuf_plugin::protoc: Protoc stderr: pulumi/plugin.proto: File not found.```
The main protobuf file in `./proto/pulumi/language.go` imports `pulumi/plugin.proto`, but to resolve this correctly, I need to be able to configure the include path from `"-I/Users/ringods/Projects/pulumi/pulumi-pact-poc/proto/pulumi"` to just `"-I/Users/ringods/Projects/pulumi/pulumi-pact-poc/proto"` (e.g. drop the last part `pulumi` from the path.
Can I do that?
uglyog
2022-12-11 21:53
You can add additional includes in the plugin manifest file, under `pluginConfig`, see https://github.com/pactflow/pact-protobuf-plugin#configuration
ringo
2022-12-13 14:31
@uglyog that?s good for now. IMO this should be a project-level configuration though.
ringo
2022-12-13 14:55
I?m struggling to express the correct grpc interaction message in my test. With a proto file containing this:
```service LanguageRuntime {
// About returns information about the runtime for this language.
rpc About(google.protobuf.Empty) returns (AboutResponse) {}
}
// AboutResponse returns runtime information about the language.
message AboutResponse {
string executable = 1; // the primary executable for the runtime of this language.
string version = 2; // the version of the runtime for this language.
map<string, string> metadata = 3; // other information about this language.
}```
I have this as my `grpcInteration` object so far:
``` grpcInteraction := `{
"pact:proto": "` + filepath.ToSlash(dir) + `",
"pact:proto-service": "LanguageRuntime/About",
"pact:content-type": "application/protobuf",
"request": {
},
"response": {
"executable": "matching(type, 'pulumi')",
"version": "matching(type, '3.48.0')",
"metadata": {
"matching(type, 'someKey')": "matching(type, 'somevalue')"
}
}
}````
The error I get is this, and I get head nor tail to it:
```2022-12-13T14:51:19.060937Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Request to configure interaction failed: Failed to process protobuf: For message fields, you need to define a Map of expected fields, got String("matching(type, 'somevalue')")```
I tried various things already for the free form `metadata` field so far, but to no avail. I haven?t found any example which has a `map` in the proto file that I can copy from. What is the correct way to write this?
ringo
2022-12-13 15:24
Or is this one of the unsupported features for now?
> ? Map fields with scalar keys.
https://github.com/pactflow/pact-protobuf-plugin#unsupported-features
matt.fellows
2022-12-18 22:29
FYI have created https://github.com/pactflow/pact-protobuf-plugin/issues/17 to track this. I got a similar feedback from another user via a DM last week
uglyog
2022-12-19 22:12
The correct form would be something like:
`"metadata": "eachKey(matching(type, 'someKey')), eachValue(matching(type, 'somevalue'))"`
ringo
2022-12-23 12:51
When writing https://github.com/ringods/pulumi-pact-poc/blob/50edb90931a6e71129530480a4632d42f12f82d0/single_resource_test.go#L29-L47, what format am I actually using? This doesn?t seem to be according the https://github.com/pact-foundation/pact-specification/tree/version-3 or https://github.com/pact-foundation/pact-specification/tree/version-4. Is there any documentation on what I have to write in my Go pact test for protobufs? Still struggling to get something working. :pray:
ringo
2022-12-23 12:52
My current problem is finding out how to get resolve this:
```2022-12-23T12:39:38.876907Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Request to configure interaction failed: Failed to process protobuf: Error: Expected a comma, got ')'
??[expression:1:17]
?
1 ? matching(boolean)
· ?
· ??? Expected a comma before this
????```
yousafn
2022-12-23 19:58
My pleasure Ringo, yeah it probably needs to be more prominent and linked to example uses in code.
matt.fellows
2022-12-24 01:21
I wonder if we could link to the expression guide in that error @uglyog?
or it might just be simpler to improve the plugin getting started guide for users (not authors which we spent a little more the on recently)
I think the error message is really good, but obviously if you didn't know the language in the first place that doesn't help!
sandy.oberoi
2023-01-18 01:13
*Context*
I am trying to write pact test for a producer/consumer setup where messages are consumed from a kinesis stream. The messages are defined in protobuf schema
*Current State*
I have the consumer side of the test working which is generating a pact json
*Issue*
I am struggling to write a test for the provider as am not sure how to setup it up. I tried following some examples provided in github but could not set it up for protobuf messages. Any help/pointers would be appriciated
I am uisng pact-go (v2.0.0-beta.17) and protobuf plugin (0.2.4)
uglyog
2023-01-18 01:21
Have a look at this example: https://github.com/pact-foundation/pact-plugins/tree/main/examples/protobuf/protobuf-provider
sandy.oberoi
2023-01-18 03:44
thank I will try this out. But as I understand this will require a call to cli to verify the message. I was hoping to be able to setup the test similar to https://github.com/pact-foundation/pact-go/blob/2.x.x/examples/provider_test.go where the verification could be done in the test itself.
uglyog
2023-01-18 03:55
@matt.fellows may be able to help here. But you'll need to provide the errors you are getting.
matt.fellows
2023-01-18 06:54
Yes please. I think the standard message interface is the way to go, but it?s possible the types are not going to allow an `[]byte` , which is probably going to be needed for that
sathish.sundararajan
2023-01-18 15:59
I have a grpc interaction which validates token. The token gets passed in header and hence request is empty. Looks like when request is empty, pact plugin is not working. I am getting an error as below. Can someone please help ?
```Error: Received unexpected error:
retreived an empty message```
```UsingPlugin(message.PluginConfig{
Plugin: "protobuf",
Version: "0.2.4",
}).```
sandy.oberoi
2023-01-18 23:21
I have the provider test setup using the `VerifyProvider`, and defined a `MessageHandler` that returns back a constructed event and the required `contentType` in `Metadata`
when I run the test I get below error
```1) Verifying a pact between Consumer and Provider Given event processing for transaction - an event for transaction processing
1.1) has a matching body
expected 'application/protobuf;message=Event' body but was 'application/json;charset=utf-8' ```
matt.fellows
2023-01-23 06:28
Thanks. Looks like we need to support the correct content type.
Could you please raise a bug for us to fix?
sathish.sundararajan
2023-01-23 14:11
@uglyog can you help me with this ? Our team is looking to use grpc contract testing and we are stuck with this issue.
jason.taylor2
2023-01-23 14:31
I think this is an issue i brought up with @matt.fellows and he pointed me to this existing issue: https://github.com/pact-foundation/pact-go/issues/250 I added a comment on the issue with a bit of detail
sathish.sundararajan
2023-01-24 14:17
Thanks @jason.taylor2 for pointing out the issue. @uglyog @matt.fellows Could you please let us know when can we expect a fix for this ?
sandy.oberoi
2023-01-26 05:11
Thanks @matt.fellows. I noticed there is a similar bug that was opened yesterday: https://github.com/pact-foundation/pact-go/issues/263
Checking if this is related above? if not I can open a new one
matt.fellows
2023-01-27 00:01
I think this is separate, as it relates to the use of messages. If they are the same I can merge, but a bug report with your setup would be really helpful thanks!
matt.fellows
2023-01-30 21:06
I?m confused Sathish. Are you trying to send an empty request? The above issue is where there is no request being sent
sathish.sundararajan
2023-01-30 21:09
Actually we are sending empty request. proto file is something like below.
message ValidateToken {
}
matt.fellows
2023-02-01 04:44
Can you please share what the code would look like that needs to be supported?
jason.taylor2
2023-02-01 18:28
i think it is simply this?
``` grpcInteraction := `{
"pact:content-type": "application/protobuf",
"request": {
},```
matt.fellows
2023-02-02 01:17
so are you expecting a response with a token or something, is that the use case? I?m confused about how the token is being sent across the wire?
sathish.sundararajan
2023-02-02 01:23
gRPC empty request with metadata containing token is sent on the wire, the gRPC response will contain response based on token validation
matt.fellows
2023-02-02 01:37
I?m just struggling with the metadata bit. It might be outside of my knowledge - where is the metadata specified? In the plugin json configuration or something?
uglyog
2023-02-02 01:40
It is the message metadata that is mapped to the gRPC metadata. I.e. the key-values that can be associated with the RPC call (not encoded in the body which is represented by the proto file.
sathish.sundararajan
2023-02-02 01:54
Yeah it would look something like below:
```"pact:proto": "` + path + `",
"pact:proto-service": "Service/ValidateToken",
"pact:content-type": "application/protobuf",
"request": {
},
"metadata" : {
"Auth": "matching(type, 'value')"
}```
jason.taylor2
2023-02-02 15:27
i think the metadata aspect can be ignored - i don?t think it is relevant.
jason.taylor2
2023-02-02 15:38
so here is the diff for the example grpc consumer in pact-go that demonstrates the issue:
``` import (
@@ -37,8 +34,6 @@ func TestGrpcInteraction(t *testing.T) {
"pact:proto-service": "RouteGuide/GetFeature",
"pact:content-type": "application/protobuf",
"request": {
- "latitude": "matching(number, 180)",
- "longitude": "matching(number, 200)"
},
"response": {```
jason.taylor2
2023-02-02 15:38
trying to run the consumer:
```$ go test .
2023-02-02T15:38:13.418161Z WARN tokio-runtime-worker pact_plugin_driver::metrics:
Please note:
We are tracking this plugin load anonymously to gather important usage statistics.
To disable tracking, set the 'pact_do_not_track' environment variable to 'true'.
--- FAIL: TestGrpcInteraction (0.66s)
grpc_consumer_test.go:89:
Error Trace: grpc_consumer_test.go:89
Error: Received unexpected error:
retreived an empty message
Test: TestGrpcInteraction
FAIL
FAIL http://github.com/pact-foundation/pact-go/v2/examples/grpc 0.670s
FAIL```
uglyog
2023-02-03 03:00
I have released version 0.2.5 of the plugin with a fix for the empty message issue
matt.fellows
2023-02-03 06:33
I think the issue is in the protobuf plugin, but if it?s in Go I can fix. Would you mind please re-testing with the latest plugin?
sathish.sundararajan
2023-02-03 14:15
Great thanks @uglyog @matt.fellows yes we will retest it and post here
jason.taylor2
2023-02-03 14:58
looks like still getting error with newer plugin:
```$ ~/.pact/bin/pact-plugin-cli list
?????????????????????????????????????????????????????????????????????????????????????????????
? Name ? Version ? Interface Version ? Directory ? Status ?
?????????????????????????????????????????????????????????????????????????????????????????????
? protobuf ? 0.2.5 ? 1 ? /home/dev/.pact/plugins/protobuf-0.2.5 ? enabled ?
?????????????????????????????????????????????????????????????????????????????????????????????
(.venv) dev ? ~/ws/pact-go/examples/grpc (2.x.x ?) $ go test -v .
=== RUN TestGrpcInteraction
2023-02-03T14:31:21.293882Z WARN tokio-runtime-worker pact_plugin_driver::metrics:
Please note:
We are tracking this plugin load anonymously to gather important usage statistics.
To disable tracking, set the 'pact_do_not_track' environment variable to 'true'.
grpc_consumer_test.go:89:
Error Trace: grpc_consumer_test.go:89
Error: Received unexpected error:
retreived an empty message
Test: TestGrpcInteraction
--- FAIL: TestGrpcInteraction (0.65s)
FAIL
FAIL http://github.com/pact-foundation/pact-go/v2/examples/grpc 0.652s
FAIL```
sathish.sundararajan
2023-02-03 15:02
Interestingly it works with version: v2.0.0-beta.14.0.20221026030030-3bb31bcc34a1
But does not work with version: v2.0.0-beta.17.0.20221222235702-6e805a539d1b - which has provider_state capability
sathish.sundararajan
2023-02-03 15:05
@matt.fellows we might need fix from go as well to add both provider_state and fix for empty request
sathish.sundararajan
2023-02-06 14:22
ah cool, yeah if newer version is made available that would be great.
matt.fellows
2023-02-07 03:39
OK looks like I had a check when fetching the bodies that threw an error if it was of zero-length. The latest in the 2.x.x should allow that use case. I?m not quite sure of the side effects of that change as this is on the boundary between Golang and the shared library. I think it?s a valid scenario though, and it was just too cautious type guarding on my part
anhtvuong
2023-02-08 03:06
Is it true that protobuf-plugin requires protobuf proto3, which requires schema-registry to work? Is there a good local docker schema-registry image I can use?
I use confluentinc/cp-schema-registry:7.3.1 from docker-compose.yml and https://github.com/codingharbour/kafka-protobuf.git but ProtobufProducer and ProtobufConsumer don't work with schema-registry via http://localhost:8081. I verify that kafka container can produce and consume messages, and it receives protobuf from ProtobufProducer.
matt.fellows
2023-02-08 07:12
I don?t think we have integrated to the schema registry. Is that a requirement to use proto3? I don?t think so.
I assume you?re referring to Kafka though, but I?m still unsure why the schema registry would be part of a pact test
anhtvuong
2023-02-08 15:53
Yes, kafka with protobuf proto3 will talk to schema-registry to create/retrieve schema id for both consumer and producer to handle compatibility.
matt.fellows
2023-02-09 01:44
Do your tests need to include that part? Presumably you could reduce the scope of your tests?
Have you looked at how message pact works? (see :point_down: howtomessagepact)
sathish.sundararajan
2023-02-14 18:25
Looks like metadata support has been added to latest plugin: https://github.com/pactflow/pact-protobuf-plugin/releases/tag/v-0.3.0. Does pact-go requires any modification to get the plugin working or latest commit should work ?
uglyog
2023-02-14 22:15
It will need an update to Pact-Go. We are going to release a Pact-JVM update first, and then one for Pact-Go
sathish.sundararajan
2023-02-14 22:19
Thanks @uglyog! Could you please let me know if we can expect the release for Go in the coming months or will it be a little longer ?
uglyog
2023-02-14 22:28
coming months!! :rolling_on_the_floor_laughing: it will probably be done next week
sathish.sundararajan
2023-02-14 22:29
:grinning: thanks, we are looking to check it out using Go soon :slightly_smiling_face:
matt.fellows
2023-02-15 00:34
If Ron makes it easy for me, we should be able to turn it around quickly :stuck_out_tongue:
jason.taylor2
2023-02-21 17:13
looks like the `pact-plugin.json` version bump was missed?
```~/.pact/bin/pact-plugin-cli install https://github.com/pactflow/pact-protobuf-plugin/releases/tag/v-0.3.0
Installing plugin protobuf version 0.2.6
Downloaded https://github.com/pactflow/pact-protobuf-plugin/releases/download/v-0.3.0/pact-protobuf-plugin-linux-x86_64.gz to /home/dev/.pact/plugins/protobuf-0.2.6/pact-protobuf-plugin-linux-x86_64.gz```
jason.taylor2
2023-02-22 12:43
no worries - just confused me for about 15m when i upgraded and the cli was still claiming 0.2.6 :rolling_on_the_floor_laughing:
sathish.sundararajan
2023-02-28 16:18
I am back :stuck_out_tongue: could you please let me know if we have an update on the metadata support in go ?
matt.fellows
2023-02-28 23:27
I need to chat with Ron on what is required and what a good interface in Go should be. Last I spoke with him we are going to use an existing FFI function for the purpose, so it?s probably just a bit of wiring that is needed
sathish.sundararajan
2023-03-01 03:21
Thanks @matt.fellows ! we are eagerly waiting to start using it :slightly_smiling_face: Can we expect it in the next couple of weeks ? Not to push you but just asking :grinning:
matt.fellows
2023-03-02 13:09
I'll raise it at planning tomorrow and let you know. Love the enthusiasm :wink:
varnit.garg2424
2023-03-07 09:04
Hello guys, I need some suggestions, feel free to bash at me if you think it?s a bad idea :smile:
So, we are working on the creation of GRPC-based micro-services using Golang.
The thing is, we haven?t created a consumer service yet, rather, we built a cli-based application that would act as a client for now. And the provider service is almost developed.
I want to test the provider service, so I?m wondering what happens if I write the pact file manually, pass it to the provider service -> spin up the provider and run pact tests to make sure that provider is able to handle the requests.
And we will later replace the ?mocked? pact file with a real one generated by the consumer in the future. I understand that it?s a kind of hacky way to test out the provider?s service, can you please provide your opinions on that?
(My goal here is not to test the contract or integration between two services, rather, I just want to make sure that my provider service works correctly - I assume that if we go this way, then it will be easy in the future to extend the tests to test the integration with consumers)
matt.fellows
2023-03-07 10:50
I wouldn?t say it?s a bad idea, but just note that Pact is not really a functional testing tool. You should probably still do additional checks - e.g. side effects etc.
varnit.garg2424
2023-03-07 11:04
Thanks for your response. Yes I understand that pact is not meant for functional testing. However, in my case provider service doesn?t even have any db.
Just need to ensure that the service works in a nutshell.
tjones
2023-03-07 11:23
I agree with Matt - depending on what you?re doing, this might be ok (or not).
sathish.sundararajan
2023-03-07 15:56
Hi Folks, we have a proto file which imports another proto for validation. Does this chaining of proto files supported with grpc plugin ? Is there a way I can make it work since its failing ?
sathish.sundararajan
2023-03-07 22:34
Something like below:
syntax = "proto3";
import "validate.proto";
message Test {
Any var1 = 1;
}
where validate.proto is another proto file
sathish.sundararajan
2023-03-07 23:14
How can i get the matching working for array containing strings inside a grpc request ? I tried eachValue but i am not sure if i am doing it right ! Basically I would like to check all values inside an array are strings : ["data1", "data2"] !
matt.fellows
2023-03-08 01:11
See the `phones` example here: https://github.com/pactflow/pact-protobuf-plugin/#the-protobuf-test-configuration
sathish.sundararajan
2023-03-08 01:46
Thanks Matt! I was looking at how to add it from go side . Do you have a similar example ? https://github.com/pact-foundation/pact-go/blob/2.x.x/examples/grpc/grpc_consumer_test.go
sathish.sundararajan
2023-03-08 17:49
Please let me know when you have an example from the go side so we could implement it :slightly_smiling_face:
sathish.sundararajan
2023-03-09 22:50
I saw pact-plugin-cli-v0.0.5 yesterday which is not there anymore. Has it been pulled back ?
sathish.sundararajan
2023-03-09 22:57
I had the `additionalIncludes` working but does not work with 0.0.4.
sathish.sundararajan
2023-03-09 23:05
Sorry i must have changed something on my end. I was testing with different versions of pact-plugin-cli and pact-plugin and must have confused between the two. Will troubleshoot more. Thanks
pjayakumar
2023-03-10 03:31
Hi All. I am looking for some examples for pact contract testing gRPC *bi-directional streaming* services. Can you please advise if this is supported at the moment? if yes, can you please point me to some examples? Thanks!
matt.fellows
2023-03-13 23:47
@uglyog I actually have forgotten how we support this, if we do.
Would we recommend people create two separate tests? One for the consumer initiated request and another for provider initiated?
IIUC streaming is really just an open pipe where consumers/providers can communicate. It doesn?t imply a given / when / then scenario.
uglyog
2023-03-14 00:12
I haven't really thought through what bi-directional streaming means in a Pact context. My assumption was that it can be reduced to a few Pact tests for the different types of messages, as streaming is just a transport concern. I was waiting for some examples of how people would use this.
If we look at bi-directional streaming in general (not gRPC specific), I think there would be a couple of different use cases:
1. 1-1: (I assume this is the general case) the requests and responses are sent as required, but they are correlated. I.e. As users need to be provisioned , the consumer sends a ProvisionUserMessage (there can be multiple of theses at any time), and when the work is done, the provider sends back ProvisionedUserStatus messages. I can see a single Pact test needed to test a correctly formed ProvisionUserMessage results in a correctly formed ProvisionedUserStatus message.
2. 1-n: the consumer can send any number of messages, but each one is correlated to a number of response messages. I guess a query for documents would be this type of interaction. Again a single Pact test is needed here.
3. n-m: This is one I'm unsure of. The consumer can send a number of messages, and each group is correlated to a number of response messages. We don't support this type of interaction.
4. n/m: Again, I'm unsure if anyone will use this option. The consumer sends messages as needed, and the provider sends messages as needed, and there is no correlation. I.e. consumer sends events to provider, and provider sends events to consumer. These can probably just be reduced to two async message Pact tests, one for C -> P and one for P -> C
pjayakumar
2023-03-14 03:49
thanks for the inputs @uglyog and @matt.fellows. I think our use case falls under `1-n` category specified above. On every consumer request message, server sends a bunch of correlated messages.
sathish.sundararajan
2023-03-20 21:20
@matt.fellows can you help me on how i can achieve this ? Dont need a full blown example, but If you can paste matchers here to achieve this, it would be helpful.
sathish.sundararajan
2023-03-21 14:05
Whats the proper way to use matcher for matching eachValue inside a list of items. I have a grpc request where one of the field value is a list of items. I am looking to properly match eachValue inside the list. The below does not seem to work. could someone please help ?
"item": ['somevalue1', 'somevalue2']
`"item": "eachValue(matching(type, 'value'))"`
sathish.sundararajan
2023-03-21 23:28
Here is the error i am getting from pact-plugin
`pact_protobuf_plugin::mock_service: Failed to match the request message - BodyMismatches({"$.item": [BodyMismatch { path: "$.item[0]", expected: Some(b""), actual: Some(b"00000000000000000000000000000000"), mismatch: "Unable to match '' using EachValue(MatchingRuleDefinition { value: \"00000000000000000000000000000000\", value_type: Unknown, rules: [Left(Type)], generator: None })" }]})`
uglyog
2023-03-21 23:45
Looks like it might be a bug with the EachValue matcher, I'll have to investigate
sathish.sundararajan
2023-03-21 23:56
Thanks @uglyog I have opened an issue with pact-plugin here: https://github.com/pact-foundation/pact-plugins/issues/22
sahoo.skumar
2023-03-27 02:39
The pact contract test for protobuf is running inside a container, the setup is absolutely fine and proper PACT_PLUGIN_DIR is set. The same test running fine without container, while running inside container the test is not getting protobuf plugin, Could some one help on this
```26-03-2023 19:33:22.140 [main] ERROR au.com.dius.pact.consumer.dsl.PactBuilder.100 - Plugin process did not output the correct startup message in 10000 ms - got null
26-03-2023 19:33:32.308 [main] ERROR au.com.dius.pact.consumer.dsl.PactBuilder.100 - Plugin process did not output the correct startup message in 10000 ms - got null
[ERROR] Tests run: 3, Failures: 1, Errors: 2, Skipped: 0, Time elapsed: 21.991 s <<< FAILURE! - in com.hpe.gravity.clientstats.ctest.ApplicationCTest
[ERROR] consumeConfigureClientStateMessage{AsynchronousMessage} Time elapsed: 11.769 s <<< ERROR!
org.junit.jupiter.api.extension.ParameterResolutionException: Failed to resolve parameter [au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage message] in method [void com.hpe.gravity.clientstats.ctest.ApplicationCTest.consumeConfigureClientStateMessage(au.com.dius.pact.core.model.V4Interaction$AsynchronousMessage) throws com.google.protobuf.InvalidProtocolBufferException]: Plugin protobuf with version any was not found
at com.hpe.gravity.clientstats.ctest.ApplicationCTest.configureClientStateMessage(ApplicationCTest.java:75)```
uglyog
2023-03-27 02:46
Startup the container with a shell, and see if you can run the plugin executable
sathish.sundararajan
2023-04-10 13:31
I am looking to understand why `_id` and `key` fields change every time a pact is created although the contents dint change. This causes the webhook to be triggered unnecessarily every time the pact is published.
```"interactions": [
{
"_id": "face3451319bf15d8cfaf1f93e432678bca8361b",
"description": "Service - Validate",
"interactionMarkup": {
...
},
"key": "2e869fcb0c3c9cc9",
"pluginConfiguration": {
"protobuf": {
...
}
},```
```"interactions": [
{
"_id": "a2678b2644ff530989b539be273597a32f9aa6e6",
"description": "Service - Validate",
"interactionMarkup": {
...
},
"key": "9b0ae9c547c8b847",
"pluginConfiguration": {
"protobuf": {
...
}
},```
sathish.sundararajan
2023-04-10 19:30
The two interactions above are real pact generated by consumer test. Check out the `_id` and `key` values which are different.
sathish.sundararajan
2023-04-10 21:53
Its written in golang following similar pattern here: https://github.com/pact-foundation/pact-go/blob/2.x.x/examples/grpc/grpc_consumer_test.go
sathish.sundararajan
2023-04-10 22:09
Also am not sure if these two fields can cause pact content to be changed causing webhook to be triggered
uglyog
2023-04-10 23:23
`_id` is generated by the Pact broker, I think it may just be a random key, but it could maybe be generated on the contents.
`key` is a unique key for the interaction, and will be generated from the contents of the interaction if not provided.
Because `key` is changing, it means your interaction contents are different.
sathish.sundararajan
2023-04-11 00:41
Interesting as i compared the two pacts which are identical but yet pact broker thinks they are different
matt.fellows
2023-04-11 01:06
I suspect the `_id` changing is not relevant to the problem, but the `key` would definitely trigger a ?change?
sathish.sundararajan
2023-04-11 01:07
yeah looks like it, we never set this value for rest or kafka interactions where it works fine
sahoo.skumar
2023-04-14 03:13
Thanks, we observed there was issued with the permission, after fixing it started working
sahoo.skumar
2023-04-15 02:38
Hi @*Sathish Sundararajan* do you have any example of *additionalIncludes, I am unable to make it work*
sahoo.skumar
2023-04-17 02:30
Hi All, is there any working example available for "additionalIncludes", I am trying to use additionalIncludes to use protos those are imported
I am struggling to make it work, I am using protobuf-0.3.0, Any help would be great
sathish.sundararajan
2023-04-18 20:00
@sahoo.skumar You pass in the directory of where the proto file decides, then it will work.
sahoo.skumar
2023-04-24 08:57
using "additionalIncludes" for setting imported proto working in protobuf-0.3.0 version, Thanks
yu.xie
2023-05-03 00:02
Cross posting this question here:
https://pact-foundation.slack.com/archives/C9UTHTFFB/p1683054915705669
yu.xie
2023-05-11 19:29
Hi, do we support reading from protobuf descriptor set binary file instead of raw `.proto` files? (Looks like we only take `-I` https://github.com/pactflow/pact-protobuf-plugin/blob/9c075769a5b5d2d09c500b850930e6011a6b66ad/src/protoc.rs#L87, but not `--descriptor_set_in` so I assume it?s not supported today, if that?s the case is there a plan to support it?)
yu.xie
2023-05-12 00:39
Another question:
In the matcher do we support list of enums?
``` "request": {
"states": [
"matching(equalTo, 'ENUM_VAL_1')"
]
},```
I?m getting this error --- is it because enum is not added to https://github.com/pactflow/pact-protobuf-plugin/blob/9c075769a5b5d2d09c500b850930e6011a6b66ad/src/message_decoder/mod.rs#L334?
```Failed to match the request message - BodyMismatches({"$.states": [BodyMismatch { path: "$.states[0]", expected: Some(b"ENUM_VAL_1"), actual: Some(b"0102"), mismatch: "Expected and actual field have different types: 1:(states, Varint, Enum) = ENUM_VAL_1 and 1:(states, LengthDelimited, Unknown) = 0102" }]})```
matt.fellows
2023-05-12 04:48
I don?t think so, but if you wanted to see it, please raise a feature request on that github repo
yu.xie
2023-05-15 23:14
Does async messaging work with protobuf in Golang? I see example for each of them (https://github.com/pact-foundation/pact-go/tree/master/examples/messages#message-pact-example, https://docs.pact.io/implementation_guides/pact_plugins/examples/protobuf/protobuf-provider), but no example that handles async protobuf messages
matt.fellows
2023-05-15 23:30
I guess it?ll be something like this, but with the async package: https://github.com/pact-foundation/pact-go/blob/2.x.x/examples/grpc/grpc_consumer_test.go#L52
matt.fellows
2023-05-15 23:31
This second part of the test shows how you would use a plugin with it: https://github.com/pact-foundation/pact-go/blob/2.x.x/message/v4/asynchronous_message_test.go#L62
yu.xie
2023-05-15 23:37
So basically you should decode from `AsynchronousMessage.Contents` manually to protobuf and then verify that?
yu.xie
2023-05-16 00:21
It looks like once you used the plugin (`WithPlugin` ), you can?t use `AsType` and `ConsumedBy`
matt.fellows
2023-05-16 01:08
`AsType` makes sense, because how would the framework know how to arbitrary convert a plugin?s contents to a narrow type?
`ConsumedBy` is replaced by `ExecuteTest`
matt.fellows
2023-05-16 01:10
I can?t remember why I changed that now I?m sorry. There was possibly a good reason for it though :slightly_smiling_face:
yu.xie
2023-05-16 01:23
I see that makes sense, it might worth marking `ConsumedBy` with `// Deprecated: Use ExecuteTest instead.` then to avoid confusions.
matt.fellows
2023-05-16 01:25
It?s not deprecated though, it?s still available for non-plugin requests.
scoplin
2023-05-25 23:38
Greetings! My team is trying to start using Pact in an ecosystem of scala-based gRPC microservices. One of the things I?m encouraging is the use of well-defined gRPC https://grpc.github.io/grpc/core/md_doc_statuscodes.html when applicable. It seems to me that these codes can be meaningful in a contract. For example, we could use `NOT_FOUND` as a means to communicate non-existence of an entity. If the provider changed behavior to return `INVALID_ARGUMENT` instead in the same situation it would clearly break clients.
Is there a way to write a consumer test matching on gRPC response status? An example would be ideal. I?m fearing there may not be based upon https://github.com/pactflow/pact-protobuf-plugin/blob/main/src/verification.rs#L67 which seems to only perform validations on non-error responses, but I am not 100% sure I?m looking in the right spot.
uglyog
2023-05-25 23:44
BTW, I found the support for gRPC status codes not very good. The Rust tonic library deals with it properly, but the gRPC Java library seems to ignore it and tries to read the messages anyway.
scoplin
2023-05-26 00:18
@uglyog Sure, I can do that. Here I assume? https://github.com/pactflow/pact-protobuf-plugin
Curious what you?ve seen from gRPC Java with respect to this? We tend to see actual non-OK status codes come back as `StatusRuntimeException`. Haven?t seen the ignoring issue, but maybe we?re using a different library (scalapb wrapper on grpc-netty) or you?re coming in at a different layer?
uglyog
2023-05-26 03:45
> Curious what you?ve seen from gRPC Java with respect to this?
I haven't really dug into it, I was expecting to get some exception with the status and associated message, but I got an IO exception stating that the Protobuf message could not be parsed. It might be due to some interceptor needing to be setup, but I expected this behaviour out of the box and it looks like it ignores the status.
yu.xie
2023-05-27 00:07
Following up on this thread:
So in async protobuf provider test, it looks like we need to have the provider implement this message handler func: (i was using pact-go)
```func(states []models.ProviderState) (message.Body, message.Metadata, error)```
Here `message.Body` is an interface ? what shall I give here? I tried both the proto Golang body, and deserialized `[]byte` , they both throw
yu.xie
2023-05-27 00:10
I?m guessing that the verifier doesn?t know it should be using the protobuf plugin to parse my provider message, what am I missing here?
rmartine
2023-05-30 21:34
Hi, dumb question -- are the consumer and provider required to have the same version of the protobuf plugin?
uglyog
2023-05-30 23:20
No, but the behaviour will be different (i.e. later versions may have a defect fixed) and that could impact things.
yu.xie
2023-05-31 19:10
Following up here, it looks like https://github.com/pact-foundation/pact-go/blob/2.x.x/message/verifier.go#L206 is setting the provider proxy to return message as `application/json; charset=utf-8` , while pact expects it to be something like `application/protobuf; message=ProtoMessageExample` .
I guess it means protobuf plugin won?t work with provider verification through https://github.com/pact-foundation/pact-go/blob/2.x.x/message/message.go#L12?
yu.xie
2023-05-31 19:50
Also, even after I updated the line above to return the proper content type, pact doesn?t seems to be able to find the protobuf matcher:
```2023-05-31T19:49:05.104537Z DEBUG ThreadId(01) verify_interaction{interaction="XXX"}: pact_matching: No content matcher defined for content type 'application/protobuf;message=SessionStatusResponse', using core matcher implementation```
```2023-05-31T19:49:05.104581Z DEBUG ThreadId(01) verify_interaction{interaction="async health information"}: pact_matching: No body matcher defined for content type 'application/protobuf;message=SessionStatusResponse', using plain text matcher```
yu.xie
2023-05-31 21:40
Digging deeper here, seems the async protobuf contract generated on the consumer side (through `AddAsynchronousMessage`) is missing plugin information.
Unlike sync message, the async message doesn?t have `plugins` and raw proto desciptors under `metadata` . Also, `pactSpecification.version` is `3.0` for async contract, (instead of `4.0` for sync grpc contracts).
That might be why the provider side can?t find the matcher from the plugin I guess.
matt.fellows
2023-06-01 07:13
Thanks for the digging Yu. So the last point, this PR should address it: https://github.com/pact-foundation/pact-go/pull/292
It was failing due to an upstream issue, but I?ve reverted the changes for the upstream dependency and will merge this shortly.
matt.fellows
2023-06-01 07:16
> ```2023-05-31T19:49:05.104537Z DEBUG ThreadId(01) verify_interaction{interaction="XXX"}: pact_matching: No content matcher defined for content type 'application/protobuf;message=SessionStatusResponse', using core matcher implementation```
>
hmm interesting. I suspect the plugin only registers `application/protobuf` and because of the additional field it?s mis matching (guess).
What happens if you hard code `application/protobuf` as the content type? I wonder if that goes a little further (but then probably fails due to the missing qualifier).
matt.fellows
2023-06-01 07:18
> ```func(states []models.ProviderState) (message.Body, message.Metadata, error)```
Hmm good point. I think we will need a variation on this that allows a `[]byte` to be returned here, and if a plugin is being used in an interaction, we use the different typed interface to signal that we require a byte array. We may also need to capture a content type here (or elsewhere) that is used to set the response mime type
matt.fellows
2023-06-01 07:18
Could you please raise an issue?
Or - and this would be amazing :sparkles: - a PR that demonstrates how you get it to work? We can then pair through a solution to this problem
yu.xie
2023-06-01 17:17
I actually didn?t get this working. I tried force returning `application/protobuf` and message name as content type header.
But eventually, pact is still falling back to plain text matcher which obviously doesn?t work. My guess is it?s because the generated contract doesn?t even include any plugin information and proto details on the consumer side
yu.xie
2023-06-01 17:21
So the sync grpc version of generated contract looks like this to me:
```{
"metadata": {
"pactRust": {
"ffi": "0.4.1",
"mockserver": "1.0.0",
"models": "1.0.4"
},
"pactSpecification": {
"version": "4.0"
},
"plugins": [
{
"name": "protobuf",
"version": "0.3.0",
"configuration": "..."
}
]
}
}```
While the async generated contract has metadata like this:
``` "metadata": {
"pactRust": {
"ffi": "0.4.1",
"models": "1.0.4"
},
"pactSpecification": {
"version": "3.0.0"
}
}```
In addition to spec version being 3.0.0, it also misses plugin information
yu.xie
2023-06-01 17:24
On the consumer side I?m doing this to generate:
```p.AddAsynchronousMessage().
GivenWithParameter(...).
ExpectsToReceive("...").
UsingPlugin(message.PluginConfig{
Plugin: "protobuf",
Version: "0.3.0",
}).
WithContents("...", "application/protobuf").
ExecuteTest(...)```
matt.fellows
2023-06-03 23:41
Oh, sorry, did you pull the latest with the spec serialisation fix?
Then if you try hard coding the content type again let's see how far you get
matt.fellows
2023-06-03 23:43
Btw part of the underlying issue is https://github.com/pact-foundation/pact-go/issues/265 I think
scoplin
2023-06-06 01:12
Hi - I was wondering if the https://github.com/pact-foundation/pact-stub-server was usable for generating a pactfile-based test double for a gRPC service? I'm _guessing_ no based upon the way it https://github.com/pact-foundation/pact-stub-server/blob/master/src/server.rs#L166 and then matches on method/path/parameters.
If the answer is in fact "no", then I wonder if there are any alternative suggestions for stubbing out gRPC providers for a client?
uglyog
2023-06-06 01:30
Technically, most gRPC interactions use HTTP 2, so with a little effort we could possibly get it to work.
rohitkrishnan
2023-06-06 14:00
Greetings, I'm trying to write a provider verification test on Android using the gRPC protobuf plugin. Since I am constrained to using jUnit4, I am trying my best to get the plugin working but having some trouble. At a high level: 1) Is this even possible? 2) Is there an easier way to do this?
So far I created a V4 pact that uses the `protobuf` plugin but the `usingPlugin` call fails with `Err(error=Init call to plugin protobuf failed: io.grpc.StatusRuntimeException: CANCELLED: Failed to read message.)` . Can I just start a mock server in some port and it will magically work, or will there be other roadblocks such as the generation of the pact at the end?
My end goal is to generate a pact that I can feed to a provider test to validate that my client contract is compatible with the latest server.
rohitkrishnan
2023-06-06 15:23
Digging deeper into the stacktrace, it looks like some classpath issue:
```cause=java.lang.NoClassDefFoundError: Could not initialize class io.pact.plugin.Plugin
at io.pact.plugin.Plugin$CatalogueEntry$ValuesDefaultEntryHolder.<clinit>(Plugin.java:1212)
at io.pact.plugin.Plugin$CatalogueEntry$Builder.mergeFrom(Plugin.java:1696)
at io.pact.plugin.Plugin$CatalogueEntry$1.parsePartialFrom(Plugin.java:2081)
at io.pact.plugin.Plugin$CatalogueEntry$1.parsePartialFrom(Plugin.java:2073)
at com.google.protobuf.CodedInputStream$ArrayDecoder.readMessage(CodedInputStream.java:889)
at io.pact.plugin.Plugin$InitPluginResponse$Builder.mergeFrom(Plugin.java:2612)
at io.pact.plugin.Plugin$InitPluginResponse$1.parsePartialFrom(Plugin.java:2986)
at io.pact.plugin.Plugin$InitPluginResponse$1.parsePartialFrom(Plugin.java:2978)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:86)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48)
at io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parseFrom(ProtoLiteUtils.java:223)
at io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(ProtoLiteUtils.java:215)
at io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(ProtoLiteUtils.java:118)
at io.grpc.MethodDescriptor.parseResponse(MethodDescriptor.java:284)```
rohitkrishnan
2023-06-06 15:29
Ah shoot, looks like it is trying to initialize a `MapEntry` with the protobuf library, but our project uses protobuf-lite :confused:
uglyog
2023-06-06 23:10
I don't think anyone has tested JUnit4 support, and we will need to investigate protobuf-lite.
andrevdrodrigues
2023-06-15 07:21
Hey @uglyog!
Today I managed to verify one pact using the pact_cli.
However, I would like to match the exact value of one field. I was checking this doc but I didnt find it:
https://docs.pact.io/implementation_guides/rust/pact_matching#matching-status-codes
When I try to match by type, its work fine:
```matching(type,true)```
andrevdrodrigues
2023-06-15 07:47
No, Im creating pact for grpc microservice? Im creating one test on the consumer side to generate the contract with this request body:
``` "request", Map.of(
"id", "matching(type,123)",
"url", "matching(type,'https://url.com/')",
"url2", "matching(type,'https://url2.com')",
"test", "matching(type,true)"
),```
However, in this case Im matching only by type and I would like match by the exact value.
For example:
If I send one request with `id = 123`, I would like to match?. Any other value, I would like to thrown an error.
yousafn
2023-06-15 09:43
Hey Andre,
Which language are you creating your consumer pact files with that contain the gRPC interactions.
yousafn
2023-06-15 10:01
Awesome, you should be able to use the jvm framework to verify your pacts aswell, but you can also use the CLI as you have found :slightly_smiling_face:
andrevdrodrigues
2023-06-15 15:15
Hi everyone!
How is the better approach to create one contract with more than one interaction?
Anyone has some example?
jason.taylor2
2023-06-15 19:28
for go, i have been able to do this with separate tests that use the same consumer and provider names. each test will append it?s interaction to the json. however, if i try to AddSynchronousMessage multiple times on the same pact then the second one fails.
jason.taylor2
2023-06-15 19:32
Are generators supported with the protobuf plugin? Don?t see it explicitly called out as supported/unsupported in the README. Haven?t been able to find any examples in any of the golang projects related to protobufs. For example, would like to generate a random number of strings from the mock for a repeated field.
uglyog
2023-06-16 00:05
It should work in theory, but I don't think anyone has tested them. There may be certain types of fields that may not work, like enums.
andrevdrodrigues
2023-06-16 07:33
Hey @jason.taylor2!
Thanks for your help!
I understood that you created new tests for each interaction, but is also necessary to define the pact using `toPact()` . In this point, Im little bit confused? So should I define 2 of these :
```@Pact(consumer = "my-consumer")
V4Pact myMethod(PactBuilder builder) ```
or should I only add one more interaction such as:
```.expectsToReceive("my interaction", "core/interaction/synchronous-message")
.with(Map.of(
))).toPact().```
?
jason.taylor2
2023-06-16 14:59
tx - is there an example of how to use the generators for gRPC from go?
jason.taylor2
2023-06-16 15:09
I suspect the two methods with the same consumer name should work from Java. if your providerName and consumer names line up across tests they would each append their interaction into the same contract. if not, will need someone with explicit java experience to chime in
yu.xie
2023-06-20 22:36
> I?ll aim to create an example next week to test it out myself
@matt.fellows Checking back on this thread ? is there an e2e working example now?
bdocherty
2023-07-05 13:21
Hi team, I'm wondering if it's possible to implement pact with grpc using pact-js? From what I can gather the plugin is not yet supported by pact-js and I cannot see any examples that do not utilise the plugin
yousafn
2023-07-05 13:42
pact-js supports v4 in beta, there is a plug-in example, you can use the pact-protobuf-plugin for grpc
bdocherty
2023-07-05 13:45
Is it possible to link the plugin example, I've had trouble finding one for pact-js?
yousafn
2023-07-05 13:54
in the examples folder in the pact-js repo under v4 subfolder, sorry i am on phone atm
yousafn
2023-07-05 13:56
here is a basic grpc test from one of my scratchpads
https://github.com/YOU54F/pluginopedia/blob/main/example-project-js-grpc-plugin/test/matt.consumer.spec.ts
bdocherty
2023-07-07 10:30
QQ Yousaf, we use Jest throughout our project. Do you know if there are plans to update `jest-pact` to include a v4 dist?
yousafn
2023-07-07 10:56
I?d really love to deprecate jest-pact, and by way of virtue mocha-pact and instead utilise pact-js directly. We don?t have the same lifecycle methods to worry about compared to spinning up a standalone process (as per the Ruby impl).
I?d be happy to consider PR?s to update jest-pact but it isn?t hugely high on my priority list to do myself. My next mission is pact-python v3/v4 + plugins with the FFI and review of all the pact-php progress towards the FFI migration
yousafn
2023-07-07 10:56
I haven?t discussed the potential deprecation with anyone yet, so I am just noodling
yousafn
2023-07-07 10:58
We did have some feedback on the v3 dsl in jest-pact https://github.com/pact-foundation/jest-pact/issues/215 which gave consideration to the change in lifecycles.
bdocherty
2023-07-07 11:26
Yes I see. I'd also like to use pact-js directly. Is that possible at this current moment? Or should I switch to using mocha for the time being?
yousafn
2023-07-07 11:36
yeah, its grand to use pact-js directly, there are some vanilla jest examples in the repo but here is a pretty simple one which exercises all the matchers
https://github.com/YOU54F/pact-matchers/blob/main/pact-js-v3-v10/matchers.v3.pact.test.js
yousafn
2023-07-07 11:36
and similar but as vscode snippets https://github.com/YOU54F/vscode-pact-snippets/blob/main/js/v3.test.js
matt.fellows
2023-07-10 23:54
Sorry I missed this - I?ve updated the issue. Can you please let me know if it addresses your issue?
rohitkrishnan
2023-07-11 15:43
I'm writing a consumer test that uses the gRPC plugin and for some reason it's having trouble with a `Timestamp` field in the request. Is this supported?
```Pact Test function failed with an exception: Bad type on operand stack
Exception Details:
Reason:
Type 'com/google/protobuf/Timestamp' (current frame, stack[2]) is not assignable to 'com/google/protobuf/GeneratedMessageLite'```
My request looks like this (Kotlin):
```"timestamp" to mapOf(
"seconds" to "0",
"nanos" to "0"
)```
My message in the proto looks like this:
```message SomeMessage {
... Other fields ...
google.protobuf.Timestamp timestamp = 4;
}```
rohitkrishnan
2023-07-11 15:45
This might be a self inflicted wound because we have both protobuf and protobuf-lite in our project, but that hasn't caused an issue so far with other fields, so wonder what's different here.
rohitkrishnan
2023-07-11 16:01
Update: The exception isn't in Pact but rather parsing the proto bytes from the V4 interaction object into a concrete proto object. Looks like this is my fault.
matt.fellows
2023-07-11 22:51
Thanks for updating/clarifying (and love the :rubberduck: debugging - this will hopefully help others!)
andrevdrodrigues
2023-07-12 07:59
Hello team!
I already managed to use the pact-cli plugin for protobuff locally, but now I?m trying to integrate it with my pipeline in github.
Does anyone have some examples of how can I install and use the plugin with github action?
Thanks a lot :slightly_smiling_face:
matt.fellows
2023-07-12 09:15
The latest versions of the tooling should automatically download at test time if not present already
matt.fellows
2023-07-12 09:16
I use it in Pact Go here: https://github.com/pact-foundation/pact-go/blob/1e8dbe423fc3053fa13913f356bb8026f1279187/Makefile#L43
andrevdrodrigues
2023-07-12 09:50
For me the tooling is not downloaded automatically? So, I need to have one step in my pipeline only to download, install the pact-plugin and after install one specific plugin for protobuf.
rohitkrishnan
2023-07-12 18:44
For posterity, we had the protobuf-javalite library as a transitive dependency and solve the issue by replacing it with the full version via a gradle configuration block.
matt.fellows
2023-07-12 22:50
Ah, looks like the auto-install was added to the driver in 0.4.0 (https://github.com/pact-foundation/pact-plugins/releases/tag/pact-jvm-driver-v0.4.0) but that doesn?t seem to be in the latest JVM package I think
sathish.sundararajan
2023-07-13 17:42
Folks, we are writing grpc contract test. For the part where we pass the proto file path to the grpc plugin, if the file is located in the same directory(or subdirectories) where the test is run from, it works. If the file is even one level up, then the plugin throws the error that it cannot find the file. Is there some restriction that the proto file has to be in or under the current folder level where the test runs from ?
sathish.sundararajan
2023-07-17 13:29
@uglyog they are absolute paths and not relative. @wesley.newcomb
uglyog
2023-07-20 23:07
Sorry, I don't have any more information here. The plugin uses gRPC protoc to compile the proto files. That is what is probably throwing the error, and I wouldn't know why it would do that.
sathish.sundararajan
2023-07-26 05:04
Yes, I can view the file from within the container using the same path that plugin is reporting unable to find
bdocherty
2023-07-31 12:36
Hey @yousafn :wave:
Appreciate all the help in getting started. I've got a flow working with our solution!
It's manual at the moment though and I'd like to hook it up to the CI and the broker. I mostly know what I'm doing here as I've done it before but one part I'm unsure on is the plugin itself. For local dev I installed through the cli, but what's the best to do this for a CI solution using github actions?
yousafn
2023-07-31 12:45
solid.
You can use a bash script for now.
https://github.com/pact-foundation/pact-js/blob/master/scripts/install-plugins.sh
https://github.com/pact-foundation/pact-plugins/blob/feat/pact-csv-plugin-linux-aarch64/scripts/install-plugin-cli.sh
https://github.com/YOU54F/pact-python/blob/feat/ffi_consumer/script/install_plugins.sh
The pact-ruby-standalone also comes with the pact-plugin-cli now installed. (note to myself:- I need to update the docs to reflect this)
https://github.com/pact-foundation/pact-ruby-standalone
andrevdrodrigues
2023-08-24 15:40
Hey folks.
Im facing a weird error when I try to verify my grpc-contract in the provider side:
```Plugin protobuf failed to validate the interaction: Plugin configuration item with key '330d8b9b2fef4480c366dd95e2ebb56e' is required. Received config ["f58d69682eec609d725e46c388aff650"]
io.pact.plugins.jvm.core.PactPluginValidationForInteractionException: Plugin protobuf failed to validate the interaction: Plugin configuration item with key '330d8b9b2fef4480c366dd95e2ebb56e' is required. Received config ["f58d69682eec609d725e46c388aff650"]```
Do you all know what about this error?
Thanks guys.
uglyog
2023-08-24 22:59
Do a clean before running your consumer tests, you probably have a stale Pact file affecting things
andrevdrodrigues
2023-08-25 08:21
Yes, I cleaned,,, and also removed.
My service is generating one contract that uses different proto files, I think that it can be the problem.
Pact is always generating only one of this files in the configurations:
Example:
```"plugins": [
{
"configuration": {
"bd82122dd37a0a29e676d1f63ef62eba": {
"protoDescriptors": "",
"protoFile": ""
}
},
"name": "protobuf",
"version": "0.3.4"
}
]```
sathish.sundararajan
2023-09-06 16:14
I am testing type matcher with requestMetadata but it seems to expect exact value instead of matching type.
```{
"pact:proto": "/home/service.proto",
"pact:proto-service": "Service/Validate",
"pact:content-type": "application/protobuf",
"request": {
"token": "matching(type, 'bj8qnqw9cnalksnd9qwnk1j9cjm')"
},
"requestMetadata": {
"x-b3-traceid": "matching(type, 'foo')"
},
"response": {
"valid": "matching(boolean, true)"
}
}```
```pact validation failed: [{Request:{Method: Path:Service/Validate Query: Headers:map[] Body:<nil>} Mismatches:[{Actual:0123456789abcdef0123456789abcdef Expected:foo Key: Mismatch:Comparison of metadata key 'x-b3-traceid' failed: Expected 'foo' to be equal to '0123456789abcdef0123456789abcdef' Type:}] Type:}]```
rohitkrishnan
2023-09-18 16:55
Hello, I found a bug in the pact protobuf plugin where it errors out like so:
```Protobuf enum value .some.package.ENUM has no value ENUM_VALUE```
The issue occurs if there are multiple top-level enums with the same package name across different files. The good news is I have a pull-request fix for this. As a process question - Should I just create a PR or do I need to file an issue as well?
The technical reason is that in http://utils.rs, it only calls `find_enum_by_name_in_message` on the _first_ message that matches the package name. In my case I have multiple of these, so the `find_enum_by_name` method returns None. The solution is to iterate through each of the messages that have the matching package and call `find_enum_by_name_in_message` for each of those.
yousafn
2023-09-18 17:40
:blobwave:
No prescribed way but if its a bug I?d normally raise an issue first, and link to a pull request that proposes a fix for said bug.
Either way you get the change in front of people, is good for collabing, if it needs anything additional, it can always be added retrospectively!
rohitkrishnan
2023-09-18 17:49
Wonderful thanks, ill throw some tests in and make a PR + an issue shortly.
rohitkrishnan
2023-09-18 20:14
PR is up! Would love someone to provide feedback :pray:
https://github.com/pactflow/pact-protobuf-plugin/issues/30
rohitkrishnan
2023-09-20 14:32
Hey folks - would you happen to know when the next release of this plugin may be? We'd like to use the fix I made above but are waiting for the next release to be cut.
andrevdrodrigues
2023-10-19 15:44
Hi team.
I have one contract very big with many endpoints and my test class to the define the pact file is becoming huge.
Does anyone know if is possible to generate the same contract using different test classes?
Thanks.
matt.fellows
2023-10-20 01:20
Are you asking if you can split tests across multiple classes? Yes, you definitely should be able to do that, and how you do it will entirely depend on what client SDK/language you?re using.
andrevdrodrigues
2023-10-20 12:33
Im asking about split test accross multiple classes but for the same contract? I mean generating the same pact file.
My language is java/jdk.
rohitkrishnan
2023-11-07 15:20
Hello, could we please get a review on this PR (and subsequently a patch release if possible) from a colleague of mine?
https://github.com/pactflow/pact-protobuf-plugin/issues/38
https://github.com/pactflow/pact-protobuf-plugin/pull/37/files
anna.nava
2023-12-04 16:51
:pray: would anyone have any suggestion or where can I see examples on how to build a protobuf matcher that checks that a property in the protobuf message is either of type UUID4 ..... or that it simply matches the regex :pray:
from my kotlin code below :point_down: you can clearly see I don't know what to pass in and I don't want to serialize from JSON, but from protobuf
or maybe I am doing it all wrong and can't be done like this as there isn't a direct equivalent to `http://au.com.dius.pact.consumer.dsl.DslPart` for protobuf.... so I wonder if Protobuf requires a different approach to what I am attempting here :questionmario:
```@Pact(consumer = "my-service")
fun createPact(builder: PactBuilder): V4Pact {
val messageFile = loadFileFromJarResource("/events/pub/a/v1/a_event.proto")
return builder
.usingPlugin("protobuf")
.expectsToReceive("an AEvent ", "core/interaction/message")
.with(
mapOf(
"message.contents" to mapOf(
"pact:proto" to messageFile.path,
"pact:message-type" to "AlertEvent",
"pact:content-type" to "application/protobuf",
"pact:protobuf-config" to mapOf("additionalIncludes" to listOf("/Users/anna.nava/dev/protobuf/proto")),
"this_amazing_property" to mapOf(
//this compiles, but passes empty string: "id" to regex("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$")
//this doesn't compile:"id" to matchRegex("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$")
//this compiles, but ends up passing in an empty string: "id" to regex("[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89aAbB][a-f0-9]{3}-[a-f0-9]{12}")
//"id" to mapOf("matchers" to mapOf("match" to "regex", "regex" to "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}\$" ))
"id" to regexp("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$", "123e4567-e89b-12d3-a456-426614174000")
),```
(apologies if this is not the right channel for, very appreciated if anyone can redirect me where is best for me to ask this, thanks!)
rholshausen
2023-12-05 00:05
I think the way it is meant to be done is something like:
```"this_amazing_property" to "matching(uuid, '123e4567-e89b-12d3-a456-426614174000')"```
anna.nava
2023-12-05 12:59
thanks! I appreciate that.... unfortunately it errors (my interpretation is uuid is only supported with JSON not protobuf)
```matching(uuid, '123e4567-e89b-12d3-a456-426614174000')
? ????
? ???? This is not a valid matcher type
?
? Note: Valid matchers are: equalTo, regex, type, datetime, date, time, include, number, integer, decimal, boolean, contentType, semver```
anna.nava
2023-12-05 15:31
in case anyone has the same problem with uuid / regex... the only way I could get this to work was:
```"this_amazing_property" to "matching(regex, '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}\$', '123e4567-e89b-12d3-a456-426614174222')"```
vinicius.gabriel
2023-12-05 18:11
Hi Team,
There are some way to use the protofile that is in another repository on `pact:proto`? For example, on my application `http://github.com/x`, I set a path to `http://github.com/y/protos` ?
rholshausen
2023-12-05 22:12
If you are asking if it can fetch a proto file from a remote repo, the answer it no. You will need to fetch it before so it is local
anna.nava
2023-12-06 09:53
actually not really :disappointed: the regex gets ignored :disappointed:
it's like in this way I am only able to get the generator working.... but not really the matcher (it simply gets ignored)
vinicius.gabriel
2023-12-07 14:55
Hey, I'm doing some testes regarding which situations the Provider test fails, and seems that when I remove a field from the Provider version of Protobuff, this do not break the contract testing (even if I put it on Consumer specification).
For example, my consumer proto msg :
```protoMessage := `{
"pact:proto": "` + path + `",
"pact:message-type": "CreateAccountMessage",
"pact:content-type": "application/protobuf",
"pact:protobuf-config": {
"additionalIncludes": [
"/Users/viniciusgabriel/Projects/common/proto"
]
},
"is_migration": "matching(boolean, true)",
"account_type": "matching(type, 'SelfDirected')",
"account_info": {
"account_id": "eachValue(matching(type, 10))",
"message_id": "eachValue(matching(type, 10))",
"reg_form": {
"reg_form": "matching(boolean, false)",
"reg_form_sign": "notEmpty('sign')",
"signed_at": {
"seconds": "matching(integer, 100)",
"nanos": "matching(integer, 100)"
}
}
}
}````
I removed the `is_migration` field from Protobuff used on Provider side:
Test:
```functionMappings := message.Handlers{
"CreateAccountMessage": func([]models.ProviderState) (message.Body, message.Metadata, error) {
msg, _ := proto.Marshal(&proto.CreateAccountMessage{
// IsMigration: true,
AccountType: sfproto.AccountType_SelfDirected,
AccountInfo: &proto.AccountMessage{
RegForm: &sfproto.RegForm{
RegFormSign: "sign",
},
},
})
return msg, message.Metadata{
"contentType": "application/protobuf;message=CreateAccountMessage",
}, nil
},
}```
Protobuff code update (used on Provider side):
```type CreateAccountMessage struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
AccountInfo *AccountMessage `protobuf:"bytes,1,opt,name=account_info,json=accountInfo,proto3" json:"account_info,omitempty"`
AccountType sfproto.ApexAccountType `protobuf:"varint,3,opt,name=account_type,json=accountType,proto3,enum=bridges.AccountType" json:"account_type,omitempty"`
}```
When I run the provider test, is not expected the test break, because a field that one Consumer expect do not exist anymore?
anna.nava
2024-01-17 16:22
that's not entirely true.
I am able to fetch it from a remote repo, with the gradle dependency on that other repo, the plugin creates "$currentWorkingDir/build/extracted-include-protos"
so I can use them from my custom gradle task which I've called contractTest like this:
```@Pact(consumer = "consumerService")
fun createPact(builder: PactBuilder): V4Pact {
val messageFile = loadFileFromJarResource("/events/messages/alert_event.proto")
val currentWorkingDir = System.getProperty("user.dir")
val absolutePath = "$currentWorkingDir/build/extracted-include-protos/contractTest"
return builder
.usingPlugin("protobuf")
.expectsToReceive("an AlertEvent", "core/interaction/message")
.with(
mapOf(
"message.contents" to mapOf(
"pact:proto" to messageFile.path,
"pact:message-type" to "AlertEvent",
"pact:content-type" to "application/protobuf",
"pact:protobuf-config" to mapOf("additionalIncludes" to listOf(absolutePath)),```
rohitkrishnan
2024-01-18 21:44
Hello folks - is there a way to pass in the `protocVersion` via the pact FFI when invoking a test? Right now it defaults to `3.19.1` which doesn't have support for M1 macs, so I need some mechanism to force it to download a later version.
rohitkrishnan
2024-01-18 21:54
Ahh I think I can pass `pact:protocVersion` to the FFI similar to how I pass `additionalIncludes`. I think I answered my own question - leaving this here in case someone else needs it.
rohitkrishnan
2024-01-19 15:37
I found a bug in the way the plugin merges values between the manifest config that's provided by the test, so I put https://github.com/pactflow/pact-protobuf-plugin/pull/42/files to fix this. I'd appreciate if someone could take a look and ideally cut a new version upon convenience - it would unblock some engineers using Apple silicon on their machines. :pray:
stan196
2024-01-25 01:29
Hey folks, I have a repeated field of a message type, ie strucutred. What's the right way to express that this field can have zero elements, or one, or many, but each matching the specific structure?
According to https://github.com/pact-foundation/pact-plugins/blob/main/docs/matching-rule-definition-expressions.md, you can do `eachValue()` but that seems to work only for simple value fields.
I'm currently doing smth like this, but this defines one element, and I'm not sure how to specify counts. @rholshausen you've added `atLeast` and `atMost` but I'm not sure where to put it, should it be just the element in the list? Thanks!
```{
"availability_zones": [
{
"cloud_provider": "notEmpty('AWS')",
"zone_name": "notEmpty('us-east-1a')"
}
]
}```
stan196
2024-01-25 01:33
ah, shows my reading comprehension is lacking:
```The final form is a reference to another key. This is used to setup type matching using an example value, and is normally used for collections. The name of the key must be a string value in single quotes.
For example, to configure a type matcher where each value in a list must match the definition of a person:
{
"pact:match": "eachValue(matching($'person'))",
"person": {
"name": "Fred",
"age": 100
}
}```
stan196
2024-01-25 02:00
so in this final form of matching syntax, is the field name looked up on the same level or across all of the proto?
Like can I lookup the field in the parent or a child structure?
e.g. with a proto similar to this, if I've already defined `config` field in my response, can I just refer to it from under the custom_config?
```message Config {}
message CustomConfig {
Config config = 1;
string whatever = 2;
}
message Response {
Config config = 1;
CustomConfig custom_config = 2;
}```
rholshausen
2024-01-25 03:55
`atLeast` and `atMost` are currently a work in progress, and will be able with the next release of the protobuf plugin
rholshausen
2024-01-25 04:19
> so in this final form of matching syntax, is the field name looked up on the same level or across all of the proto?
It's defined on the same level
stan196
2024-01-26 02:09
Hey folks, I've run into what I suspect may be a bug.
I have a grpc service which responds with a message that has a single boolean field (proto has more fields, but those are not populated by this service).
Whenever the server responds with `false` in that field (i can adjust the state in the test), I'm getting this error from the protobuf plugin:
```1) Verifying a pact between and -
2024-01-25 17:53:27.1661 ? 1.1) has a matching body
2024-01-25 17:53:27.1661 ? $ -> Expected message '' but was missing or empty```
This happens regardless of whether consumer expects true or false. The server seems to respond with a correct message, seeing how changing the state to return `true` makes the plugin work correctly, passing if the consumer requested true and failing if the consumer requested false.
This could be because boolean false is default, so the plugin cannot distinguish empty response from the explicit false, or am I misreading this?
stan196
2024-01-26 02:13
According to the logs, `decode_message` returns empty for such a proto, here:
https://github.com/pactflow/pact-protobuf-plugin/blob/main/src/message_decoder/mod.rs#L258
```2024-01-26T01:32:33.300435Z INFO tokio-runtime-worker request{method=POST uri=http://127.0.0.1:57435/io.pact.plugin.PactPlugin/VerifyInteraction version=HTTP/2.0 headers={"te": "trailers", "content-type": "application/grpc", "authorization": Sensitive, "user-agent": "tonic/0.9.2"}}:decode_message: pact_protobuf_plugin::message_decoder: return=Ok([])```
rholshausen
2024-01-28 22:29
Oh, it's probably because false is the default, Protobuf is dropping the field from the payload. It does the same thing with enums.
christopher.tonog
2024-01-30 05:09
:wave: Hi all, I'm having trouble with getting the grpc plugin to work. I'm on version 0.3.9 and I'm running into this error when running a pact test for a JS consumer.
```2024-01-30T05:05:04.269707Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Request to configure interaction failed: Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1
2024-01-30T05:05:04.270658Z ERROR ThreadId(01) pact_ffi::mock_server: Failed to start mock server - Mock server failed to start: Failed to start gRPC mock server: Pact file does not contain any Protobuf descriptors```
I'm assuming the failure to start the mock server is because of the failed protoc invocation. Has anyone run into this? I added debug logging, but nothing really struck at me. Thanks so much!
stan196
2024-01-30 05:25
I'm sorry, but I seem to have stumbled upon another protobuf related bug :slightly_smiling_face: Please let me know how I can help reproduce or fix
https://github.com/pactflow/pact-protobuf-plugin/issues/44
stan196
2024-01-30 06:58
I have tested this without any pact files present - or is there some caching happening somewhere?
eddjlsh94
2024-01-30 10:19
Hey! Is there a way that I can test protobuf files that import other proto files? I pass one into the protobuf plugin with `"pact:proto", filepath("proto1.proto)` but inside `proto1.proto` I have `import './proto2.proto'` which also has its own dependency `import './proto3.proto'`. Can I pass multiple proto files in to the protobuf plugin so I can handle multiple imports like this?
eddjlsh94
2024-01-30 10:20
Right now it'll fail with a protobuf error if I `pact:proto` the first file and have all of them in the same directory
eddjlsh94
2024-01-30 10:23
A second question, albeit this is more Java related perhaps (although any support would be <3'd!), if I try to return a builder with more than 10 key,values Java complains. If I include the keys for `pact:proto`, `pact:message-type`, and `pact:content-type`, I only have 7 keys remaining for `message.contents` values. If I add an 8th key, Java explodes. Is there a nice way around this? How are people dealing with messages including more than 7 values?
eddjlsh94
2024-01-30 10:24
This explains the problem: https://typeofnan.dev/how-to-create-a-map-of-more-than-10-in-java/
matt.fellows
2024-01-30 10:36
Is this helpful at all?
https://github.com/pactflow/pact-protobuf-plugin/issues/21
rholshausen
2024-01-30 21:44
Oh, wait, it is for a provider state. That might be a regression. I'll take a look
stan196
2024-01-30 22:45
yeah, it happens for the state with params, I think I've seen it once for the state without params too, but not 100% sure, can try to confirm later today
christopher.tonog
2024-01-31 22:09
Thanks! I was able to get it to work by updating the config `"protocVersion"` to recent version. I think because I have a newer Mac and it wasn't able to find the arch for the older protobuf version.
christopher.tonog
2024-01-31 22:10
Well, still running into the same protoc error, but now I know the protoc binary is being run :smile:
stan196
2024-02-01 01:26
Hey folks, for `atLeast(x)` condition here - does this mean that empty lists/maps are ok unless I restrict them using `atLeast` ? Thanks!
stan196
2024-02-01 07:08
It might be a dumb question, but are there any good examples on how to specify matching rules for values in a map? I'm struggling to understand the syntax tbh, like where do you put the matching rules.
Like how can we say the map accepts 0+ fields and both key and value are strings (basically `map labels = 4` in the .proto file).
I've tried:
```"labels": {"pact:match": "eachKey(matching(type, 'some-key')), eachValue(matching(type, 'some-value'))"}```
and got `rpc error: code = FailedPrecondition desc = Failed to match the request message - BodyMismatches({"$.destination": [BodyMismatch { path: "$.destination.labels", expected: None, actual: None, mismatch: "Expected repeated field 'labels' to be empty but received 1 values" }]})`
```"labels": {"pact:match": "atLeast(0), eachKey(matching(type, 'some-key')), eachValue(matching(type, 'some-value'))"},```
and it gave me an error `2024-01-31 22:44:50.5126 ? 2024-02-01T06:44:50.498712Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - min-type is not a valid matching rule type` (or at least that's the error I found in the debug logs, in info it was the usual 'pact file contains no protobuf descriptors')
stan196
2024-02-01 07:28
also, conceptually, is it better to do multiple pacts, one with 0 values, one with 1 value, one with multiple values?
rholshausen
2024-02-02 04:24
Yes, a sperate test for each different combination you want to assert. But I would normally just do 2: empty and not empty
rholshausen
2024-02-02 04:27
Docs can be foud here: https://docs.rs/pact_models/latest/pact_models/matchingrules/expressions/index.html
stan196
2024-02-02 17:27
Thanks! Yeah, I've seen those docs, they're good but miss important point - how to apply those matchers for list and map fields, e.g. where do you put `eachKey` and `eachValue`? I was kinda semi-guessing based on your previous comments :slightly_smiling_face:
rholshausen
2024-02-05 05:44
This is the default behavior with the `eachKey` and `eachValue` functions, you don't need anything else
eddjlsh94
2024-02-06 12:49
When I run a test I get a few errors:
`io.grpc.StatusRuntimeException: UNAVAILABLE: io exception` and `... Connection refused/[0:0:0:0:0:0:1]:63856`
The tests work but the logs are full of these. Any advice on why?
rholshausen
2024-02-06 22:10
O would expect that your gRPC client is trying to use some gRPC service that is unavailable during the test.
stan196
2024-02-07 00:12
What's the right way to specify matching `[]byte` fields in grpc requests/responses? I'm currently converting it to string, smth like:
```"bytes": "matching(type, '" + string(bytes) + "')",```
is this the right way to go?
stan196
2024-02-07 00:14
Seems to work, but there's one issue. Certain bytes get converted in the response, so the response I receive from the mock server is not the one I specify in the `response` field, so I was wondering if could be related to converting bytes back and forth to/from string
stan196
2024-02-07 00:16
Specifically `0xc0` and `0x86`, maybe others too. Both are always replaced with the same set of bytes: `0xef, 0xbf, 0xbd` and then the test fails because you assert to receive `0xc0` and receive `0xef, 0xbf, 0xbd` instead:
```expected: []byte{0xc0}
actual : []byte{0xef, 0xbf, 0xbd}```
I wonder if it has something to do with trimming the string or empty characters or something, but they get replaced even when they're in the middle of the sequence.
idk if there are some special bytes?
stan196
2024-02-07 00:16
I don't particularly care for specific bytes in this particular test, so I can easily work around it, but was curious
rholshausen
2024-02-07 00:17
It would be treated as UTF-8, which means the bytes are converted to UTF code points. They would have to be escaped correctly.
stan196
2024-02-07 02:28
Another issue found, sorry! :sweat_smile:
https://github.com/pactflow/pact-protobuf-plugin/issues/45
@ryoshida has also ran into a similar issue with strings and booleans, but I haven't been able to reproduce on my end yet, will update the ticket if I can.
stan196
2024-02-07 02:29
I think @ryoshida tested this and empty collections are ok unless atLeast is specified
stan196
2024-02-07 02:31
ooh, seems like the first issue here is the same as the one for which I've opened a separate ticket :sweat_smile:
https://pact-foundation.slack.com/archives/CEQBDD5U4/p1707272916302469
eddjlsh94
2024-02-07 09:56
My provider is throwing this error: `body: $ Messages with StartGroup wire type fields are not supported`. Anyone know what that means? I can't find anything obvious online
eddjlsh94
2024-02-07 09:57
It's thrown during the `verifyInteraction()` function, on the first contract test
christopher.tonog
2024-02-07 20:41
Hi all :wave: I'm having some issues getting the protobuf/gRPC plugin to work. I've been specifically getting these error messages when running a js consumer test:
```ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Request to configure interaction failed: Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1
ERROR ThreadId(01) pact_ffi::mock_server: Failed to start mock server - Mock server failed to start: Failed to start gRPC mock server: Pact file does not contain any Protobuf descriptors```
I can imagine a number of things can be the cause of this (and I unfortunately can't post much more in the public channel), but I was wondering if there was a way to get more information on what specifically caused the protoc binary to exit? I set log levels to debug and trace but nothing really stood out to me. Thanks!
rholshausen
2024-02-07 22:11
The gRPC plugin does not support groups. Can you provide the proto file so we can see how you are using them?
matt.fellows
2024-02-07 22:17
You may need to clear our any pact files before a new run.
There should be logs in the `~/.pact/plugins/protobuf-<version>/` folder
rholshausen
2024-02-07 22:19
How have you setup the tests? This is a gRPC exception: `io.grpc.StatusRuntimeException`
stan196
2024-02-07 23:37
search the plugin logs for the word `protoc` there should be a line smth like `invoking protoc` or similar, it will print the exact command used to invoke `protoc` as well as its output.
Also if you run your test with PACT_LOG_LEVEL=debug env variable it will print the same logs in the console, but they're quite verbose so it's usually easier to read them from the plugin's log file
stan196
2024-02-07 23:39
According to the doc here: https://github.com/pactflow/pact-protobuf-plugin?tab=readme-ov-file#verifying-grpc-error-responses-031 you can specify a matcher on metadata fields, but when I ran the test with pact-go, my pact file doesn't contain any matchers for the metadata value. When I read through the pact v4 spec, it doesn't look like you can specify matchers for metadata at all, but I could've missed it, it's a big doc :sweat_smile:
Are matchers supported on metadata fields?
``` "response": [
{
"contents": {
"content": ""
},
"metadata": {
"contentType": "application/protobuf;message=Feature",
"grpc-message": "no feature was found at latitude:-1 longitude:-1",
"grpc-status": "NOT_FOUND"
}
}
],```
rholshausen
2024-02-07 23:52
rholshausen
2024-02-07 23:53
Hmm, it shouldn't be doing anything, but passing the JSON strait on through
stan196
2024-02-07 23:55
if it's not too much to ask and if you have already set up the pact-plugins repo on your machine, could you please run the consumer-jvm example and share the resulting pact json file so I can compare with what pact-go generated? Could be user error on my side.
If it'll take you more than 2 minutes, it's ok tho, I can take a look too. Thanks!
rholshausen
2024-02-08 00:01
```{
"consumer": {
"name": "grpc-consumer-jvm"
},
"interactions": [
{
"comments": {
"testname": "io.pact.example.grpc.consumer.PactConsumerTest.testMetadata(MockServer)"
},
"description": "validate token request",
"interactionMarkup": {
"markup": "```protobuf\nmessage ValidateTokenRequest {\n}\n```\n\n```protobuf\nmessage ValidateTokenResult {\n bool ok = 1;\n}\n```\n",
"markupType": "COMMON_MARK"
},
"key": "124feec5",
"pending": false,
"pluginConfiguration": {
"protobuf": {
"descriptorKey": "21e4eba65e4d1100ce2fb63b5e6b2c3e",
"service": "Test/ValidateToken"
}
},
"request": {
"contents": {
"content": ""
},
"matchingRules": {
"metadata": {
"Auth": {
"combine": "AND",
"matchers": [
{
"match": "regex",
"regex": "[A-Z]{3}\\d+"
}
]
}
}
},
"metadata": {
"Auth": "AST00004",
"contentType": "application/protobuf;message=ValidateTokenRequest"
}
},
"response": [
{
"contents": {
"content": "CAE=",
"contentType": "application/protobuf; message=ValidateTokenResult",
"contentTypeHint": "BINARY",
"encoded": "base64"
},
"matchingRules": {
"body": {
"$.ok": {
"combine": "AND",
"matchers": [
{
"match": "boolean"
}
]
}
},
"metadata": {
"code": {
"combine": "AND",
"matchers": [
{
"match": "integer"
}
]
}
}
},
"metadata": {
"code": "100",
"contentType": "application/protobuf;message=ValidateTokenResult"
}
}
],
"transport": "grpc",
"type": "Synchronous/Messages"
}
],
"metadata": {
"pact-jvm": {
"version": "4.4.6"
},
"pactSpecification": {
"version": "4.0"
},
"plugins": [
{
"configuration": {
"21e4eba65e4d1100ce2fb63b5e6b2c3e": {
"protoDescriptors": "Ct8BCg5tZXRhZGF0YS5wcm90bxIMbWV0YWRhdGF0ZXN0IhYKFFZhbGlkYXRlVG9rZW5SZXF1ZXN0IiUKE1ZhbGlkYXRlVG9rZW5SZXN1bHQSDgoCb2sYASABKAhSAm9rMmAKBFRlc3QSWAoNVmFsaWRhdGVUb2tlbhIiLm1ldGFkYXRhdGVzdC5WYWxpZGF0ZVRva2VuUmVxdWVzdBohLm1ldGFkYXRhdGVzdC5WYWxpZGF0ZVRva2VuUmVzdWx0IgBCFloUaW8ucGFjdC9tZXRhZGF0YXRlc3RiBnByb3RvMw==",
"protoFile": "syntax = \"proto3\";\n\npackage metadatatest;\noption go_package = \"io.pact/metadatatest\";\n\nmessage ValidateTokenRequest {\n}\n\nmessage ValidateTokenResult {\n bool ok = 1;\n}\n\nservice Test {\n // This call requires the Auth header to be correctly set\n rpc ValidateToken(ValidateTokenRequest) returns (ValidateTokenResult) {}\n}\n"
}
},
"name": "protobuf",
"version": "0.3.12"
}
]
},
"provider": {
"name": "validate-token-provider"
}
}```
christopher.tonog
2024-02-08 00:17
thanks for the replies, for some reason nothing is getting written to the `plugin.log.2024-02-07` file (likely a separate issue), nor do I see the `protoc` binary getting invoked. Seems to fail before then :thinking2:
This is the debug output that gets written out along with that failed invocation:
```2024-02-07T15:31:39.104258Z DEBUG ThreadId(01) pact_plugin_driver::content: Got response: ConfigureInteractionResponse { error: "Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1", interaction: [], plugin_configuration: None }
2024-02-07T15:31:39.104275Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_write: send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) }
2024-02-07T15:31:39.104286Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::proto::connection: Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library)
2024-02-07T15:31:39.104369Z ERROR ThreadId(01) pact_ffi::plugins: Failed to call out to plugin - Request to configure interaction failed: Failed to process protobuf: Failed to invoke protoc binary: exit code exit status: 1
2024-02-07T15:31:39.104525Z DEBUG ThreadId(01) pact_ffi::mock_server: pact_ffi::mock_server::pactffi_create_mock_server_for_transport FFI function invoked```
dumb question, but the same logs that get sent to the terminal output would be the same logs that would be sent to the that plugin log file, correct?
stan196
2024-02-08 00:33
thank you, I'll look into what's different in how pact-go processes it (or if I'm doing something wrong maybe)
stan196
2024-02-08 00:35
anything interesting in the logs before that?
Also, do you have a single protobuf plugin under that folder or multiple versions? I had a case on my machine where I had multiple protobuf plugins installed and was looking for logs in a wrong one
christopher.tonog
2024-02-08 00:55
Not anything in particular, if there was, I wasn't able to decipher them :sweat_smile:
```24-02-07T15:31:39.050631Z DEBUG ThreadId(01) pact_plugin_driver::plugin_models: IP6 connection failed, will try IP4 address - transport error
2024-02-07T15:31:39.050736Z DEBUG ThreadId(01) hyper::client::connect::http: connecting to 127.0.0.1:53033
2024-02-07T15:31:39.051238Z DEBUG ThreadId(01) hyper::client::connect::http: connected to 127.0.0.1:53033
2024-02-07T15:31:39.051316Z DEBUG ThreadId(01) h2::client: binding client connection
2024-02-07T15:31:39.051363Z DEBUG ThreadId(01) h2::client: client connection bound
2024-02-07T15:31:39.051380Z DEBUG ThreadId(01) h2::codec::framed_write: send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384 }
2024-02-07T15:31:39.051582Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_write: send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
2024-02-07T15:31:39.051718Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_read: received frame=Settings { flags: (0x0), initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16777216 }
2024-02-07T15:31:39.051769Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_write: send frame=Settings { flags: (0x1: ACK) }
2024-02-07T15:31:39.051792Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_read: received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 }
2024-02-07T15:31:39.051823Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_read: received frame=Settings { flags: (0x1: ACK) }
2024-02-07T15:31:39.051837Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::proto::settings: received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384 }
2024-02-07T15:31:39.051936Z DEBUG ffi-setup_contents tower::buffer::worker: service.ready=true processing request
2024-02-07T15:31:39.052141Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_write: send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
2024-02-07T15:31:39.052224Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_write: send frame=Data { stream_id: StreamId(1) }
2024-02-07T15:31:39.052246Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_write: send frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) }
2024-02-07T15:31:39.104147Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_read: received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
2024-02-07T15:31:39.104179Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_read: received frame=Data { stream_id: StreamId(1) }
2024-02-07T15:31:39.104195Z DEBUG ffi-setup_contents Connection{peer=Client}: h2::codec::framed_read: received frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }```
I do have multiple versions. But the others didn't have log files (this one was created, it was just empty). Also logs seem to show I'm using the correct one:
```2024-02-07T15:31:38.536558Z DEBUG ThreadId(01) pact_plugin_driver::plugin_manager: Starting plugin using "/Users//.pact/plugins/protobuf-0.3.11/pact-protobuf-plugin"```
stan196
2024-02-08 01:03
yeah, I don't see anything suspicious here. Do you have `protoc` binary under ~/.pact/plugins/protobuf-/bin ? I've had issues in the past where it was trying to download a version of protoc that's not compatible with arm macs.
Alternatively it will try to invoke the protoc in your path
yousafn
2024-02-08 10:45
Can you check which version of protoc `protoc --version` you have running? it may be worth upgrading to the latest, assuming you installed with brew `brew upgrade protobuf`
If you try and invoke the pact-protobuf-plugin executable directly, does that work
`/Users/<myuser>/.pact/plugins/protobuf-0.3.11/pact-protobuf-plugin`
> dumb question, but the same logs that get sent to the terminal output would be the same logs that would be sent to the that plugin log file, correct?
Not a dumb question if you don?t know the answer! I had assumed that, but no, currently the plugins (executables in the plugin folder) , don?t send their logs to the plugin driver (in your pact client library), so the logs aren?t joined up.
I tend to tail the logs inside the plugin folder, if I am diagnosing a test, so I can see the std output from the main test (which tells me its communicating with the plugin) and the plugins output from its own log file
christopher.tonog
2024-02-08 15:06
I updated protoc by setting `"protocVersion":"25.1"` in the `pact-plugin.json` config
```bin % pwd
/Users/<user>/.pact/plugins/protobuf-0.3.11/protoc/bin
bin % ./protoc --version
libprotoc 25.1```
Plugin appears to work (I kept it running for a bit)
```protobuf-0.3.11 % ./pact-protobuf-plugin
{"port":59464, "serverKey":"<maybe secret? redacted just in case>"}
2024-02-08T14:10:35.633062Z INFO tokio-runtime-worker pact_protobuf_plugin: No activity for more than 600 seconds, sending shutdown signal
2024-02-08T14:10:35.633457Z INFO main pact_protobuf_plugin: Received shutdown signal, shutting plugin down```
I also have a m2 mac, not sure if that changes anything.
yousafn
2024-02-08 16:08
good idea on the caution, that server key is just a uuid which is sent to the pact plugin driver, so they can somewhat securely communicate without anything else listening in on your machine
that looks promising on the plugin side (it executes) and we are on the same protoc version, I?m running a mac m1, shouldn?t make too much of a diff, I?ve got a machine with sip enabled, and one without.
Do you see anything in your log files about being unable to find a protobuf descriptor, something like that, it usually means you?ve provided it a file but it can?t find it.
You can share logs in a private PM if you prefer.
I popped a JS example in the plugins repo, that you might like to try and run?
https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator/js
christopher.tonog
2024-02-08 16:47
Thanks @yousafn for the help, I was finally able to get past this issue with help internally. Turns out there was a secondary import in the provider .proto file. I didn't see the other proto file in the same directory but thought it might've been implicitly imported some other way so that I didn't need to add the extra `additionalIncludes` proto config (I don't have much context in the provider code base so I missed that). :face_with_spiral_eyes: Adding the extra proto file via `additionalIncludes`fixed the issue! Thanks for helping walk through troubleshooting, I learned a couple things along the way. :bow:
stan196
2024-02-09 01:14
Reproduced the issue inside pact-protobuf plugin itself:
https://github.com/pactflow/pact-protobuf-plugin/pull/46
I'll experiment some more with it and try to walk through this with a debugger. Would appreciate any debugging suggestions
stan196
2024-02-09 01:50
I've found this spot in the pact-reference that seems to populate metadata matching rules, but not sure yet where it's called, there are quite a lot of various message types between sync/async and v3/v4 and what not:
https://github.com/pact-foundation/pact-reference/blob/master/rust/pact_consumer/src/builders/sync_message_builder.rs#L463
rholshausen
2024-02-09 01:53
I normally use the trace logs or the `dbg!` macro for debugging. If you use the JetBrains IDEs (CLion or Rust Rover), then the debugger in the IDE will work. I've tested Rust Rover, it works ok.
rholshausen
2024-02-09 03:42
Looks like it might be a pact_consumer issue. I can see the values passed back from the plugin in the logs:
```metadata_rules: {"key": MatchingRules { rule: [MatchingRule { r#type: "type", values: None }] }}, metadata_generators: {} })
metadata_rules: {"grpc-message": MatchingRules { rule: [MatchingRule { r#type: "type", values: None }] }}, metadata_generators: {} }]```
rholshausen
2024-02-09 03:57
The issue is here: https://github.com/pact-foundation/pact-reference/blob/master/rust/pact_consumer/src/builders/sync_message_builder.rs#L133
But that is for Rust tests, there must be another place with the same problem if it is happening in your Go tests.
stan196
2024-02-09 04:07
> Looks like it might be a pact_consumer issue. I can see the values passed back from the plugin in the logs:
yep, can confirm the same, plugin seems to respond with the correct interaction response with go tests too. Go is just passing values to/from ffi lib and is doing very minimal processing on its own it seems
stan196
2024-02-09 04:09
Let me try a fixed pact consumer locally. I think I can just refer to a local path of the pact_consumer in my cargo.toml in the protobuf-plugin and that should work, right?
rholshausen
2024-02-09 04:26
Are you going to try fix the pact_consumer one? Or do you need me to do it.
stan196
2024-02-09 04:36
I can try to fix the pact_consumer in an hour or two, will send a PR. Can you pls point me to the pact ffi issue (so I can learn more)?
rholshausen
2024-02-09 04:37
stan196
2024-02-09 08:21
https://github.com/pact-foundation/pact-reference/pull/384
pls let me know if you want commits squashed beforehand, or if we can squash them during merge
stan196
2024-02-09 08:26
I'm not sure if there are any linter/formatter/code-quality tasks I should run to make sure PR is ok? the docs mention cargo build and that works; also tests that I wrote work too
yousafn
2024-02-09 13:49
Thanks for the change @stan196, really appreciate it, and it?s nice seeing the journey too!
> learning how to write rust using copy paste and copilot :smile:
:sweat_smile: I feel you man! It has been a bit of help whilst polyglotting.
> pls let me know if you want commits squashed beforehand, or if we can squash them during merge
I think it might be better squashing prior as I think you lose some attribution to the change if the merger squashes. I might not be quite correct but I taught to try and tell others providing contributions to squash first.
> I?m not sure if there are any linter/formatter/code-quality tasks I should run to make sure PR is ok? the docs mention cargo build and that works; also tests that I wrote work too
Should get picked up in the existing test suite and with the additions you?ve included. I don?t think there is any standardised formatting rules existing in the repo that you can apply, and I think modern day formatting will change the files alot, which I don?t believe Ron is in favour of, as it makes the commit history messy when looking at why a line was changed.
BTW - How are you finding the plugin stuff in general? Are you still at a POC stage, or have you got it reasonably well embedded at the moment?
stan196
2024-02-09 19:20
> I think it might be better squashing prior as I think you lose some attribution to the change if the merger squashes. I might not be quite correct but I taught to try and tell others providing contributions to squash first.
ok, I'll squash them on my side :slightly_smiling_face:
> BTW - How are you finding the plugin stuff in general? Are you still at a POC stage, or have you got it reasonably well embedded at the moment?
we've started onboarding more projects, but it's still an early phase. It's been mostly a smooth journey so far, couple of bugs here and there but overall easy to onboard! :+1: I can share some findings and places we had to work around in a different thread after we finish onboarding more projects
stan196
2024-02-09 23:10
I've also updated the PR in protobuf-plugin to actually verify the metadata rules programmatically vs just looking at the pact file