yousafn
2020-03-04 19:40
Link to the initial discussion
https://pact-foundation.slack.com/archives/C5F4KFKR8/p1583222077124100
yousafn
2020-03-06 00:12
Thought I would have a little play at invoking pact via `cy.task` based on the examples from @sergiy & @enrique?s youtube video.
Repo here https://github.com/YOU54F/cypress-pact
CircleCI run here https://circleci.com/gh/YOU54F/cypress-pact/4
It is a bit contrived, as I am using `cy.request` to make the request that is hitting the pact mock server, rather than it being the request from the webpage under test.
There are four files of note
`contains tasks to call node processes outside of the browser`
https://github.com/YOU54F/cypress-pact/blob/master/cypress/plugins/index.js
`sets global commands which will call cy.tasks with options passed in for the pact mock server and addInteracxtion`
https://github.com/YOU54F/cypress-pact/blob/master/cypress/support/commands.ts
`calls verify afterEach and finalise after all`
https://github.com/YOU54F/cypress-pact/blob/master/cypress/support/index.ts
`spec file`
https://github.com/YOU54F/cypress-pact/blob/master/cypress/integration/test/pact.ts
`generated pact from circleci run`
https://4-245270571-gh.circle-artifacts.com/0/pacts/test-app-reqres.json
matt.fellows
2020-03-06 00:47
Awesome stuff @yousafn, so if my calculations were correct, this got you into about 1am your time - :clap:
matt.fellows
2020-03-06 01:45
OK so I had a little bit of time today to look at this Yousef - this is - pretty cool!
matt.fellows
2020-03-06 01:45
So I?m new to Cypress, but found the experience very good (and envious - we have so much work on the docs/tutorial/dev experience front!)
matt.fellows
2020-03-06 01:46
I?ve had a bit of a read through the API and can see there are some hooks for network and server requests - but only a cursory glance at this stage
matt.fellows
2020-03-06 01:47
I can see at least 2 uses cases for Pact here:
1. To actually drive out the Pact interactions for some of the ?integration? scenarios (not all, because I imagine there are a lot of scenarios that will be boundary testing/duplicates of existing interactions - although I suppose if we built in the smarts it could just collapse any duplicates?)
2. To use as a mock/stub server (i.e. canned responses, not for pact generation) to drive UI behaviour that relies on a backend
matt.fellows
2020-03-06 01:49
> It is a bit contrived, as I am using `cy.request` to make the request that is hitting the pact mock server, rather than it being the request from the webpage under test.
So yes I saw this - why isn?t the click of ?Get Comments? triggering the request to the actual Pact mock server? (does it trigger it to the real site, is that the point?)
matt.fellows
2020-03-06 01:50
I can also see that when Pact test fails, the display in the UI isn?t very useful - there is work to do on that front perhaps to send back a JSON format that Cypress knows about, to render it nicely in their UI
matt.fellows
2020-03-06 01:50
BUT, that looks very promising - thanks so much for putting that together
yousafn
2020-03-06 09:06
Morning - yeah it was a bit late :joy: wife is working nights so I had a couple of spare hours after dropping her off at work
RE: point 2 I?d like to have Cypress hit the pack mock server when a request is made that matches the route. I think that is the preferable option and I believe what the guys have done in their Cypress-pact plugin. Hopefully we can use http://cy.route to intercept the request and redirect it to the pact mock server and return the pact mock servers response back to the front end
For the error side. I have seen cypress plugins send messages to the ui so that is possible. I?ll fish out some examples.
Really neat idea though in general so has been fun to experiment
matt.fellows
2020-03-06 09:48
Ok cool yeah. I think I'm board with how it might all hang together and some changes in Pact JS etc that could support it. I can see the use in it also. I'm still not fully convinced it's the best idea to have the contracts generated this way - or rather, I think Pact in its current form is not ideally suited. It's too likely that duplicate interactions will be created and an explosion of interactions.
matt.fellows
2020-03-06 09:49
Maybe it's not a bad thing. Ideally, the pact framework (or similar tool) could just detect the interactions intelligently (including provider states etc)
abhi358
2020-03-09 01:12
wow !! some interesting points here. Gr8 job @yousafn and @matt.fellows. keep up the good work :+1: Really excited about the developments in this area
matt.fellows
2020-03-09 03:57
Thanks Abhi - any feedback on the feature request would be much appreciated, even if it's just the use case you plan to use it in. It all helps
abhi358
2020-03-10 03:30
My use case is :
In my organization, we are trying to test our front-end in complete isolation from the real services.
The way we are trying to achieve is we are mocking the responses from the backend using Swagger and then using it as fixture data to provide to the Cypress mocks and stubs which then hands over the response data to the front end.
I was thinking if there would be a way to use Pact files from PactBroker to use instead of swagger mocks then it would be way easier and consistent
evanrosal
2020-05-08 04:34
Any follow through on this? I think pact would be good to go hand-in-hand with our cypress e2e tests. We are trying to centralise our test framework around cypress. We are already using the jest plugin for FE unit tests and this would be a great addition to complete our test pyramid.
@abhi358 mentioned a really good use case.
bethskurrie
2020-05-08 04:35
We're super keen on making something work with cypress - we've just got some other things that are currently in play that we need to finish before we pick up other things (eg. getting pact-js moved to the rust implementation, messaging pact in the various languages, pact-jvm using the new verifications api so it can do pending/wip pacts... there's a lot on the go!)
evanrosal
2020-05-08 04:36
Thanks for the swift response @bethskurrie and totally understand the priorities. Thanks for being transparent on where you guys are on this.
matt.fellows
2020-05-08 05:04
So, I don?t think that any changes to the core Pact libraries themselves would need changing to integrate with Cypress, so it?s a really good candidate for new contributors - anybody keen?
matt.fellows
2020-05-08 05:04
There may be some changes / enhancements to support it, but I don?t see it requiring a major overhaul (unlike things like protobufs/gRPC, for example)
yousafn
2020-05-08 21:29
so at work we have api?s which have defined swagger schemas. our consumers implement pact contracts and publish to the broker. Before they are published to the broker, the pacts are validated against the swagger schema to avoid drift. (we can work from feature branch swaggers for expected changes)
we then use pact stub server to load in pact file, which has a set of matchers in, and our front-end uses these endpoints to provide e2e test environments locally and to mock our dependencies in certain test envs.
For some things where we need more dynamic and rich mocks, we use wiremock. You can run a pact verifier against the generated mocks to avoid drift.
yousafn
2020-05-08 21:31
It would be interesting in hearing about particular use cases, and any particular issues in achieving them but sometimes they may be able to be achieved by other means :slightly_smiling_face:
bethskurrie
2020-05-08 23:39
> You can run a pact verifier against the generated mocks to avoid drift.
How do you do this @yousafn ?
yousafn
2020-05-09 12:22
for a couple of our services, we use wiremock to stand in as the provider as we needed looser matching. I use a pact provider verification task to hit the wiremock provider, otherwise there was nothing in place to ensure that the non pact-based mock services, we?re still adhering to the expectations used in the pact tests that sat in the clients codebase.
yousafn
2020-05-09 12:23
there are wiremock plugins available by atlassian although I?ve not really found the time to look into them but they looked promising when I last looked.
bethskurrie
2020-05-14 22:03
You're verifying the pact against a wire mock instance of the provider?! Meta!!
bethskurrie
2020-05-14 22:04
Funny, in the CI/CD workshop I have a slide where I mentioned that you could verify a pact against a completely bogus implementation of the provider in theory, but I hadn't thought of doing it in practice @yousafn :laughing:
yousafn
2020-07-16 13:26
DId anyone keep a copy of the original thread with the initial ideas / thoughts
matt.fellows
2020-07-16 13:42
This workspace is growing so fast the even horizon is only like 3 months now!
matt.fellows
2020-07-28 04:49
I have basically created a working plugin for this in a demo app https://github.com/pactflow/example-consumer-cypress
matt.fellows
2020-07-28 04:50
(next step is to extract it into its own plugin and publish on the Cypress website)
matt.fellows
2020-07-28 04:52
1. First run goes to the `/` page which calls out to an Product listing API `/products`
2. Second run extends the test to click through to a product page which needs to hit `/products/:id`. The test fails, because the interaction hasn?t been configured in Cypress or Pact
3. Add the `product/:id` interaction in and voila!
matt.fellows
2020-07-28 11:45
Draft blog on the matter: https://pactflow.io/p/5dd80538-c983-4ee1-b99e-81c1a70f6141/
matt.fellows
2020-07-28 11:45
Again, props to @yousafn for the inspiration and getting the first cut of this up. It may have been a little too overwhelming without that spike
mcruzdrake
2020-07-28 15:24
@matt.fellows Interesting! Just finished reading the blog post and I agree that your UI tests should use data that is reliable and using the contracts itself is a great data source. I'm excited to see how this progresses :slightly_smiling_face:
mcruzdrake
2020-07-28 15:25
Will it be alright to share the blog post or shall I wait when it's been published as it's still a draft?
yousafn
2020-07-28 15:49
Article looks great @matt.fellows, will check out the repo one eve this week. Had a quick scan over the code and it looks good :)
phil.endsley
2020-07-29 00:12
Great work @matt.fellows and @yousafn! This has come up a few times in our organization as teams are picking up pacts. Looking forward to being early adopters. Very excited for this integration
matt.fellows
2020-07-29 00:56
@mcruzdrake I?ve just published it (with a stronger word of caution because this is still rather controversial and experimental): https://pactflow.io/blog/cypress-pact-front-end-testing-with-confidence/
matt.fellows
2020-07-29 00:56
At this stage, before any official plugin is created, I?d like to seek more feedback from teams wanting to do this so that I can genuinely understand the use cases, the potential challenges etc.
matt.fellows
2020-07-29 00:57
If you?re in this camp and are keen for a chat - DM me and we?ll setup a time
bethskurrie
2020-07-29 00:57
It's worth mentioning that Matt and I have just had a big discussion about stopping people abusing this feature, and creating pacts that are bloated and drive their provider teams nuts with slight variations that don't provide meaningful extra coverage - so please don't prove my fears right!
bethskurrie
2020-07-29 00:58
I have a half finished blog post entitled "A disastrous UI testing experience using Pact" which is basically all about not testing the UI using pact. Keep in mind, you're trying to test the _integration point_ through the UI, not the UI itself.
matt.fellows
2020-07-29 00:59
I think once we get a bit more feedback and we?re convinced it can work without making Pact that tool everybody hates, we?ll be able to add proper guides on doing it well
matt.fellows
2020-07-29 01:01
> I have a half finished blog post entitled ?A disastrous UI testing experience using Pact? which is basically all about not testing the UI using pact. Keep in mind, you?re trying to test the _integration point_ through the UI, not the UI itself.
Expanding on that, because I can see 2 clear use cases for Pact here:
1. Using generated contracts as stubs explicitly for _testing the UI_
2. Using e2e UI testing tools like Cypress to _generate_ contracts
The latter (2) is where we are concerned about abuse
phil.endsley
2020-07-29 01:03
I see where you all are coming from now. Speaking for myself, we're only interested in the first scenario
bethskurrie
2020-07-29 01:05
It might be a more UI centric way to say "test one screen/action" at a time, don't chain them.
matt.fellows
2020-07-29 01:06
Cypress is pretty great, so there is no reason why you couldn?t stub out any network requests required to render a screen, and only apply a Pact test for the single http interaction you care about - that may be for the whole screen, or a single element
mcruzdrake
2020-07-29 08:40
Same here, I am interested to apply use case 1 in one of the teams here as they have a lot of data issue and using the generated contracts might help them :slightly_smiling_face:
vladyslav
2020-08-18 15:10
HI All!
Just went thought https://pactflow.io/blog/cypress-pact-front-end-testing-with-confidence/ and wondering if you guys have an example of React-Apollo Graphql integration. Thank you in advance!
matt.fellows
2020-08-19 00:15
I wrote a blog on these a couple of years ago http://blog.pact.io/2018/07/24/contract-testing-a-graphql-api/
heytaco
2020-08-25 03:57
Hi there! My name is HeyTaco!, and you can use me to give people tacos to show your appreciation. My tacos will spread joy through Slack!
evanrosal
2020-09-24 13:15
Hey Matt, sorry new to Pact itself (more on Cypress) but can the ?make mocked? be run locally on my windows machine? I tried running it and it gives me error: not found: make-.env
Even with the error above, I tried to run cypress and it throws an error: cy.task(?stopMockServer?) failed: cannot ready property ?writePact? of undefined.
I tried stackoverflowing but it wasnt helping so just thought popping up the question here lean me to the right direction.
matt.fellows
2020-09-24 13:35
It should. It may be the windows path length issues. Worth googling on pact-node issues
matt.fellows
2021-02-27 11:39
Yep! So, the cypress feature has been in test at Pactflow (as in, we use it) for quite some months now. We?ve had a conversation with the VP eng at Cypress so have a good idea of how to do it better with the newer cy2.route
matt.fellows
2021-02-27 11:39
Also, we just released our bi-directional contracts feature at Pactflow - which makes a Cypress integration much nicer. Stay tuned!
ebanster
2021-02-28 06:35
Thanks @matt.fellows@matt.fellows@matt.fellows Just a note that cy.route2 has now been renamed to cy.intercept
giuseppe.salvatore
2021-04-29 11:43
I have just read this https://pactflow.io/blog/cypress-pact-front-end-testing-with-confidence/
yousafn
2021-04-29 15:02
just found we can view all the historic posts from this channel via this url - which is pretty cool
https://docs.pact.io/slack/cypress.html
yousafn
2021-04-29 15:06
Here is a link to @sergiy?s original post and video with the cypress-pact idea https://docs.pact.io/slack/general.html#1583231060.124900
Props for the work :ok_hand:
yousafn
2021-04-29 15:08
> sergiy https://docs.pact.io/slack/general.html#1583231060.124900
> Hi there :wave:, we?ve implemented *pact-cypress* integration lib at our company and using it on a regular basis in our test suites. We?ve been talking about it and more here - https://www.youtube.com/watch?v=cs5ebVzG_Q4 at one of the testing meetups organized by SauceLabs. I had a short chat with @matt.fellows about it, and we were thinking about open sourcing our tooling, seems like folks here could be interested?
yousafn
2021-04-29 15:09
just noticed you work at contentful @sergiy! So does my old colleague and good friend!
sergiy
2021-04-29 15:17
Hey @yousafn! Thanks for the recognition :slightly_smiling_face:
Embarrassing enough, we still didn?t managed to release our promised `cypress-pact` implementation public, we?ve been working with @enrique on and which is running actively for more than a year on our delivery pipelines. Were busy with lots of other interesting projects? Maybe I should ask James to carry it on :zany_face:
?.
Also we?ve promised a case study to @matt.fellows, sorry for that too :facepalm:
maybe we should restore our conversation again..
yousafn
2021-04-29 16:05
Ahhh man, that?s just the way it goes, there is always many spinning plates! I?m sure JB is busy with many other cool projects. Looking forward to catching up in the future
marcin.baranowski953
2021-07-20 12:02
Hello community!
I already have working project with few working cypress tests.
Now I'm trying to add contract tests using Cypress + Pact
In developer console I can see that app is calling `/api/v1/document-service` , but I get:
`Pact verification failed - expected interactions did not match actual.`
Part of logs:
```W, [2021-07-20T12:49:37.157389 #34805] WARN -- : Verifying - actual interactions do not match expected interactions.
Missing requests:
POST /api/v1/document-service
W, [2021-07-20T12:49:37.157489 #34805] WARN -- : Missing requests:
POST /api/v1/document-service```
I'm using:
``` cypress: 7.5.0
@pact-foundation/pact: 9.16.0```
Steps I've done:
1. Added cypress plugin (https://github.com/pactflow/example-consumer-cypress/blob/master/cypress/plugins/cypress-pact.js)
2. Added commands (https://github.com/pactflow/example-consumer-cypress/blob/master/cypress/support/commands.js)
3. Added config to cypress.json (https://github.com/pactflow/example-consumer-cypress/blob/master/cypress.json) - not sure what to put to baseUrl, if I don't want interactions with real server.
4. Added test (post below)
Tried both using deprecated cy.server()/cy.route() and new cy.intercept(), but still verification failures
Can You help me with this issue?
matt.fellows
2021-07-21 01:54
Does the example project above you referenced work? If so, then we know it?s something different about your project
matt.fellows
2021-07-21 01:54
if it doesn?t, it may need to be modified to support a later version of Cypress
marcin.baranowski953
2021-07-21 15:50
I've tried to run https://github.com/pactflow/example-consumer-cypress. It works like a charm both for `"cypress": "^4.11.0"` and `"cypress": "7.5.0"`
I guess the problem is with my `cypress.json` config and pact broker url. Will check it tommorow.
marcin.baranowski953
2021-07-22 07:32
1. During cypress-pact test - do I have to have real provider's server? I wanted to mock all the responses in tests.
2. At first, I don't want any pact broker - just generated stubs to check them on my local dev machine with provider - how to set it up?
matt.fellows
2021-07-22 07:36
If the cypress pact integration works, it should produce a pact file on your hard drive
marcin.baranowski953
2021-07-22 07:49
It's not working yet.
(1) by mocking I mean using `cy.intercept()` . I've tried it on example project, but as soon as I use - it produces the same error I have in my project:
```Pact verification failed!
Actual interactions do not match expected interactions for mock MockService.
Missing requests:
GET /products```
(2) I will use pact-broker as soon as I will make sure pact is a tool that works for us
matt.fellows
2021-07-22 08:11
You might need to add some more debugging to the commands etc to see what's happening
marcin.baranowski953
2021-07-27 16:42
Sorry for not responding. I've been debuging quite long with my team mate, but we couldn't find good reason, why such thing happens.
Seeing this answer from Beth: https://stackoverflow.com/a/68492932/11463373
we decided to follow her suggestion and do separate, non-cypress tests.
Thanks for your support :slightly_smiling_face:
charles.kuo
2022-02-11 20:30
@matt.fellows
I have API test written in cypress (no UI interaction, only API call by `cy.request()` against deployed enviornment)
and would like to convert to cypress + pact test
It seems like that I should still be able to use Pact mock service to generate pacts?
charles.kuo
2022-02-11 20:31
(also API test is much closer to unit test run time relatively to front-end Cypress test)
charles.kuo
2022-02-11 21:49
I think I might have get it wrong
During test development stage:
1. For API test, I would need to run cypress test suite against a live API service to get Pact generated
2. The contract will then be published to Pactflow
3. Provider build pipeline will use Pact::verify to check for inconsistency?
charles.kuo
2022-02-12 04:35
I have setup the repo and it ran as expected
https://github.com/pactflow/example-consumer-cypress
I would like to use this project as sandbox to learn how or see if I can create pact against an API endpoint. However, instead of triggering GET request to `/products/10` from the app, I would like to use something like ...
```cy.request({
url: "http://localhost:3000/products/10",
auth: {
bearer: "Bearer 2019-01-14T11:34:18.045Z",
},
headers: {
"content-type": "application/json, text/plain, */*",
},
}).then((resp) => {
cy.log(resp);
// basic assertion here?
});```
The request got 200 status but the response body looks like html template, instead of the JSON I was looking for
```const expectedProduct = {
id: "10",
type: "CREDIT_CARD",
name: "28 Degrees",
};```
My end goal is to use the generated Pact to build contract test on the producer side (a Ruby backend) later.
I was wondering how should I approach this?
charles.kuo
2022-02-12 05:15
However, `cy.wait("@product")` did generate the response I was looking for with from the same URL http://localhost:3000/products/10
I am sure this is due to my misunderstanding or knowledge gap
charles.kuo
2022-02-12 05:25
The html response I got from my cy.request() response is
from public/index.html
charles.kuo
2022-02-12 06:47
I think I know the actual API response data would come from
https://test.pactflow.io/pacts/provider/pactflow-example-provider/consumer/example-cypress-consumer/latest/stub/product/10
but I don't know how http://localhost:3000/products/10 can also return the expected JSON response when it's trigger by app
charles.kuo
2022-02-12 09:43
Or maybe this should just be a consumer w/ http client like in your video demo, instead of using cy.request() to send request
charles.kuo
2022-02-13 07:33
Thanks - I think my usage might be a bit of edge case - but I for sure appreciate the help!
jwang
2022-08-23 20:05
Is there more documentation out there beyond https://github.com/pactflow/pact-cypress-adapter for using contracts with cypress?
matt.fellows
2022-08-24 03:22
It?s designed for use with our BDCT feature. See https://docs.pactflow.io/docs/bi-directional-contract-testing and https://docs.pactflow.io/docs/workshops/bi-directional-contract-testing.
It?s _not_ designed for use with standard Pact verification.
Do you have any specific questions on it?
yousafn
2022-08-24 12:44
Matt wrote a blog post about it a while back
https://pactflow.io/blog/cypress-pact-front-end-testing-with-confidence/
and we have an e2e demo that you can see in action and run
https://docs.pactflow.io/docs/examples/bi-directional/consumer/cypress/
This is an alternative to our initial proposal, which when designed, bi-directional contract testing wasn't a thing, in Pactflow. This technique was used by Contentful, as described https://www.youtube.com/watch?v=cs5ebVzG_Q4&ab_channel=Contentful
https://docs.pactflow.io/docs/examples/cypress/
HTH
jwang
2022-09-19 23:16
Is there a functional difference between using the cy.intercept & cy.usePactWait() vs cy.usePactRequest?
yousafn
2022-09-20 11:44
`cy.usePactWait` listens to aliased `cy.intercept` commands https://github.com/pactflow/pact-cypress-adapter#cyusepactwaitalias--alias
`cy.usePactRequest` will use Pact to issue the request and `cy.usePactGet` to serialise into a pact file
The former will be used when you app actually issues the request, the second is issuing the request itself, which may not be the same as your app issues (not sure why one would use this use case)
yousafn
2022-11-16 22:00
Using Pact & Cypress?
Been itching for provider states and matchers?
You might want to get involved in a review of the following PR's
? https://github.com/pactflow/pact-cypress-adapter/pull/19
? https://github.com/pactflow/pact-cypress-adapter/pull/22
:rockon:
anandhiemail
2023-01-10 23:21
Hi Team, I am a QE and have just started learning about Pact and looking for the possibilities to get benefit out of it.
Do we have to write Pact tests close to the dev code only? Can we write contract tests in a different repo (other than dev repo) using Pact/Cypress? Would it still serve the purpose?
matt.fellows
2023-01-11 00:54
For others, I?ve answered that here: https://pact-foundation.slack.com/archives/C5F4KFKR8/p1673381550927689
yousafn
2023-05-02 11:03
:ralph-wave:
I've archived this repo
https://github.com/pact-foundation/cypress-pact
as it was just a placeholder with no info, and provided links to the current adapter implementation.
