has joined #pact-beam

has joined #pact-beam

I don't know anyone who can pick this up at the moment, but for coordination if/when someone wants to pick up this work, here is a channel.

thanks @bethskurrie this is the project i mentioned in the #general channel https://github.com/elitau/pact_elixir

has joined #pact-beam

has joined #pact-beam

has joined #pact-beam

Hi folks! I?m looking into creating a new Elixir implementation with the Rust core. The repo will live at https://github.com/andrewjensen/pact_elixir for now, although I would prefer to transfer it to an organization (Pact? My employer, https://github.com/podium?) as it gets further along.

I?m planning on copying some patterns out of this repo: https://github.com/discord/sorted_set_nif (Discord wrote https://blog.discord.com/using-rust-to-scale-elixir-for-11-million-concurrent-users-c6f19fc029d3 a while back introducing the repo and its performance benefits)

has joined #pact-beam

@pact-slack should probably be in here!

has joined #pact-beam

@andrew.jensen was there a reason why you created a new library? Ideally we can focus our efforts together and have two maintainers of a single repo (instead of multiple). Something we can work through here?

For reference, there is a potential client of DiUS (my empolyer, the parent company of Pactflow) that could do with this. We may be able to contribute some effort into this to move it forward also

@matt.fellows thanks for raising the question. I am now realizing that I misunderstood some messages in other channels, and incorrectly thought that https://github.com/elitau/pact_elixir was either unmaintained or implemented without bindings around the shared core. But I took some time to read through the library today and I can see that it?s looking great! I agree, let?s focus efforts on one implementation and make it the best it can be.

@pact-slack thanks for all your hard work on pact_elixir! Are you open to outside PRs and issues?

Also @mpglover, are you interested in standardizing on https://github.com/elitau/pact_elixir as well? If I?m remembering correctly, you had started an implementation of your own a while back but didn?t get too far.

Sure, every PR and issue is welcome! I've started pact_elixir as as side project with the goal to use it in our company. But we are still using another testing approach thus the development of this library never got good traction.

~chro~

has joined #pact-beam

I think you all should move forward with whatever makes sense for the folks actively involved. I don't know that I'll have a lot of time to dedicate to an Elixir integration in the near term. I'll still keep an eye on the channel and maybe I'll try to contribute here and there. But I doubt I'll be super involved over the next year or two. I started a project several years ago with a specific focus on supporting the spec without dependencies from outside the core Elixir ecosystem. This is why I didn't start out trying to enhance or fork the Rust-based integration. A variety of things changed shortly after starting the project and I no longer had a use case for Pact in Elixir so I left it by the wayside.

Cool, thanks for the context!

Some updates: ? Teamed up with @pedroefajardo_pactio yesterday to check out https://github.com/elitau/pact_elixir ? The library?s tests fail intermittently on my machine - seeing Rust panics like `Os { code: 24, kind: Other, message: "Too many open files" }` that are likely the root cause, might have to do with FFI code not cleaning up properly between each test run ? We have narrowed the corrupted state down to a few specific test cases involving writing pact files and stopping the mock server ? The library currently points to `pact_mock_server@0.4.0` and `rustler@0.21.0` so we are trying to upgrade both dependencies to see if it will resolve the errors and test failures (and because we?ll want to use spec v3, etc.) ? As we try to upgrade `pact_mock_server@0.7.20`, we are not able to compile the transitive dependency `pact_models`, and we?re seeing compile-time errors like `cannot find macro `json_internal` in this scope` and `the trait bound `serde_json::Value: Eq` is not satisfied` so we?re trying to debug those @matt.fellows do you have any recommendations for getting past those compile errors? I can compile `pact_models` in isolation, but the fancy Elixir -> Rust abstraction of Rustler might be configuring rustc differently in a way that?s causing the errors.

Apologies I missed this Andrew, will take a look next time I can

Worth join/raising in #libpact_ffi-users too

we were able to address the ?Too many open files? by using `ulimit -n 2048` the correct way to do it is, I think, to configure Rust to limit parallelization but I haven?t gotten to how to do that yet.

has joined #pact-beam

I've tried to use the `pact_elixir` library but not having much luck. After adding the dependency, when trying to run `mix test` for the first time I get this: ```$ mix test The `:rustler` compiler has been deprecated since v0.22.0 and will be removed in v1.0. To remove this warning, please consult the CHANGELOG for v0.22.0. ==> pact_elixir Compiling 15 files (.ex) Compiling crate pactmockserver in release mode (native/pactmockserver) Updating http://crates.io index Compiling lib/pact_elixir/mock_server/rust_pact_mock_server_facade.ex (it's taking more than 10s) Downloading crates ... Downloaded lazy_static v1.4.0 Downloaded quote v0.6.13 Downloaded void v1.0.2 Downloaded rand v0.4.3 Downloaded unreachable v1.0.0 Downloaded libc v0.2.66 Downloaded proc-macro2 v0.4.30 Downloaded semver-parser v0.7.0 Downloaded syn v0.11.11 Downloaded synom v0.11.3 Downloaded safemem v0.2.0 Downloaded typeable v0.1.2 Downloaded uuid v0.6.5 Downloaded utf8-ranges v1.0.0 Downloaded url v1.7.2 Downloaded unreachable v0.1.1 Downloaded which v2.0.1 Downloaded rustc-serialize v0.3.24 Downloaded language-tags v0.2.2 Downloaded hex v0.2.0 Downloaded failure v0.1.6 Downloaded unicode-bidi v0.3.4 Downloaded unicase v1.4.2 Downloaded unicode-xid v0.1.0 Downloaded unicode-xid v0.0.4 Downloaded syn v0.15.44 Downloaded peresil v0.3.0 Downloaded quote v0.3.15 Downloaded mime v0.2.6 Downloaded itoa v0.3.4 Downloaded typed-arena v1.3.0 Downloaded cookie v0.2.5 Downloaded httparse v1.2.3 Downloaded dtoa v0.4.2 Downloaded backtrace v0.3.12 Downloaded rustler v0.21.0 Downloaded num-traits v0.1.40 Downloaded aho-corasick v0.6.3 Downloaded env_logger v0.4.3 Downloaded rustc-demangle v0.1.16 Downloaded memchr v1.0.2 Downloaded thread_local v0.3.4 Downloaded lazy_static v0.2.9 Downloaded rustler_codegen v0.21.0 Downloaded pact_mock_server v0.4.0 Downloaded matches v0.1.6 Downloaded indextree v1.3.0 Downloaded ansi_term v0.9.0 Downloaded percent-encoding v1.0.0 Downloaded unicode-segmentation v1.2.1 Downloaded num_cpus v1.7.0 Downloaded traitobject v0.0.1 Downloaded heck v0.3.0 Downloaded p-macro v0.2.0 Downloaded version_check v0.1.3 Downloaded maplit v0.1.5 Downloaded cfg-if v0.1.5 Downloaded itertools v0.6.5 Downloaded either v1.3.0 Downloaded hpack v0.2.0 Downloaded serde_derive v1.0.15 Downloaded time v0.1.38 Downloaded semver v0.7.0 Downloaded uuid v0.5.1 Downloaded byteorder v1.1.0 Downloaded sxd-document v0.2.4 Downloaded rustler_sys v2.0.0 Downloaded pact_matching v0.4.4 Downloaded base64 v0.7.0 Downloaded serde_derive_internals v0.16.0 Downloaded serde_json v1.0.3 Downloaded solicit v0.4.4 Downloaded itertools v0.7.8 Downloaded rand v0.3.17 Downloaded regex-syntax v0.4.1 Downloaded serde v1.0.15 Downloaded log v0.3.8 Downloaded regex v0.2.2 Downloaded hyper v0.9.18 Downloaded unicode-normalization v0.1.5 Downloaded difference v1.0.0 Downloaded idna v0.1.4 Compiling libc v0.2.66 Compiling void v1.0.2 Compiling matches v0.1.6 Compiling log v0.3.8 Compiling version_check v0.1.3 Compiling cfg-if v0.1.5 Compiling proc-macro2 v0.4.30 Compiling rustc-demangle v0.1.16 Compiling unicode-normalization v0.1.5 Compiling unicode-xid v0.0.4 Compiling unicode-xid v0.1.0 Compiling lazy_static v0.2.9 Compiling percent-encoding v1.0.0 Compiling quote v0.3.15 Compiling regex-syntax v0.4.1 Compiling syn v0.15.44 Compiling either v1.3.0 Compiling utf8-ranges v1.0.0 Compiling unicode-segmentation v1.2.1 Compiling rustc-serialize v0.3.24 Compiling itoa v0.3.4 Compiling safemem v0.2.0 Compiling lazy_static v1.4.0 Compiling traitobject v0.0.1 Compiling dtoa v0.4.2 Compiling httparse v1.2.3 Compiling language-tags v0.2.2 Compiling typeable v0.1.2 Compiling typed-arena v1.3.0 Compiling byteorder v1.1.0 Compiling peresil v0.3.0 Compiling serde v1.0.15 Compiling num-traits v0.1.40 Compiling rustler_sys v2.0.0 Compiling semver-parser v0.7.0 Compiling p-macro v0.2.0 Compiling ansi_term v0.9.0 Compiling difference v1.0.0 Compiling maplit v0.1.5 Compiling hex v0.2.0 Compiling indextree v1.3.0 Compiling unicode-bidi v0.3.4 Compiling synom v0.11.3 Compiling unreachable v1.0.0 Compiling unreachable v0.1.1 Compiling itertools v0.7.8 Compiling itertools v0.6.5 Compiling hpack v0.2.0 Compiling mime v0.2.6 Compiling unicase v1.4.2 Compiling heck v0.3.0 Compiling sxd-document v0.2.4 Compiling base64 v0.7.0 Compiling semver v0.7.0 Compiling thread_local v0.3.4 Compiling syn v0.11.11 Compiling idna v0.1.4 Compiling solicit v0.4.4 Compiling quote v0.6.13 Compiling memchr v1.0.2 Compiling time v0.1.38 Compiling rand v0.3.17 Compiling rand v0.4.3 Compiling num_cpus v1.7.0 Compiling backtrace v0.3.12 Compiling url v1.7.2 error: failed to run custom build command for `rustler_sys v2.0.0` Caused by: process didn't exit successfully: `/Users/erikdanielsen/work/dius/pact-workshop-elixir/consumer/_build/test/lib/pact_elixir/native/pactmockserver/release/build/rustler_sys-27d371f25823a0e2/build-script-build` (exit status: 101) --- stdout This OTP release uses the unsupported Erlang NIF version "2.16". Please report at https://github.com/rustlerium/rustler. --- stderr thread 'main' panicked at 'gen_api.erl encountered an error.', /Users/erikdanielsen/.cargo/registry/src/github.com-1ecc6299db9ec823/rustler_sys-2.0.0/build.rs:34:18 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace warning: build failed, waiting for other jobs to finish... error: build failed == Compilation error in file lib/pact_elixir/mock_server/rust_pact_mock_server_facade.ex == ** (RuntimeError) Rust NIF compile error (rustc exit code 101) (rustler 0.22.0) lib/rustler/compiler.ex:36: Rustler.Compiler.compile_crate/2 lib/pact_elixir/mock_server/rust_pact_mock_server_facade.ex:10: (module) (stdlib 3.15.2) erl_eval.erl:685: :erl_eval.do_apply/6 could not compile dependency :pact_elixir, "mix compile" failed. You can recompile this dependency with "mix deps.compile pact_elixir", update it with "mix deps.update pact_elixir" or clean it with "mix deps.clean pact_elixir"```

I tried again with `RUST_BACKTRACE=full mix test` ```error: failed to run custom build command for `rustler_sys v2.0.0` Caused by: process didn't exit successfully: `/Users/erikdanielsen/work/dius/pact-workshop-elixir/consumer/_build/test/lib/pact_elixir/native/pactmockserver/release/build/rustler_sys-27d371f25823a0e2/build-script-build` (exit status: 101) --- stdout This OTP release uses the unsupported Erlang NIF version "2.16". Please report at https://github.com/rustlerium/rustler. --- stderr thread 'main' panicked at 'gen_api.erl encountered an error.', /Users/erikdanielsen/.cargo/registry/src/github.com-1ecc6299db9ec823/rustler_sys-2.0.0/build.rs:34:18 stack backtrace: 0: 0x10cfc9ede - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h0b3cb0f81eb6c032 1: 0x10cff37ac - core::fmt::write::h01b240c0519e8c85 2: 0x10cfc54ca - std::io::Write::write_fmt::h146f386a5b35d587 3: 0x10cfc5daf - std::panicking::default_hook::{{closure}}::h5f8b7ffe7769db61 4: 0x10cfc593f - std::panicking::default_hook::h9f87fc2fdf4f1125 5: 0x10cfc6472 - std::panicking::rust_panic_with_hook::h32f3432eccbbcf03 6: 0x10cfb9617 - std::panicking::begin_panic::{{closure}}::h39f2f6cd54707c07 7: 0x10cfb94c8 - std::sys_common::backtrace::__rust_end_short_backtrace::h9be66abe58513ae8 8: 0x10cff56b9 - std::panicking::begin_panic::h2352c1c97fb9a865 9: 0x10cfb9e89 - build_script_build::main::hb625c3f24e169ba2 10: 0x10cfb6c4a - core::ops::function::FnOnce::call_once::h30ac50f2b0695325 11: 0x10cfb94fd - std::sys_common::backtrace::__rust_begin_short_backtrace::he563f2a24eeaafd0 12: 0x10cfb5f90 - std::rt::lang_start::{{closure}}::h060f609a7f2cbfeb 13: 0x10cfc7aa5 - std::rt::lang_start_internal::h6f56f13a742d6e93 14: 0x10cfb5f62 - std::rt::lang_start::hf6ab5b518932e40e 15: 0x10cfb9f56 - _main warning: build failed, waiting for other jobs to finish... error: build failed == Compilation error in file lib/pact_elixir/mock_server/rust_pact_mock_server_facade.ex == ** (RuntimeError) Rust NIF compile error (rustc exit code 101) (rustler 0.22.0) lib/rustler/compiler.ex:36: Rustler.Compiler.compile_crate/2 lib/pact_elixir/mock_server/rust_pact_mock_server_facade.ex:10: (module) (stdlib 3.15.2) erl_eval.erl:685: :erl_eval.do_apply/6 could not compile dependency :pact_elixir, "mix compile" failed. You can recompile this dependency with "mix deps.compile pact_elixir", update it with "mix deps.update pact_elixir" or clean it with "mix deps.clean pact_elixir"```

I assume it's a version incompatibility issue but it's hard to say as I'm not an Elixir developer

```[127] $ elixir -v ? Erlang/OTP 24 [erts-12.0.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit] [dtrace] Elixir 1.12.3 (compiled with Erlang/OTP 24)```

Rust v1.55.0 installed with homebrew ```$ rustc --version rustc 1.55.0```

Does anyone have any idea what's going on here?

see also https://pact-foundation.slack.com/archives/C02AG6CML5T/p1629996144008800

I can see some recent updates on the repo, are you pulling in the dependency from a package manager or using the code?

Yeah I saw that

It's the standard elixir build tool and a dependency added through that (`mix`)

Yes, my recommendation would be to remove any rust code from Elixr completely, in favour of the FFI (c) interface

Looking at the code, this is basically replacing https://github.com/elitau/pact_elixir/blob/master/native/pactmockserver/src/lib.rs with calls from Elixr directly to FFI methods

In `mix.exs`: ``` defp deps do [ {:httpoison, "~> 1.8"}, {:jason, "~> 1.2"}, {:pact_elixir, "~> 0.5.2"} # {:dep_from_hexpm, "~> 0.3.0"}, # {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"} ] end```

(see also #libpact_ffi-users for clarifying questions on that interface)

gotcha

For the mock server, that?s specifically this interface: https://docs.rs/pact_ffi/0.0.2/pact_ffi/mock_server/index.html

Also modified `project` according to the readme: ``` def project do [ app: :consumer, version: "0.1.0", elixir: "~> 1.12", start_permanent: Mix.env() == :prod, escript: [main_module: Commandline.CLI], compilers: [:rustler] ++ Mix.compilers(), rustler_crates: rustler_crates(Mix.env()), deps: deps() ] end``` and added these: ``` defp rustler_crates(mix_env) when mix_env in [:test, :dev] do [ pactmockserver: [ path: "deps/pact_elixir/native/pactmockserver", mode: (:debug), ] ] end defp rustler_crates(_prod) do [] end``` Also based on readme

For an example of this, see Go?s native mapping: https://github.com/pact-foundation/pact-go/blob/2.x.x/internal/native/mock_server.go

(I?ve just added some feedback to the thread above which I stupidly missed a while ago - sorry @andrew.jensen!)

Trying to read the compilation error, it seems to be erroring outside of the rust code. If it was a rust failure, I could probably point to the problem. But because Elixr is built on erlang, it looks like there are 3 languages being compiled here :scream:

If I can read the code correctly, it looks like if you can replace https://github.com/elitau/pact_elixir/blob/master/lib/pact_elixir/mock_server/rust_pact_mock_server_facade.ex with calls directly to an FFI, things might start working and you could remove the Rustler stuff

I think we need a NIF: https://andrealeopardi.com/posts/using-c-from-elixir-with-nifs/

Hey @matt.fellows and @edanielsen! The `rustler` library is an abstraction on top of NIFs. My vote is to keep using rustler if at all possible, since the abstraction includes some extra glue code that makes the bridge between the Rust and Elixir code much safer than raw NIFs. > This OTP release uses the unsupported Erlang NIF version ?2.16? This error message in particular sounds like the version incompatibility might be between OTP 24.x and Rustler. My guess is that the Rustler library will ship a new version soon that supports a new range of NIFs, but in the meantime, could you try compiling with OTP 23 and see if that works? @pedroefajardo_pactio and I have been developing (for now) on Elixir 1.11.x and OTP 23.x. (If you?re looking for a convenient way to switch between different versions of a single system dependency, you might look into https://github.com/asdf-vm/asdf. @pedroefajardo_pactio and I both run it at work for Elixir/Erlang/JS/Ruby toolchains and it works great.)

Oh man. I?m dreading the work to rip out Rustler and do raw FFI instead. Maybe it?s a better direction in the long term though. What do you think @pedroefajardo_pactio and @pact-slack?

If that's the case then you should look to use the FFI exported functions as they will have better stability guarantees and will provide consistency with the other libs

Does rustler mean everyone who uses it has to have cargo/rust installed?

Does it work cross platform also?

For a testing tool, I feel like safety guarantees are less important if that makes the devex worse. It may not be the case with Elixr, but we used a similar tool an JS and it's been quite complicated supporting newer node version, and all of the OS/Watch combinations (plus it's super slow because rust compilation is now part of every build)

Not saying you should drop it, but wanted to share our experience

@edanielsen You could also try to use the current https://github.com/elitau/pact_elixir/commit/90928e5666a1fdb5a73da626fb8adb97808b26e5 of pact_elixir. @andrew.jensen and @pedroefajardo_pactio https://github.com/elitau/pact_elixir/commit/90928e5666a1fdb5a73da626fb8adb97808b26e5 the rustler dependency so that the project config is https://github.com/rusterlium/rustler/blob/master/CHANGELOG.md#deprecations. It compiles on my Mac OS machine with Erlang 24.0.5 and Elixir 1.12.1 (installed via asdf).

Hmm, I'm not quite sure what that would mean. How can one bundle/embed the executable FFI Version of pact_mock_server if it is not compiled during the installation of pact_elixir?

@matt.fellows Yes, currently a user of pact_elixir needs a working cargo/rust installed. As the compilation is done on the user machine it should be cross plattform - as long as rust, erlang and elixir compiles.

I'm not sure how Elixr works. Can you bundle the FFI library (.so/dll/dylib) with the package?

That's how others do it

that doesn?t sound ideal. I?ve found the rust compilation to be fairly slow, so presumably this could dramatically increase CI build times for end users. If you look at the current Pact JS build times, they take almost 30 minutes (granted, we have other checks, but the Rust bits account for a big chunk of that)

It will also mean contributors may need to know Rust, or at least enough to map to existing Rust functions, which we?ve found to be a barrier, over a well defined (but admittedly, not ideal) C interface

Thanks guys, I'll give it a go and report back

Elixir allows to bundle any files into a package. So the approach would be to build cross platform binaries from the rust source, include them in the elixir package and call the C FFI. Sounds doable.

That's what we do already (not binaries, libraries)

Despite it being less stable (perhaps), I'm pretty certain it will be a much better process for developers

The kids of stability needs for a testing package is different to a runtime tho, were need designing a telephony system here :rolling_on_the_floor_laughing:

This thread was started before we finished the work of compiling the project using rustler v0.22, libc v0.2.101, and removing the Rustler warning. the project is now compiling and all the tests are green. take my comment below with a grain of salt and with the understanding that I am new to Elixir and to Rustler. TL;DR - I don?t know for certain what?s impossible so at the same time I don?t know for certain what?s possible. I meant that ^^^^ comment semi-humorously At the moment I am familiar enough with how Rustler does things and I think we should keep going that way instead of going with straight FFI. My plan for this week is to see if I can get the project to compile using pact_mock_server v0.7.20. @andrew.jensen @matt.fellows @pact-slack

Thanks for the update Pedro, sounds like awesome progress! :clap:

I also just realised my update before made no sense whatsoever: > The kids of stability needs for a testing package is different to a runtime tho, were need designing a telephony system here ?we?re _not_ designing a telephony system? was what I meant to say :stuck_out_tongue:

As long as Elixr can speak a c interface (which it obviously can, because that is what Rustler ultimately does) and you can add a c compatible library to your package (which it sounds like you can), I?d strongly encourage you move away from having the need for rust on a target machine

that need not mean you do that right away, which is understandable

I think the c interface is a lot scarier than it sounds. Happy to point you at other language examples to follow if/when that time comes

@matt.fellows please point me to other language examples. maybe that?s something else I can focus on this week. I?ll speak with @andrew.jensen later today about this

Matt,

Will you please elaborate a bit more on the why of ```"I'd strongly encourage you move away from having the need for rust on a target machine"```

I would +1 on Matts opinion. As a user of the pact_elixir library I would like to have a good and easy developer expirience. Currently you need a working rust compiler installed and you can get Rust compiler errors/warnings during the installation of the library. And this is not what you expect if you use a elixir library.

ok, thanks went and searched for ?elixir ffi? one of the projects I found is this one https://github.com/joshnuss/elixir-ffi, granted it hasn?t been touched in several years, but I found reading the warning interesting. :slightly_smiling_face: ```*** Warning: Turns out this is a really bad idea, do not use! When the BEAM calls native functions it needs to know how many reductions should be used. Otherwise calling a long running function can break the scheduling completly, because with this approach a long running function is considered 1 reduction ***``` ^^^^ that is very early searching. i am sure things are different by now. I just thought I would share the comment in that project here. I?ll talk with Andrew about this and come up with a plan that we will share with y?all.

I saw that too

I wouldn't use his abstraction

FFI is inherently dangerous because you're crossing memory boundaries

But that's not really a concern for a test framework that is used in small discrete runs

A crash indicates a bug in the tool and presumably is catchable so we can encourage bug reports

A crash in a runtime app is bad

Here are some references: https://github.com/pact-foundation/pact-go/projects/2

FFI wrapper code (provides functions for internal use - this represents your facade essentially): https://github.com/pact-foundation/pact-go/tree/2.x.x/internal/native

JS https://github.com/pact-foundation/pact-js-core/tree/master/src/ffi

Pact .NET https://github.com/pact-foundation/pact-net/tree/feature/4.0.0

Hi all, here is an update on the current status of Elixir bindings as of October 2021, and the next steps for developing the library. *HIGH-LEVEL WORK TO BE DONE* The https://github.com/elitau/pact_elixir library is a community-built implementation of Pact in Elixir. Thank you to @pact-slack for the great work on this library so far! In order to make the library fully-featured, we have these general tasks to complete: ? Change the Elixir FFI implementation (see details below) ? Re-implement consumer-side mocks with new FFI ? Implement provider verification ? Implement messaging equivalents ? Implement other workflows like publishing and downloading pacts from a broker *FFI: PROBLEM AND CONSTRAINTS* The pact_elixir library currently uses a Elixir+Rust library called https://github.com/rusterlium/rustler to create safe bindings between the Elixir wrapper and the Rust reference implementation of Pact. Rustler is a great library for creating interop between the two languages because it minimizes the risks of FFI. However, rustler requires the app developer to have a Rust toolchain installed, and to compile the core Rust code at the same time as the app and the pact_elixir library are compiled. This leads to more system dependencies and slower build performance. The developer should not need to know that the internals of pact_elixir are actually written in a different language. The Elixir and Erlang ecosystem has another FFI concept called NIFs, or Natively-Implemented Functions. Note that NIFs require special instrumentation of the native code in order to communicate properly with the Elixir bindings that are calling in. This does not seem like an ideal solution either, because it requires the pact_elixir library to maintain hand-written shim code in C to add this instrumentation. *FFI: NEXT STEPS* We intend to follow the same approach that pact-js and pact-go now follow: the pact_elixir library should download the FFI bundles for each OS architecture at library publish time, and then link to them using whatever strategy is most unobtrusive to the app developer. We can reuse the pipeline scripts from these other implementations in order to download the bundles. We need to pick an appropriate FFI strategy from the Elixir/Erlang ecosystem to allow a good developer experience. Rustler and handwritten NIF instrumentation do not seem to be good options. Erlang Ports or Port Drivers may be viable options. We may also be able to find a library to automatically *generate NIF bindings around the FFI bundles*, and if this is possible, this may be the best option. I am not very familiar with this part of the Elixir and Erlang ecosystem, so if anyone is more familiar with what is available, please share what you know! Once we have researched and picked the right FFI strategy, we can create the bindings and have a fully-featured Elixir implementation of Pact! *RESOURCES* ? https://github.com/elitau/pact_elixir: Elixir bindings for Pact, let?s focus our efforts to evolve this repo! ? https://hex.pm: package manager for the ecosystem ? https://erlang.org/doc/tutorial/users_guide.html: Erlang Interoperability Tutorial User?s Guide, explains NIFs, ports, port drivers, etc. ? https://sudonull.com/post/97911-How-to-write-your-NIF-in-Elixir: Tutorial on writing NIFs. Note that the C code requires instrumentation, so hand-writing a NIF is likely not our best option.

Thx Andrew for the extensive description - this approach sounds good to me! We should experiment with calling the C library via Erlang Ports and via NIFs. Besides replacing all existing Rust calls with C calls, we could list all missing user facing features like matchers, error handling, provider side and message based communication DSL.

has joined #pact-beam

https://twitter.com/josevalim/status/1463605416861081602?s=21 could become an alternative approach. "With this work, libraries using Rustler can precompile native code and everything will continue to work transparently for users!" What do you think?

That does sound like what we want!

One of these rare moments of the week having time for software side projects and GitHub is down :grin:

has joined #pact-beam

has joined #pact-beam

Hi! I see there's been some discussion about NIFs and FFIs for the pact-elixir project. Has there been any resolution on this front? I'm interested in using Pact at my company and we have several elixir applications, so I'm curious about the current status of the project.

I'm interested in contributing if I can, as well.

Hi Austin, thanks for your interest in the library. I?m currently playing with prebuilt NIFs, so that users of the library don?t need to have Rust installed - based on the work at https://github.com/rusterlium/html5ever_elixir/pull/28. I think this could be a viable solution to the problems regarding NIFs and FFIs discussed in this channel and I?ll try to release a new version of pact_elixir in the next days. The current solution could for sure be simplified as soon as rustler comes with support for prebuilt NIFs.

The library has basic functions but lacks a lot of the features present in more mature pact libraries. I would suggest to give it a try, see what?s missing for your use cases and discuss your suggestions in this channel or create a PR. Also have a look at the https://github.com/elitau/pact_elixir/issues in Github.

As a note for upcoming devs of the library. I?ve encountered the problem of `Too many open files` raised by Rust on Mac Os during when executing `mix test` and solved it with `ulimit -S -n 2048` Full error message: ```thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.9.18/src/net.rs:217:41```

Just published version 0.5.2-rc.8 to http://hex.pm with prebuilt NIFs (version numbers are totally messed up for now). Give it a try :)

that?s? strange

is this during compilation or executing the c code? I?ve run into a lot of different problems but never that one (I may have previously bumped my ulimit of course), but still doesn?t sound right

(update: my ulimit is `unlimited` :laughing: )

Do you have a roadmap or priority list of issues to tackle?

No. I?m currently not using the library in the products I?m involved. I would suggest first to update all dependencies, foremost the pactmockserver Rust library. Also not all possible matching features are currently supported. And the error handling is not catching all possible errors thrown by the underlying Rust pack lib.

I was getting the errors during executing the rust code and still not sure, what exactly is causing it.

has joined #pact-beam

Hey all! I noticed this library uses the old rust app. Is it still worth picking up if I get started with Pact today?

There was some activity around this a few months back with several folks looking to update and generally revamp the Rust-based approach. Last I saw a release candidate was shipped that uses a prebuilt NIF for the Rust lib integration: https://hex.pm/packages/pact_elixir/0.5.2-rc.8 so you could give that a shot. I haven't used it myself/can't speak to exactly how well it will work. I had ideas of a native Elixir implementation years ago but have not found the time to work on it unfortunately. So I'd say give the Rust app wrapper option a shot if you're open to that. Otherwise you're likely stuck using the prepackaged tools for some degree of verification https://docs.pact.io/implementation_guides/cli

Hi All, was wondering if pact-elixir libraries are in a working state? cc: @pact-slack @andrew.jensen @pedroefajardo_pactio

idk @sheyan.rizfee I haven?t touched them in a while. We were waiting for an implementation that was going to bypass Rustler. I don?t know what the state of that work is. I haven?t followed up on it.

thanks for the update..

https://github.com/philss/rustler_precompiled was recently released. I plan to built it into pact_elixir in the coming weeks.

This is awesome! Rustler?s previous requirement of the developer having a Rust toolchain was a big blocker before, but this should resolve it

has joined #pact-beam

has joined #pact-beam

FYI we?ll be dropping a Pactflow POST very soon. Worth a shout out here for people to get involved / contribute or best to keep this a little quiet until you folks get it to a point that others will be able to build upon?

has joined #pact-beam

has joined #pact-beam

has joined #pact-beam

How?s this effort going?

has joined #pact-beam

Not good. I?m too occupied with other things.

Is there official support for Elixir yet?

has joined #pact-beam

has joined #pact-beam

hey folks! Are there any plans of supporting elixir?

Unfortunately the history of this channel is lost to the depths of the free tier. There are no official plans, albeit there have been starts

you can see some of that history here: https://docs.pact.io/slack/pact-elixir.html

We (maintainers) are of course happy to support a community initiative to bring one up to speed

thanks @matt.fellows

has joined #pact-beam

has joined #pact-beam

has joined #pact-beam

Should we rename this to pact-beam and include discussions of the new pact_erlang library as well?

Makes sense to me. This channel has been pretty quiet since I joined it long ago. I think discussing any beam-related stuff here makes sense.

We can rename if needed. We are working to get a paid workspace, hopefully that will be soon, and then the elixir history will be back

I?d expect (in time) that both Elixir and Erlang would have separate DSLs, specific to each language. Despite the common runtime, they are separate. I say that without knowing much. Is ?beam? the common platform that would be obvious to both languages / the ecosystem to join?

The inaccurate but quickest approximation I can give is like the JVM for Java, Scala, Clojure, etc.

that?s what I was thinking

so `#pact-beam` would cover both Erlang and Elixir?

So we can share/reuse stuff across the board if we choose. But yes you do end up with language-specific DSLs for stuff.

Yeah it'd cover those among some others

@priyaranjanmudliar - where did you land with the Erlang bindings? (and welcome :blobwave: )

has joined #pact-beam

erlang bindings? Do you mean erlang to c function calls ?

The erlang wip is at https://github.com/greyorange-labs/pact-erlang With the package at https://hex.pm/packages/pact_erlang

@matt.fellows I think this can be used by Elixir apps as well

Awesome, I?ll put in the channel description so it?s obvious to newcomers

Are you folks using it in your org or is it still very early experimental days?

Also, cool! https://www.greyorange.com/

just started trying to use it.. but still figuring out the right API/DSL

Request for comments: DSL and project structure for pact-erlang: https://github.com/greyorange-labs/pact-erlang/issues/11

has joined #pact-beam

pact_erlang version 0.2.0 released with the new map-based DSL: https://github.com/greyorange-labs/pact_erlang Sample usage: ```PactRef = pact:v4(<<"myapp">>, <<"animal_service">>), AnimalObject = #{<<"name">> => <<"Mary">>, <<"type">> => <<"alligator">>}, {ok, Port} = pact:interaction(PactRef, #{ given => <<"an alligator with the name Mary exists">>, upon_receiving => <<"a request to GET an animal: Mary">>, with_request => #{ method => <<"GET">>, path => <<"/animals/Mary">> }, will_respond_with => #{ status => 200, headers => #{ <<"Content-Type">> => <<"application/json">> }, body => thoas:encode(AnimalObject) } }), ?assertMatch({ok, AnimalObject}, animal_service_interface:get_animal(Port, "Mary")), {ok, matched} = pact:verify(PactRef), pact:write(PactRef, <<"./pacts">>), pact:cleanup(PactRef).```

Thanks @srijan.c for all the help, this is awesome ! :taco: for you..

Nice stuff fella! Will make sure we get this in this months pact blog!

has joined #pact-beam

@lewiscowles has left the channel

has joined #pact-beam