Shipping Rust Binaries with GoReleaser

As consumers, we like to get a binary release in one of the following ways:

• OS native package format such as deb, rpm and so on.
• Language native package format such as pip, npm and so on. It's not common to supply a pure binary artifact this way but some do it, such as puppeteer which downloads a latest binary of a headless Chrome.
• A curl shell combo such as curl http://acme.org/bin | sh which is fast, has no dependencies, but receives criticism because you're essentially letting your users execute remote code. This is why I avoid these, but if I must, verify the shell script by hand first.
• A modern binary packager such as Homebrew.

While the process may differ from one technique to another, they all require a way to host the binaries or packages, except perhaps the language-native package formats, where you can get away with pushing large binaries into the registries (but it may be frowned upon).

In principle, these are the steps required do package and release:

1. Cross compile your binaries (e.g. Linux, Windows, macOS), since we’re distributing binaries we have to have a ready-made version per platform and architecture.
2. Package (e.g. tar, zip), some platforms expect a specific packaging format.
3. Digest your packages (e.g. sha256) — while we’re all enjoying a reliable internet connection and file hosting solutions — we’re not assuming it’s always the case for all users. It’s wise to verify downloads by having it digested.
4. Upload to <provider> using some kind of path format scheme (e.g. to S3 as `v0.0.1/project-name/tarball).
5. Set up the distribution artifact (e.g. generate a shellfile for download and verification from S3 using the given format scheme).

For an open source project, there are many tools to use, because Github Releases and Git tags make it a breeze to construct a great release process. For other projects, Amazon S3 would be perfect for storing, and an opinionated release formatting rules helps. This is where GoReleaser shines.

Using GoReleaser with Rust

Although GoReleaser supports building just Go projects, it does so much more in the packaging and distribution department that it is extremely hard to ignore.

It can package, generate sha256 sums, upload to Github releases or Amazon S3 or custom HTTP endpoints, as well as generate a changelog, semantic versions and a lot more. The website doesn’t show everything, so I recommend looking at the repo for docs and to find out what more it can support.

So it has everything we want, but it can’t build anything other than Go.

Rust cross-compilation is easy from a Linux host with cross, and even easier with TravisCI with trust but it’s not easy from an OSX host, not to mention if your project is not open source.

To build cross-platform with the convenience of your OSX system, this is how I do both OSX and Linux:

docker run --rm -v`pwd`:/app liuchong/rustup:nightly sh -c 'cd /app && cargo build --release --target=x86_64-unknown-linux-gnu'
cargo build --release --target=x86_64-apple-darwin

When you finish building, there are Linux and OSX (darwin) builds in your target/ folder.

For Windows builds, you might want to go back to trust and use a CI system.

There has been efforts of making GoReleaser generic such as multiple language support with Rust, and some more generalization discussion and finally a generic infrastructure.

There still is missing infrastructure to support a plug-in architecture for custom builders that would work with GoReleaser, so for now Rust still isn’t supported.

Hacking the GoReleaser Build Process

This hack revolves around satisfying GoReleaser by dropping a dummy.go file for it to build, and hooking into the post stage of the build.

Here is a working .goreleaser.yaml set up:

project_name: cep
builds:
  -
    main: dummy.go
    goarch:
      - amd64
    binary: cep
    hooks:
      post: sh build-cross-release.sh
archive:
  replacements:
    darwin: Darwin
    linux: Linux
    windows: Windows
    386: i386
    amd64: x86_64
checksum:
  name_template: 'checksums.txt'
s3:
  -
    bucket: cep-dev
    acl: public-read
release:
  disable: true

And the Go dummy program is trivially:

package main

func main() {
}

Using the post stage we actually perform the necessary Rust builds and copying the artifacts to where GoReleaser would expect them to be, overwriting the Go artifacts (for a project named cep):

# jondot: https://github.com/goreleaser/goreleaser/issues/962

docker run --rm -v`pwd`:/app liuchong/rustup:nightly sh -c 'cd /app && cargo build --release --target=x86_64-unknown-linux-gnu'
cargo build --release --target=x86_64-apple-darwin

mkdir -p dist/darwin_amd64 && cp target/x86_64-apple-darwin/release/cep dist/darwin_amd64
mkdir -p dist/linux_amd64 && cp target/x86_64-unknown-linux-gnu/release/cep dist/linux_amd64

GoReleaser would carry on happily with the rest of the stages of the build, meaning we get everything for free!

Bonus Points: GoDownloader

Probably not as well-known, godownloader is a sister project to GoReleaser. It will read a .goreleaser.yaml and generate a curl-shell combo for you.

If distributing binaries via curl http://your-binary.com/binary | sh is your cup of tea, you can use godownloader and you get a clever shell script for installing your binary for free.

Exploring Other Rust Tools for Release

There is existing Rust work that aims to build something similar to GoReleaser, but frankly, it’s just not complete enough. You can take a look at the following:

• cargo-release
• cargo-deliver
• cargo-hublish

Finally, if you want to cross-build Rust from OSX, take a look at docker-rustup.