Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build statically linked Linux binaries in CI #5872

Merged
merged 2 commits into from
Dec 22, 2022

Conversation

cknitt
Copy link
Member

@cknitt cknitt commented Dec 3, 2022

This PR provides statically linked Linux binaries to resolve all those glibc compatibility issues once and for all. 😃

This is achieved by

  1. Performing the Linux build in an Alpine Linux Docker container. Alpine uses musl libc which is a glibc replacement that can be statically linked.
  2. Adding the flags -cclib -static -cclib -no-pie to the dune files for the executables built with OCaml.
  3. Adding LDFLAGS=-static to the build for the ninja executable.

(See https://ocamlpro.com/blog/2021_09_02_generating_static_and_portable_executables_with_ocaml for more information.)

The Docker image used is in the GitHub Container Registry as ghcr.io/rescript-lang/rescript-ci-build. Sources are at https://github.com/rescript-lang/docker-rescript-ci-build.

Docker support in Github actions has many limitations. While it is possible to run a single build step in Docker, it is then run as root in the Docker container, resulting in permission issues with the generated files. The only approach I finally got to work was to build the compiler and ninja binaries in a separate job and upload them as artifacts, to be used by the for the Linux platform in the existing build job (in place of building them from source there).

This additional job for building the static binaries needs to complete before the normal build job (for all platforms) can be started. It is fairly quick though (~2 minutes) as all dependencies are already preinstalled in the Docker container used. Still, this unfortunately increases the total execution time for the CI workflow.

I also had to make a cleaner separation in the Makefile and scripts between the dune build and the subsequent stdlib build/tests. These should not use dune exec to avoid triggering a rebuild of the executables with different settings. (Also dune exec at least checks for changes and so comes with some performance cost anyway.)

The generated binaries were tested successfully on NixOS, see #5694 (comment).

@cknitt cknitt force-pushed the build-container branch 28 times, most recently from bb524b8 to 1efa927 Compare December 10, 2022 15:40
@cknitt cknitt force-pushed the build-container branch 2 times, most recently from 5db585f to 0a678a1 Compare December 10, 2022 17:13
@cknitt cknitt force-pushed the build-container branch 12 times, most recently from 277229e to 457b6ae Compare December 11, 2022 06:42
@cknitt cknitt changed the title Experiment: Try running the Linux build in a container Build statically linked Linux binaries in CI Dec 11, 2022
@cknitt cknitt force-pushed the build-container branch 2 times, most recently from 84a7a07 to bc76943 Compare December 21, 2022 16:19
@cknitt cknitt marked this pull request as ready for review December 22, 2022 06:44
@cknitt cknitt requested a review from cristianoc December 22, 2022 06:44
Copy link
Collaborator

@cristianoc cristianoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On linux, do we still need the steps for ocaml and opam, given that the compiler has been built already?

@cknitt
Copy link
Member Author

cknitt commented Dec 22, 2022

On linux, do we still need the steps for ocaml and opam, given that the compiler has been built already?

Yes, for several reasons. From what I remember, one was ocamldep being needed for the stdlib build, then there is dune for the format check and also reanalyze for the tests.

@cknitt cknitt merged commit a713ff7 into rescript-lang:master Dec 22, 2022
@cknitt cknitt deleted the build-container branch December 22, 2022 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants