Add dist::Gcc build step
#137667
Merged
Add dist::Gcc build step
#137667
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustbot
added
S-waiting-on-review
|
@bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Feb 26, 2025
Add `dist::Gcc` build step To distribute the prebuilt libgccjit.so from CI. Opening as a draft to examine the impact on CI times. r? `@ghost`
|
☀️ Try build successful - checks-actions |
|
@bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Feb 26, 2025
Add `dist::Gcc` build step To distribute the prebuilt libgccjit.so from CI. Opening as a draft to examine the impact on CI times. r? `@ghost`
|
☀️ Try build successful - checks-actions |
Kobzol
force-pushed
the
gcc-dist-build
branch
from
bea7b2f to
6f37480
Compare
|
r? @onur-ozkan |
|
Some changes occurred in src/tools/opt-dist cc @Kobzol |
onur-ozkan
reviewed
Feb 27, 2025
To distribute the prebuilt libgccjit.so from CI.
Kobzol
force-pushed
the
gcc-dist-build
branch
from
6f37480 to
e4bfad2
Compare
rustbot
added
A-testsuite
|
@rustbot ready |
|
Feel free to r=me once CI is green. |
|
@bors r=onur-ozkan rollup=iffy |
bors
added
S-waiting-on-bors
jieyouxu
added a commit
to jieyouxu/rust
that referenced
this pull request
Mar 4, 2025
Add `dist::Gcc` build step This PR adds a `dist:Gcc` bootstrap step to distribute a prebuilt `libgccjit.so` from CI on x64 Linux. With primed sccache, the build takes ~4 minutes on CI, and produces a 50 MiB archive. I want to land this before adding something akin to `[gcc] download-ci-gcc = true`, to already have the artifacts available on CI, to make it easier to setup the download merge-base logic. r? `@ghost`
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 4, 2025
Rollup of 10 pull requests Successful merges: - rust-lang#134063 (dec2flt: Clean up float parsing modules) - rust-lang#136662 (Count char width at most once in `Formatter::pad`) - rust-lang#137011 (Promote ohos targets to tier2 with host tools.) - rust-lang#137077 (Postprocess bootstrap metrics into GitHub job summary) - rust-lang#137327 (Undeprecate env::home_dir) - rust-lang#137373 (Compile run-make-support and run-make tests with the bootstrap compiler) - rust-lang#137463 ([illumos] attempt to use posix_spawn to spawn processes) - rust-lang#137477 (uefi: Add Service Binding Protocol abstraction) - rust-lang#137569 (Stablize `string_extend_from_within`) - rust-lang#137667 (Add `dist::Gcc` build step) r? `@ghost` `@rustbot` modify labels: rollup
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 5, 2025
…kingjubilee Rollup of 6 pull requests Successful merges: - rust-lang#137077 (Postprocess bootstrap metrics into GitHub job summary) - rust-lang#137373 (Compile run-make-support and run-make tests with the bootstrap compiler) - rust-lang#137634 (Update `compiler-builtins` to 0.1.149) - rust-lang#137667 (Add `dist::Gcc` build step) - rust-lang#137722 (`librustdoc`: 2024 edition! 🎊) - rust-lang#137947 (Do not install rustup on Rust for Linux job) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 5, 2025
Rollup merge of rust-lang#137667 - Kobzol:gcc-dist-build, r=onur-ozkan Add `dist::Gcc` build step This PR adds a `dist:Gcc` bootstrap step to distribute a prebuilt `libgccjit.so` from CI on x64 Linux. With primed sccache, the build takes ~4 minutes on CI, and produces a 50 MiB archive. I want to land this before adding something akin to `[gcc] download-ci-gcc = true`, to already have the artifacts available on CI, to make it easier to setup the download merge-base logic. r? ``@ghost``
matthiaskrgr
added a commit
to matthiaskrgr/rust
that referenced
this pull request
Mar 11, 2025
Add support for downloading GCC from CI This PR adds a new bootstrap config section called `gcc` and implements a single config `download-ci-gcc`. Its behavior is similar to `download-ci-llvm`. Since rust-lang#137667, we distribute a CI component that contains the prebuilt `libgccjit.so` library on x64 Linux. With `download-ci-gcc`, this component is downloaded from CI to avoid building GCC locally. This is an MVP of this functionality, designed for local usage. This PR does not enable this functionality on the LLVM 18 PR CI job which builds `cg_gcc`, and does not implement more complex detection logic. It simply uses `false` (build locally) or `true` (download from CI if you're on the right target, if CI download fails, then bootstrap fails). The original LLVM CI download functionality has a lot of features and complexity, which we don't need for GCC (yet). I don't like how the LLVM CI stuff is threaded through multiple parts of bootstrap, so with GCC I would like to take a more centralized approach, where the `build::Gcc` step handles download from CI internally. This means that: - For the rest of bootstrap, it should be transparent whether GCC was built locally or downloaded from CI. - GCC is not downloaded eagerly unless you actually requested GCC (either you requested `x build gcc` or you asked to build/test the GCC backend). This approach will require some modifications once we extend this feature, but so far I like this approach much more than putting this stuff into `Config[::parse]`, which already does a ton of stuff that it arguably shouldn't (but it's super difficult to extract its logic out). This PR is an alternative to rust-lang#130749, which did a more 1:1 copy of the `download-ci-llvm` logic. r? `@onur-ozkan`
matthiaskrgr
added a commit
to matthiaskrgr/rust
that referenced
this pull request
Mar 11, 2025
Add support for downloading GCC from CI This PR adds a new bootstrap config section called `gcc` and implements a single config `download-ci-gcc`. Its behavior is similar to `download-ci-llvm`. Since rust-lang#137667, we distribute a CI component that contains the prebuilt `libgccjit.so` library on x64 Linux. With `download-ci-gcc`, this component is downloaded from CI to avoid building GCC locally. This is an MVP of this functionality, designed for local usage. This PR does not enable this functionality on the LLVM 18 PR CI job which builds `cg_gcc`, and does not implement more complex detection logic. It simply uses `false` (build locally) or `true` (download from CI if you're on the right target, if CI download fails, then bootstrap fails). The original LLVM CI download functionality has a lot of features and complexity, which we don't need for GCC (yet). I don't like how the LLVM CI stuff is threaded through multiple parts of bootstrap, so with GCC I would like to take a more centralized approach, where the `build::Gcc` step handles download from CI internally. This means that: - For the rest of bootstrap, it should be transparent whether GCC was built locally or downloaded from CI. - GCC is not downloaded eagerly unless you actually requested GCC (either you requested `x build gcc` or you asked to build/test the GCC backend). This approach will require some modifications once we extend this feature, but so far I like this approach much more than putting this stuff into `Config[::parse]`, which already does a ton of stuff that it arguably shouldn't (but it's super difficult to extract its logic out). This PR is an alternative to rust-lang#130749, which did a more 1:1 copy of the `download-ci-llvm` logic. r? ``@onur-ozkan``
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a
dist:Gccbootstrap step to distribute a prebuiltlibgccjit.sofrom CI on x64 Linux.With primed sccache, the build takes ~4 minutes on CI, and produces a 50 MiB archive.
I want to land this before adding something akin to
[gcc] download-ci-gcc = true, to already have the artifacts available on CI, to make it easier to setup the download merge-base logic.r? @ghost