Split crates requiring nightly Rust into separate `unstable` workspace #1227

joshtriplett · 2025-05-07T09:43:24Z

No description provided.

joshtriplett · 2025-05-07T15:45:30Z

Marking as WIP, because it'll need some CI debugging to make sure tests don't fail.

scripts/docker_build.sh

Cargo.lock

kkysen · 2025-05-08T02:22:02Z

Also I merged #1225 so you can rebase now.

joshtriplett · 2025-05-08T12:26:47Z

Currently attempting a full implementation of #1220. Still WIP; will work on ensuring CI passes.

joshtriplett · 2025-05-08T12:29:09Z

Side note: analysis/tests/minimal/reference_pdg.bc was checked in despite being .gitignored. It doesn't look like that test is using "compare" mode, so I don't think that file is being used, only written to. Nonetheless, I moved it across to unstable/analysis/tests/minimal/reference_pdg.bc. If there's any potential cleanup there, it can happen separately from this PR.

joshtriplett · 2025-05-08T15:23:20Z

@kkysen For some reason, the Azure pipeline doesn't seem to be kicking off for pushes anymore, and I'm not sure why. I think I've fixed most of the issues at this point, but I can't test that until the Azure pipeline runs again.

EDIT: Seems to be working again.

joshtriplett · 2025-05-08T15:44:18Z

@kkysen I just realized that the Docker images probably don't get rebuilt as a result of this pipeline; I updated the scripts that build Docker images, and then the failure is happening because the CI runs using the old Docker images.

I think this may be easiest to fix by going ahead and switching away from Azure pipelines. I'll work towards that.

kkysen · 2025-05-08T21:01:24Z

Side note: analysis/tests/minimal/reference_pdg.bc was checked in despite being .gitignored. It doesn't look like that test is using "compare" mode, so I don't think that file is being used, only written to. Nonetheless, I moved it across to unstable/analysis/tests/minimal/reference_pdg.bc. If there's any potential cleanup there, it can happen separately from this PR.

It's used in a test here:

c2rust/c2rust-analyze/tests/analyze.rs

Line 74 in 4b7691e

let pdg_path: PathBuf = "../analysis/tests/minimal/reference_pdg.bc".into();

As long as you keep the relative paths the same, it should still work fine. We in general gitignored *.bc files but added this one manually for this one test.

kkysen · 2025-05-08T21:03:23Z

@kkysen I just realized that the Docker images probably don't get rebuilt as a result of this pipeline; I updated the scripts that build Docker images, and then the failure is happening because the CI runs using the old Docker images.

I think this may be easiest to fix by going ahead and switching away from Azure pipelines. I'll work towards that.

Oh yes, that's always been one of the very annoying pain points with Azure pipelines. We have to update the docker images separately, and so we can't test multiple in CI at the same time either. One of the many reasons I've always wanted to move away from Azure pipelines. I agree it's probably easier to just move to GitHub Actions in one go and things will work better there. Thanks again for helping with all of this!

joshtriplett · 2025-07-10T22:23:44Z

(Apologies that this change stalled out. Some other things came up. I'll get back to it eventually, but someone else who wants to work on it should not hesitate to pick it up and run with it.)

now that the transpiler builds on stable, tests need to specify their own rust toolchain as generated code still requires nightly

Fixes: immunant#1220 Co-authored-by: Khyber Sen <kkysen@gmail.com>

At this point, we care more about the stable (`c2rust-transpiler`) workspace, and its tests are faster as well, so run them separately and first.

…ot needed

…for cargo's sparse registry The old git-based registry fetching is extremely slow in CI, and due to stable crates now building on stable but testing their output on nightly, is very difficult to correctly cache. Updating the nightly of their output to the earliest to have the sparse registry should fix this.

…for `syn v2.0.107` and `cargo`'s sparse registry `c2rust-bitfields-derive` depends on `syn v2.0`. Without a `Cargo.lock`, like in transpiled projects (as opposed to `c2rust` itself), this will resolve to `syn v2.0.107`, which requires Rust 1.68, but the generated `rust-toolchain.toml` uses Rust 1.65 (the same as our own nightly). This is currently breaking CI for the `c2rust-instrument` tests, and will also break CI for `c2rust-testsuite` once #1418 lands (a bug fix). Furthermore, the old git-based registry fetching is extremely slow in CI, so when #1227 lands and switches to stable, we also want Rust 1.70 for `cargo`'s sparse registry. Due to stable crates now building on stable but testing their output on nightly, it is very difficult to correctly cache. Updating the nightly of their output to the earliest to have the sparse registry should fix this. The combination of this, #1418, #1419, #1420, and #1421 has been tested to work on `libxml2`: ```sh cargo build --release (cd ../c2rust-testsuite/tests/libxml2/repo/ && $OLDPWD/target/release/c2rust-refactor --cargo --rewrite-mode inplace --lib reorganize_definitions )

…for `syn v2.0.107` and `cargo`'s sparse registry `c2rust-bitfields-derive` depends on `syn v2.0`. Without a `Cargo.lock`, like in transpiled projects (as opposed to `c2rust` itself), this will resolve to `syn v2.0.107`, which requires Rust 1.68, but the generated `rust-toolchain.toml` uses Rust 1.65 (the same as our own nightly). This is currently breaking CI for the `c2rust-instrument` tests, and will also break CI for `c2rust-testsuite` once #1418 lands (a bug fix). Furthermore, the old git-based registry fetching is extremely slow in CI, so when #1227 lands and switches to stable, we also want Rust 1.70 for `cargo`'s sparse registry. Due to stable crates now building on stable but testing their output on nightly, it is very difficult to correctly cache. Updating the nightly of their output to the earliest to have the sparse registry should fix this. The combination of this, #1418, #1419, #1420, and #1421 has been tested to work on `libxml2`: ```sh cargo build --release (cd ../c2rust-testsuite/tests/libxml2/repo/ && $OLDPWD/target/release/c2rust-refactor --cargo --rewrite-mode inplace --lib reorganize_definitions ) ```

…for `syn v2.0.107` and `cargo`'s sparse registry (#1422) `c2rust-bitfields-derive` depends on `syn v2.0`. Without a `Cargo.lock`, like in transpiled projects (as opposed to `c2rust` itself), this will resolve to `syn v2.0.107`, which requires Rust 1.68, but the generated `rust-toolchain.toml` uses Rust 1.65 (the same as our own nightly). This is currently breaking CI for the `c2rust-instrument` tests, and will also break CI for `c2rust-testsuite` once #1418 lands (a bug fix). Furthermore, the old git-based registry fetching is extremely slow in CI, so when #1227 lands and switches to stable, we also want Rust 1.70 for `cargo`'s sparse registry. Due to stable crates now building on stable but testing their output on nightly, it is very difficult to correctly cache. Updating the nightly of their output to the earliest to have the sparse registry should fix this. The combination of this, #1418, #1419, #1420, and #1421 has been tested to work on `libxml2`: ```sh cargo build --release (cd ../c2rust-testsuite/tests/libxml2/repo/ && $OLDPWD/target/release/c2rust-refactor --cargo --rewrite-mode inplace --lib reorganize_definitions ) ```

joshtriplett force-pushed the try-stable branch from ba9b028 to ee319ba Compare May 7, 2025 15:45

joshtriplett changed the title ~~Temporarily exclude crates that require nightly~~ WIP: Temporarily exclude crates that require nightly May 7, 2025

kkysen reviewed May 8, 2025

View reviewed changes

scripts/docker_build.sh Outdated Show resolved Hide resolved

kkysen reviewed May 8, 2025

View reviewed changes

Cargo.lock Show resolved Hide resolved

joshtriplett force-pushed the try-stable branch from ee319ba to b5e1b78 Compare May 8, 2025 12:26

joshtriplett changed the title ~~WIP: Temporarily exclude crates that require nightly~~ WIP: Split crates requiring nightly Rust into separate unstable workspace May 8, 2025

joshtriplett force-pushed the try-stable branch 5 times, most recently from bb14e51 to 444ca79 Compare May 8, 2025 15:13

joshtriplett closed this May 8, 2025

joshtriplett force-pushed the try-stable branch from 444ca79 to 4b7691e Compare May 8, 2025 15:27

joshtriplett reopened this May 8, 2025

This was referenced May 9, 2025

Allow running multiple transpiles in parallel #1241

Merged

Fixes to rust-toolchain.toml handling #1225

Merged

joshtriplett force-pushed the try-stable branch from 444ca79 to e0ee1d0 Compare May 14, 2025 15:16

fw-immunant mentioned this pull request Jul 23, 2025

ci: test on aarch64 Linux #1283

Open

fw-immunant force-pushed the try-stable branch from e0ee1d0 to 819675a Compare July 23, 2025 18:30

kkysen force-pushed the try-stable branch from 22cd116 to 65d9988 Compare July 24, 2025 10:21

kkysen mentioned this pull request Jul 25, 2025

Translate builtin_assume and unaligned SIMD intrinsics #1132

Open

kkysen force-pushed the try-stable branch 3 times, most recently from a00fa22 to 768a099 Compare August 4, 2025 07:31

This was referenced Aug 8, 2025

Apply clippy fixes to stable crates #1308

Merged

transpile: tests: snapshots: hard-code nightly toolchain for now #1310

Merged

Raise the MSRV? #1324

Open

bungcip mentioned this pull request Sep 5, 2025

cleanup AST builder functionality #1350

Merged

kkysen mentioned this pull request Sep 12, 2025

transpile: Use raw borrows instead of normal borrows #1355

Merged

kkysen force-pushed the try-stable branch 2 times, most recently from b647886 to c57c608 Compare October 3, 2025 08:33

kkysen mentioned this pull request Oct 8, 2025

Can we have git commit hash displayed when calling c2rust --help? #1396

Open

kkysen force-pushed the try-stable branch 3 times, most recently from d890b5c to d4e510b Compare October 9, 2025 04:54

fw-immunant and others added 8 commits October 8, 2025 22:43

tests: add rust-toolchain.toml file for generated code

34f6f3a

now that the transpiler builds on stable, tests need to specify their own rust toolchain as generated code still requires nightly

Split crates requiring nightly Rust into separate unstable workspace

63b9d7c

Fixes: immunant#1220 Co-authored-by: Khyber Sen <kkysen@gmail.com>

ci: run cargo fmt --check for unstable workspace, too

897628e

ci: split up stable and unstable steps and run unstable steps afterward

575c8b6

At this point, we care more about the stable (`c2rust-transpiler`) workspace, and its tests are faster as well, so run them separately and first.

ci: split up rustup component adds for stable vs unstable workspaces

4904c6b

ci: don't build unstable workspace for internal-testsuite as it's n…

4e96d31

…ot needed

ci: use separate jobs for stable and unstable builds

0eb5db7

kkysen force-pushed the try-stable branch from d4e510b to 6052ab4 Compare October 9, 2025 05:46

kkysen mentioned this pull request Oct 20, 2025

transpile: update generated nightly to nightly-2023-04-15 (1.70.0) for syn v2.0.107 and cargo's sparse registry #1422

Merged

kkysen mentioned this pull request Dec 1, 2025

Add definition splitting to transpiler #1477

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Split crates requiring nightly Rust into separate `unstable` workspace #1227

Split crates requiring nightly Rust into separate `unstable` workspace #1227

Uh oh!

joshtriplett commented May 7, 2025 •

edited

Loading

Uh oh!

joshtriplett commented May 7, 2025

Uh oh!

Uh oh!

Uh oh!

kkysen commented May 8, 2025

Uh oh!

joshtriplett commented May 8, 2025

Uh oh!

joshtriplett commented May 8, 2025

Uh oh!

joshtriplett commented May 8, 2025 •

edited

Loading

Uh oh!

joshtriplett commented May 8, 2025

Uh oh!

kkysen commented May 8, 2025

Uh oh!

kkysen commented May 8, 2025

Uh oh!

joshtriplett commented Jul 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Split crates requiring nightly Rust into separate unstable workspace #1227

Are you sure you want to change the base?

Split crates requiring nightly Rust into separate unstable workspace #1227

Uh oh!

Conversation

joshtriplett commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshtriplett commented May 7, 2025

Uh oh!

Uh oh!

Uh oh!

kkysen commented May 8, 2025

Uh oh!

joshtriplett commented May 8, 2025

Uh oh!

joshtriplett commented May 8, 2025

Uh oh!

joshtriplett commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshtriplett commented May 8, 2025

Uh oh!

kkysen commented May 8, 2025

Uh oh!

kkysen commented May 8, 2025

Uh oh!

joshtriplett commented Jul 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Split crates requiring nightly Rust into separate `unstable` workspace #1227

Split crates requiring nightly Rust into separate `unstable` workspace #1227

joshtriplett commented May 7, 2025 •

edited

Loading

joshtriplett commented May 8, 2025 •

edited

Loading