Skip to content

Conversation

@johnathan79717
Copy link
Contributor

@johnathan79717 johnathan79717 commented Jan 9, 2026

Summary

Multiple fixes for ci-barretenberg-full mode:

  1. Create libbb-external.a - Single static library for barretenberg-rs that bundles barretenberg + env + vm2_stub
  2. Propagate AVM env var - Pass AVM environment variable to isolated docker containers so tests use the correct bb binary
  3. Denoise barretenberg-rs - Add denoise to build and test commands for cleaner CI output

Problem

ci-barretenberg-full mode was failing with multiple issues:

  1. barretenberg-rs build failed with:

    error: could not find native static library `vm2`, perhaps an -L flag is missing?
    

    This happened because ci-barretenberg-full builds with AVM=0, so libvm2.a is not produced.

  2. acir_tests failed with:

    barretenberg/cpp/build/bin/bb-avm: No such file or directory
    

    The AVM environment variable wasn't being passed to isolated docker containers, causing find-bb to incorrectly select bb-avm instead of bb.

Solution

  1. Create libbb-external.a that includes everything barretenberg-rs needs:

    • All objects from libbarretenberg.a
    • env module (logstr, throw_or_abort_impl, env_hardware_concurrency)
    • vm2_stub (provides stub for create_avm2_recursion_constraints_goblin)

    This simplifies linking in barretenberg-rs to just one library and removes the need for --allow-multiple-definition and --start-group/--end-group.

  2. Propagate AVM env var to isolated docker containers in ci3/docker_isolate

  3. Use denoise for barretenberg-rs build and test commands

Caching

libbb-external.a is built to build/lib/ and the native build cache uploads build/{bin,lib}, so it will be cached automatically.

@johnathan79717 johnathan79717 added the ci-barretenberg-full Run all barretenberg checks. label Jan 9, 2026
@johnathan79717 johnathan79717 force-pushed the jh/fix-bb-rs-vm2-library branch from 7b2e604 to a7d9cd9 Compare January 9, 2026 17:21
@johnathan79717 johnathan79717 requested a review from ludamad January 9, 2026 17:33
@AztecBot AztecBot force-pushed the jh/fix-bb-rs-vm2-library branch from 2cfc72b to 9fc3aef Compare January 9, 2026 17:40
@AztecBot AztecBot enabled auto-merge January 9, 2026 17:40
@johnathan79717 johnathan79717 added ci-full Run all master checks. and removed ci-barretenberg-full Run all barretenberg checks. labels Jan 9, 2026
@AztecBot AztecBot added this pull request to the merge queue Jan 9, 2026
@ludamad ludamad removed this pull request from the merge queue due to a manual request Jan 9, 2026
Copy link
Collaborator

@ludamad ludamad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above

@johnathan79717 johnathan79717 changed the title fix: make vm2 library linking optional in barretenberg-rs fix: link vm2_stub in barretenberg-rs for AVM recursion constraints Jan 13, 2026
@johnathan79717 johnathan79717 changed the base branch from next to merge-train/barretenberg January 13, 2026 12:31
@johnathan79717 johnathan79717 added ci-barretenberg-full Run all barretenberg checks. and removed ci-full Run all master checks. labels Jan 13, 2026
johnathan79717 and others added 2 commits January 13, 2026 12:54
## Summary
- The vm2 library is only present when barretenberg is built with AVM enabled
- When AVM is disabled (e.g., in some CI modes), libbarretenberg contains vm2_stub instead
- This change makes build.rs check if libvm2.a exists before linking it

## Problem
barretenberg-rs build was failing in CI with:
```
error: could not find native static library `vm2`, perhaps an -L flag is missing?
```

This happened because the cpp build was done without AVM enabled, so libvm2.a wasn't present.

Co-authored-by: Jonathan Hao <jonathan@aztec-labs.com>
Addresses review feedback:
- Add static library version of vm2_stub for external consumers
- Always link vm2_stub instead of conditionally linking vm2
- This avoids caching issues and provides consistent behavior
- vm2_stub throws at runtime if AVM recursion is attempted
@johnathan79717 johnathan79717 force-pushed the jh/fix-bb-rs-vm2-library branch from 710539f to 512a859 Compare January 13, 2026 12:55
@johnathan79717
Copy link
Contributor Author

Will ask for review again once ci-barretenberg-full passes.

@johnathan79717 johnathan79717 removed the request for review from ludamad January 13, 2026 13:16
@johnathan79717 johnathan79717 requested review from ludamad and removed request for charlielye January 13, 2026 13:36
@johnathan79717 johnathan79717 changed the title fix: link vm2_stub in barretenberg-rs for AVM recursion constraints fix: ci-barretenberg-full mode fixes Jan 13, 2026
@johnathan79717
Copy link
Contributor Author

Re: why linking barretenberg.a isn't enough:

From barretenberg/cpp/src/CMakeLists.txt:

# For this library we include everything but the env and wasi modules, as it is the responsibility of the
# consumer of this library to define how and in what environment its artifact will run.

libenv.a is intentionally excluded from libbarretenberg.a because it provides environment-specific functions (logstr, throw_or_abort_impl, env_hardware_concurrency) that different consumers may need to implement differently (WASM vs native vs FFI).

So the current linking is:

  • libbarretenberg.a - main library (excludes env/wasi by design)
  • libenv.a - native environment implementation
  • libvm2_stub.a - AVM recursion stub
  • libstdc++ - C++ runtime (required for any C++ code)

Creating a single libbb-external.a that bundles everything is a good idea for the future, but would require CMake changes to create that combined archive.

@ludamad
Copy link
Collaborator

ludamad commented Jan 13, 2026

And why not cmake changes?

@johnathan79717 johnathan79717 force-pushed the jh/fix-bb-rs-vm2-library branch from 1c8d0da to 4e7fe0a Compare January 13, 2026 15:51
Create a single static library that bundles everything barretenberg-rs needs:
- All objects from libbarretenberg.a
- env module (logstr, throw_or_abort_impl, env_hardware_concurrency)
- vm2_stub (provides stub for create_avm2_recursion_constraints_goblin)

This simplifies linking in barretenberg-rs to just one library and
removes the need for --allow-multiple-definition and --start-group/--end-group.
@johnathan79717 johnathan79717 force-pushed the jh/fix-bb-rs-vm2-library branch from 4e7fe0a to 1afc054 Compare January 13, 2026 15:54
@johnathan79717
Copy link
Contributor Author

And why not cmake changes?

You're right. I'm not sure what I was thinking. Done.

@ludamad
Copy link
Collaborator

ludamad commented Jan 13, 2026

LGTM now :)

@johnathan79717 johnathan79717 merged commit 3854d2f into merge-train/barretenberg Jan 13, 2026
6 of 7 checks passed
@johnathan79717 johnathan79717 deleted the jh/fix-bb-rs-vm2-library branch January 13, 2026 16:03
github-merge-queue bot pushed a commit that referenced this pull request Jan 14, 2026
BEGIN_COMMIT_OVERRIDE
chore: Improve Chonk debug info (#19538)
chore: translator non-native and decomp relations audit (#19081)
chore: add safety to derive_generators and tweak pedersen scope (#19525)
fix: ci-barretenberg-full mode fixes (#19466)
test: use WASM backend for bbjs-test acir tests (#19529)
fix: use absolute path in run_test.sh for CI
fix: use env_objects in bb-external library
fix: completeness issue in cycle scalar constructor from bigfield
(#19475)
chore: review a few minor files for ultra/mega audit (#19513)
END_COMMIT_OVERRIDE
github-actions bot pushed a commit that referenced this pull request Jan 14, 2026
## Summary
Multiple fixes for `ci-barretenberg-full` mode:

1. **Link vm2_stub in barretenberg-rs** - Add static library version of
vm2_stub and always link it instead of conditionally linking vm2
2. **Propagate AVM env var** - Pass `AVM` environment variable to
isolated docker containers so tests use the correct `bb` binary
3. **Denoise barretenberg-rs** - Add `denoise` to build and test
commands for cleaner CI output

## Problem
`ci-barretenberg-full` mode was failing with multiple issues:

1. barretenberg-rs build failed with:
   ```
error: could not find native static library `vm2`, perhaps an -L flag is
missing?
   ```
This happened because `ci-barretenberg-full` builds with `AVM=0`, so
libvm2.a is not produced.

2. acir_tests failed with:
   ```
   barretenberg/cpp/build/bin/bb-avm: No such file or directory
   ```
The `AVM` environment variable wasn't being passed to isolated docker
containers, causing `find-bb` to incorrectly select `bb-avm` instead of
`bb`.

## Solution
1. Create a static library `libvm2_stub.a` from the existing vm2_stub
OBJECT library and always link it in barretenberg-rs (the stub throws at
runtime if AVM recursion is attempted)
2. Propagate `AVM` env var to isolated docker containers in
`ci3/docker_isolate`
3. Use `denoise` for barretenberg-rs build and test commands

## Caching
Verified that `libvm2_stub.a` will be included in the cache upload since
it's built to `build/lib/` and the native build cache uploads
`build/{bin,lib}`.

---------

Co-authored-by: johnathan79717 <511655+johnathan79717@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-barretenberg-full Run all barretenberg checks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants