Implement `bootc image copy-to-storage` for composefs backend #1890

Johan-Liebert1 · 2026-01-06T10:23:51Z

Export a composefs repository as an OCI image. This would be helpful for running tests that we have for ostree backend. Tests for sealed UKI would be separate, I think, but tests for Type1 entries can be shared between the two backends

cgwalters

Awesome, thanks for working on this! Nothing truly blocking, but some cleanups

crates/lib/src/bootc_composefs/export.rs

cgwalters · 2026-01-06T22:31:36Z

crates/lib/src/bootc_composefs/export.rs

+
+    let size = header.entry_size()?;
+
+    let item = match reader.read_exact(size as usize, ((size + 511) & !511) as usize)? {


This seems to be relying on size = 0 for anything except regular files. I think we should probably have at least an anyhow::ensure! for that.

cgwalters · 2026-01-06T22:35:29Z

crates/lib/src/bootc_composefs/export.rs

+    reader: &mut SplitStreamReader<R, ObjectID>,
+) -> anyhow::Result<Option<(Header, TarItem<ObjectID>)>> {
+    let mut buf = [0u8; 512];
+    if !reader.read_inline_exact(&mut buf)? || buf == [0u8; 512] {


Also isn't there a bigger problem in that iterating by raw headers in this way means we aren't handling e.g. GNU long links and PAX extended headers?

I did run into the truncation problem when using the parsing code in composefs-rs (hence the copying of part of the code here). The PAX and GNU extended headers are just passed through

Which parsing code? You mean https://github.com/containers/composefs-rs/blob/1498349e11d0e4bd545feab533f0b7ab5294ae9a/crates/composefs-oci/src/tar.rs#L202 ?

Yes, that's the one

Ah, I think the logic we have here should replace that in get_entry ideally...there's no reason AFAIK for us to synthesize a new tarstream.

Actually though in this case there's the cat function which I think already does what we want in just giving us back the raw byte stream.

So: I think all of this just be replaced by reader.cat(layer_writer, || repository.load_object())

Though note the cat implementation has the same suboptimal "read entire file into memory" thing but that's fixable.

I think the issue lies with GNU long names and PAX headers which need to be parsed and then reconstructed to the full path. That could be inlined, but it's a bit cleaner as a separate struct

crates/lib/src/bootc_composefs/export.rs

cgwalters

Looks like this is failing CI by breaking the ostree-based path here, though it's not at all obvious to me why that is

cgwalters · 2026-01-07T14:19:41Z

crates/lib/src/bootc_composefs/repo.rs

    let transport = transport.strip_suffix(":").unwrap_or(&transport);

-    if transport == "registry" {
+    if transport == "registry" || transport == "docker://" {


Unrelated to this change but it'd be cleaner to parse this back into the Transport enum which already handles this canonicalization

Johan-Liebert1 · 2026-01-07T15:27:23Z

The tests seem to be timing out, but I can't find the actual error. Weird. I'll try it locally once

Export a composefs repository as an OCI image. In this iteration the outputted files are in OCI Directory format and are plain TARs, i.e. not compressed Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>

Instead of handling the history,metadata,annotations ourselves, delegate them to `ocidir` crate. Also take into account the source and target image references Finally call `skopeo::copy` to copy to containers-storage Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>

Inline the tar parsing/unpacking Check for two NULL 512 blocks instead of just one Share source image and target image generating code between composefs and ostree Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>

The sysroot lock was being taken by `get_host` before it was released by the caller. Move the `get_host` function up the stack of calls Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>

cgwalters · 2026-01-08T15:18:14Z

BTW not a big problem but something I noticed post-merge: my personal preference is to squash critical fixes to prior commits. "critical" has a sliding scale from "compiles" to "unit tests pass" to "integration tests pass" and beyond that of course to "docs are coherent" etc.

In this case the earlier commit didn't pass our integration tests, and I think that argues for squashing.

Johan-Liebert1 · 2026-01-09T09:46:37Z

Ah, my bad, it would've been better to squash. I'll keep this in mind

Johan-Liebert1 requested a review from cgwalters January 6, 2026 10:23

github-actions bot added the area/ostree Issues related to ostree label Jan 6, 2026

Johan-Liebert1 added area/composefs Issues related to composefs and removed area/ostree Issues related to ostree labels Jan 6, 2026

Johan-Liebert1 force-pushed the copy-to-storage branch from 1d344dc to 72d7942 Compare January 6, 2026 10:45

github-actions bot added the area/ostree Issues related to ostree label Jan 6, 2026

Johan-Liebert1 removed the area/ostree Issues related to ostree label Jan 6, 2026

cgwalters requested changes Jan 6, 2026

View reviewed changes

github-actions bot added the area/ostree Issues related to ostree label Jan 7, 2026

Johan-Liebert1 requested a review from cgwalters January 7, 2026 09:39

cgwalters requested changes Jan 7, 2026

View reviewed changes

Johan-Liebert1 added 3 commits January 8, 2026 10:29

composefs: Implement bootc image copy-to-storage

813da0c

Export a composefs repository as an OCI image. In this iteration the outputted files are in OCI Directory format and are plain TARs, i.e. not compressed Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>

copy-to-storage: Refactor tar parsing

4b7c12d

Inline the tar parsing/unpacking Check for two NULL 512 blocks instead of just one Share source image and target image generating code between composefs and ostree Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>

Johan-Liebert1 force-pushed the copy-to-storage branch from 04aaa2e to 92ec678 Compare January 8, 2026 08:02

copy-to-storage: Fix deadlock in ostree path

c83639d

The sysroot lock was being taken by `get_host` before it was released by the caller. Move the `get_host` function up the stack of calls Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>

Johan-Liebert1 force-pushed the copy-to-storage branch from 92ec678 to c83639d Compare January 8, 2026 08:23

Johan-Liebert1 requested a review from cgwalters January 8, 2026 11:04

cgwalters enabled auto-merge (rebase) January 8, 2026 13:37

cgwalters approved these changes Jan 8, 2026

View reviewed changes

cgwalters merged commit fd7f1b6 into bootc-dev:main Jan 8, 2026
63 of 65 checks passed


		let size = header.entry_size()?;

		let item = match reader.read_exact(size as usize, ((size + 511) & !511) as usize)? {

Implement bootc image copy-to-storage for composefs backend #1890

Implement bootc image copy-to-storage for composefs backend #1890

Uh oh!

Conversation

Johan-Liebert1 commented Jan 6, 2026

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Johan-Liebert1 commented Jan 7, 2026

Uh oh!

Uh oh!

cgwalters commented Jan 8, 2026

Uh oh!

Johan-Liebert1 commented Jan 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Implement `bootc image copy-to-storage` for composefs backend #1890

Implement `bootc image copy-to-storage` for composefs backend #1890