bcvk libvirt / to-disk fails with images that run quadlets at boot

[petracvv]

I am using bcvk to test bootc containers locally via libvirt and to create disk images for deployment. However both of those functions fail when the bootc image I am trying to test uses quadlets that start at boot. This is due to an embedded script in the to_disk.rs function:

bcvk/crates/kit/src/to_disk.rs

Lines 258 to 264 in 79f105b

	echo "Setting up temporary filesystems..."
	# Mount /var/tmp as a large tmpfs, then symlink /var/lib/containers to it
	# to consolidate temporary storage in one location
	mount -t tmpfs -o {TMPFS_SIZE} tmpfs /var/tmp
	mkdir -p /var/tmp/containers
	rm /var/lib/containers -rf
	ln -sr /var/tmp/containers /var/lib/containers

It tries to delete /var/lib/containers after booting the image but that directory is in use by the quadlet that is trying to start at boot.

Is this a bug in bcvk or a limitation I have to workaround? I am not familiar enough with the to_disk process yet to try to figure out a fix.

To reproduce this:

• Create a test container file:

FROM quay.io/centos-bootc/centos-bootc:stream10

COPY --chown=0:0 <<EOF /usr/share/containers/systemd/sleep.container
[Container]
Image=docker.io/library/alpine:latest
Exec=sleep 600

[Service]
# Ensures the service only runs once per boot instead of restarting
Type=oneshot
RemainAfterExit=yes

[Install]
# Triggers the container to start automatically when the system boots
WantedBy=default.target
EOF

RUN <<EOF
rm -rfv /var/roothome/buildinfo
rm -rfv /run/cloud-init /run/rhsm
EOF

RUN bootc container lint --fatal-warnings

• Build the image with: podman build -t test-quadlet
• Try to create a libvirt VM with bcvk: bcvk libvirt run --firmware uefi-insecure --graphical-console localhost/test-quadlet
• See an error message like:

❯ bcvk libvirt run --firmware uefi-insecure --graphical-console --karg 'ds=None' localhost/test-quadlet
Creating libvirt domain 'test-quadlet' (install source container image: localhost/test-quadlet)
 INFO Creating base disk: "/home/petracvv/.local/share/libvirt/images/bootc-base-b84a10fe4b78230c.qcow2"
 INFO Connected (12 seconds elapsed), beginning installation...
Setting up temporary filesystems...
rm: cannot remove '/var/lib/containers/storage/overlay-containers/860faec176acec9248ca836befa92717c00f0842c8ea7a8e8cd68f3a5c75e00b/userdata/shm': Device or resource busy
rm: cannot remove '/var/lib/containers/storage/overlay/f8775c33eda6401ea31cf595c8c7ae8d3894d83bcd06686d9bdb69206c10e72a/merged': Device or resource busy
Error:
   0: Failed to find or create base disk
   1: Failed to install bootc to base disk: "/home/petracvv/.local/share/libvirt/images/bootc-base-b84a10fe4b78230c.66JvT0.tmp.qcow2"
   2: SSH installation command failed with exit code: Some(1)

Location:
   crates/kit/src/to_disk.rs:593

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Let me know if I can test anything more.