Erofs overlay patches - #3042
Conversation
The read-only root filesystem of an overlayroot image can be either squashfs or erofs, selected via overlayroot_readonly_filesystem. Both overlay code paths in DiskBuilder (the read-only partition size estimation and the actual root sync), however, hard-coded get_squashfscompression() when constructing the FileSystem, so the erofscompression attribute was silently ignored for an erofs overlay root. As a consequence an erofs overlay base was always written uncompressed regardless of erofscompression -- e.g. an Amazon Linux 2023 root grew to ~1.4G instead of a few hundred MB. PR OSInside#2648 fixed the same class of bug (OSInside#2647) for the standalone erofs filesystem builder, but the overlayroot disk-builder path was missed. Route the read-only filesystem compression through a small helper (_get_root_readonly_compression) that returns get_erofscompression() for an erofs read-only type and get_squashfscompression() otherwise, so both attributes are honored consistently. Since erofscompression is free-form text this also allows requesting e.g. "lz4hc,level=12". Refs: OSInside#2647, OSInside#2648 Assisted-by: Kiro:claude-opus-4.8
The tool that builds the overlay read-only root filesystem (mkfs.erofs or mksquashfs) accepts options beyond the compressor that materially affect image size -- most notably the erofs physical cluster size (mkfs.erofs -C), which widens the compression window. There was no way to pass such options from the image description: fscreateoptions targets the read-write root filesystem, not the overlay read-only layer. Add an overlayroot_readonly_createoptions attribute (free-form, whitespace-split, mirroring fscreateoptions) and pass it as create_options to both overlay read-only FileSystem builds. An image description can then request e.g. overlayroot_readonly_createoptions="-C 1048576" for a larger erofs pcluster which, together with erofscompression="lz4hc,level=12" (see previous commit), noticeably shrinks the read-only base without relying on an external mkfs.erofs wrapper. Updates the schema (rnc/rng), the generated xml_parse.py, a new XMLState.get_fs_readonly_create_option_list() accessor, and a unit test. Assisted-by: Kiro:claude-opus-4.8
There was a problem hiding this comment.
Thanks much for looking into this.
I don't think we need a new attribute to differentiate compression options for erofs between different image layouts. The schema already provides k.type.erofscompression.attribute. So it should be possible to utilize this in the same way as it was done for builder/live.py and builder/filesystem.py. Just add the missing part to builder/disk.py as you you did but without the need for a new attribute in the schema
Thoughts ?
under which condition you would need to specify both for one image type ? |
|
From my perspective you only need to apply the code we have in the other builders to the at several places |
mkfs.erofs carves out I am not seeing a way to express that with today's schema, am I missing it? |
|
@martinp26 ok, sorry I was on the wrong track. Yes makes perfect sense as we are using All good, sorry for the noise |
|
I will build Staging packages at https://build.opensuse.org/project/show/Virtualization:Appliances:Staging |
|
Thanks! |
disk: honor erofscompression and allow create options for the overlay read-only root fs
Issue#3038
Issue#3039
What
Two commits fixing/extending compression handling for the
overlayrootread-only filesystem:
disk: honor erofscompression for overlay read-only root filesystem
The overlay read-only fs can be
squashfsorerofs(
overlayroot_readonly_filesystem), but both overlay code paths inDiskBuilderhard-codedget_squashfscompression(), soerofscompressionwas ignored and an erofs overlay base was writtenuncompressed. Routes the read-only compression through a small helper
(
_get_root_readonly_compression) that returnsget_erofscompression()for an erofs read-only type and
get_squashfscompression()otherwise.Complements Add missing erofscompression validation #2648 (which fixed only the standalone erofs builder).
disk: allow extra create options for the overlay read-only root fs
Adds a new free-form
overlayroot_readonly_createoptionsattribute(whitespace-split, analogous to
fscreateoptions) passed ascreate_optionsto both overlay read-onlyFileSystembuilds. Thislets a description request e.g.
overlayroot_readonly_createoptions="-C 1048576"for a larger EROFSpcluster. Updates schema (rnc + rng), the generated
xml_parse.py,XMLState.get_fs_readonly_create_option_list(),disk.py, docs andtests.
Why
Building an AL2023
overlayrootEROFS image,erofscompressionhad noeffect (uncompressed ~1.4 GB base). With these two changes the recipe can
use
erofscompression="lz4hc,level=12"+overlayroot_readonly_createoptions="-C 1048576"directly.Testing
make check— flake8 clean (shell unchanged)make test—mypy kiwiclean; full unit suite passes at 100%coverage (new code covered; existing overlay test updated)
make docs— sphinx html builds, new attribute documentedtrangreproduceskiwi.rngbyte-for-byte from the edited
kiwi.rnc; the generatedxml_parse.pyadditions match the generateDS output pattern
Assisted-by: Kiro:claude-opus-4.8