Repository navigation
test: add model-based Linux ABI nightly coverage - #208
Conversation
Greepolis
left a comment
There was a problem hiding this comment.
Review: changes requested — the ABI module is not covered yet
The test itself is good. It drives the real linux_syscall() dispatcher and handlers on ksvc_fake. Its model of file bytes, descriptors and shared offsets is independent of the code it checks, and it never reads the kernel's state. All three sabotage cases are caught: dup resetting the offset, pread64 moving it, and lseek returning 0. CI is green on the branch after the merge of main.
The problem is the claim. The test is small; the module it removes from the coverage debt is not.
Why abi cannot leave the debt with this test
check-nightly-coverage.py counts # module: abi as coverage of all of kernel/abi/:
| Part | Lines | Covered here |
|---|---|---|
kernel/abi/linux/ |
~18,700 | 8 file/descriptor syscalls |
kernel/abi/nt/ |
~9,100 | none |
kernel/abi/files/ |
~4,100 | regular files only |
kernel/abi/mem/ |
~330 | none |
That is roughly 5% of the module. The rule in CLAUDE.md is one intensive, randomised test per module, checked against an independent model. Lowering the baseline from 3 to 2 on this test makes the ratchet green by construction: the checker sees the tag, not what the job exercises.
What is missing before abi can be counted
The model must be extended, with seeded sequences through ksvc_fake against an independent model. Each new area needs at least one production mutation that the model catches.
- Paths and directories (
kernel/abi/linux/fs.c):openatwithO_CREAT,O_EXCL,O_TRUNCandO_APPEND;mkdir,rmdir,unlink,rename(includingRENAME_NOREPLACE),linkandsymlink/readlink;getdents64checked against the model's tree;stat/fstatsizes and types;ftruncate;chdirand relative paths withdirfd.
- Descriptor table semantics:
dup3, andO_CLOEXEC/FD_CLOEXECthroughfcntl;fcntl(F_DUPFD)andclose_range;- the 1024-descriptor limit (
EMFILE).
- Other file types (
kernel/abi/files/): pipes, at least. That meanspipe2, partial reads and writes,EAGAINwithO_NONBLOCK, and EOF after the last writer closes, all against a byte-queue model. - Vector and positional I/O:
pwrite64,readv/writevandpreadv/pwritev, which share the offset rules already modelled. - Processes and descriptors:
forksharing descriptions with the child,exitreleasing them, andwait4statuses, as far as the fake supports them. If the fake cannot run these, document it in the "why it remains" table rather than leaving them out silently. - Memory (
kernel/abi/mem/, the mm rows inlinux/):mmap/munmap/mprotectranges against an interval model, includingMAP_FIXEDover an existing mapping and theEINVALcases. - NT personality (
kernel/abi/nt/): a seeded sequence ofNtCreateFile/NtReadFile/NtWriteFile/NtCloseand handle duplication against the same kind of model, with at least one mutation, such as a handle granted rights it was not asked for. Alternatively, splitabiintoabi-linuxandabi-ntin the coverage checker and keepabi-ntin the debt explicitly. Either is acceptable; counting NT as covered by a Linux file test is not.
Smaller points
- The syscall numbers are bare literals (
2,0,17,32, …). Use named constants so a reader knows which call each one is. scripts/dev/cases/abi-coverage.txtis missing a blank line before the third case.- The PR description says "no merge performed". The branch now contains a merge of
main(c7322de). - PRs #209 and #210 sit on top of this branch, and #210 says the final baseline reaches zero. Recompute both claims once ABI's scope is settled.
When this can be approved
When the model covers the areas above, or the ones left out are written down as a split module that stays in the debt, each with a mutation the model catches. The baseline may then go to 2.
vibeos_abi_torture runs seeded syscall sequences through linux_syscall() on ksvc_fake. Each result is held against a model that owns its own file bytes, directory tree, descriptors, shared offsets and mappings, and never reads the fake's state. The model covers files and descriptors (open, read, write, lseek, pread/pwrite, the vector forms, dup/dup2/dup3, fcntl, close_range, EMFILE). It covers pipes, including non-blocking writes at capacity; and paths (openat, O_APPEND, ftruncate, stat, getdents64, renameat2, chdir/getcwd, link, symlink, readlinkat). It also covers memory (anonymous and file-backed mmap, mprotect, munmap) and fork (descriptor inheritance, shared offsets, copy-on-write isolation). Each area has production mutations in scripts/dev/cases/abi-*.txt, with verify-abi-torture.sh as their verifier. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A seeded sequence of NtCreateFile, NtReadFile, NtWriteFile, NtDuplicateObject and NtClose runs against a model of the file's bytes and the shared position: a duplicate names the same description, closing the original leaves it working, and explicit and current offsets interleave. Mutation cases cover create, duplicate, read, write and close; verify-nt-abi-torture.sh is their verifier. The model's buffer is taken after fresh(). kf_reset() starts the fake's user memory again from its first byte, so a buffer taken before it was handed out again to the status block and the ByteOffset. That read as NtWriteFile changing its input buffer. Fixes #211 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
verify-module-mutations.sh runs the Linux and NT ABI cases against their own verifiers, not through the IPC/object torture executable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A nightly job runs the Linux ABI model under ASan and UBSan on many seeds. The module-coverage workflow runs the short models and their mutations when kernel/abi or the fake kernel changes. The ABI module stays in the nightly coverage debt: the model is scoped to the areas above, not the whole of kernel/abi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6cd6236 to
3b848ea
Compare
Greepolis
left a comment
There was a problem hiding this comment.
Re-review: ready to merge, with one wording fix
What changed since the last review
The earlier review asked for one of two outcomes: cover the module, or write the uncovered parts down and keep abi in the debt. The PR now takes the second path, and does it honestly:
- the baseline stays at 3, and
abiremains one uncovered module; module_torture.mdandcheck-nightly-coverage.pysay plainly that most ofkernel/abi/linux/andkernel/abi/nt/is not modelled.
The 17 new commits add real value:
- MAP_FIXED. Replacing an existing page must give zero-filled memory, not keep the old page's bytes. An unaligned fixed address and a mapping with no type are now rejected with
EINVAL. - Legacy pathname wrappers.
mkdir,unlinkandreadlinkare now exercised as table entries of their own, each with a mutation. - NT handle rights. There is a mutation for them, run against the NT suite.
- NT verifier fix. The NT mutations now run
verify-nt-abi-torture.sh. Before, they ran the Linux torture, which cannot reach NT code, so those cases could not go red for the right reason.
Verification
I merged the PR with current main (11 commits ahead, which change nt/file.c, futex.c, timer.c and ext2.c) and ran:
| Check | Result |
|---|---|
Merge with main |
clean, no conflicts |
check-sabotage-anchors.py |
ok, nothing unresolved |
check-nightly-coverage.py |
ok (baseline 3) |
ctest |
16/16 pass |
verify-module-mutations.sh |
rc 0, 63 cases across 15 files, every one red, every file restored |
CI on the branch is green on all 18 checks.
One point to fix before merging
module_torture.md says the anonymous mappings are "tracked by an independent interval model". The interval set in test_memory_model() is updated by the test next to each call, and vm_model_access() asks the model, never the kernel. The checks that constrain the kernel are still the per-page kf_peek/kf_poke calls the scenario already made. As written, the model cannot disagree with the kernel, so it is not an oracle yet.
Either of these closes the point:
- After each operation, walk every page of the range and require
kf_peekto succeed exactly where the model says a page is mapped, andkf_pokeexactly where it says the page is writable. The model then becomes a real oracle. - Or reword the sentence to say the interval set documents the expected layout, without claiming that it checks the kernel.
Option 1 is better: it is a small loop and it turns the model into evidence.
Minor
- The branch is 11 commits behind
main. It merges cleanly and passes, so this needs no action.
The interval set was updated by the test next to each call and asked only about itself, so it could never disagree with the kernel (#208 review). After every map, protect and unmap, each page of the range and the one past it must now read exactly where the model maps it and take a write exactly where the model says it is writable. The write probe stores back the byte it read. A new mutation makes mprotect narrow the page before its range. The old checks missed it, because the next mprotect restores that page before anything writes to it. The cross-check catches it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scope
Adds a replayable Linux ABI syscall test on
ksvc_fake, using the reallinux_syscall()dispatcher and handlers with independent reference models. The ABI module remains in the uncovered debt: the current tests cover useful slices, not the wholekernel/abi/tree.Modelled slices
open,read,write,lseek,pread64,pwrite64,dup,dup2,close; the oracle owns file bytes, open-description references and shared offsets.dup3,fcntl(F_DUPFD),F_DUPFD_CLOEXEC,F_GETFD/F_SETFD,O_CLOEXEC,close_range, the 1024-descriptor limit andEMFILE, including first-free-slot reuse.pipe2, queue ordering, partial reads, capacity-limited nonblocking writes,EAGAIN, and EOF after the last writer closes.readv,writev,preadv,pwritev, with checks that positional operations leave the shared offset unchanged.mmaptracked by an independent interval model,MAP_FIXED_NOREPLACE,MAP_FIXEDreplacement of an existing page, unaligned-address and missing-mapping-typeEINVALchecks, independent page contents,mprotectread-only enforcement and restoration, partialmunmap, file-backedMAP_PRIVATEcopy isolation, tmpfsMAP_SHAREDpage aliasing in both directions, and access checks against the real fake page tables.wait4(WNOHANG)while the child is live, private-page COW isolation after a child write, and survival of the child's mapping after the parent'smunmap.mkdir/mkdirat,unlink/unlinkat, legacyreadlink/readlinkat,chdir/getcwd(includingERANGEand cwd-relative lookup), relativeopenatthrough a directory descriptor,O_CREAT,O_EXCL,O_TRUNC,O_APPEND,linkat(shared inode/link count, directory refusal and cross-mountEXDEV),symlinkat/readlinkat(raw target bytes, truncation, follow/no-follow behavior),renameat2(RENAME_NOREPLACE), nested two-level directories with relative dirfds, nestedgetdents64/newfstatat,ENOTEMPTYrefusal and cleanup,unlinkat, directory removal, one-directorygetdents64enumeration,ftruncate,fstatand relativenewfstatatsize/type checks.The models compare returned values and user-visible bytes/metadata against state they own; they do not read the fake filesystem or kernel descriptor table.
Production mutations
The Linux model is paired with production mutations for
lseek,dupshared offsets,pread64/pwrite64position preservation,pipe2nonblocking mode,dup3andF_DUPFD_CLOEXECflags,F_SETFD,close_range,EMFILE,O_APPEND,ftruncate, stat size,getdents64enumeration, therenameat2no-replace result,chdirworking-directory updates,linkat,symlinkat,readlinkatand the legacy pathname wrappers,mkdirat,unlinkat, anonymous and file-backed private/sharedmmapincluding fixed replacement and mapping-type validation,mprotect,munmap, fork descriptor-table copying and fork COW-page marking. A separate NT dispatcher suite (nt_syscall, not the Linux selector) models handle close/duplication and seeded file read/write sequences; its mutations coverNtClose,NtDuplicateObject,NtReadFile,NtWriteFile,NtCreateFiledispositions and per-handle access-right enforcement, and each runs against the NT host-test suite.Remaining ABI debt
Not yet modelled: most Linux ABI syscall handlers, full directory-tree operations and more complex rename cases, broader file types, child exit/reap, exit-time descriptor release and the portable process table (the fake's
ks_task_exitonly escapes and does not model zombie/reap lifecycle),brk/mremapand the other memory handlers, and most of the NT personality. NT has a separatent_syscallhost-test harness, with a limited handle/file model and targeted mutations, but that does not cover the rest ofkernel/abi/nt/. Unsupported areas remain documented as uncovered. Each newly claimed area needs a production mutation caught by its model.The ABI nightly job remains enabled as a useful scoped test, but carries no module-coverage annotation. The checker baseline remains 3 (ABI, PROC and TXN).
Stacked follow-ups
Branch safety
test/abi-intensive-coverage-201c7322de3frommain.main; no changes from this branch have been applied tomain.