Report how much room is left in a directory reply - #717
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1e5db0102
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| /// when [`add()`](Self::add) reports the buffer full. Each entry costs its name plus a | ||
| /// fixed header, padded for alignment, so this is a budget rather than an entry count; | ||
| /// `add()` remains the authority on whether a particular entry fits. | ||
| pub fn remaining_capacity(&self) -> usize { |
There was a problem hiding this comment.
Forward capacity through the async readdir builder
For filesystems implemented via experimental::AsyncFilesystem, this accessor is not reachable: TokioAdapter wraps the ReplyDirectory in DirEntListBuilder, whose public API only forwards add() (src/experimental.rs:70-87). Those users are still a readdir call path but cannot see the kernel budget, so the new capacity feature only works for synchronous Filesystem::readdir; please add a forwarding remaining_capacity() on the builder as well.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Correct, and I had missed that path entirely. DirEntListBuilder holds &mut ReplyDirectory but re-exports only add(), and AsyncFilesystem::readdir receives the builder rather than the reply, so the accessor was unreachable for async filesystems -- the feature would have shipped working on one of its two call paths.
Fixed in c54597d with a forwarding remaining_capacity() on the builder. I checked whether readdirplus needed the same treatment: the experimental API has no async readdirplus, so readdir is the only path that needed it.
Rather than trust that a pub method on a pub struct is reachable, I confirmed it from the outside -- a temporary builder.remaining_capacity() call in examples/async_hello.rs compiles clean under --features=experimental, and the example is reverted.
One thing I ran into while verifying, unrelated to this PR: cargo clippy --all-targets --all-features fails on AsyncFilesystem::read with too many arguments (9/7). It reproduces on a clean checkout of this branch's base, and this PR does not touch that trait. It stays hidden because make pre runs clippy with --all-targets and --no-default-features but never --all-features, and mac-ci runs an all-features build rather than clippy. Worth its own issue if you want the experimental module linted.
Generated by Claude Code
The size the kernel asks for in readdir and readdirplus was used only to size the reply buffer and never reached the filesystem. Paging already worked, since add() refuses an entry that does not fit, but a filesystem could only discover the limit by hitting it: work spent building the refused entry is wasted, which matters for readdirplus, where every entry carries a full set of attributes that may have to be fetched. Expose the remaining room on both reply types, and on the builder the experimental async API hands to readdir, so that work can be sized to the budget up front on either path. It is reported as bytes rather than an entry count because an entry costs its name plus a fixed header, padded for alignment; add() stays the authority on whether a given entry fits. Fixes #214 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjYWzn3mu8Sc2y2eACtJM9
b1e5db0 to
c54597d
Compare
Fixes #214.
What the issue gets right, and what it doesn't
The issue asks for the readdir/readdirplus buffer size, on the grounds that "there's no proper way to page the calls". The first half holds: the kernel's
sizeis passed toReplyDirectory::new()to size the buffer and never reaches the filesystem.The second half does not.
push()returnstruewithout adding when an entry does not fit (src/ll/reply.rs:401), so loopingadd()until it returns true already pages correctly. Confirmed against a real mount: a 2000-entry directory is served in 5 calls of 585 entries and all 2000 names are listed.What is actually missing is knowing the budget before doing the work. Looping on
add()wastes exactly one entry's preparation per call -- about 0.2% for plain readdir, which is not much. It matters more for readdirplus, where every entry carries a full set of attributes that may have to be fetched, and for a filesystem that fetches entries from a remote source in batches and wants to size the batch to the reply.So this exposes the remaining room rather than the raw
size.The change
remaining_capacity()onReplyDirectoryandReplyDirectoryPlus. This follows the shape suggested in the issue thread rather than adding a parameter to the trait methods: it is a pure addition with no API break, and it is strictly more informative, since it tracks downward as entries are added instead of only reporting the total.It is reported in bytes rather than as an entry count because an entry costs its name plus a fixed header, padded for alignment. The docs say so and point at
add()as the authority on whether a particular entry fits, so nothing here promises byte-exact prediction.Testing
Three unit tests: the budget starts at the size the kernel asked for, decreases as entries are added, and agrees with
add()(entries keep fitting while room remains, and the entry finally refused did not fit in what was left). A readdirplus test covers the much larger per-entry cost there.Verified end to end against a real mount, using the new accessor to size the batch up front:
The budget matches the
sizethe kernel sends, and planning from it hits the limit exactly (585 x 56 = 32760, 8 bytes left).cargo test --all(78 lib tests),fmt,clippy --all-targetsandclippy --no-default-featuresclean underRUSTFLAGS=--deny warnings;x86_64-apple-darwinchecks clean.Generated by Claude Code