Conversation
Global filesystem synchronization previously waited for each mapping's durable data completion before starting the next mapping. With ext4's synchronous journal requests, this repeatedly sealed tiny transactions and paid the complete journal/checkpoint barrier cost for small files. Start bounded windows using the existing asynchronous writeback budget, then request synchronous progress and drain every mapping in the window. Submit the first claimed Token batch on the sync caller's stack so its acceptance is not deferred behind the window's forced commit. Reuse the existing continuation for successors; keep Legacy and ordinary range WRITE dispatch asynchronous and preserve journal durability barriers. Retain the canonical inode and page cache across data completion and metadata writeback. Preserve the frozen range after a partial dispatch failure, sample mapping errseq before starting, and report errors only after draining the accepted work. Replace the completion scan's temporary allocation with a bounded array so memory pressure cannot interrupt drain. Add PageCache success/submission-error/deferred-error selftests and ext4 multi-window, mixed-size, concurrent-redirty, and remount coverage. Validation: - make kernel and focused rustfmt/diff checks passed. - KVM guests with both one and two vCPUs passed 46 ext4, one PageCache accounting (including kernel selftests), and four sync_file_range tests. - Repeated DragonOS repository clone/checkout/sync/fsck validation passed. - The measured checkout sync barrier count fell from about 20,500 to about 2,700 without changing the journal protocol or single-file fsync. - Three-role adversarial review found no unresolved blocking findings. Signed-off-by: longjin <longjin@dragonos.org>
Member
Author
|
@codex review |
PageCache domain completion only records the write_inode error and passes it through unchanged. Use inspect_err rather than map_err to express that side effect and satisfy the denied clippy::manual_inspect lint invoked by make fmt. Validated with make fmt, FMT_CHECK=1 make fmt (the CI command), and make kernel. The error value, superblock reporting, and writeback completion ordering are unchanged. Signed-off-by: longjin <longjin@dragonos.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix the small-file transaction amplification in filesystem-wide sync by separating bounded cross-mapping submission from completion waits.
No journal format, commit/checkpoint protocol, flush barrier, worker pool, or retry timer is changed. An inline first batch may still block in an existing backend; it is not an unbounded per-file drain or a promise of nonblocking I/O.
Root cause
The previous mount loop called a complete synchronous PageCache operation for each inode. ext4 completes those pages only after journal durability, while an active synchronous request forces newly published work to seal promptly. Small files therefore repeatedly paid a full transaction/checkpoint barrier sequence before the next mapping was started.
The new split follows Linux's
for_syncprinciple: organize writeback across mappings before the outer completion wait, while retaining metadata and filesystem durability boundaries. Partial-start errors keep their completion target instead of discarding an already-started range.Validation
make kernel, focused Rust formatting checks, andgit diff --checkpassed.file://clone, checkout, sync, andgit fsck --fullsucceeded.a4aaa794dcontrol measured checkout-following sync at 30.718 / 31.817 seconds, with 20,551 / 20,516 flushes. Four patched runs measured 5.491–7.854 seconds, with 2,659–2,716 flushes (about 87% fewer).Boundaries
These are controlled functional checks and repeated measurements, not a statistically cache/layout-normalized benchmark. Clone itself remains variable: a patched run showed a roughly 10-second post-child-exit gap, and the unmodified control also showed a roughly 5-second gap in that phase. This change does not claim to diagnose or eliminate that separate tail-latency behavior, or to accelerate every Git phase.
Power-cut testing and the full gVisor suite were not run. Remount tests are not a substitute for power-loss testing, because unmount can perform additional synchronization. The existing journal durability protocol and barriers are unchanged.