Skip to content

fix(block): make virtio BIO backpressure event driven - #2311

Merged
fslongjin merged 1 commit into
DragonOS-Community:masterfrom
fslongjin:codex/virtio-blk-event-backpressure
Sep 21, 2026
Merged

fslongjin merged 1 commit into
DragonOS-Community:masterfrom
fslongjin:codex/virtio-blk-event-backpressure

Conversation

@fslongjin

Copy link
Copy Markdown
Member

Summary

Replace VirtIO BIO budget cooldown with event-driven descriptor backpressure, and close related ownership/publication gaps.

  • Limit dequeue to the worker's remaining budget. A partially filled earlier batch must not cause a later batch's tail to be dropped.
  • Treat unpublished QueueFull as retryable, retaining the current BIO in order. Reuse WaitQueue condition checking rather than adding an epoch or descriptor-count mirror.
  • Wake resource waiters after descriptor reclamation and before callbacks. Shutdown wakes them after publishing non-Online and before joining the worker.
  • Keep default fair scheduling, retain bounded work rounds and use cond_resched(), removing the fixed 20 ms sleep and FIFO override.
  • Serialize submission, token-context publication and Submitted state against completion on other CPUs with the existing device lock.
  • Keep callbacks outside the device lock and explicitly retain DMA context on a fatal duplicate-token invariant failure, since panic may unwind.

No journal, flush ordering, VirtQueue size, rate-limiting framework or public syscall ABI changes. The optional diagnostic counter queue_full_retries counts failed admission attempts, not failed BIOs. budget_hits now counts exhausted work rounds, not timed cooldowns.

Before/after evidence

Check Before After
Production BioQueue, uneven batches Dequeued 33 with budget 32 Dequeues 32, leaves one queued in FIFO order
32 independent readers, 64 LBA0 reads each QueueFull errors and approximately 60-second timeout Five rounds pass on two vCPUs, five on one vCPU
Resource-pressure coverage QueueFull becomes EIO Retries observed; errors, short completions and remaining in-flight BIOs are zero

The guest regression is read-only and verifies stable LBA0 contents. Each reader opens its own file description to avoid VFS per-file serialization. It skips explicitly when the VirtIO block device is unavailable and is included in the dunitest whitelist.

Validation

  • make kernel with diagnostic statistics enabled: pass.
  • Static dunitest build: pass.
  • Host production-queue tests: 2/2, covering budget ownership, FIFO, zero limit, quiescing, stopping and rejected submissions.
  • QEMU two-vCPU read test: 5/5, approximately 68–95 ms per round, with 8 QueueFull admission retries.
  • QEMU one-vCPU read test: 5/5, approximately 76–95 ms per round, with 68 retries. A concurrent normal CPU workload also finishes.
  • Both configurations reach peak in-flight 32 and finish with submits equal to completes, zero I/O errors and zero in-flight requests.
  • Existing guest ext4 inode suite: 44/44.
  • Git full object checking, local up-to-date pull and sync: pass.
  • A 32 MiB file write/fsync probe on the guest root filesystem: pass.
  • Three-role adversarial review completed; one panic-unwind DMA lifetime finding was fixed and rechecked.

Host test command:

rustc --edition=2021 --test tools/block/test_bio_queue.rs -o /tmp/test-bio-queue
/tmp/test-bio-queue

Limits

The host harness substitutes synchronization primitives and validates actual queue logic, not kernel scheduling. The guest tests do not exhaustively inject hot-removal races, media errors or all wakeup interleavings. The single-core CPU workload is a progress smoke test, not a fairness or tail-latency guarantee. This PR does not claim to remove the remaining ext4 small-file synchronization cost.

The VirtIO BIO worker dequeued up to 16 requests even when fewer requests
remained in its 32-request budget. Breaking the consuming iterator at the
budget boundary could discard unpublished BIOs. Hardware QueueFull was
also converted to a final EIO instead of retaining the request. A fixed
20 ms cooldown after each full round neither prevented queue exhaustion
nor responded to descriptor reclamation.

Bound each dequeue by the remaining work budget and process the entire
returned batch. Retain the current BIO on QueueFull and retry through a
device-local WaitQueue whose condition attempts submission. Completion
reclamation wakes the queue before invoking callbacks; shutdown publishes
non-Online and wakes it before joining the worker. Registration followed
by condition rechecking closes the wake-before-sleep race without a
duplicate capacity counter or completion epoch.

Keep the worker in the default fair scheduling class and use cond_resched
at bounded work-round boundaries instead of FIFO plus a fixed cooldown.
Idle waits remain event driven. Preserve the journal, flush protocol and
VirtQueue size; add an optional QueueFull retry diagnostic counter.

Hold the device lock through hardware submission, token-context insertion
and Submitted-state publication so another CPU cannot consume a completion
before its context exists. Keep callbacks outside that lock. On the fatal
duplicate-token invariant path, retain the exposed DMA context explicitly
because kernel panic can unwind rather than halt the whole machine.

Add a lightweight host test including the production BioQueue and a
whitelisted, read-only dunitest using independently opened block-device
descriptions. The test checks completion and contents without writing the
raw disk, and bounds its controller if a BIO becomes stuck.

Validation:
- Before: deterministic dequeue test took 33 requests for budget 32;
  guest concurrent reads reported QueueFull errors and timed out.
- After: two host queue tests pass, including FIFO, zero budget and stop.
- Kernel and static dunitest builds pass.
- Guest concurrent reads pass five rounds on two vCPUs and five on one;
  QueueFull retries are exercised with zero errors or remaining BIOs.
- Full guest ext4 inode suite passes 44/44; Git object checking, local
  up-to-date pull and a 32 MiB root-filesystem sync probe pass.
- Three-role adversarial review completed; the DMA unwind finding was
  fixed and rechecked.

Hot-removal races and strict scheduler tail-latency bounds were not
exhaustively fault-injected by this validation.

Signed-off-by: longjin <longjin@dragonos.org>
@github-actions github-actions Bot added Bug fix A bug is fixed in this pull request test Unitest/User space test labels Sep 21, 2026
@fslongjin

Copy link
Copy Markdown
Member Author

@codex review

@fslongjin
fslongjin merged commit a4aaa79 into DragonOS-Community:master Sep 21, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug fix A bug is fixed in this pull request test Unitest/User space test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant