Skip to content

fix(net): preserve TCP state across local address migration - #2323

Merged
fslongjin merged 3 commits into
DragonOS-Community:masterfrom
fslongjin:codex/fix-ngc011-tcp-namespace-ownership
Sep 23, 2026
Merged

fslongjin merged 3 commits into
DragonOS-Community:masterfrom
fslongjin:codex/fix-ngc011-tcp-namespace-ownership

Conversation

@fslongjin

@fslongjin fslongjin commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Summary

Fix TCP connections and listeners becoming unreachable when a local IP address moves between devices (NGC-011).

TCP state previously belonged to each interface's SocketSet, while ingress followed the current address owner. Moving an IP redirected incoming segments to a different socket table, stranding established connections and TIME_WAIT state.

Design

  • Own a TCP-only protocol domain in NetNamespace; leave UDP/raw sockets and link/IP admission on interfaces.
  • Queue validated TCP input with its original ingress device metadata and bounded packet/byte capacity. Reuse the existing tuple, listener, device binding and TIME_WAIT matching.
  • Route TCP output through the existing FIB, MTU, neighbor and transmit-admission paths. Do not use a synthetic netdev or make transport lifetime depend on loopback configuration.
  • Schedule transport progress and real protocol/close deadlines through the existing namespace poller. Bound syscall assistance so unrelated traffic cannot prevent nonblocking operations from returning.
  • Preserve connecting publication/cancellation and deferred close ownership; remove obsolete per-interface TCP bookkeeping.

Integration tests also exposed two existing socket-layer gaps that the asynchronous transport must handle correctly: finish successful nonblocking connections without requiring a first send, and preserve Linux shutdown receive/event semantics. Receive shutdown no longer freezes a byte quota; complete event masks are published from a single locked transport snapshot.

Dependency

smoltcp #34 has merged into dragonos/v0.12.0. The dependency is pinned to merged commit 2afecd80594f324cc81b803cf9fc457b564a311f; its source tree is identical to the previously validated dependency commit.

Validation

  • Follow-up integration CI fixes: keep asynchronous handshake completion separate from userspace connect confirmation, and honor logical listen backlog with lazily allocated full-size transport buffers. Serialize transport input preparation without holding protocol locks across socket callbacks; trim obsolete slots before rearming after backlog reduction.

  • Final QEMU guest: unmodified complete gVisor tcp_socket_test passes 184/184, including all six previously failing integration cases. New connect-confirmation (12) and backlog (5) tests pass on both Linux and DragonOS. Existing targeted networking and nginx regressions below were rerun successfully. No reference tests or timeouts were weakened.

  • Reproduced established IPv4 connection failure on the unmodified baseline after moving the destination IP; the same probe passed on Linux and passes with this change.

  • make kernel and make fmt.

  • The repository dunitest make build-suites succeeds. Both new migration and connection-completion executables provide their own entry point, as required by the Makefile's gtest-all.o link rule. Their actual Makefile-built binaries and the namespace-progress binary pass 15 tests in an isolated Linux network namespace.

  • QEMU/KVM guest: address migration (10), nonblocking connection completion (4), self-connect/shutdown (18), namespace progress under background traffic (1), port lifetime (31), and other networking regressions (152).

  • nginx: IPv4/IPv6 HTTP and 10 graceful shutdown/restart cycles.

  • TIME_WAIT expiry with both retained and closed descriptors, without shortening the protocol timeout.

  • smoltcp library tests: 677 passed.

  • Independent design review and two rounds of logic, security/concurrency and system/performance review; findings addressed and rechecked.

The existing custom self-connect progress test now explicitly waits for data to be queued before testing SHUT_RD, preserving its byte/content/EOF assertions. Additional tests cover data arriving after SHUT_RD. No gVisor reference tests were weakened.

Scope and limitations

This changes TCP ownership, not arbitrary transport protocols or the entire routing model. Moving an IP does not retarget SO_BINDTODEVICE. Input/output queues remain bounded; egress still scans the socket set, so a finite polling batch is not a constant-time CPU guarantee. Validation covers targeted guest workloads rather than claiming exhaustive absence of regressions.

Move TCP protocol ownership from interface SocketSets to one TCP-only domain per network namespace. Retain interface IP admission and device metadata, bounded ingress queues, route-specific MTU and the existing neighbor/transmit admission paths. Keep UDP and raw socket ownership unchanged.

Use namespace transport deadlines and finite syscall polling batches. Preserve connection registration cancellation and deferred close ownership, wake the worker when the final socket reference disappears, and reap compact TIME_WAIT with its original port protection.

Complete successful nonblocking Connecting states through the common event path so first peer data and FIN are observable without a preceding send. Match Linux receive shutdown semantics by removing frozen byte quotas and publish complete shutdown event masks atomically from a locked transport snapshot.

Pin the validated public smoltcp transport-ingress revision. Add migration, connection completion, namespace progress and post-SHUT_RD regression coverage; retain self-connect byte assertions with explicit receive-readiness setup.

Validation: make kernel and make fmt; 216 guest regression tests; nginx IPv4/IPv6 requests over 10 graceful restart cycles; real 60-second TIME_WAIT expiry with open and closed descriptors; 677 smoltcp tests. Design and final implementation reviewed independently for logic, concurrency/security and system/performance.
Signed-off-by: longjin <longjin@dragonos.org>
@github-actions github-actions Bot added the Bug fix A bug is fixed in this pull request label Sep 23, 2026
@fslongjin

Copy link
Copy Markdown
Member Author

@codex review

Add explicit GoogleTest main functions to the migration and connection-completion suites. The dunitest Makefile links each source with gtest-all.o and therefore requires its own entry point. Standalone validation had linked gtest_main.cc, masking the missing entry points and causing all three x86_64 CI build jobs to fail before guest tests could run.

Update the smoltcp pin and lockfile to the merged PR DragonOS-Community#34 commit 2afecd80594f324cc81b803cf9fc457b564a311f. Its source tree is identical to the previously validated dependency revision.

Reproduced the linker failure with the repository Makefile, then verified the full build-suites target and 15 tests from the actual Makefile-built executables in an isolated Linux network namespace. make kernel, make fmt and independent patch review pass.

Signed-off-by: longjin <longjin@dragonos.org>
@fslongjin

Copy link
Copy Markdown
Member Author

@codex review

Separate successful TCP transport establishment from userspace connect confirmation. Keep readiness and ordinary I/O from consuming that confirmation, and restore retryable socket state when an unconfirmed connection has closed.

Replace the eight-slot listener cap with logical backlog capacity and on-demand full-size buffer allocation before validated SYN processing. Serialize input progression without holding protocol locks during listener preparation, and trim excess slots before rearming after backlog reduction.

Add mandatory IPv4/IPv6 connect confirmation and backlog regressions. Verify all 184 gVisor TCP tests, 17 new tests, existing network and nginx regressions in DragonOS, plus Linux reference execution, kernel build and formatting.

Signed-off-by: longjin <longjin@dragonos.org>
@fslongjin

Copy link
Copy Markdown
Member Author

@codex review

@fslongjin
fslongjin merged commit c917a92 into DragonOS-Community:master Sep 23, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug fix A bug is fixed in this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant