Skip to content

feat(router): configurable relay circuit limits - #498

Merged
aojea merged 1 commit into
google:mainfrom
kaisoz:kaisoz/bug-router-relay-limits
Sep 24, 2026
Merged

aojea merged 1 commit into
google:mainfrom
kaisoz:kaisoz/bug-router-relay-limits

Conversation

@kaisoz

@kaisoz kaisoz commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

The router started circuit-relay-v2 with go-libp2p's default resources, so every relayed connection was reset after 2 minutes or 128 KiB per direction. On a mesh where nodes only reach each other through the relay (every node behind NAT, e.g. sam-one on Cloud Run), long blocking calls and large transfers died mid-flight.

  • sam-router: --relay-limit-duration (default 1h) and --relay-limit-data (default no limit; accepts sizes like 512MiB). 0 means no limit.
  • sam-one: --router-relay-limit-duration / --router-relay-limit-data; 0 keeps the component default, like the other --router-* tunables.
  • go-libp2p only treats a nil limit as unlimited and a zero field cuts every circuit immediately, so relayLimit maps a single 0 to the maximum value.
  • Tests: the mapping and size parsing, plus an in-process circuit test showing a data cap cuts the circuit and a duration-only limit does not.

Part of #483

The router relay used go-libp2p's defaults, cutting every relayed
connection after 2 minutes or 128 KiB per direction. Add
--relay-limit-duration (default 1h) and --relay-limit-data (default no
limit, accepts sizes like 512MiB) to sam-router, and the matching
--router-relay-limit-* tunables to sam-one. 0 means no limit.

go.mod: go-humanize and golang.org/x/sys become direct dependencies
(the latter was already imported by the mobile FFI).

Part of google#483

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces configuration options and command-line flags to cap relayed connections by duration and data limits across sam-router and sam-one, utilizing the go-humanize library to parse human-readable byte sizes. The review feedback highlights a critical issue where limits exceeding math.MaxUint32 will silently wrap around during go-libp2p serialization, potentially cutting connections immediately; the reviewer suggests capping these values to math.MaxUint32 and updating the test suite accordingly. Additionally, a mismatched protocol ID ('test' instead of '/test') was identified in the unit tests for the limited connection stream.

Comment on lines +31 to +44
func relayLimit(duration time.Duration, data int64) *relay.RelayLimit {
if duration <= 0 && data <= 0 {
return nil
}
limit := &relay.RelayLimit{Duration: duration, Data: data}
if duration <= 0 {
// The relay advertises seconds as uint32; anything larger wraps.
limit.Duration = math.MaxUint32 * time.Second
}
if data <= 0 {
limit.Data = math.MaxInt64
}
return limit
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In the libp2p circuit relay v2 protocol, both duration (in seconds) and data (in bytes) are represented as uint32 on the wire. If the configured limits exceed math.MaxUint32 (e.g., math.MaxInt64 or a user-configured value like 5GB), they will silently wrap around when serialized by go-libp2p. For example, a 5GB limit wraps around to 1GB, and a 4GB limit wraps to 0 (which cuts the connection immediately). To prevent silent wrap-around, we should explicitly cap both duration and data to math.MaxUint32 (or math.MaxUint32 * time.Second for duration).

Suggested change
func relayLimit(duration time.Duration, data int64) *relay.RelayLimit {
if duration <= 0 && data <= 0 {
return nil
}
limit := &relay.RelayLimit{Duration: duration, Data: data}
if duration <= 0 {
// The relay advertises seconds as uint32; anything larger wraps.
limit.Duration = math.MaxUint32 * time.Second
}
if data <= 0 {
limit.Data = math.MaxInt64
}
return limit
}
func relayLimit(duration time.Duration, data int64) *relay.RelayLimit {
if duration <= 0 && data <= 0 {
return nil
}
limit := &relay.RelayLimit{Duration: duration, Data: data}
if duration <= 0 || duration > math.MaxUint32*time.Second {
// The relay advertises seconds as uint32; anything larger wraps.
limit.Duration = math.MaxUint32 * time.Second
}
if data <= 0 || data > math.MaxUint32 {
limit.Data = math.MaxUint32
}
return limit
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only Duration is uint32 on the wire; Data is uint64 (go-libp2p v0.49.0 p2p/protocol/circuitv2/pb/circuit.pb.go:454), so byte limits don't wrap and capping them at 4 GiB would add a hidden limit. Also, the relay enforces its local rc.Limit (deadline + io.LimitReader, relay/relay.go:473-478); the wire Limit is only advertised to the client, so a wrapped value couldn't cut a connection. The only case that would advertise a wrong number is a duration above ~136 years. Leaving as is.

Comment on lines +34 to +53
func TestRelayLimit(t *testing.T) {
unlimitedDuration := math.MaxUint32 * time.Second
for _, tc := range []struct {
name string
duration time.Duration
data int64
want *relay.RelayLimit
}{
{"no limits", 0, 0, nil},
{"duration only", time.Hour, 0, &relay.RelayLimit{Duration: time.Hour, Data: math.MaxInt64}},
{"data only", 0, 4096, &relay.RelayLimit{Duration: unlimitedDuration, Data: 4096}},
{"both", time.Minute, 4096, &relay.RelayLimit{Duration: time.Minute, Data: 4096}},
} {
t.Run(tc.name, func(t *testing.T) {
if got := relayLimit(tc.duration, tc.data); !reflect.DeepEqual(got, tc.want) {
t.Errorf("relayLimit(%v, %d) = %+v, want %+v", tc.duration, tc.data, got, tc.want)
}
})
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the test cases to align with the new capping behavior in relayLimit (where unlimited or overflowing values are capped to math.MaxUint32 instead of math.MaxInt64), and add test cases to verify that overflow is correctly handled.

func TestRelayLimit(t *testing.T) {
    unlimitedDuration := math.MaxUint32 * time.Second
    for _, tc := range []struct {
        name     string
        duration time.Duration
        data     int64
        want     *relay.RelayLimit
    }{
        {"no limits", 0, 0, nil},
        {"duration only", time.Hour, 0, &relay.RelayLimit{Duration: time.Hour, Data: math.MaxUint32}},
        {"data only", 0, 4096, &relay.RelayLimit{Duration: unlimitedDuration, Data: 4096}},
        {"both", time.Minute, 4096, &relay.RelayLimit{Duration: time.Minute, Data: 4096}},
        {"duration overflow", 200 * 365 * 24 * time.Hour, 0, &relay.RelayLimit{Duration: unlimitedDuration, Data: math.MaxUint32}},
        {"data overflow", 0, 5 << 30, &relay.RelayLimit{Duration: unlimitedDuration, Data: math.MaxUint32}},
    } {
        t.Run(tc.name, func(t *testing.T) {
            if got := relayLimit(tc.duration, tc.data); !reflect.DeepEqual(got, tc.want) {
                t.Errorf("relayLimit(%v, %d) = %+v, want %+v", tc.duration, tc.data, got, tc.want)
            }
        })
    }
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follows from the thread above: no capping change, so the expectations stay.

t.Fatal(err)
}

s, err := src.NewStream(network.WithAllowLimitedConn(ctx, "test"), dst.ID(), "/test")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The protocol ID registered on the destination host is "/test" (line 97), and the stream is opened with "/test" (line 120). However, WithAllowLimitedConn is called with "test". Since protocol IDs must match exactly, this should be "/test" to correctly allow the protocol on the limited connection.

Suggested change
s, err := src.NewStream(network.WithAllowLimitedConn(ctx, "test"), dst.ID(), "/test")
s, err := src.NewStream(network.WithAllowLimitedConn(ctx, "/test"), dst.ID(), "/test")

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second argument of network.WithAllowLimitedConn is a free-form reason string, not a protocol ID (go-libp2p v0.49.0 core/network/context.go:100), so "test" is fine.

@aojea
aojea merged commit 2b41a67 into google:main Sep 24, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants