Skip to content

perf: raise the default connection pool size from 5 to 100 - #97

Merged
mogita merged 1 commit into
masterfrom
perf/connection-pool-default-100
Oct 1, 2026
Merged

mogita merged 1 commit into
masterfrom
perf/connection-pool-default-100

Conversation

@mogita

@mogita mogita commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The default net_http_persistent adapter used pool_size: 5, which caps each client at 5 connections across all threads. A 6th concurrent thread waits at most 0.5s for a free connection (net-http-persistent checks out without a timeout, so connection_pool's 0.5s default applies) and then raises ConnectionPool::TimeoutError, so Puma or Sidekiq apps with more than 5 threads fail under load.

Solution

  • Raise the max_conns_per_host default (and STREAM_MAX_CONNS_PER_HOST fallback) from 5 to 100. Connections open lazily, so the pool grows only to the peak number of threads in a request at once.
  • net-http-persistent checks idle_timeout only when a connection is checked out, so connections opened for a burst stay open (in CLOSE_WAIT once the load balancer closes them) until the next burst reconnects them. That is at most one socket per thread busy at the peak, and apps with 5 or fewer threads see no change.
  • 100 matches stream-py (#299).
  • Apps that relied on the old limit to throttle traffic can pass max_conns_per_host: 5.

How to verify

  1. bundle exec rspec spec/connection_pooling_spec.rb spec/getstream_ruby_spec.rb
  2. All 20 examples pass and assert pool_size: 100 and the client.initialized log line at 100.

The default of 5 capped each client at 5 concurrent requests, so busy servers queued behind the pool. 100 matches httpx's default and stream-py.
env_request_timeout = ENV.fetch('STREAM_REQUEST_TIMEOUT', nil) || ENV.fetch('STREAM_TIMEOUT', nil)
@request_timeout = (request_timeout || timeout || env_request_timeout || 30).to_i
@max_conns_per_host = (max_conns_per_host || ENV.fetch('STREAM_MAX_CONNS_PER_HOST', nil) || 5).to_i
@max_conns_per_host = (max_conns_per_host || ENV.fetch('STREAM_MAX_CONNS_PER_HOST', nil) || 100).to_i

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

net-http-persistent 4.0.8 checks idle_timeout only when a connection is checked out, and its pool hands back the most recently used connection first. After a burst, only the connections needed for steady traffic get reused, and the rest stay open past the LB idle close (CLOSE_WAIT) until the next burst. With 50 threads settling at about 5 concurrent requests, that is 45 dead sockets per process per client, where the old cap bounded it at 5. The PR body says an idle app opens no extra sockets, which holds only before the first burst.

@mogita mogita Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kept 100. The extra sockets are bounded by the peak number of threads in a request, and past 5 those threads used to wait at most 0.5s on the pool and then raise ConnectionPool::TimeoutError. Corrected the PR body.

@mogita
mogita merged commit 958dbbc into master Oct 1, 2026
9 checks passed
@mogita
mogita deleted the perf/connection-pool-default-100 branch October 1, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant