perf: raise the default connection pool size from 5 to 100 - #97
Conversation
The default of 5 capped each client at 5 concurrent requests, so busy servers queued behind the pool. 100 matches httpx's default and stream-py.
| env_request_timeout = ENV.fetch('STREAM_REQUEST_TIMEOUT', nil) || ENV.fetch('STREAM_TIMEOUT', nil) | ||
| @request_timeout = (request_timeout || timeout || env_request_timeout || 30).to_i | ||
| @max_conns_per_host = (max_conns_per_host || ENV.fetch('STREAM_MAX_CONNS_PER_HOST', nil) || 5).to_i | ||
| @max_conns_per_host = (max_conns_per_host || ENV.fetch('STREAM_MAX_CONNS_PER_HOST', nil) || 100).to_i |
There was a problem hiding this comment.
net-http-persistent 4.0.8 checks idle_timeout only when a connection is checked out, and its pool hands back the most recently used connection first. After a burst, only the connections needed for steady traffic get reused, and the rest stay open past the LB idle close (CLOSE_WAIT) until the next burst. With 50 threads settling at about 5 concurrent requests, that is 45 dead sockets per process per client, where the old cap bounded it at 5. The PR body says an idle app opens no extra sockets, which holds only before the first burst.
There was a problem hiding this comment.
Kept 100. The extra sockets are bounded by the peak number of threads in a request, and past 5 those threads used to wait at most 0.5s on the pool and then raise ConnectionPool::TimeoutError. Corrected the PR body.
Problem
The default
net_http_persistentadapter usedpool_size: 5, which caps each client at 5 connections across all threads. A 6th concurrent thread waits at most 0.5s for a free connection (net-http-persistent checks out without a timeout, so connection_pool's 0.5s default applies) and then raisesConnectionPool::TimeoutError, so Puma or Sidekiq apps with more than 5 threads fail under load.Solution
max_conns_per_hostdefault (andSTREAM_MAX_CONNS_PER_HOSTfallback) from 5 to 100. Connections open lazily, so the pool grows only to the peak number of threads in a request at once.idle_timeoutonly when a connection is checked out, so connections opened for a burst stay open (in CLOSE_WAIT once the load balancer closes them) until the next burst reconnects them. That is at most one socket per thread busy at the peak, and apps with 5 or fewer threads see no change.max_conns_per_host: 5.How to verify
bundle exec rspec spec/connection_pooling_spec.rb spec/getstream_ruby_spec.rbpool_size: 100and theclient.initializedlog line at 100.