[majkelx]

Stacked on #54. Fixes a reader which stayed open and deaf for good after a recovery cut short by the network (seen in a long-running service after a power outage), and the review finding on #54.

The failure

MsgReader._reopen() removed the old subscription before the new one existed. When the JetStream API timed out in the middle (consumer info, then three attempts to create a consumer), the reader was left holding the unsubscribed handle while its consumer stayed alive on the server. From then on the health check passed (the consumer exists), every pull request kept that consumer alive, and the answers went to an inbox nobody listened on: Timeout after 2.0s waiting for server response, for ever.

What changes

• A consumer and its subscription are one session, replaced as a whole, the new one first. A failed or cancelled replacement leaves the session in use untouched. A half-made session leaves no subscription behind.
• A session is judged by the server's answers to the reader's own pull requests, not by "the consumer exists". The periodic consumer_info poll is gone: one JetStream API request per reader every 10 s less.
• One retry authority. The inner three attempts with their own pauses are removed; RetryPolicy paces everything.
• Only a missing stream is fatal. A 404 is classified by err_code: a missing consumer (10014) is ordinary life of ephemeral consumers, a missing stream (10059, or the client library's own "no stream for subject") is a deployment error, as before.
• RetryPolicy.total_timeout of a reader bounds an attempt to recover, not only the pauses between attempts (review finding on Deadlines, cancellation and the reader session, Python 3.11+ (2.5.1) #54).

Server versions differ

A pull request to a deleted consumer is answered with status 503 by nats-server 2.14 and with nothing at all by 2.10.x. The reader handles both: after an unanswered pull it asks about the consumer once, to tell an expired consumer (replaced without an error) from lost answers (a failure, paced by the retry policy).

Tests

New tests/test_reader_session.py, 10 cases: lost subscription, lost subscription while no consumer can be created, failed replacement of a working session, cancelled replacement, half-made session, expired consumer under OnError.RAISE, consumer-level 404, missing stream at open, deadline of a recovery attempt.

Full suite: 457 passed on nats-server 2.10.22, 2.10.7 (container) and 2.14.0.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

Replacement-session cleanup can delete a durable consumer still used by the new session, invalidating the replacement and potentially losing the durable cursor.

3 open issues

• Cleanup deletes the durable consumer still used by the replacement · New
• Queued messages are discarded as a transport timeout · New
• Recovery reschedules an expired timeout and loses the valid message · New

What changed in this PR

This PR strengthens reader recovery for version 2.5.1, building on #54’s deadline work to prevent interrupted recovery from leaving readers open but unable to receive messages.

Changes:

• Replaces consumer/subscription pairs together and checks session health through pull responses.
• Centralizes retries, bounds recovery attempts, and distinguishes missing consumers from missing streams.
• Adds recovery regression tests and updates documentation and release metadata.

File	Description
tests/​test_resl_consumer_expiry.py	Updates comments to describe session replacement.
tests/​test_reader_session.py	Adds session recovery, cancellation, and deadline tests.
tests/​test_delivery_policies.py	Uses the new session-replacement method.
serverish/​messenger/​msg_reader.py	Implements session lifecycle, response-based health checks, and recovery deadlines.
serverish/​connection/​nats_errors.py	Distinguishes missing streams from missing consumers.
README.md	Adds the 2.5.1 release notes.
pyproject.toml	Bumps the package version to 2.5.1.
doc/​messenger.md	Documents automatic recovery and fatal configuration errors.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Recovery can permanently lose unread messages, fail at the consumer limit, and enter an unpaced retry loop on persistent pull rejections.

1 open issue

• Durable handoff can discard undelivered messages · New

2 previously missed issues

In code that hasn't changed since last review

Unexpected pull rejections cause an unpaced retry loop

serverish/​messenger/​msg_reader.py:755

A persistent unexpected closing status such as a 409 rejection returns an empty successful batch, so read_batch() calls recovered() and can immediately repeat the pull after its blocking fetch is also rejected. This creates a round-trip-speed polling loop that bypasses retry pacing and total_timeout. Raise an error for these unexpected statuses so OnError and RetryPolicy handle the rejection.

Consumer capacity exhaustion makes reconnect recovery fatal

serverish/​messenger/​msg_reader.py:947

Creating a replacement ephemeral consumer requires a spare slot, so a reconnect or lost subscription cannot recover when the stream is already at max_consumers (for example, one reader on a stream limited to one consumer). JetStream rejects creation with BadRequestError, which stops the reader as a configuration error even under OnError.WAIT. Add a capacity-aware recovery path, such as safely rebinding the existing consumer, without making the temporary need for an extra slot fatal.

3 resolved since last review

• Cleanup deletes the durable consumer still used by the replacement
• Recovery reschedules an expired timeout and loses the valid message
• Queued messages are discarded as a transport timeout

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Named ephemeral readers are omitted from delete-before-create handling, so recovery can either reject a delivery-policy change or delete the consumer used by the replacement session.

1 open issue

• Delete named ephemeral consumers before recreating them · New

1 resolved since last review

• Durable handoff can discard undelivered messages

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Human assessment is needed because session-recovery edge cases can permanently skip queued messages, block consumer replacement, and reset retry budgets.

No open issues 🎉

1 resolved since last review

• Delete named ephemeral consumers before recreating them

_{Review effort: Balanced}