Skip to content

Do not cache a plain HTTP client when Docker server credentials are missing - #1366

Merged
krisstern merged 6 commits into
jenkinsci:masterfrom
pguinet:fix-missing-docker-credentials-plain-http
Sep 30, 2026
Merged

krisstern merged 6 commits into
jenkinsci:masterfrom
pguinet:fix-missing-docker-credentials-plain-http

Conversation

@pguinet

@pguinet pguinet commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1365

When a cloud's credentialsId cannot be resolved (typically for a few seconds at startup, while JCasC replaces the credentials store and a startup job recreates them), DockerAPI.toSSlConfig() returned null and a client without TLS was built and cached. The cache key ignores the credential's content, and the watchdog and the provisioner keep the entry alive, so every call kept failing with 400 Client sent an HTTP request to an HTTPS server long after the credentials existed.

Four commits:

  • ContainerNodeNameMap.merge() dropped the containerListIncomplete flag. When a failing cloud was followed by a healthy one, the merged map looked complete and cleanUpSuperfluousComputer() could remove agents of the unreachable cloud. This is a prerequisite for the second commit.
  • toSSlConfig() now throws when the credentials are missing, so nothing is cached and the first call made once they exist builds a TLS client. The watchdog skips the affected cloud and marks the container list as incomplete instead of aborting its run.
  • The watchdog catches that exception around getClient() only, so an unrelated IllegalStateException raised once the client is obtained still propagates.
  • The exception is now a dedicated MissingDockerServerCredentialsException (a subclass of IllegalStateException), and the watchdog catches only that type, so other IllegalStateExceptions thrown by getClient(), such as a UsageTrackingCache invariant failure, propagate. Credentials are only resolved for tcp endpoints: docker-java ignores the SSL config for unix and npipe sockets, so a leftover credentials id there keeps working as before.

Behaviour change

This is intended: for a tcp endpoint whose credentialsId does not resolve to global Docker Server Credentials, getClient() now fails loudly instead of silently dropping TLS. This also applies to a stale credentials id on a plain-HTTP daemon (:2375), and to credentials stored in a folder, which toSSlConfig() never looked up. Endpoints without a credentials id, and unix/npipe endpoints, are unaffected.

Testing done

  • New tests in DockerAPITest, DockerContainerWatchdogTest and ContainerNodeNameMapTest. They fail on master and pass with the fix, except three that guard against regressions rather than reproduce the bug: testOtherIllegalStateExceptionFromGetClientIsNotSwallowed, testMissingCredentialsExceptionAfterObtainingClientIsNotSwallowed (both fail if the catch is widened again) and getClientIgnoresAnUnresolvableCredentialsIdOnAUnixSocket (fails without the tcp guard).
  • mvn verify passes (201 tests, SpotBugs clean).
  • The bug was reproduced on a production controller (plugin 1327): no Docker agent for about 1h40 after a restart, until the cloud URI was changed to force a new cache key.

AI disclosure: the investigation, the patch and its tests were produced with the help of an AI assistant, Claude Opus 5.5 (claude-opus-5-5) by Anthropic, used through Claude Code. I reviewed the changes and take responsibility for them.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests that demonstrate the feature works or the issue is fixed

🤖 Generated with Claude Code

pguinet and others added 2 commits September 24, 2026 11:44
ContainerNodeNameMap.merge() dropped containerListIncomplete. When a cloud
failing with ContainersRetrievalException was followed by a healthy cloud,
the merged map looked complete and cleanUpSuperfluousComputer() could remove
agents of the unreachable cloud.

Refs jenkinsci#1365

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…issing

When a cloud's credentialsId could not be resolved, toSSlConfig() returned
null and a client without TLS was built and cached. The cache key ignores the
credential's content and the watchdog and provisioner keep the entry alive,
so the client kept failing with "400 Client sent an HTTP request to an HTTPS
server" long after the credentials were created.

toSSlConfig() now throws, so nothing is cached and the first call made once
the credentials exist builds a TLS client. The watchdog skips the affected
cloud and marks the container list as incomplete instead of aborting the run.

Fixes jenkinsci#1365

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Narrow the IllegalStateException handling so unrelated failures are not suppressed.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

Fixes Docker client caching when Docker server credentials are temporarily unavailable.

Changes:

  • Rejects unresolved credentials before client caching.
  • Preserves incomplete watchdog state across cloud merges.
  • Adds regression tests for credentials, watchdog handling, and map merging.
File Description
src/​test/​java/​io/​jenkins/​docker/​client/​DockerAPITest.java Tests credential resolution and client caching.
src/​test/​java/​com/​nirima/​jenkins/​plugins/​docker/​DockerContainerWatchdogTest.java Tests watchdog failure handling and recovery.
src/​test/​java/​com/​nirima/​jenkins/​plugins/​docker/​ContainerNodeNameMapTest.java Tests merge behavior and completeness state.
src/​main/​java/​io/​jenkins/​docker/​client/​DockerAPI.java Rejects missing credentials before caching.
src/​main/​java/​com/​nirima/​jenkins/​plugins/​docker/​DockerContainerWatchdog.java Handles affected clouds while continuing processing.
src/​main/​java/​com/​nirima/​jenkins/​plugins/​docker/​ContainerNodeNameMap.java Preserves incomplete-list state during merges.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/main/java/com/nirima/jenkins/plugins/docker/DockerContainerWatchdog.java Outdated
krisstern and others added 2 commits September 27, 2026 19:43
The IllegalStateException catch in the watchdog covered the whole
try-with-resources block, so an unrelated IllegalStateException raised
after the client was obtained (e.g. closing the client) was swallowed,
logged as a connection failure and marked the container list
incomplete. Catch it around getClient() only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Narrow the watchdog catch to handle only missing credentials and propagate unrelated failures.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread src/main/java/com/nirima/jenkins/plugins/docker/DockerContainerWatchdog.java Outdated
pguinet and others added 2 commits September 29, 2026 08:47
getClient() can also throw IllegalStateException from the client cache
bookkeeping (UsageTrackingCache.cacheAndIncrementUsage), which the
watchdog would have logged as an unreachable cloud and skipped. Throw a
MissingDockerServerCredentialsException instead, a subclass of
IllegalStateException for compatibility, and catch only that.

Only resolve the credentials for tcp endpoints: docker-java ignores the
SSL configuration for unix and npipe sockets, so a leftover credentials
id there must not turn a working cloud into a failing one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

One or more issues must be addressed before approval.

Review effort: Lite
Findings: None

Resolved since last review (1)

@krisstern
krisstern merged commit 3611e07 into jenkinsci:master Sep 30, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docker client silently falls back to plain HTTP when the cloud's credentials are missing, and the broken client stays cached

4 participants