Skip to content

[release-4.20] OCPBUGS-113989: frr-k8s: use Recreate strategy for webhook server deployment on SNO - #3141

Closed
jechen0648 wants to merge 6 commits into
openshift:release-4.20from
jechen0648:OCPBUGS-113989
Closed

[release-4.20] OCPBUGS-113989: frr-k8s: use Recreate strategy for webhook server deployment on SNO#3141
jechen0648 wants to merge 6 commits into
openshift:release-4.20from
jechen0648:OCPBUGS-113989

Conversation

@jechen0648

Copy link
Copy Markdown

Manual backport of #3135 to release-4.20, superseding the failed auto cherry-pick #3140.

On SNO clusters, the frr-k8s webhook server Deployment uses hostNetwork with a fixed port (9123). With the default RollingUpdate strategy, upgrades get stuck because the new pod cannot bind the host port already held by the old pod. This PR:

Switches the frr-k8s-webhook-server Deployment to Recreate strategy on SNO clusters
Adds the networkoperator.openshift.io/pre-patch annotation so CNO can atomically switch strategy type and clear rollingUpdate fields via strategic-merge-patch before SSA (required on upgrade when the API server has already defaulted rollingUpdate)
Explicitly sets RollingUpdate with rollingUpdate fields on non-SNO clusters so CNO owns those fields going forward
Wires bootstrapResult into renderAdditionalRoutingCapabilities to determine SNO via ControlPlaneReplicaCount
Adds unit tests for the pre-patch apply path and deployment strategy rendering
4.20-specific changes
On release-4.20, the hostNetwork workload is frr-k8s-webhook-server (in webhook.yaml). The separate frr-k8s-statuscleaner Deployment (node-status-cleaner.yaml) was only introduced in 4.21.

The auto cherry-pick in #3140 failed unit tests because it cherry-picked the 4.21 test Test_renderFRRStatusCleanerStrategy, which looks for a Deployment that does not exist on 4.20. This PR fixes that by:

Renaming the test to Test_renderFRRWebhookServerStrategy
Targeting frr-k8s-webhook-server instead of frr-k8s-statuscleaner

Test plan
go test ./pkg/network/
go test ./pkg/apply/
CI unit job on PR

tpantelis and others added 5 commits August 25, 2026 15:33
This commit adds helper functions to testutil_test.go that are used
by component rendering tests to verify rendered Kubernetes objects:

- mustFindRenderedObj: Finds and converts unstructured objects using generics
- mustFindContainer: Finds a container by name in a container list
- findExecCommand: Extracts exec command strings from container command args

These utilities facilitate testing of rendered manifests and container
configurations across multiple components.

Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
Co-Authored-By: Ori Braunshtein <obraunsh@redhat.com>
(cherry picked from commit c617131)

Cherry-pick not clean:
- dropped definiton of mustFindContainer and findExecCommand (otherwise
  lint fails)

(cherry picked from commit cedf0d7)
The statuscleaner deployment uses hostNetwork with a fixed port (9123).
With the default RollingUpdate strategy, upgrades on SNO clusters get
stuck because the new pod cannot bind the host port already held by the
old pod. Switching to Recreate ensures the old pod is terminated before
the new one starts.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
Co-Authored-By: Ori Braunshtein <obraunsh@redhat.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
(cherry picked from commit 3d86b75)

Cherry-pick not clean:
- added bootstrap result wiring to additionalRoutingCapibilities (that
  was already there for 5.0), aligned existing unit test with
  fakeBootstrapResult() as in 5.0

(cherry picked from commit a5752d9)

Conflicts:
	pkg/network/render.go preserving
Very small conflict due to the presence of isSupportedDualStackPlatform.
Resolution: keep isSupportedDualStackPlatform + add bootstrapResult to
renderAdditionalRoutingCapabilities.
On upgrade, the frr-k8s-statuscleaner Deployment has rollingUpdate
fields defaulted by the API server. SSA cannot remove fields it does
not own, so switching strategy.type to Recreate fails with:

    spec.strategy.rollingUpdate: Forbidden: may not be specified when
    strategy `type` is 'Recreate'

Add a generic pre-patch annotation:

    networkoperator.openshift.io/pre-patch

that applies a strategic-merge-patch to the live object before SSA. This
lets the template atomically set type=Recreate and remove rollingUpdate
in a single patch, before SSA takes over.

Also explicitly set strategy to RollingUpdate with rollingUpdate fields
on non-SNO clusters so that CNO owns these fields going forward,
preventing the issue from recurring in case we have to make the same
switch for non-SNO clusters in the future.

Signed-off-by: Andreas Karis <ak.karis@gmail.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
(cherry picked from commit 8ebb78e)
(cherry picked from commit d59aba6)
Add tests for ApplyObject's pre-patch behavior: strategic-merge-patch
runs before SSA, NotFound is tolerated, other errors stop reconciliation,
and objects without the annotation skip pre-patch entirely.

Remove fakeRESTMapper and replace it with
testrestmapper.TetsOnlyStaticRESTMapper to return a proper mapper,
and return scheme.Scheme from Scheme(). Both changes needed by
ApplyObject under test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Andreas Karis <ak.karis@gmail.com>
(cherry picked from commit 6ca59b1)
(cherry picked from commit b1775c9)

Not clean cherry-pick:
Make the linter happy by applying the changes to pkg/apply from
openshift#3138. See
that PR for further details.
On release-4.20 the hostNetwork deployment is frr-k8s-webhook-server, not
frr-k8s-statuscleaner (which was split out in 4.21). Update the strategy
unit test to target the correct deployment name.

Signed-off-by: Jean Chen <jechen@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Aug 25, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@jechen0648: This pull request references Jira Issue OCPBUGS-113989, which is invalid:

  • expected dependent Jira Issue OCPBUGS-112789 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is MODIFIED instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Manual backport of #3135 to release-4.20, superseding the failed auto cherry-pick #3140.

On SNO clusters, the frr-k8s webhook server Deployment uses hostNetwork with a fixed port (9123). With the default RollingUpdate strategy, upgrades get stuck because the new pod cannot bind the host port already held by the old pod. This PR:

Switches the frr-k8s-webhook-server Deployment to Recreate strategy on SNO clusters
Adds the networkoperator.openshift.io/pre-patch annotation so CNO can atomically switch strategy type and clear rollingUpdate fields via strategic-merge-patch before SSA (required on upgrade when the API server has already defaulted rollingUpdate)
Explicitly sets RollingUpdate with rollingUpdate fields on non-SNO clusters so CNO owns those fields going forward
Wires bootstrapResult into renderAdditionalRoutingCapabilities to determine SNO via ControlPlaneReplicaCount
Adds unit tests for the pre-patch apply path and deployment strategy rendering
4.20-specific changes
On release-4.20, the hostNetwork workload is frr-k8s-webhook-server (in webhook.yaml). The separate frr-k8s-statuscleaner Deployment (node-status-cleaner.yaml) was only introduced in 4.21.

The auto cherry-pick in #3140 failed unit tests because it cherry-picked the 4.21 test Test_renderFRRStatusCleanerStrategy, which looks for a Deployment that does not exist on 4.20. This PR fixes that by:

Renaming the test to Test_renderFRRWebhookServerStrategy
Targeting frr-k8s-webhook-server instead of frr-k8s-statuscleaner

Test plan
go test ./pkg/network/
go test ./pkg/apply/
CI unit job on PR

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f3493306-d886-4595-85d0-c82eaa466842

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from jcaamano and tssurya August 25, 2026 20:00
@openshift-ci

openshift-ci Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jechen0648
Once this PR has been reviewed and has the lgtm label, please assign jcaamano for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

The apply backport replaced fakeRESTMapper with testrestmapper.
TestStatusManager_set used a fake test/test GVK that the real mapper
cannot resolve. Use ConfigMap instead so related-object deletion is
exercised with a scheme-known type.

Signed-off-by: Jean Chen <jechen@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@openshift-ci

openshift-ci Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@jechen0648: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/security 8b97f6f link false /test security

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

}

func Test_renderFRRStatusCleanerStrategy(t *testing.T) {
func Test_renderFRRWebhookServerStrategy(t *testing.T) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to squash this commit into commit frr-k8s: use Recreate strategy for statuscleaner deployment and add a note at the end of the commit about the change that was made.


func Test_renderFRRStatusCleanerStrategy(t *testing.T) {
func Test_renderFRRWebhookServerStrategy(t *testing.T) {
// On 4.20 the hostNetwork workload is frr-k8s-webhook-server; statuscleaner was split out in 4.21.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that comment here isn't necessary

Group: "test",
Version: "test",
Kind: "test",
Group: "",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this commit here makes no sense - the unit test is exactly the same in 4.21 where it passes without problems, meaning if there are problems, they lie elsewhere

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a better solution (IMO). I'll push it in a bit

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #3142

@jechen0648

Copy link
Copy Markdown
Author

prefer #3142

@jechen0648 jechen0648 closed this Aug 26, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@jechen0648: This pull request references Jira Issue OCPBUGS-113989. The bug has been updated to no longer refer to the pull request using the external bug tracker.

Details

In response to this:

Manual backport of #3135 to release-4.20, superseding the failed auto cherry-pick #3140.

On SNO clusters, the frr-k8s webhook server Deployment uses hostNetwork with a fixed port (9123). With the default RollingUpdate strategy, upgrades get stuck because the new pod cannot bind the host port already held by the old pod. This PR:

Switches the frr-k8s-webhook-server Deployment to Recreate strategy on SNO clusters
Adds the networkoperator.openshift.io/pre-patch annotation so CNO can atomically switch strategy type and clear rollingUpdate fields via strategic-merge-patch before SSA (required on upgrade when the API server has already defaulted rollingUpdate)
Explicitly sets RollingUpdate with rollingUpdate fields on non-SNO clusters so CNO owns those fields going forward
Wires bootstrapResult into renderAdditionalRoutingCapabilities to determine SNO via ControlPlaneReplicaCount
Adds unit tests for the pre-patch apply path and deployment strategy rendering
4.20-specific changes
On release-4.20, the hostNetwork workload is frr-k8s-webhook-server (in webhook.yaml). The separate frr-k8s-statuscleaner Deployment (node-status-cleaner.yaml) was only introduced in 4.21.

The auto cherry-pick in #3140 failed unit tests because it cherry-picked the 4.21 test Test_renderFRRStatusCleanerStrategy, which looks for a Deployment that does not exist on 4.20. This PR fixes that by:

Renaming the test to Test_renderFRRWebhookServerStrategy
Targeting frr-k8s-webhook-server instead of frr-k8s-statuscleaner

Test plan
go test ./pkg/network/
go test ./pkg/apply/
CI unit job on PR

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants