Skip to content

oadp: split kubevirt e2e job (5.0) and add Claude failure analysis to kdm-controller/plugin - #84337

Merged
openshift-merge-bot[bot] merged 2 commits into
openshift:mainfrom
kaovilai:worktree-oadp-split-kubevirt-kdm-job
Sep 1, 2026
Merged

oadp: split kubevirt e2e job (5.0) and add Claude failure analysis to kdm-controller/plugin#84337
openshift-merge-bot[bot] merged 2 commits into
openshift:mainfrom
kaovilai:worktree-oadp-split-kubevirt-kdm-job

Conversation

@kaovilai

@kaovilai kaovilai commented Sep 1, 2026

Copy link
Copy Markdown
Member

Summary

ci/prow/5.0-e2e-test-kubevirt-aws was hitting Prow's 2h step timeout running both the non-kdm CSI virt suite and the kdm suite serially (openshift/oadp-operator#2413). openshift/oadp-operator#2423 makes TEST_VIRT_KDM=false explicitly exclude kdm specs while leaving existing jobs that don't set it unaffected (no regression).

Splits the oadp-dev__5.0 presubmit into two parallel jobs, matching the naming used for the kdm-controller repo's own split job (virt-kdm-e2e-test-aws):

  • virt-nokdm-e2e-test-aws: make TEST_VIRT=true TEST_VIRT_KDM=false test-e2e
  • virt-kdm-e2e-test-aws: make TEST_VIRT_KDM=true test-e2e

Each comfortably clears the 2h default, and CI feedback now arrives in parallel instead of serially.

Also wires the Claude-based oadp-analyze-e2e-failure post-step (added for oadp-operator in #83957) into the kubevirt-datamover-controller/kubevirt-datamover-plugin virt-kdm-e2e-test-aws jobs (both oadp-1.6 and oadp-dev configs), for the same failure-analysis coverage.

Depends on:

Note

Responses generated with Claude

Summary by CodeRabbit

  • Splits the oadp-dev__5.0 KubeVirt AWS presubmit into parallel KDM and non-KDM jobs to avoid Prow’s two-hour step timeout.
  • Adds Claude-based E2E failure analysis to KubeVirt Datamover Controller and Plugin AWS jobs for oadp-1.6 and oadp-dev.
  • Configures the analysis step with the required Claude helper image, OADP branch, and test-name variables.

ci/prow/5.0-e2e-test-kubevirt-aws was hitting Prow's 2h step timeout
running both the non-kdm CSI virt suite and the kdm suite serially
(openshift/oadp-operator#2413). openshift/oadp-operator#2423 makes
TEST_VIRT_KDM=false explicitly exclude kdm specs while leaving
existing jobs that don't set it unaffected.

Split the presubmit into two parallel jobs, matching the naming used
for the kdm-controller repo's own split job:
- virt-nokdm-e2e-test-aws: TEST_VIRT=true TEST_VIRT_KDM=false
- virt-kdm-e2e-test-aws: TEST_VIRT_KDM=true

Each comfortably clears the 2h default, and CI feedback now arrives
in parallel instead of serially.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
@kaovilai

kaovilai commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

/pj-rehearse pull-ci-openshift-oadp-operator-oadp-dev-5.0-virt-nokdm-e2e-test-aws pull-ci-openshift-oadp-operator-oadp-dev-5.0-virt-kdm-e2e-test-aws

Note

Responses generated with Claude

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor
ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 65c98033-dffd-4bc8-b65d-bf22f41de434

📥 Commits

Reviewing files that changed from the base of the PR and between 4d306b7 and a5c2e60.

📒 Files selected for processing (4)
  • ci-operator/config/migtools/kubevirt-datamover-controller/migtools-kubevirt-datamover-controller-oadp-1.6.yaml
  • ci-operator/config/migtools/kubevirt-datamover-controller/migtools-kubevirt-datamover-controller-oadp-dev.yaml
  • ci-operator/config/migtools/kubevirt-datamover-plugin/migtools-kubevirt-datamover-plugin-oadp-1.6.yaml
  • ci-operator/config/migtools/kubevirt-datamover-plugin/migtools-kubevirt-datamover-plugin-oadp-dev.yaml

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

The AWS virtualization CI configuration now separates KDM-disabled and KDM-enabled tests. OADP controller and plugin jobs also configure test metadata and run oadp-analyze-e2e-failure post-steps.

Changes

AWS virtualization CI

Layer / File(s) Summary
Separate KDM virtualization jobs
ci-operator/config/openshift/oadp-operator/openshift-oadp-operator-oadp-dev__5.0.yaml
The existing AWS job now runs with TEST_VIRT_KDM=false. A parallel job runs with TEST_VIRT_KDM=true.
Configure controller test analysis
ci-operator/config/migtools/kubevirt-datamover-controller/*
The configurations add claude-ai-helpers:latest, set AWS test metadata, and run oadp-analyze-e2e-failure.
Configure plugin test analysis
ci-operator/config/migtools/kubevirt-datamover-plugin/*
The configurations add claude-ai-helpers:latest, set AWS test metadata, and run oadp-analyze-e2e-failure.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to a5c2e

This change splits the KubeVirt end-to-end workload into separate jobs and adds failure analysis to related jobs without introducing an actionable merge-blocking risk; it is merge-ready after normal checks and review.

Suggested reviewers: sseago, mpryc, joeavaikath

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PASS. The pull request changes only CI YAML configuration and generated Prow YAML; it adds no Ginkgo It, Describe, Context, or When titles. The introduced TEST_NAME, as, context, and job i…
Test Structure And Quality ✅ Passed PASS: The pull request changes only six YAML CI configuration files. The diff contains no Go test files or Ginkgo test code, and no It, BeforeEach, AfterEach, Eventually, or Consistently con…
Microshift Test Compatibility ✅ Passed PASS: The pull request adds or changes CI YAML job definitions only. The verified PR diff contains no added Go files and no new Ginkgo declarations (It, Describe, Context, or When). The new jo…
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: The pull request changes only six CI YAML files. The diff adds no Go or other test source files and no Ginkgo declarations such as It(), Describe(), Context(), or When(). It only changes…
Topology-Aware Scheduling Compatibility ✅ Passed PASS — The PR changes only ci-operator configuration and generated Prow job YAML. The changes rename/split E2E jobs, set environment variables, add a Claude post-step, and add CI pod wiring. No deploy…
Ote Binary Stdout Contract ✅ Passed PASS. The pull request changes six CI YAML files only; it adds or updates job configuration, environment variables, shell commands, and generated Prow jobs. It does not change an OTE binary or suite s…
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PASS — The PR adds no Ginkgo test code or test files. The diff contains only CI YAML changes, and the new jobs invoke the existing make ... test-e2e target. No new IPv4 assumptions or external-conne…
No-Weak-Crypto ✅ Passed PASS: The complete PR diff adds CI YAML metadata, job names, Make commands, environment variables, the existing claude-ai-helpers image reference, and secret mounts. Searches of all added lines foun…
Container-Privileges ✅ Passed No changed container or Kubernetes manifest introduces a listed privilege. The PR diff adds CI image references, environment variables, post-step references, and job renames. The generated Prow job ch…
No-Sensitive-Data-In-Logs ✅ Passed No changed line logs a password, token, API key, PII, session ID, hostname, or customer data. The PR adds CI configuration, test names, branch names, and references an existing `oadp-analyze-e2e-failu…
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes both main changes: splitting the KubeVirt E2E job for OADP 5.0 and adding Claude failure analysis to the KDM controller and plugin jobs.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (4 skipped: 4 unsupported.)

Full details: Stable And Deterministic Test Names

Explanation

PASS. The pull request changes only CI YAML configuration and generated Prow YAML; it adds no Ginkgo It, Describe, Context, or When titles. The introduced TEST_NAME, as, context, and job identifiers are literal static strings such as virt-nokdm-e2e-test-aws and virt-kdm-e2e-test-aws. They contain no pod suffixes, timestamps, UUIDs, node names, namespaces, IP addresses, or generated values.

Full details: Test Structure And Quality

Explanation

PASS: The pull request changes only six YAML CI configuration files. The diff contains no Go test files or Ginkgo test code, and no It, BeforeEach, AfterEach, Eventually, or Consistently constructs. Therefore, the listed Ginkgo test-quality requirements are not applicable.

Full details: Microshift Test Compatibility

Explanation

PASS: The pull request adds or changes CI YAML job definitions only. The verified PR diff contains no added Go files and no new Ginkgo declarations (It, Describe, Context, or When). The new jobs run on openshift-org-aws, and the changed files do not introduce MicroShift-incompatible test code. Therefore, the custom check does not apply.

Full details: Single Node Openshift (Sno) Test Compatibility

Explanation

PASS: The pull request changes only six CI YAML files. The diff adds no Go or other test source files and no Ginkgo declarations such as It(), Describe(), Context(), or When(). It only changes CI job names, Makefile arguments, images, environment variables, and post-steps. The SNO test-compatibility check is therefore not applicable.

Full details: Topology-Aware Scheduling Compatibility

Explanation

PASS — The PR changes only ci-operator configuration and generated Prow job YAML. The changes rename/split E2E jobs, set environment variables, add a Claude post-step, and add CI pod wiring. No deployment manifest, operator/controller source, or scheduling constraint is added. The changed lines contain no affinity, topology spread, nodeSelector, control-plane/worker targeting, tolerations, replica-count logic, or PDB settings. Therefore none of the custom check's topology-failure conditions applies.

Full details: Ote Binary Stdout Contract

Explanation

PASS. The pull request changes six CI YAML files only; it adds or updates job configuration, environment variables, shell commands, and generated Prow jobs. It does not change an OTE binary or suite setup code. No changed lines contain process-level stdout patterns such as fmt.Print*, klog, RunSpecs, BeforeSuite, or TestMain. The check's failure condition is therefore not introduced by this pull request.

Full details: Ipv6 And Disconnected Network Test Compatibility

Explanation

PASS — The PR adds no Ginkgo test code or test files. The diff contains only CI YAML changes, and the new jobs invoke the existing make ... test-e2e target. No new IPv4 assumptions or external-connectivity logic was introduced in test source.

Full details: No-Weak-Crypto

Explanation

PASS: The complete PR diff adds CI YAML metadata, job names, Make commands, environment variables, the existing claude-ai-helpers image reference, and secret mounts. Searches of all added lines found no MD5, SHA1, DES, 3DES, RC4, Blowfish, or ECB usage, custom cryptography, or secret/token comparisons. The referenced oadp-analyze-e2e-failure step uses secret redaction and does not introduce weak cryptography.

Full details: Container-Privileges

Explanation

No changed container or Kubernetes manifest introduces a listed privilege. The PR diff adds CI image references, environment variables, post-step references, and job renames. The generated Prow job changes contain no privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, allowPrivilegeEscalation, securityContext, or root-user setting. OO_PSA_ENFORCE_PRIVILEGED: "true" is pre-existing in the KDM CI configs and is an environment variable, not a container security field.

Full details: No-Sensitive-Data-In-Logs

Explanation

No changed line logs a password, token, API key, PII, session ID, hostname, or customer data. The PR adds CI configuration, test names, branch names, and references an existing oadp-analyze-e2e-failure step. That step redirects Claude output to artifacts and applies secret redaction before retaining the report and transcript. The existing OADP test credential mounts are not printed or changed to emit values.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@kaovilai
kaovilai marked this pull request as draft September 1, 2026 04:15
@kaovilai

kaovilai commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

/pj-rehearse abort

Note

Responses generated with Claude

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 1, 2026
@openshift-ci
openshift-ci Bot requested review from Joeavaikath and sseago September 1, 2026 04:18
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 1, 2026
@kaovilai
kaovilai marked this pull request as ready for review September 1, 2026 14:52
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 1, 2026
@openshift-ci
openshift-ci Bot requested a review from mpryc September 1, 2026 14:53
…post-step

Wires the Claude-based e2e failure analysis post-step (added for
oadp-operator in openshift#83957) into the kubevirt-datamover
virt-kdm-e2e-test-aws jobs too, for both oadp-1.6 and oadp-dev configs
of kubevirt-datamover-controller and kubevirt-datamover-plugin:

- base_images.claude-ai-helpers (ci namespace, shared image)
- steps.env.OADP_BRANCH / TEST_NAME (per config file)
- steps.post: oadp-analyze-e2e-failure ref, before the ipi-aws-post
  teardown chain

Reuses the existing OADP-scoped ref as-is since these jobs install and
test against oadp-operator itself (test-oadp-operator base image,
oadp-operator e2e suite via TEST_VIRT_KDM=true).

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@kaovilai: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-oadp-operator-oadp-dev-5.0-virt-kdm-e2e-test-aws openshift/oadp-operator presubmit Presubmit changed
pull-ci-openshift-oadp-operator-oadp-dev-5.0-virt-nokdm-e2e-test-aws openshift/oadp-operator presubmit Presubmit changed
pull-ci-migtools-kubevirt-datamover-plugin-oadp-1.6-images migtools/kubevirt-datamover-plugin presubmit Ci-operator config changed
pull-ci-migtools-kubevirt-datamover-plugin-oadp-1.6-virt-kdm-e2e-test-aws migtools/kubevirt-datamover-plugin presubmit Ci-operator config changed
pull-ci-migtools-kubevirt-datamover-plugin-oadp-dev-images migtools/kubevirt-datamover-plugin presubmit Ci-operator config changed
pull-ci-migtools-kubevirt-datamover-plugin-oadp-dev-virt-kdm-e2e-test-aws migtools/kubevirt-datamover-plugin presubmit Ci-operator config changed
pull-ci-migtools-kubevirt-datamover-controller-oadp-1.6-images migtools/kubevirt-datamover-controller presubmit Ci-operator config changed
pull-ci-migtools-kubevirt-datamover-controller-oadp-1.6-virt-kdm-e2e-test-aws migtools/kubevirt-datamover-controller presubmit Ci-operator config changed
pull-ci-migtools-kubevirt-datamover-controller-oadp-dev-images migtools/kubevirt-datamover-controller presubmit Ci-operator config changed
pull-ci-migtools-kubevirt-datamover-controller-oadp-dev-virt-kdm-e2e-test-aws migtools/kubevirt-datamover-controller presubmit Ci-operator config changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Sep 1, 2026
@openshift-ci

openshift-ci Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kaovilai, shubham-pampattiwar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kaovilai
kaovilai marked this pull request as draft September 1, 2026 15:57
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 1, 2026
@kaovilai
kaovilai marked this pull request as ready for review September 1, 2026 15:57
@kaovilai
kaovilai marked this pull request as draft September 1, 2026 15:57
@openshift-ci openshift-ci Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. and removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Sep 1, 2026
@kaovilai kaovilai changed the title oadp-operator: split e2e-test-kubevirt-aws into kdm/non-kdm jobs (5.0) oadp: split kubevirt e2e job (5.0) and add Claude failure analysis to kdm-controller/plugin Sep 1, 2026
@kaovilai
kaovilai marked this pull request as ready for review September 1, 2026 17:06
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 1, 2026
@kaovilai

kaovilai commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

/pj-rehearse pull-ci-migtools-kubevirt-datamover-controller-oadp-1.6-virt-kdm-e2e-test-aws pull-ci-migtools-kubevirt-datamover-plugin-oadp-1.6-virt-kdm-e2e-test-aws

(openshift/oadp-operator#2424, cherry-pick of openshift/oadp-operator#2423 to oadp-1.6, has merged — rehearsing the oadp-1.6 kdm-controller/plugin jobs now. Holding the 5.0/oadp-dev jobs until openshift/oadp-operator#2423 itself merges.)

Note

Responses generated with Claude

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@kaovilai

kaovilai commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

/pj-rehearse ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Sep 1, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit 746f261 into openshift:main Sep 1, 2026
16 of 18 checks passed
kaovilai added a commit to kaovilai/release that referenced this pull request Sep 2, 2026
…p-1.6)

Same fix as the earlier oadp-dev/5.0 split (openshift#84337),
now backported to oadp-1.6's own e2e-test-kubevirt-aws presubmit across
all three variants (4.22, 4.23, 5.0). openshift/oadp-operator#2424
(cherry-pick of openshift#2423 to oadp-1.6) gave this branch's Makefile the same
TEST_VIRT_KDM support, so the split is safe here too.

- virt-nokdm-e2e-test-aws: TEST_VIRT=true TEST_VIRT_KDM=false
- virt-kdm-e2e-test-aws: TEST_VIRT_KDM=true

Prompted by openshift/oadp-operator#2427 (unpends two more kdm restore
specs on oadp-1.6), which risks pushing the still-unsplit job back over
the 2h step timeout the same way oadp-dev/5.0 hit it originally.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
openshift-merge-bot Bot pushed a commit that referenced this pull request Sep 2, 2026
…p-1.6) (#84490)

Same fix as the earlier oadp-dev/5.0 split (#84337),
now backported to oadp-1.6's own e2e-test-kubevirt-aws presubmit across
all three variants (4.22, 4.23, 5.0). openshift/oadp-operator#2424
(cherry-pick of #2423 to oadp-1.6) gave this branch's Makefile the same
TEST_VIRT_KDM support, so the split is safe here too.

- virt-nokdm-e2e-test-aws: TEST_VIRT=true TEST_VIRT_KDM=false
- virt-kdm-e2e-test-aws: TEST_VIRT_KDM=true

Prompted by openshift/oadp-operator#2427 (unpends two more kdm restore
specs on oadp-1.6), which risks pushing the still-unsplit job back over
the 2h step timeout the same way oadp-dev/5.0 hit it originally.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
openshift-merge-bot Bot pushed a commit that referenced this pull request Sep 4, 2026
…jobs (#84592)

Same split already applied to the presubmits (#84337,
#84490), now extended to the daily periodics we'd deliberately left
unsplit at the time. All branches now have TEST_VIRT_KDM support
(openshift/oadp-operator#2423, #2424), so the periodics are safe to
split too:

- virt-nokdm-e2e-test-aws-periodic: TEST_VIRT=true TEST_VIRT_KDM=false
- virt-kdm-e2e-test-aws-periodic: TEST_VIRT_KDM=true

Applied to all 5 variant configs that carry this periodic:
oadp-dev (5.0, 5.1) and oadp-1.6 (4.22, 4.23, 5.0). Both split jobs
keep the same cron schedule and Slack reporter_config as the job they
replace.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants