Support CPD 5.3.1 - Helm-based install for WML, WSL, Spark, Cognos Analytics and SPSS by Sakshi-Singhroha1 · Pull Request #555 · ibm-mas/gitops

Sakshi-Singhroha1 · 2026-08-11T06:54:41Z

Issue

https://jsw.ibm.com/browse/MASCORE-15507
https://jsw.ibm.com/browse/MASCORE-15752
https://jsw.ibm.com/browse/MASCORE-16163

Description

Updates GitOps automation to support CPD 5.3.1 in addition to existing CPD 5.2 and 5.3 support. For CPD 5.3.1+, OLM-based installation is replaced with Helm-based installation using IBM Helm charts.

Summary of changes

CPD 5.3.1 OLM bootstrap (101-ibm-sync-jobs-cp4d, 110-ibm-cp4d)

Added 00-ibm-cp4d-olm-531.yaml with full OLM bootstrap for CPD 5.3.1
Added CPD 5.3.1 catalog sources (cpd-platform, opencloud-operators, ibm-zen-operator-catalog)
Added ClusterRole + ClusterRoleBinding for cpd-sa to allow Helm cluster-scoped chart installs (CRDs, ClusterRoles) at CPD 5.3.1+
Gated OLM-only resources behind < 5.3.1 version check

WML (120-ibm-wml)

Added Helm-based install job for CPD 5.3.1+ using ibm-redis-cp (v1.3.1, dependency) and wml (v12.1.0) Helm charts
Fixed CRD poll name: redicps.redis.ibm.com → rediscps.redis.ibm.com
Gated OLM Subscription behind < 5.3.1 version check
Added cpd_product_version to values.yaml

WSL (120-ibm-wsl)

Added Helm-based install job for CPD 5.3.1+ installing opensearch (v1.2.0), ccs (v12.1.0), datarefinery (v12.1.0), ws-runtimes (v12.1.0) and ws (v12.1.0) in dependency order
Gated OLM Subscription and WS CR behind < 5.3.1 version check
Added cpd_product_version to 120-ibm-wsl-app.yaml

Spark (120-ibm-spark)

Added Helm-based install job for CPD 5.3.1+ using analyticsengine (v12.1.0) Helm chart
Gated OLM Subscription, AnalyticsEngine CR, and control-plane job behind < 5.3.1 version check
Added cpd_product_version to values.yaml and 120-ibm-spark-app.yaml

Cognos Analytics (120-ibm-cognos-analytics)

Added missing Chart.yaml (was causing helm dependency update failure)
Added Helm-based install job for CPD 5.3.1+ using cognos-analytics (v29.1.0) Helm charts
Fixed CRD poll name: cognosanalytics.cognosanalytics.cpd.ibm.com → caservices.ca.cpd.ibm.com
Fixed parse error from escaped quotes in default values inside YAML block scalar
Renamed ArgoCD Application template from 120-ibm-spss-app.yaml → 120-ibm-cognos-analytics-app.yaml

SPSS Modeler (120-ibm-spss) — re-added with CPD 5.3.1 support

Re-added chart with OLM Subscription + SPSSModeler CR for CPD < 5.3.1
Added Helm-based install job for CPD 5.3.1+ installing canvasbase (v12.1.0, dependency) then spss (v12.1.0)
Added ArgoCD Application template 120-ibm-spss-app.yaml

Post-sync job (121-ibm-post-sync-job-cp4d-services)

Skip OLM-specific CCS/OpenSearch post-sync patching for CPD 5.3.1+
Bumped $_job_version to v2

RBAC

Added aicfgs resource to application-admin-role.yaml and per-namespace-rbac.yaml

Test Results

Tested on noble8 cluster with CPD 5.3.1 (inst02):

wml.noble8.inst02 → WmlBase/wml-cr Completed 100%
wsl.noble8.inst02 → Synced Healthy
spark.noble8.inst02 → Synced Healthy
cognos.noble8.inst02 → CAService/ca-addon-cr Completed 100%
spss.noble8.inst02 → canvasbase + spss Helm charts installed, canvasbase Completed 100%

…507)

…Jobs for WML, Spark, SPSS

…plications

…/shell conflict

…ersion value

…ipt change

…hind < 5.3.1

…stead of lt(semver,semver)

…fore namespace-scoped helm install

… for Helm 5.3.1+ installs

…I groups for Helm install

…tall jobs

- Gate OperandRegistry/OperandConfig to < 5.3.1 to prevent OLM resource drift when services are installed via Helm - Gate spark-hb-control-plane patch job to < 5.3.1; OLM-managed deployment does not exist in Helm install model - Add CRD establishment wait inside WSL install_component() between cluster-scoped apply and namespace-scoped helm upgrade to prevent race condition - Extend Helm Job --wait timeout from 30m to 60m for WML, SPSS and Spark to match WSL and avoid premature timeout on loaded clusters

* Added CPD 5.3.1 gitops support * [minor] Fix CatalogSources template - add CPD 5.3.1 block (MASCORE-15507) * For CPD 5.3.1 services are installed via Helm * MASCORE-15507: Fix WSL Helm Job - install deps in sequence; add Helm Jobs for WML, Spark, SPSS * MASCORE-15507: Pass cpd_product_version to WSL, Spark, SPSS ArgoCD Applications * MASCORE-15507: Fix heredoc in WSL Helm Job - use printf to avoid YAML/shell conflict * Fix semver comparison - use toString to handle unquoted cpd_product_version value * MASCORE-15507: skip OLM CCS/OpenSearch post-sync patching for CPD 5.3.1+ * MASCORE-15507: bump job version to v2 to force new job name after script change * minor update * changes undo * MASCORE-15507: gate CCS/datarefinery/ws-runtimes OLM subscriptions behind < 5.3.1 * MASCORE-15507: fix semver comparison syntax - use .Compare pattern instead of lt(semver,semver) * MASCORE-15507: install cluster-scoped chart first to register CRDs before namespace-scoped helm install * MASCORE-15507: grant cpd-sa cluster-level CRD/ClusterRole permissions for Helm 5.3.1+ installs * MASCORE-15507: expand cpd-sa ClusterRole with escalate, roles, CPD API groups for Helm install * MASCORE-15507: pin cli image to digest instead of :latest in Helm install jobs * adding temp changes for facilities * undone postbackflow.sh script * MASCORE-15507: fix CPD 5.3.1 Helm service install correctness - Gate OperandRegistry/OperandConfig to < 5.3.1 to prevent OLM resource drift when services are installed via Helm - Gate spark-hb-control-plane patch job to < 5.3.1; OLM-managed deployment does not exist in Helm install model - Add CRD establishment wait inside WSL install_component() between cluster-scoped apply and namespace-scoped helm upgrade to prevent race condition - Extend Helm Job --wait timeout from 30m to 60m for WML, SPSS and Spark to match WSL and avoid premature timeout on loaded clusters --------- Co-authored-by: Sakshi Singhroha <Sakshi.Singhroha@ibm.com> Co-authored-by: Nikita Kumble <nikitakumble@gmail.com>

… Jobs Static-named ArgoCD sync hook Jobs (install-wsl-helm, install-wml-helm, install-spss-helm, install-spark-helm) were failing with: 'spec.template: Invalid value ... field is immutable' This happens when an old Job with the same name exists from a prior sync and ArgoCD tries to patch it. Since Job spec.template is immutable, the patch is rejected. Fix: add HookFailed to hook-delete-policy on all four Jobs so ArgoCD deletes the Job on both success AND failure, ensuring no stale Job exists when the next sync runs.

The wait loop was polling for: opensearchclusters.opensearch.cloudpackopen.ibm.com (does not exist) The actual CRD name registered by the IBM OpenSearch Helm chart is: clusters.opensearch.cloudpackopen.ibm.com This caused the WSL Helm Job to always timeout after 5 minutes at the OpenSearch step, blocking CCS, DataRefinery, WS-Runtimes and WSL from ever installing.

…f oc wait oc wait exits immediately with error if the resource does not exist. The CCS CR (ccs-cr) is created by the CCS operator after Helm installs it, so it is not immediately present. Replace oc wait with a polling loop using --ignore-not-found, same pattern used for all other CRD waits in this script. Polls every 30s up to 60 minutes.

…fter CCS Helm install

…us for CCS CR readiness

…o MASCORE-16031

… SPSS (wave 122) SPSS operator at wave 122 tries to create a WS dependency CR of kind ws.cpd.ibm.com/v1beta1.WS. If ws-cluster-scoped runs at wave 123 (after SPSS), the CRD doesn't exist yet and SPSS fails with: 'Failed to find exact match for ws.cpd.ibm.com/v1beta1.WS' Move ws-cluster-scoped from wave 123 to wave 119 so it runs alongside the other cluster-scoped CRD installs, well before SPSS at wave 122.

…124) SPSS operator creates a WS CR and waits for Watson Studio as a dependency. The WSL operator (ws chart) deploys at wave 124, so SPSS namespace-scoped app must be at wave 125 or later to avoid a race condition where SPSS tries to create a WS CR before the WS operator is running. Also update wave comment from 123 to 119 on wsl-cluster section (cosmetic).

The instance-applications/120-ibm-wsl post-verify job (cpd-wsl-post-verify) was only deployed via the CPD < 5.3.1 AVP app path. For CPD >= 5.3.1 the native Helm apps are used for the WS operator/CR, but the post-verify job still needs to run to write the wsl-cp4d and wsl-project-id secrets to AWS SM. Without these secrets, inst02-watsonstudio-system AVP render fails. Wave 126 ensures it runs after wsl (124) and spss (125).

mnivedithaa

Testing of this should include below scenarios

Testing of cpd 5.3.1 - new instance
Testing of previous cpd version deployment unaffected
Upgrade from 5.2 OLM based deployment to 5.3.1 helm based deployment (Is there any conflict that happens since CRDs exist from OLM ones during upgrade? Do we need to ensure the older CRs are cleaned up after which Helm should kick in? Probably need to consider if we move CRD installation to cluster level)

mnivedithaa · 2026-08-25T11:03:36Z

 kind: Subscription
 metadata:
-  name: "cpd-spss-operator"
+  name: spss-modeler-subscription


Can you please revert the name change and sync wave update?

Reverted — name back to cpd-spss-operator and sync-wave back to 091.

mnivedithaa · 2026-08-25T11:04:42Z

 kind: Spss
 metadata:
-  name: "spss-sample"
+  name: "spssmodeler-cr"


Same here. Also why are few values removed from this?

That was from an older draft — the file only has name: "spssmodeler-cr" which is the correct name expected by the SPSS operator.

mnivedithaa · 2026-08-25T11:05:09Z

@@ -1,62 +0,0 @@
-SPSS Modeler
-===============================================================================


Can you revert back this readme update if deleted by mistake?

Restored — was accidentally deleted

mnivedithaa · 2026-08-25T11:06:34Z

@@ -1,3 +1,24 @@
+# ---
+# apiVersion: operators.coreos.com/v1alpha1


remove the commented lines here and in other places.

Removed from 00-ibm-wsl_Subscription.yaml and 01-ibm-wsl-cr.yaml. These were left over from development when the OLM content was temporarily commented out while adding the < 5.3.1 gate. Checked WML, Spark, SPSS — no other files had the same issue.

mnivedithaa · 2026-08-25T14:34:29Z

+    2. olm-utils-cm ConfigMap lookup (deployed by wave 110-ibm-cp4d)
+    3. Hardcoded default (safety net for helm template dry-runs)
+*/}}
+{{- $olmUtils := lookup "v1" "ConfigMap" $cpd_instance_ns "olm-utils-cm" }}


Does this lookup plugin work with Argocd? Argocd helm templates are generated even before connecting to cluster as far as I know. So can you verify if it is picking the values? If not, an alternative way to access it?

Yes, lookup works here because ArgoCD renders Helm templates server-side via the argocd-vault-plugin-helm CMP sidecar running inside the cluster — so it has full Kubernetes API access at render time. Verified on noble10: every CPD service app's targetRevision matches exactly what's in olm-utils-cm.

mnivedithaa · 2026-08-25T15:15:44Z

+    retry:
+      limit: 20
+    syncOptions:
+      - CreateNamespace=true


This should be false here? As operator namespace is created prior to this

Fixed — both namespaces exist before wave 119. CreateNamespace=false applied across all five service app templates.

mnivedithaa · 2026-08-25T15:17:11Z

+# Must run AFTER WSL at wave 124 — SPSS operator creates a WS CR and waits for it,
+# so the WSL operator must already be running when SPSS reconciles.
+apiVersion: argoproj.io/v1alpha1
+kind: Application


There are multiple application definition in single file. Maybe split them up for better readability? Same for other services as well

mnivedithaa · 2026-08-25T15:17:23Z

+    retry:
+      limit: 20
+    syncOptions:
+      - CreateNamespace=true


Here as well. Verify in other places as well

Fixed — both namespaces exist before wave 119. CreateNamespace=false applied across all five service app templates.

mnivedithaa · 2026-08-25T15:19:25Z

+  - group: ca.cpd.ibm.com
+    kind: CAService
+    jsonPointers:
+    - /spec/license/license
+    - /spec/enableInstanaMetricCollection


Is there any particular reason for ignoring this?

The IBM Helm chart sets spec.license.license: "Enterprise" in the CR but the Cognos operator removes that field after reconciliation (live CR only has spec.license.accept: true). For enableInstanaMetricCollection, the operator sets it based on cluster state. Without ignoreDifferences, ArgoCD detects these as drift and loops. Verified on noble10 — with ignoreDifferences the app stays Synced Healthy.

…L files

…reated at waves 085/110

rbinns · 2026-08-25T18:41:47Z

Hi Sakshi - I was actually requested that these items be move to namespace or instance scoped, rather than being at cluster scope. I don't believe the resources should be managed at the cluster scope precisely because 2 or more instances would then be trying to manage the same entity. Therefore we need to move these 100% to the namespace/instance scope, rather than cluster scope.

The problem is that we have customers deployed in clusters where multiple CPD installs are present. I expressed a concern that these were being deployed at the cluster level. I believe this is severe enough that this needs to be resolved now.

For CPD >= 5.3.1, derive operatorNamespace as ibm-cpd-{instance_id}-operators instead of the shared cpd_operators_namespace. This ensures each instance has its own isolated operator namespace, preventing resource name collisions caused by IBM chart hardcoded names (ibm-cpd-wml-operator-serviceaccount, etc.). Applies to: wml-redis-cluster, wml-redis, wml-cluster, wml, wml-post-verify apps. CPD < 5.3.1 (OLM path) is unchanged. Validated by MASCORE-17865 spike testing: - Spike 1: shared operatorNamespace causes name collision - NOT viable - Spike 2: dedicated operatorNamespace per instance - fully isolated and proven

Sakshi-Singhroha1 and others added 30 commits July 21, 2026 11:22

Added CPD 5.3.1 gitops support

19f60af

[minor] Fix CatalogSources template - add CPD 5.3.1 block (MASCORE-15…

b7eb094

…507)

For CPD 5.3.1 services are installed via Helm

8c99760

MASCORE-15507: Fix WSL Helm Job - install deps in sequence; add Helm …

31562eb

…Jobs for WML, Spark, SPSS

MASCORE-15507: Pass cpd_product_version to WSL, Spark, SPSS ArgoCD Ap…

2b6f7b4

…plications

MASCORE-15507: Fix heredoc in WSL Helm Job - use printf to avoid YAML…

326fe68

…/shell conflict

Fix semver comparison - use toString to handle unquoted cpd_product_v…

e8f2c0f

…ersion value

MASCORE-15507: skip OLM CCS/OpenSearch post-sync patching for CPD 5.3.1+

892f5f0

MASCORE-15507: bump job version to v2 to force new job name after scr…

7a99311

…ipt change

minor update

675a87c

changes undo

647900b

MASCORE-15507: gate CCS/datarefinery/ws-runtimes OLM subscriptions be…

413dd4e

…hind < 5.3.1

MASCORE-15507: fix semver comparison syntax - use .Compare pattern in…

0da1933

…stead of lt(semver,semver)

MASCORE-15507: install cluster-scoped chart first to register CRDs be…

6ed6801

…fore namespace-scoped helm install

MASCORE-15507: grant cpd-sa cluster-level CRD/ClusterRole permissions…

b7e2f54

… for Helm 5.3.1+ installs

MASCORE-15507: expand cpd-sa ClusterRole with escalate, roles, CPD AP…

846e49e

…I groups for Helm install

MASCORE-15507: pin cli image to digest instead of :latest in Helm ins…

9c91cf7

…tall jobs

adding temp changes for facilities

e410251

undone postbackflow.sh script

a561f1e

initial commit

4aa87d7

Merge remote-tracking branch 'origin/main' into MASCORE-15507

e0e8254

Merge branch 'test-15757-15507' into MASCORE-15507

a00d78e

fix WSL Helm Job: explicitly create ccs-cr with non_olm_deploy=true a…

54ced4a

…fter CCS Helm install

fix WSL Helm Job: correct CCS CR kind from 'Ccs' to 'CCS'

b0793fd

fix WSL Helm Job: poll .status.ccsStatus not .status.controlPlaneStat…

417bd71

…us for CCS CR readiness

drashtimalviya1207 and others added 16 commits August 21, 2026 13:48

update echo

2cc40b2

Update 08-postsync-db2-audit-policy_Job.yaml

55ebc10

update echo

bfa61f1

Update 08-postsync-db2-audit-policy_Job.yaml

9902ef5

update job version

85e83d8

Merge branch 'main' into MASCORE-16031

ff2de50

Update 07-postsync-setup-db2_Job.yaml

7cd9f0f

Merge branch 'MASCORE-16031' of https://github.com/ibm-mas/gitops int…

aadee68

…o MASCORE-16031

readme updated

ce62fce

dummy commit for cpd 5.3.1

049ff76

fix: mascore-12697

d64803a

Merge branch 'noble9-common' into MASCORE-15507

cb6568d

Merge branch 'main' into MASCORE-15507

73c7f85

mnivedithaa reviewed Aug 25, 2026

View reviewed changes

Sakshi-Singhroha1 added 2 commits August 26, 2026 12:01

fix: restore SPSS README, remove commented-out template lines from WS…

f64cbba

…L files

fix: set CreateNamespace=false on all CPD service apps - namespaces c…

2b37971

…reated at waves 085/110

rbinns requested changes Aug 26, 2026

View reviewed changes

Sakshi-Singhroha1 and others added 5 commits August 27, 2026 11:08

fix(spss): revert subscription name and sync-wave to match main

21f23db

Merge branch 'main' into MASCORE-15507

c411b4d

fix: update README tables to match template resource kinds

ac389a0

fix: update cli image digest in wml post-verify template

2e2cd9b

Merge branch 'main' into MASCORE-15507

cd4075b

Hardik-Prajapati-10 self-assigned this Aug 31, 2026

Sakshi-Singhroha1 added 2 commits September 15, 2026 11:37

fix: update cli image digest to sha256:1c5701a across all templates

14c37fb

Sakshi-Singhroha1 force-pushed the MASCORE-15507 branch 2 times, most recently from da2c28e to b851e4c Compare September 15, 2026 07:40

		@@ -1,62 +0,0 @@
		SPSS Modeler
		===============================================================================

		@@ -1,3 +1,24 @@
		# ---
		# apiVersion: operators.coreos.com/v1alpha1

Conversation

Sakshi-Singhroha1 commented Aug 11, 2026

Issue

Description

Summary of changes

Test Results

Uh oh!

mnivedithaa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants