Mount ServiceAccount token when dcgm-exporter env needs API access - #2721
Open
rajathagasthya wants to merge 1 commit into
Open
Mount ServiceAccount token when dcgm-exporter env needs API access#2721rajathagasthya wants to merge 1 commit into
rajathagasthya wants to merge 1 commit into
Conversation
rajathagasthya
force-pushed
the
fix-dcgm-exporter-token-for-configmap-data
branch
from
August 6, 2026 16:49
eed3a44 to
bccb920
Compare
rajathagasthya
marked this pull request as ready for review
August 6, 2026 20:56
rajathagasthya
requested review from
cdesiniotis,
karthikvetrivel,
rahulait,
shivamerla and
tariq1890
as code owners
August 6, 2026 20:56
The dcgm-exporter DaemonSet sets automountServiceAccountToken=false and re-enables it only when pod metadata enrichment is turned on through the typed enablePodLabels/enablePodUID fields. Users who configure the same features through raw dcgmExporter.env are missed by that condition, and setting DCGM_EXPORTER_CONFIGMAP_DATA crashes the exporter on startup since it cannot read the ServiceAccount token. This worked before the token was made opt-in. Mount the token when the user-provided env reads a ConfigMap or enables pod labels / pod UID. A DCGM_EXPORTER_CONFIGMAP_DATA value of "none" is the exporter default meaning no ConfigMap is read, so it does not count. Configurations that make no API calls keep the default. Apply the same env check to the cluster-scoped RBAC gate. Enrichment through raw env otherwise got a token but no matching permissions, leaving the pod informer with only the namespaced Role, which grants pod reads in the operator namespace without watch. The ConfigMap case keeps a separate condition since it needs no cluster-scoped RBAC: the namespaced Role already grants configmaps get/list and its RoleBinding is ungated. The daemonset test compared gpuv1.EnvVar against the container's corev1.EnvVar, which can never be equal. Both render as v1.EnvVar in failure output, so the mismatch stayed invisible until a test case set env on the ClusterPolicy. Signed-off-by: Rajath Agasthya <ragasthya@nvidia.com>
rajathagasthya
force-pushed
the
fix-dcgm-exporter-token-for-configmap-data
branch
from
August 7, 2026 13:28
bccb920 to
0cc3160
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The dcgm-exporter DaemonSet sets
automountServiceAccountToken=falseand re-enables it only when pod metadata enrichment is turned on throughenablePodLabels/enablePodUIDfields. Users who configure the same features throughdcgmExporter.envare missed by that condition, and settingDCGM_EXPORTER_CONFIGMAP_DATAcrashes the exporter on startup since it cannot read the ServiceAccount token. This worked before the token was made opt-in.Mount the token when the user-provided env reads a ConfigMap or enables pod labels/pod UID. A
DCGM_EXPORTER_CONFIGMAP_DATAvalue of none` is the exporter default meaning no ConfigMap is read, so it does not count. Configurations that make no API calls keep the default.Apply the same env check to the cluster-scoped RBAC gate. Enrichment through raw env otherwise got a token but no matching permissions, leaving the pod informer with only the namespaced Role, which grants pod reads in the operator namespace without watch. The ConfigMap case keeps a separate condition since it needs no cluster-scoped RBAC: the namespaced Role already grants configmaps get/list and its RoleBinding is ungated.
Testing
Metrics are available when
DCGM_EXPORTER_CONFIGMAP_DATAis set:Metrics are available when pod enrichment is set via env:
kubectl patch clusterpolicy cluster-policy --type merge -p \ '{"spec":{"dcgmExporter":{"env":[{"name":"DCGM_EXPORTER_KUBERNETES_ENABLE_POD_LABELS","value":"true"}]}}}'Checklist
make lint)make validate-generated-assets)make validate-modules)