[petruanica]

Description of the issue

The Container Insights pipelines are dropping two groups of attributes that duplicate values already being sent: the deprecated semantic conventions net.host.name, net.host.port and http.scheme, and the resource-scope copies of the raw Prometheus labels. Neither had integration coverage.

The deprecated-semconv half has a specific failure worth guarding. The karpenter, keda, kserve-controlplane and knative-controlplane promote transforms delete url.scheme — the current name — while keeping http.scheme, the deprecated one, and no test detects it.

Description of changes

Adds a shared pair of checks in util/otelmetrics, then calls them from every suite whose pipeline is affected. Following the existing style of that package, the checks return an error rather than taking a *testing.T, so each suite wraps them in its own require.NoError.

CheckDeprecatedSemconvRemoved fails if any series carries one of the three deprecated attributes at resource scope. CheckCurrentSemconvPresent fails if no series carries url.scheme. The second is the valuable half: it fails if the inversion described above ever returns. CheckRawLabelsNotAtResourceScope fails if a raw Prometheus label appears at resource scope as well as datapoint scope.

Coverage per suite:

Suite	Deprecated removed	url.scheme present	Raw labels only at datapoint
standard (node-exporter, cadvisor, apiserver, kube-state-metrics)	yes	yes	cadvisor, kube-state-metrics
solutions/karpenter	yes	yes	yes
solutions/keda	yes	yes	yes
gpu (dcgm)	yes	no	yes
neuron	yes	no	n/a
ebs_csi	yes	no	n/a
lis_csi	yes	no	n/a

Two of those columns need explaining.

url.scheme is asserted present only where a missing value could not be explained by something other than a bug. The attribute-limit processor drops attributes by tier above max_total_attributes, and url.scheme is not on its protected list, so on a GPU, Trainium or heavily-labelled node a missing url.scheme would be ambiguous. The karpenter and keda metrics are cluster-scoped with small attribute sets, and they are also the pipelines that carried the inversion, so that is where the assertion earns its place.

The raw-label check is marked n/a where the pipeline runs no groupbyattrs, so no raw name reaches resource scope to begin with. neuron is the one exception: it runs groupbyattrs, but over the semantic-convention names plus runtime_tag, which neuron_promote renames and then deletes.

The datapoint copies, which are what dashboards read, must survive. Those are already covered by TestCadvisorHasRawPromotedKeys, TestDCGMHasRawPromotedKeys and the TestKSM_*_HasRaw*Label family, so the new tests check only the resource side.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

The three shared checks have unit tests in util/otelmetrics/attribute_scope_test.go, which run without a cluster: 12 subtests, all passing. They cover each key individually, a clean series, a single offending series among several, empty results, and the de-duplication of repeated failure messages.

gofmt clean and go vet -tags integration passes on all eight affected packages; every integration test binary compiles with the new tests registered.

The integration tests themselves need a live cluster running the paired configuration change. They will fail against a cluster that has not yet taken it — that is the intended behaviour, and the reason this PR is paired one-to-one with the config change.

Coverage for the vllm and kserve_controlplane pipelines is not in this PR. There is no suite for either yet, and the cluster used for validation runs neither solution, so a test added now would pass vacuously. The chart-side change for both is asserted on the rendered configuration in aws-observability/helm-charts#384.

Related pull requests

Chart change	aws-observability/helm-charts#384
Sibling Container Insights changes	#775, #777

PR Checklist

• Commits are squashed into a logical, reviewable set (one commit for a single change)
• Commits and PR description comply with Amazon internal guidelines
• gofmt clean, go vet -tags integration passes, test binary builds
• All GitHub Actions checks on the PR are passing
• Integration test evidence: N/A — this PR is the integration test coverage. The tests require a cluster running the paired configuration change and will fail until it is deployed, which is intentional.
• New or updated integration test coverage: N/A — this is the corresponding amazon-cloudwatch-agent-test PR
• New functionality has unit tests; bug fixes have a reproducing test — the three shared checks have unit tests that run without a cluster
• Config translation changes include updated golden files — N/A, no translator in this repo
• Breaking or customer-visible changes are called out in the PR description — N/A, test-only change

[miconeilaws]

• Squash commits
• I don't think these tests cover the integration pipelines like karpenter etc, do we also need these checks there?
• Other pipelines have no coverage. dcgm, neuron, ebs-csi and lis-csi also gained the semconv drop in #384, and dcgm also gained the resource-copy delete. Their suites exist (gpu, neuron, ebs_csi, lis_csi), so with a shared helper, one call per suite closes the gap.