Skip to content

feat(gha): run the affected unit tests against OpenTofu - #447

Open
jsteinich wants to merge 2 commits into
open-constructs:mainfrom
jsteinich:feat/opentofu-unit-axis-restored
Open

jsteinich wants to merge 2 commits into
open-constructs:mainfrom
jsteinich:feat/opentofu-unit-axis-restored

Conversation

@jsteinich

Copy link
Copy Markdown
Contributor

Related issue

Re-lands #439, reverted in #446. Stacked on #443 — merge that first; this branch contains its commit.

If this is approved before #446, #446 can be dropped. Otherwise this is the follow-up to it.

Why #439 failed

Its diff touched only .github/workflows/pr-unit.yml. The job it adds runs nx affected -t test --exclude='*,!tag:unit-test:terraform', so on a workflow-only change nx affected resolved to no terraform-tagged projects and the OpenTofu job ran zero tests. The green check proved nothing, and it then failed on every PR that touched a tagged project.

This branch touches provider-generator and provider-schema, so the tofu job runs for real on its own PR. That is the point — a change to pr-unit.yml alone can never demonstrate this.

What actually had to be fixed

Four fixtures, in two classes. The first was expected; the second was not, and is the more interesting one.

Providers OpenTofu will not install

  • versions-file.test.ts pinned kreuzwerker/docker 2.16.0, provider.test.ts pinned 3.0.2. Both rejected as unsigned. Moved to 3.9.0 — signing starts at 3.7.0, not at the 3.x major (test(tests): move the python providers fixture to Docker provider 3.9.0 #440).
  • provider.test.ts used andsafe-AG/bitbucket, which is on the Terraform registry but 404s on OpenTofu's. That test needs two same-named providers from different namespaces, so it moves to zahiar/bitbucket, which is on both.

Fully qualified names leaking the fetching CLI into expectations

The schema JSON is keyed by FQPN, so the host records which binary fetched it, not anything about the code under test. Three places baked Terraform's in:

file was now
provider-schema.test.ts snapshots raw schema incl. FQPN keys keys stubbed in the existing sanitizer
versions-file.test.ts asserted registry.terraform.io/${fqn} compares the provider part
provider.test.ts directory snapshot incl. versions.json FQPN keys normalized

For provider-schema.test.ts this extends a pattern already there: its sanitizeJson stubs format_version, cli_name and cli_version for exactly this reason — cli_name is already stubbed because it stamps the fetching binary, and the FQPN host is the same signal through a different field.

Keys only. A registry URL appearing inside provider documentation text is identical under either CLI and stays as authored.

Normalizing happens in the tests, not the product: the FQPN host is meaningful at runtime — #443 and #445 depend on it — so stripping it in the product would destroy information.

Verified against both CLIs

Not assumed, and not inferred from a green matrix job:

package terraform OpenTofu 1.12.6
@cdktn/provider-generator 23 suites / 117 tests / 101 snapshots same, all pass
@cdktn/provider-schema 6 suites / 58 tests / 10 snapshots same, all pass

The generated documentation links that also differed by CLI are fixed by #443, which this is stacked on — without it, provider.test.ts still diverges.

🤖 Generated with Claude Code

@sakul-learning

sakul-learning commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

after looking into Jest snapshot capabilities to mask out variants.

Short answer: yes, but not as a config-level pattern rule — the hook is a custom snapshot serializer. There is no regex/replace option for snapshot content, and snapshotFormat cannot host one. A serializer can, and it applies to object keys as well as values, which is what this case needs.

What Jest actually offers

mechanism scope fit for a registry host
expect.addSnapshotSerializer({ test, print }) / snapshotSerializers config every value the test callback claims ✅ the pattern rule, in the right layer: it changes what is written and compared, not the value under test
snapshotFormat (escapeString, printBasicPrototype, maxDepth) pretty-format formatting options only ❌ docs explicitly exclude plugins and compareKeys, so no substitution hook
snapshotResolver resolves test → .snap file paths ❌ paths only, never content
property matchers, toMatchSnapshot({ createdAt: expect.any(Date) }) named properties at known paths ❌ FQPN keys are arbitrary; the docs' advice for volatile strings is to normalise before snapshotting, which is exactly the alternative below

Keys were the thing worth checking, since the variance here is in keys. Verified with the repo's jest 30: a serializer whose test matches strings is applied to object keys, not only to values.

// registry-serializer.cjs
module.exports = {
  test: (val) => typeof val === "string" && val.includes("registry.opentofu.org"),
  print: (val) =>
    JSON.stringify(val.replaceAll("registry.opentofu.org", "registry.terraform.io")),
};
// jest.config.js
module.exports = { snapshotSerializers: ["<rootDir>/registry-serializer.cjs"] };

Snapshotting { "registry.opentofu.org/hashicorp/null": "3.1.0" } writes:

exports[`value and key handling 1`] = `
{
  "registry.terraform.io/hashicorp/null": "3.1.0",
}
`;

Nested FQPN keys included, from one shared module. Note the serializer must be a JS/CJS file the snapshot runtime can require — a .ts serializer needs a require hook, so a plain .cjs next to the preset is the low-friction shape.

How I'd apply it to the registry-host variance

The variance is a single fact — the host records which CLI fetched the schema, not anything about the generated code — and it reaches expectations through three snapshot sites plus one assertion:

site how it leaks serializer handles it?
packages/@cdktn/provider-schema/src/__tests__/provider-schema.test.ts FQPN keys in provider_schemas / provider_versions ✅
packages/@cdktn/provider-generator/src/__tests__/provider.test.ts FQPN keys inside the directory snapshot (versions.json) ✅
packages/@cdktn/provider-generator/src/get/__tests__/generator/versions-file.test.ts:67 plain toEqual against registry.terraform.io/${fqn} ❌ assertions never go through serializers — this one still needs value-level normalisation

So the pragmatic split is: keep the value-level normalisation for the non-snapshot assertion, and consider moving the snapshot-side handling into one serializer registered where both packages pick it up (shared preset, or snapshotSerializers per package config) rather than a helper per file per package. Both work today; the serializer also covers the next test that snapshots schema JSON, which would otherwise surface as a red tofu job long after the change that introduced it — the same late-surprise shape as #439 itself.

Two notes on the existing attempt, neither blocking:

  • Rewriting both hosts to a STUBBED_REGISTRY/ token is honest and registry-neutral, but it invalidates the committed snapshots and forces their regeneration. Mapping registry.opentofu.org → registry.terraform.io keeps existing expectations valid with a zero-line snapshot diff. Either is defensible; it is a diff-size trade.
  • The provider pins are right: verified against OpenTofu 1.12.6 that kreuzwerker/docker 2.16.0 fails with the provider is not signed with a valid signing key while 3.9.0 installs cleanly. For the registry-absence case, andsafe-AG/bitbucket is genuinely absent from registry.opentofu.org (the registry API 404s), and pinning the host explicitly (source: "registry.terraform.io/andsafe-AG/bitbucket") also installs under OpenTofu — so keeping the original provider is an option if the fixture/snapshot churn from switching to zahiar/bitbucket is unwanted.

Separate finding on this PR: the tofu row appears twice

.github/workflows/pr-unit.yml on this branch has two identical matrix entries:

          - terraform_version: "1.12.6"
            terraform_binary: tofu1.12.6
          - terraform_version: "1.12.6"
            terraform_binary: tofu1.12.6

That fans the job out to four rows and produces two identically named Affected Unit Tests (terraform) (1.12.6, tofu1.12.6) checks — one of which is pure duplicate work. Looks like the row re-added for the revert hasn't been dropped now that the branch sits on a main that already carries #439's version of it.

Docs: snapshotSerializers · snapshotFormat · snapshotResolver · expect.addSnapshotSerializer · property matchers

@jsteinich
jsteinich force-pushed the feat/opentofu-unit-axis-restored branch from 7d67654 to b996c63 Compare September 19, 2026 14:50
@jsteinich
jsteinich marked this pull request as ready for review September 23, 2026 01:44
@jsteinich
jsteinich requested a review from a team as a code owner September 23, 2026 01:44
@sakul-learning

Copy link
Copy Markdown
Contributor

The duplicate tofu1.12.6 matrix row from my earlier comment is resolved at b996c63. I found two remaining issues in the registry-targeting changes:

Follow-up PR: changing registry targets can leave generated documentation links stale

packages/@cdktn/provider-generator/src/get/generator/models/resource-model.ts now makes ResourceModel.linkToDocs depend on targetVersions: OpenTofu-only projects emit search.opentofu.org links, while Terraform or dual-target projects emit registry.terraform.io links. However, ConstructsMaker.filterAlreadyGenerated() in packages/@cdktn/provider-generator/src/get/constructs-maker.ts still decides freshness only from the provider-version map and CDKTN version; it does not compare the targetVersions written to constraints.json.

As a result, if a project runs cdktn get, switches between Terraform/dual-target and OpenTofu-only without changing its provider versions, and runs cdktn get again, the provider is skipped and its generated README/JSDoc links retain the old registry. This can be handled in a follow-up PR by including the registry-selection class (OpenTofu-only versus Terraform/default) in the generation cache key. A regression test should cover that cross-registry transition; changing only a version range within the same registry class does not need to regenerate bindings.

Fix in this PR: OpenTofu URLs are still labeled “Terraform Registry”

For an OpenTofu-only project, ResourceModel.linkToDocs now returns an OpenTofu URL, but TerraformProviderGenerator.emitResourceReadme() still emits Refer to the Terraform Registry for docs, and StructEmitter.emitInterface() still emits Docs at Terraform Registry. The generated text is therefore contradictory, for example labeling a https://search.opentofu.org/... link as Terraform Registry documentation.

Please fix this in the current PR, either by making the display name part of the registry abstraction or by using registry-neutral wording such as “provider documentation.”

@so0k

so0k commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

for the first finding.. if changing the project targetVersions doesn't follow a change of the binary used to run get (which .. would trigger a new fetch and gen?)

them not doing a churn on the generated docstrings is - by design?

jsteinich and others added 2 commits September 23, 2026 08:15
Re-lands open-constructs#439 with the fixtures actually fixed. That PR passed vacuously -
its diff touched only pr-unit.yml, so `nx affected` resolved to no
terraform-tagged projects and the tofu job ran zero tests. It then failed on
every PR that touched one, and was reverted in open-constructs#446.

This branch touches provider-generator and provider-schema, so the tofu job
runs for real on its own PR.

Four things had to change, in two classes.

Unsigned providers OpenTofu rejects:

- versions-file.test.ts pinned kreuzwerker/docker 2.16.0 and provider.test.ts
  pinned 3.0.2. Both move to 3.9.0; signing starts at 3.7.0, not at the 3.x
  major (open-constructs#440).
- provider.test.ts used andsafe-AG/bitbucket, which exists on the Terraform
  registry but not on OpenTofu's. It needs two same-named providers from
  different namespaces, so it moves to zahiar/bitbucket, which is on both.

Fully qualified names leaking the fetching CLI into expectations. The schema
JSON is keyed by FQPN, so the host records which binary fetched it rather
than anything about the code under test:

- provider-schema.test.ts snapshots the raw schema. Its sanitizer already
  stubs format_version, cli_name and cli_version for this exact reason;
  provider_schemas and provider_versions keys now get the same treatment.
  Keys only - a host inside provider documentation text is identical either
  way and stays as authored.
- versions-file.test.ts asserted keys equal to `registry.terraform.io/${fqn}`;
  it now compares the provider part.
- provider.test.ts snapshots a generated directory including versions.json;
  FQPN keys are normalized there too.

Verified against both CLIs rather than assumed: @cdktn/provider-generator
23 suites / 117 tests / 101 snapshots and @cdktn/provider-schema 6 suites /
58 tests / 10 snapshots pass under terraform and under OpenTofu 1.12.6.

The generated docs links that also differed by CLI are fixed by open-constructs#443, which
this is stacked on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nts at

Review finding on open-constructs#447. open-constructs#443 made linkToDocs follow the project's declared
targetVersions, but the prose around it stayed hardcoded: an OpenTofu-only
project got "Refer to the Terraform Registry for docs" above a
search.opentofu.org link, and "Docs at Terraform Registry" on every struct
attribute.

The registry now carries its own display name, and ResourceModel exposes it
beside the link it already derives, so the two cannot disagree. Taking the
abstraction option rather than neutral wording keeps the text specific -
readers of an OpenTofu project see "OpenTofu Registry", not "provider
documentation".

No snapshots move: every fixture declares no targetVersions, which resolves
to Terraform, so the emitted text is unchanged. provider-generator stays at
23 suites / 117 tests / 101 snapshots. A unit test covers the OpenTofu branch
that no fixture exercises, asserting the name and the link host together.

Not changed: module-generator's "Docs at Terraform Registry" for registry
modules. That one hardcodes its own registry.terraform.io module URL, so it
is self-consistent; module sources are a separate question from provider
docs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants