test: Add conformance fixtures for two template-parsing divergences [DO NOT MERGE YET] - #156
Conversation
leongdl
changed the title
There was a problem hiding this comment.
Quorum verdict: NOT-GOOD (3/5; 2 GOOD-WITH-NITS) — claimed spec: RFC 0005 §3 / EL §1.2.1 int64 bounds.
Majority position: timeout: 9223372036854775808 is a plain YAML <posinteger> schema field containing no expression. RFC 0005's int64 rule governs the expression language; Template Schemas §5 states no upper bound on <posinteger>, so no normative text mandates rejection — an arbitrary-precision implementation accepting it is arguably conformant. Where an implementation does reject, it may reject at the YAML-parse layer, not via the named rule. Naming: the bare 3-- prefix means Template Schemas §3 (StepTemplate) under the suite convention, which isn't what's tested; EL references need the expr prefix. Recommend: reclassify as a spec question (does the base/EXPR schema bound literal integer fields?), rename accordingly, or drop from this PR. Minority notes rejection is service-safe (fixed-width data planes benefit), but that's an argument for a spec erratum, not for pinning it now.
There was a problem hiding this comment.
Quorum verdict: GOOD (5/5) — spec: §7.3/§7.3.1 Task.File.<name> is path-typed; EL §2.3.1 .name/.suffix properties; direct member access in a format string (RFC 0005 shows this exact form).
Explicit filename: run.txt makes both expected values deterministic; SUFFIX:.txt can't be satisfied by a wrong SUFFIX:run.txt. Distinct from the existing let-binding fixture (7.3--task-file-expr-properties): the inline {{Task.File.Run.name}} form exercises the validator path that misparses the dotted tail as an embedded-file key — exactly the divergence this PR documents. Header comment is a model of maintainer-in-a-year readability.
|
Quorum review (5 independent agents). Net: 1 NOT-GOOD, 1 GOOD. The Task.File property-access fixture is solid and pins a real validator divergence (the DO-NOT-MERGE flag reflects failing implementations, not a fixture defect). The int-literal fixture doesn't have a normative basis as written and needs renaming/reclassification. |
…e [DO NOT MERGE YET]
- 7.3--task-file-direct-property-access.test: direct property access on a
path-typed Task.File reference inside a format string
({{Task.File.Run.name}} / .suffix) is valid per Template Schemas 7.3.1
and Expression Language 2.3.1, with a pinned filename making both
expected values deterministic. Pins the known validator divergence that
mis-resolves the dotted tail as an embedded-file key.
Review: quorum-review fixes — dropped 3--int-literal-above-int64-max
.invalid (a plain <posinteger> schema field carries no expression; no
normative text bounds it at int64, so the rejection had no spec basis and
mis-used the section-number namespace). If literal schema fields should be
int64-bounded, that is a spec decision to make first.
Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
9a3da21 to
9aebc10
Compare
|
Quorum-review fixes applied and pushed (rebased onto mainline 3069673):
|
3 participants
Important
DO NOT MERGE YET. Opened as a draft on purpose. Each fixture here is
currently red on exactly one implementation, so merging before the two owning
bugs are fixed puts a permanent failure in both conformance suites. See
Merge sequencing below.
Adds two conformance fixtures for template-parsing (validation-time)
divergences found while reviewing RFC 0008 wrap-action support across the two
implementations. Neither was covered by the existing suite, which is why both
survived.
1.
EXPR/jobs/7.3--task-file-direct-property-access.test.yamlRFC 0005 declares
Task.File.<name>andEnv.File.<name>aspathtyped, andshows property access on one directly inside a format string:
An implementation whose template validator resolves a
Task.File.*reference bylooking the whole dotted tail up as an embedded-file key rejects that form:
…even though its own runtime resolves the expression correctly, and its
environment-template validator accepts the equivalent
Env.File.<name>.name.Why it was uncovered. The existing
7.3--task-file-expr-propertiesfixturereaches the property through a
letbinding (cfg = Task.File.config, thencfg.parent), and a property reached inside a function call(
len(Task.File.Run.name)) parses differently again. I verified all three forms:only the direct member-access form the RFC itself uses triggers the rejection,
and nothing exercised it.
filenameis set in the fixture so the expected value of.nameisdeterministic.
2.
EXPR/job_templates/3--int-literal-above-int64-max.invalid.yamlRFC 0005 §3 ("64-bit Signed Integer Type") bounds integers to −2⁶³ … 2⁶³−1 and
says any value outside that range is an error. This template's
timeoutis 2⁶³(
9223372036854775808), one past the maximum, so validation must reject it.An implementation whose model has no upper bound accepts the template and only
discovers the problem at run time — or converts it into a native duration type
and overflows. Rejecting it at validation is what lets a submitter see the error
before the job is ever scheduled.
Verified against both reference CLIs
Task.File.*property accessEach red is a tracked defect in a different implementation, and each fixture is
green on the other — which is the intended shape of a conformance test.
Merge sequencing
Merging this PR makes one suite red on each implementation until the owning bugs
land. Suggested order:
Task.File.<name>property-access validator in openjd-rs.Alternatively merge it first, deliberately, as the failing test that drives both
fixes — happy either way, but it should be a conscious choice rather than a
surprise.
Related, not included
An INT parameter default above 2⁶³−1 (
type: INT, default: 9223372036854775808)is accepted by both implementations today, verified with
openjd checkonboth CLIs. RFC 0005 §3 says it should not be. I deliberately did not add a
fixture for it, because it would sit permanently red on both suites and needs a
decision first: either fix both models, or clarify that the integer domain
constrains expression evaluation rather than literal parameter defaults.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.