Skip to content

test: [expected failures] parameter-type fixtures needing implementation fixes - #166

Draft
leongdl wants to merge 1 commit into
OpenJobDescription:mainlinefrom
leongdl:conformance-param-types-expected-failures
Draft

test: [expected failures] parameter-type fixtures needing implementation fixes#166
leongdl wants to merge 1 commit into
OpenJobDescription:mainlinefrom
leongdl:conformance-param-types-expected-failures

Conversation

@leongdl

@leongdl leongdl commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Important

Expected-failure fixtures — do not expect these to pass. Spec-correct fixtures that FAIL against at least one current implementation, parked across two proposed/ directories (EXPR and base) that the runner does not discover, so merging keeps the suite green. Per-fixture output, spec citation, and classification in each proposed/README.md. Promote each fixture unchanged when its fix lands.

Contents (5 fixtures, dual-implementation verified)

  • 2.13--list-int-item-int64-overflow.invalid and 2.16--list-list-int-inner-item-int64-overflow.invalid — a LIST[INT] element / LIST[LIST[INT]] inner element of 2^63 accepted at check by BOTH implementations. The known int64 acceptance bug, one container deeper: both validate int64 bounds for expression literals but not for values arriving as YAML data in list defaults.
  • 2.10--range-expr-endpoint-int64-max — openjd-rs FALSELY REJECTS a RANGE_EXPR default with endpoints near int64 max; probing shows an undocumented 2^62 endpoint cap. Python accepts. The spec grammar admits any int64 endpoint. Rust-side bug. (Side effect: the live companion negative at 2^63 currently rejects for the wrong reason.)
  • 3.4.1--task-param-type-case-insensitive — schema §2 says job AND task parameter type names become case-insensitive with EXPR; BOTH implementations reject mixed-case task parameter types ("unknown variant int") while accepting them for job parameters. Both-implementation gap.
  • base/.../2--type-lowercase-string.invalidtype: string in base (no extensions) must be rejected; openjd-rs accepts (unconditional case-insensitivity), Python correctly rejects. Rust-side bug; base is case-sensitive by spec.

Companion to the green-fixture PR from branch conformance-param-types-gaps.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quorum verdict: GOOD (5/5) — spec: §2.10 + §3.4.1.1.1 + EL §1.2.1 (2^63-1 endpoints must validate). Pins openjd-rs's undocumented 2^62 endpoint cap (rs-only per sweep); fix widens acceptance only. The README's note that the companion negative (PR #158's 2.10--range-expr-endpoint-int64-overflow) passes for the wrong reason until this accept is green is exactly right — promote as a pair. Nit: the quoted validation error looks pydantic-style (Python) for an rs-attributed bug — double-check which CLI produced it, and state that Python passes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quorum verdict: GOOD-WITH-NITS — spec: §2.13 + EL §1.2.1 (LIST[INT] element bound). Matches sweep (fails BOTH — say so; README says 'the current openjd CLI' without attribution). Nits: only fixture family in the five PRs without in-file header comments — add one (the bound comes from the EL type table, not §2.13); carry the same 'pending restoration of the RFC 0005 overflow text' hedge the expr-lang README uses for the identical bound. Accept twin: PR #158's 2.13--list-int-item-int64-max.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quorum verdict: GOOD-WITH-NITS — spec: §2.16 (inner element, one container deeper — a declared sweep axis, kept despite likely sharing 2.13's recursive code path). Same nits as 2.13: no header comment, no per-implementation attribution, needs the spec-restoration hedge. Accept twin: PR #158's 2.16--list-list-int-inner-item-int64-max.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quorum verdict: GOOD (5/5, one attribution fix) — spec: §2 intro (with EXPR, job AND task parameter type names are case-insensitive — the sentence explicitly covers task types). Acceptance-widening only; queue environments unaffected (they carry job params). Promotion-ready. Fix: the README quotes a serde/Rust-style error and diagnoses rs-specifically, but the sweep shows BOTH implementations fail — add Python's observation. Inventory note: the reject twin (task-param lowercase WITHOUT EXPR) doesn't exist anywhere — consider adding it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quorum verdict: GOOD-WITH-NITS (dedupe action) — spec: §2 (base type names case-sensitive; the gate is otherwise undetectable suite-wide — good justification). Matches sweep (rs-only; Python correctly rejects — README omits this, its PR #163 sibling states it). Action: duplicates PR #163's base/proposed/2--type-lowercase.invalid.yaml — same construct, same defect, two different proposed/ paths; if both PRs land the same pin exists twice. Keep one or differentiate the type token.

@leongdl

leongdl commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Quorum review (5 independent agents). 5 fixtures + 2 READMEs. Net: 3 GOOD, 2 GOOD-WITH-NITS. This branch's kind-level placement (EXPR/job_templates/proposed/, base/job_templates/proposed/) is the better convention of the two in use — promotion is 'move up one directory unchanged' — and the quorum recommends the other four expected-failures branches adopt it. Actions: dedupe 2--type-lowercase-string against PR #163's identical fixture; add per-implementation attribution to the README (sweep shows several entries fail BOTH or rs-only where the README is vague); add the missing header comments to the two list-overflow fixtures and carry the same spec-restoration hedge the expr-lang README uses for the identical int64 bound.

Five parked fixtures across two proposed/ dirs: LIST[INT] and LIST[LIST[INT]] element 2^63 acceptance (both implementations - the known int64 bug one container deeper), the openjd-rs 2^62 endpoint cap on RANGE_EXPR defaults, task-parameter type case-insensitivity unimplemented in both, and the base lowercase type-name acceptance in openjd-rs (Python correctly rejects). Per-fixture details in each proposed/README.md. Companion to conformance-param-types-gaps.

Review: quorum-review fixes — dropped 2--type-lowercase-string.invalid
(identical pin exists on the base expected-failures PR; that copy is the
single source); README renamed to README-param-types.md (family-named to
avoid add/add conflicts with co-located expected-failures PRs) and
corrected with per-implementation attribution (2.13/2.16 and 3.4.1 fail
BOTH implementations; 2.10 is rs-only — all re-verified against the
current upstream/main rs build this session); added the dropped RFC 0005
overflow-text hedge to the 2.13/2.16 entries to match the expr-lang
README; added header comments to the two list-overflow fixtures.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
@leongdl
leongdl force-pushed the conformance-param-types-expected-failures branch from 04735b7 to 2cb3fca Compare August 15, 2026 19:26
@leongdl

leongdl commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Quorum-review fixes applied and pushed (rebased onto mainline 3069673):

  • Dropped 2--type-lowercase-string.invalid.yaml and its README — it duplicated test: [expected failures] base fixtures needing implementation fixes #163's 2--type-lowercase.invalid.yaml (same construct, same rs-only defect); that branch's copy is the single source.
  • README renamed to README-param-types.md (family-named, kind-level convention) so co-located expected-failures PRs can't add/add-conflict on a shared README path.
  • Per-implementation attribution added: 2.13/2.16 list-overflow and 3.4.1 task-param case-insensitivity fail both implementations; 2.10--range-expr-endpoint-int64-max is rs-only (the quoted "Model validation error" format is genuinely openjd-rs output — same format as pydantic — re-verified against the current upstream/main build, where all four defects still reproduce).
  • Added the dropped-RFC-0005-overflow-text hedge to the 2.13/2.16 entries (matching the expr-lang README) and header comments to both list-overflow fixtures.
  • Noted the missing reject twin (task-param lowercase type WITHOUT EXPR) as a future addition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant