Skip to content

Strengthen the union × protocol conformance suites (follow-up to #48 phase 3) #56

Description

@aaylward

Follow-up from the review discussion on the phase-3 union suites (protocol-tests/unions/, added in PR #55). The suites are mutation-checked (a flipped expectation fails the run), but a mutation check only proves the assertions are load-bearing — these are the places where the assertions themselves are weaker than they should be.

  • Reject cells assert only !ok() / "an error envelope", not which error. A rejection for the wrong reason — e.g. a generic parse failure instead of the union's exactly-one-member diagnostic, or a jsonRpc2 error envelope with the wrong code — passes today. Pin the error kind and at least a message substring per cell ("expected exactly one union member", "unknown or missing union member", the jsonRpc2 error code/data.__type), so the reject paths are pinned to their diagnoses, not just their outcomes.
  • Member-type coverage is string/int/struct only. SinkChoice has text/count/nested members, so union × {blob, timestamp (each format), list, map, enum/intEnum, recursive} cells are untested for cbor and jsonRpc2. Rather than growing the shared roundtrip fixture (which churns three protocol variants of goldens), generate a union-heavy model through the Bazel smithy_cpp_*_library path — the same pattern as codegen/compile-tests/ — and run the four-direction cells against it. A recursive union member also exercises the smithy::Boxed path inside a variant.
  • Wire comparison is Document-level, not byte-level. EncodesEachVariant… decodes the captured body and compares smithy::Documents, so structure is pinned but encoding is not (map ordering, integer widths). The codec's determinism is separately pinned in cbor_test.cc, but one representative byte-exact vector per protocol (hex/base64 of a full union request body) would close the seam between "codec is deterministic" and "serde feeds the codec what we think".
  • The __type-tolerance expectation is implementation-derived. The exactly-one count deliberately excludes a __type key (error-payload discriminators), and the test pins that from reading the serde, not from a spec clause. Document the intended contract (where __type may appear next to a union member and why) in the serde/codegen docs or as a code comment the generator owns, so the test has an authority to point at.
  • Error-shape unions are untested. The tolerance exists for error payloads, but no test round-trips a modeled error carrying a union member over cbor/jsonRpc2. If any fixture grows one, add that cell — it's the path the __type arithmetic actually protects.

Related: #48 (parent tracking issue), PR #55 (the suites as merged), and the same critique applies in milder form to the phase-2 malformed-server suites (statuses and messages were reverse-engineered from the generated servers; the enum/pattern/length messages are pinned but the SerializationException messages are not).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions