Skip to content

test(gateway): cover all APIs with real-service E2E - #241

Merged
jialez0 merged 3 commits into
mainfrom
codex/gateway-api-e2e
Oct 16, 2026
Merged

jialez0 merged 3 commits into
mainfrom
codex/gateway-api-e2e

Conversation

@jialez0

@jialez0 jialez0 commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add real-service, contract-gated E2E coverage for all 45 logical Trustee Gateway API operations. The suite starts Gateway, KBS, gRPC AS, RESTful AS and RVPS production binaries; no upstream HTTP/gRPC services are mocked.

  • Exercise LocalFs, EncryptedLocalFs, EncryptedDb/SQLite, SharedStore/MySQL and EncryptedDb/MySQL. Both shared profiles run two independent copies of every component.
  • Run the full matrix with both the interpreter and Regorus VM in a new CI workflow.
  • Cover RCAR, nonce/client-key binding, independently verified JWT signatures, actual JWE decryption, resource CRUD, both AS aliases, policy CRUD, reference registration/import/append/refresh/delete, certificates, challenge-key rotation, configured OIDC/JWKS/discovery, resource-key reload/rewrap/rotate, audit, health, credentials and heartbeats.
  • Check denial paths, cross-replica session/data visibility, concurrent resource writes/reference merges, cache invalidation, and persistence after restarting a complete replica.
  • Fail ordinary Go CI when registered routes, logical RVPS dispatch, the E2E contract or API documentation drift. Every contract operation must have an actual successful service call; unsupported responses do not count.
  • Replace outdated API documentation with the live wire formats, authentication/state ownership, backend limits and current error semantics.

Defects exposed and fixed

  1. Gateway AS policy-list aliases were forwarded to the wrong backend path.
  2. KBS attestation-policy management and resource listing omitted administrator authentication.
  3. New wide Gateway audit tables inherited MySQL COMPACT and could fail startup; explicitly create them with DYNAMIC without altering server defaults or existing tables.
  4. Concurrent distinct-key MySQL KV writes could exhaust immediate deadlock retries. Add bounded jittered exponential backoff and a 32-writer regression without changing CAS semantics.

Validation

  • Interpreter: all 45 operations across all five profiles passed, with the Go race detector.
  • Regorus VM: the complete five-profile suite passed three consecutive times, with the Go race detector.
  • KV: all 14 tests passed against disposable MySQL; the 32-writer distinct-key regression passed 20 consecutive rounds.
  • Gateway: all Go tests with race detection, ordinary/tagged go vet, route/documentation guard passed.
  • Rust formatting, shell syntax and git diff checks passed.

CI creates isolated test databases, deliberately exercises MySQL COMPACT defaults, and uploads only credential-free API operation counters. Test keys/configuration/tokens/cookies/evidence and service logs are not uploaded.

Validation boundaries

The software sample verifier exercises the real protocol/policy/storage chain and report-data binding, not physical TDX/SNP/CSV/TPM authenticity. This does not claim hardware quote generation, production PKI, live cloud KMS/HSM or external OCI/Rekor validation. There is no generic Gateway cache-management endpoint: the suite checks observable policy/RVPS/key-cache invalidation. WASM in-memory persistence and existing WASM checks are retained.

Merge using rebase and merge only, after review and all CI checks pass.

Exercise all 45 Gateway API operations with both policy engines and five storage profiles, including two complete shared-state replicas. Guard route/documentation coverage and refresh the API guide against live protocols.

Fix missing KBS management authentication, AS policy-list routing, wide MySQL audit table initialization and burst-write transaction retry starvation exposed by the suite.

Signed-off-by: Jiale Zhang <zhangjiale@linux.alibaba.com>
@ostest-bot

Copy link
Copy Markdown

@jialez0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@jialez0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start 。

@jialez0 jialez0 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review of a8d1740 completed: checked route/contract/docs parity, real upstream processes, isolated loopback-only database ownership and cleanup, independent JWT verification and JWE decryption, shared sessions and persistence, cache invalidation, and bounded deadlock backoff without CAS changes. Gateway DYNAMIC table options affect new tables only. Artifacts contain operation counters only. Local E2E passed across five profiles with both engines (three full VM repetitions); 32-writer regression passed 20 rounds. No blocking findings. Hardware/external-service and deployment-auth boundaries are documented. Waiting for all CI before rebase-only merge.

Signed-off-by: Jiale Zhang <zhangjiale@linux.alibaba.com>
@ostest-bot

Copy link
Copy Markdown

@jialez0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@jialez0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start 。

@jialez0 jialez0 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up self-review: 682610d changes only Actionlint reporting. The earlier failure was reviewdog PATCH /check-runs returning 404, with zero lint annotations. Local reporting plus fail_level=error and nofilter preserves full workflow validation while removing check-run write dependence; contents remain read-only. All functional source/tests are unchanged from the locally verified revision.

Signed-off-by: Jiale Zhang <zhangjiale@linux.alibaba.com>
@ostest-bot

Copy link
Copy Markdown

@jialez0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@jialez0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start 。

@jialez0 jialez0 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final workflow self-review at 5faf4cf: upgraded the existing setup-go action to v6 because full Actionlint correctly rejects deprecated Node20 actions. Workflow Files and Go Test/Vet are now green. Re-ran KV Clippy with -D warnings and all 14 KV tests successfully. No functional source changed after the local E2E matrix. Awaiting remaining checks before rebase merge.

@jialez0
jialez0 merged commit fca1f56 into main Oct 16, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants