Skip to content

feat(objectstore): expose checksum provenance - #504

Merged
ajroetker merged 6 commits into
mainfrom
feature/objectstore-checksums
Aug 19, 2026
Merged

ajroetker merged 6 commits into
mainfrom
feature/objectstore-checksums

Conversation

@ajroetker

@ajroetker ajroetker commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

What changed

  • adds checksum algorithm, value, scope, and full-object/composite type metadata to object-store objects
  • propagates checksum metadata through memory, filesystem, S3, and GCS adapters
  • requests S3 checksum metadata and recognizes AWS checksum algorithms, including the default CRC64NVME representation
  • keeps S3 and GCS metadata probes pinned to the requested object version
  • distinguishes whole-object checksums from checksums returned for ranged or part responses
  • retries safe S3 GET/HEAD reads without checksum mode when checksum retrieval needs additional permissions
  • adds a focused object-store test target and allocation-failure coverage

Why

Lake and sidecar work from #145 needs trustworthy checksum provenance, but that provider contract is independently useful and reviewable.

Impact

Callers can distinguish checksum encodings, stored-object versus response-body scope, and full-object versus composite calculations without adopting lake manifests or scanners. Versioned reads no longer risk pairing bytes with checksum metadata from the latest version, and optional checksum retrieval does not break otherwise-authorized reads.

Validation

  • zig build lib-objectstore-test --summary all: 53 passed, 2 provider integration tests skipped by environment
  • checksum-mode, permission fallback, versioned metadata, CRC32C/CRC64NVME, composite/full-object, ranged scope, and allocation-failure regressions covered
  • git diff --check

@ajroetker
ajroetker marked this pull request as ready for review August 19, 2026 21:34
@ajroetker
ajroetker merged commit 82a06e4 into main Aug 19, 2026
13 checks passed
@ajroetker
ajroetker deleted the feature/objectstore-checksums branch August 19, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant