Store only object metadata headers

[peterjan]

Currently metadataHeaders keep every X-Amz- request header as object metadata and s3d serves it all back on GET and HEAD. This PR adds an allowlist to prevent that considering it leaks secrets on public buckets. It also eats into the metadata size budget and returns headers that might claim things we don't even support.

Master, unauthenticated GET on a public bucket:

HTTP/1.1 200 OK
X-Amz-Content-Sha256: ee44b2a3ae5e84e9fde84e85ddad5b90f11e43b2ee22a8e30187254b2d100ef5
X-Amz-Date: 20260916T140543Z
X-Amz-Security-Token: FwoGZXIvYXdzEBYaDFAKESTSSESSIONTOKENabc123
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key: pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=
X-Amz-Server-Side-Encryption-Customer-Key-Md5: DWygnHRtgiJ77HCm+1rvHw==

Kopia, a tool that I was testing, sends exotic headers that get accepted and then repeated on GET/HEAD:

PutObject with x-amz-object-lock-mode GOVERNANCE   accepted
HeadObject reports ObjectLockMode                  "GOVERNANCE"
GetObjectRetention on the same object              501
DeleteObject inside the retain window              accepted, object gone

[Copilot]

🟡 Changes recommended

SSE header validation currently relies on Header.Get(...) != "", which can fail to reject security-sensitive headers when they are present with empty values.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Implements S3 bucket encryption configuration endpoints and ensures object-related responses consistently report x-amz-server-side-encryption: AES256, reflecting that objects are encrypted before leaving the gateway. It also prevents sensitive request plumbing headers (including SSE-C and request-signing headers) from being persisted and echoed back as object metadata.

Changes:

• Add bucket encryption subresource routing plus backend/store plumbing for persisting/retrieving the bucket encryption configuration.
• Enforce/reject SSE request headers (reject SSE-C and KMS rather than ignoring) and always emit the SSE-S3 response header (AES256) on object responses.
• Narrow persisted object metadata to an allowlist and filter legacy stored metadata to avoid serving sensitive headers back.

File summaries

File	Description
tox/tox_test.go	Enables running encryption-related tox tests by removing marker exclusions.
sia/sia.go	Extends the persistence Store interface with bucket encryption configuration methods.
sia/persist/sqlite/migrations.go	Adds SQLite migration to create bucket_encryption_configurations table.
sia/persist/sqlite/init.sql	Adds initial schema definition for bucket_encryption_configurations.
sia/persist/sqlite/encryption.go	Implements SQLite persistence for bucket encryption configuration CRUD.
sia/encryption.go	Implements Sia backend methods to marshal/unmarshal and store bucket encryption config.
s3/validation_test.go	Adds a test guarding the object-metadata allowlist behavior.
s3/s3.go	Extends the S3 backend interface with bucket encryption configuration APIs.
s3/objects.go	Adds SSE header validation on reads/writes, emits SSE response header, and implements metadata allowlisting (including legacy filtering on reads).
s3/multipart.go	Validates SSE headers for multipart operations and emits SSE response header on relevant responses.
s3/messages.go	Adds XML message types for bucket encryption configuration documents.
s3/encryption.go	Adds SSE header validation + bucket encryption subresource handlers and config validation logic.
s3/encryption_test.go	Adds end-to-end tests for bucket encryption config, SSE response headers, SSE rejection behavior, and legacy-metadata filtering.
s3/buckets.go	Routes ?encryption bucket subresource and removes it from unsupported list.
README.md	Documents SSE behavior and updates the API support table for bucket encryption endpoints.
internal/testutil/testutil.go	Adds test helpers for bucket encryption APIs and shared SSE-C raw headers.
.changeset/add_server_side_encryption_headers.md	Adds a changeset documenting the new SSE/bucket-encryption behavior.

Review details

Suppressed comments (2)

s3/encryption.go:104

• validateSSEReadHeaders only rejects the SSE algorithm header when Header.Get(headerSSE) is non-empty. If the header is present with an empty value, it will be treated as absent and not rejected.

func validateSSEReadHeaders(h http.Header) error {
	if h.Get(headerSSE) != "" {
		return fmt.Errorf("%q is not valid on a read: %w", headerSSE, s3errs.ErrInvalidArgument)
	} else if hasAnySSECustomerHeader(h) {

s3/encryption.go:90

• The bucket-key-enabled check is gated on Header.Get(...) != "". If the header is present but empty, the request will bypass validation instead of returning an InvalidArgument error.

	// only a request to enable a bucket key has to be refused
	if v := h.Get(headerSSEBucketKeyEnabled); v != "" {
		enabled, err := strconv.ParseBool(v)
		if err != nil {
			return fmt.Errorf("invalid %s value %q: %w", headerSSEBucketKeyEnabled, v, s3errs.ErrInvalidArgument)

• Files reviewed: 17/17 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

SSE-KMS/bucket-key headers are still effectively ignored on GET/HEAD requests, conflicting with the stated goal of rejecting unsupported encryption headers rather than ignoring them.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 18/18 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The changes are well-scoped, add targeted validation and tests, and the remaining feedback is limited to a minor comment wording issue.

Review details

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

KMS read headers and multipart completion can bypass rejection; checksum controls are persisted and website redirects are dropped.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

s3/encryption.go:108

• The read validator only rejects the target SSE header and SSE-C headers. A GET or HEAD carrying X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id or X-Amz-Server-Side-Encryption-Context therefore reaches the backend and silently ignores the KMS request, contrary to this change's promise that KMS is rejected. Reject the KMS-specific read headers before object access as well.

	} else if hasAnySSECustomerHeader(h) {
		return fmt.Errorf("customer provided encryption keys are not supported: %w", s3errs.ErrNotImplemented)

s3/multipart.go:558

• CompleteMultipartUpload is the remaining object-write path, but unlike multipart creation and part upload it never calls validateSSEWriteHeaders. A client can send SSE-C or KMS headers here and the backend still commits the object, so those headers are silently ignored. Validate the request before decoding or completing the upload.

	setSSEResponseHeader(w)

s3/objects.go:873

• X-Amz-Website-Redirect-Location is object metadata and was previously retained by the X-Amz- rule, but the new allowlist drops it on PUT and it is consequently missing from GET/HEAD responses. Add this supported object-metadata header to the allowlist rather than silently discarding a client's setting.

var objectMetadataHeaders = map[string]struct{}{
	"Cache-Control":       {},
	"Content-Disposition": {},
	"Content-Encoding":    {},
	"Content-Type":        {},
	"Expires":             {},
}

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Lite

[ChrisSchinnerl]

My problem with this is that

1. it effectively just lies to clients about the encryption since we don't apply AES
2. I'm not sure what use-case this enables apart from supporting something that other backend support. e.g. is there an app that doesn't work without it?
3. this is a massive PR for faking AES headers on a self-hosted tool for which you control the backend anyway

Looking at Cloudflare's compatibility it seems like they also don't support SSE. Which doesn't mean we can't have anything that Cloudflare doesn't support but if a player of their size doesn't feel like the demand is there maybe it's fine not to have it.

If we did add it I might honestly lean towards actually applying additional AES256 encryption simply for the sake that users who explicitly want that feature actually get what they want.
e.g. by using a configurable secret that never gets sent to our backend including snapshots. So even if you used an app like Revelio to access the same account, you wouldn't be able to decrypt any of the objects.
Although the need for this still feels quite unlikely.

[chris124567]

I agree with Chris and if we do really we want to support this just for the sake of some s3 clients we should implement the absolute bare minimum

[peterjan]

My problem with this is that
...

Hm, that's all fair points. I've reduced the PR to what I think is worth salvaging, the metadata headers allowlist. That's actually how I ended up at SSE in the first place.

We store every X-Amz- request header as object metadata and serve it back on GET/HEAD reqs, potentially leaking secrets on public buckets. It also eats into the metadata budget and makes s3d claim features it doesn't support because it's echoing whatever was sent on upload.

Master, unauthenticated GET on a public bucket:

HTTP/1.1 200 OK
X-Amz-Content-Sha256: ee44b2a3ae5e84e9fde84e85ddad5b90f11e43b2ee22a8e30187254b2d100ef5
X-Amz-Date: 20260916T140543Z
X-Amz-Security-Token: FwoGZXIvYXdzEBYaDFAKESTSSESSIONTOKENabc123
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key: pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=
X-Amz-Server-Side-Encryption-Customer-Key-Md5: DWygnHRtgiJ77HCm+1rvHw==

Kopia, a tool that I was testing, sends exotic headers that get accepted and then repeated on GET/HEAD:

    PutObject with x-amz-object-lock-mode GOVERNANCE   accepted
    HeadObject reports ObjectLockMode                  "GOVERNANCE"
    GetObjectRetention on the same object              501
    DeleteObject inside the retain window              accepted, object gone

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

🟡 Changes recommended

The new allowlist still includes several checksum headers that don’t appear to be supported/validated elsewhere, so clients can persist and later observe potentially misleading “supported” checksum metadata.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

Copilot review overview

🟢 Approval recommended

The changes directly address the stated security/correctness issue, include a data migration to prevent legacy metadata propagation, and add targeted tests covering the new filtering behavior.

Review effort: Lite
Findings: None

Resolved since last review (1)

• The object-metadata allowlist includes checksum headers (X-Amz-Checksum-Crc64nvme / -Md5 / -Sha512)…

[ChrisSchinnerl]

Got conflicts now