Skip to content

Support federated and temporary S3 / SigV4 credentials #47 - #48

Open
achtsnits wants to merge 1 commit into
stac-extensions:mainfrom
achtsnits:s3_flows_credentialprovider
Open

achtsnits wants to merge 1 commit into
stac-extensions:mainfrom
achtsnits:s3_flows_credentialprovider

Conversation

@achtsnits

@achtsnits achtsnits commented Oct 18, 2026 •

Copy link
Copy Markdown

Addresses #47.

STAC login and S3 access are separate. GDAL/rasterio and fsspec/s3fs still
need S3 credentials. Today, that usually means extra setup for each provider.

Adds optional credentialProvider to the existing s3 scheme: describes how
a helper gets credentials for S3 SigV4 requests.

Changes

Three examples:

  1. s3_public: public reads.
    No credentials, signing, or Asset auth:refs.
  2. s3_sts: OIDC ID token →
    AssumeRoleWithWebIdentity → temporary S3 credentials.
    stsUrl and roleArn in metadata; RoleSessionName generated by the client.
    Use the returned AccessKeyId, SecretAccessKey, and SessionToken;
    renew before Expiration.
  3. s3_direct: JWT access token
    with the requested scope. Tenant signing values; JWT in SessionToken.
    Gateway checks the JWT and access policy. No STS call; login and token
    renewal still needed.

Both signed examples use type: "s3". Provider type:
assumeRoleWithWebIdentity or jwtSessionToken.
Input scheme linked through scheme-level auth:refs. Token kind fixed by the
profile; optional subjectTokenType to state it explicitly.

Also added the
Storage extension
to the examples: object_storage for endpoint, bucket, and region.
Hope this use is fine.

@cayetanobv

Copy link
Copy Markdown
Contributor

Thanks for building on the scheme-level auth:refs; I added those in #44 with the tokenExchange flow, so it's good to see them reused here. The STS profile looks good to me: stsUrl and roleArn are all the document needs to add on the S3 side, and the temporary keys only come back at runtime.

A few concerns on jwtSessionToken:

  1. Field names. secretAccessKey holds a public value here, but the name says secret. It may trip secret scanners and invite people to paste real keys into it. Could the fields be renamed, e.g. signingKeyId / signingValue?
  2. Secrets in metadata. "Private tenant values MAY appear only in metadata restricted to clients authorized to receive them" still allows a real secret in a STAC document. The profile already says signing values must not grant access without a valid JWT, but metadata gets cached, copied and indexed, so I'd go further and say signing values in metadata MUST be public.
  3. Scope of the PR. STS is a standard flow (AWS, MinIO, Ceph), while jwtSessionToken describes a gateway-specific convention; as the README text says, native AWS S3 does not support it. Splitting it into a separate PR would let STS land sooner while the direct profile is discussed.
  4. Where scope lives. scope on the provider is both what the client requests at sign-in and what the gateway checks in the JWT. With several input schemes in auth:refs, is the same scope string expected to work for all of them? It may be worth stating that explicitly.

On "Login and audience still need client configuration": the oidc schemes in the examples have no client ID, so a client can't sign in from the document alone. #50 proposes an optional clientId on openIdConnect schemes (issue #49). Once it lands, the examples could add it to oidc. For STS it would also cover the audience, since the ID token's aud is the client ID that the STS checks.

@achtsnits

Copy link
Copy Markdown
Author

Great input, thanks! Agree on splitting this PR. STS is definitely the more established flow and seems closer to acceptance, and we can discuss direct JWT separately.

Also agree on renaming the signing fields and requiring public values. We use the public tenant ID for both. Tokens and temporary credentials are obtained at runtime and stay out of STAC metadata.

On scope, we need to explain where the coupling sits:

  • With STS, the STS endpoint and S3/Gateway service must agree on credentials and policy enforcement. The external IdP can remain largely independent of S3, it supplies an ID token that STS trusts.

  • In our direct JWT flow, the coupling is between the IdP (Keycloak in our case) and S3/Gateway. This partly follows from our platform design, where IAM building block and platform services are sitting more closely together. The S3/Gateway expects a specific audience and client roles in the token, checks those claims rather than the scope, and applies its policy.

In our example storage permissions come from the assumed role or the gateway’s role checks. Our direct flow therefore needs no required storage scopes and can use an empty scope. But yes, other gateways may authorize through scopes and when required storage scopes are declared, they should be present in the access token and enforced by the gateway. So agree as well that provider-level scopes must work for all alternatives in auth:refs (otherwise, separate definitions are needed).

Thanks for pointing to #50. If I understand correctly, for STS the ID token’s audience must include that client ID, which STS must accept. For direct JWT access, the gateway audience can differ from the login client ID.

For next steps, I suggest:

  1. Let Add clientId to openIdConnect schemes #50 land
  2. Open a separate PR for the STS flow with clientId added (and state that STS must trust)
  3. Keep this PR focused on the direct JWT flow and address the feedback above.

@cayetanobv

Copy link
Copy Markdown
Contributor

Sounds good, the plan works for me. And yes, you read #50 correctly: for STS the ID token's aud is the login client ID, which the STS must accept, while for direct JWT the gateway audience can differ.

One suggestion on the order: the STS PR doesn't need to wait for #50. You could open it now and add clientId to the example once #50 is merged, so the two can be reviewed in parallel.

For the direct JWT PR, since your gateway authorizes on audience and client roles rather than scopes, it may be worth making scope optional and loosening the "gateway MUST check required scopes" wording so it covers both cases.

Before going further, it would be good to hear @m-mohr's opinion on the overall direction: the split into two PRs, the credentialProvider object on the s3 scheme, and clientId in #50.

@m-mohr

m-mohr commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

As with #44, going through OpenAPI first would make sense. Do they have existing plans to align to?

Haven't looked into the proposal myself yet.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants