Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ Implemented:
`docs/spec/done/0020-explicit-file-system-selection.md`
- FS token authentication and configuration-free access from
`docs/spec/done/0018-fs-token-auth-and-config-free-access.md`
- Server-backed Filesystem token lifecycle management from
`docs/spec/done/0030-file-system-token-lifecycle-management.md`
- install and update distribution from
`docs/spec/done/0012-install-and-update-distribution.md`
- English PingCAP Preview documentation from
Expand All @@ -96,6 +98,13 @@ Implemented:
- `ti db execute-sql-statement`
- `ti fs create-file-system`
- `ti fs import-file-system-token`
- `ti fs generate-file-system-token`
- `ti fs generate-file-system-scoped-token`
- `ti fs list-file-system-tokens`
- `ti fs enable-file-system-token`
- `ti fs disable-file-system-token`
- `ti fs delete-file-system-token`
- `ti fs refresh-file-system-token`
- `ti fs delete-file-system`
- `ti fs list-file-systems`
- `ti fs describe-file-system`
Expand Down Expand Up @@ -264,6 +273,12 @@ vault grant reads, vault mount read on macOS/Linux hosts when available,
journal create/append/read/search/verify, public Git clone/hydrate/worktree
flows, mount and drain through the companion runtime, and explicit WebDAV
fallback when the platform supports it.
The live FS family also generates one uniquely named finite token on the test
Filesystem, verifies secret-free list metadata and data-plane access, then
disables, enables, refreshes, and deletes only that token while tolerating the
documented authentication-cache convergence delay. It must verify that refresh
preserves token ID, invalidates the old plaintext, and never mutates a
provisioning or pre-existing token.
If remote inventory has no resource with a local token, the suite creates one
temporary ti fs resource, records the server-assigned ID, and deletes only
that ID before the DB lifecycle needs the Starter slot or when the process
Expand Down Expand Up @@ -325,6 +340,7 @@ internal/dryrun/ shared dry-run result envelope
internal/fs/ ti fs control-plane, data-plane, and mount use cases
internal/fs/fscred/ ID-keyed ti fs credentials, selection, and legacy migration
internal/fs/mountlocator/ non-secret Drive9 background mount routing state
internal/fs/tokenmgmt/ Filesystem token lifecycle and local rotation safety
internal/oplog/ local JSONL operation log writer
internal/output/ structured JSON/text/raw rendering
internal/query/ JMESPath query application
Expand Down Expand Up @@ -388,6 +404,17 @@ Follow these rules unless `docs/priciples.md` is updated:
resource and local credentials when waiting fails.
- `ti fs delete-file-system` is asynchronous. After Drive9 accepts deletion,
output status is `deleting`, not `deleted`.
- One remote Filesystem can have multiple tokens, but one profile stores at
most one selected local token per Filesystem. Remote token inventory and
lifecycle state are authoritative; local credentials are not a token wallet.
- Filesystem token generation returns plaintext once and does not change local
selection unless `--store-locally` is explicit. Replacing local selection
never revokes the previous remote token.
- Filesystem token refresh is bearer-only and non-idempotent. Do not retry it
after an ambiguous network failure. Generate, list, enable, disable, and
delete use only TiDB Cloud public/private keys.
- Reject refresh, disable, or deletion of a token correlated with a known
active local mount. The error must show exact drain and unmount commands.
- Read-only commands reject `--dry-run`.
- Apply `--query` after command execution and before rendering.
- Users provide cloud placement as one canonical `region_code`, never as
Expand Down Expand Up @@ -505,6 +532,14 @@ Implemented command behavior:
- `ti fs create-file-system --wait`
- `ti fs create-file-system --dry-run`
- `ti fs import-file-system-token --from-file ./fs-token`
- `ti fs generate-file-system-token --file-system-id <file-system-id> --token-name ci --ttl 24h`
- `ti fs generate-file-system-token --file-system-id <file-system-id> --token-name local --no-expiration --store-locally --replace`
- `TI_FS_TOKEN=<owner-token> ti fs generate-file-system-scoped-token --ttl 24h --allow /workspace:read,list,write --subject sandbox-agent`
- `ti fs list-file-system-tokens --file-system-id <file-system-id>`
- `ti fs disable-file-system-token --file-system-id <file-system-id> --token-id <token-id>`
- `ti fs enable-file-system-token --file-system-id <file-system-id> --token-id <token-id>`
- `ti fs delete-file-system-token --file-system-id <file-system-id> --token-id <token-id>`
- `ti fs refresh-file-system-token --file-system-id <file-system-id>`
- `ti fs delete-file-system --file-system-id <file-system-id>`
- `ti fs delete-file-system --file-system-id <file-system-id> --dry-run`
- `ti fs list-file-systems`
Expand Down Expand Up @@ -603,6 +638,13 @@ Registered command surface:
- `ti db execute-sql-statement`
- `ti fs create-file-system`
- `ti fs import-file-system-token`
- `ti fs generate-file-system-token`
- `ti fs generate-file-system-scoped-token`
- `ti fs list-file-system-tokens`
- `ti fs enable-file-system-token`
- `ti fs disable-file-system-token`
- `ti fs delete-file-system-token`
- `ti fs refresh-file-system-token`
- `ti fs delete-file-system`
- `ti fs list-file-systems`
- `ti fs describe-file-system`
Expand Down Expand Up @@ -879,6 +921,14 @@ profile. `ti fs create-file-system`, remote list/describe, and
`ti fs delete-file-system` remain TiDB Cloud-authenticated. Delete requires an
ID but does not require a locally stored owner token.

Filesystem token generate/list/enable/disable/delete commands also use only
TiDB Cloud public/private keys and always require an explicit File System ID.
Refresh instead resolves exactly one bearer token from `--fs-token`,
`TI_FS_TOKEN`, or the selected local credential. A local refresh atomically
replaces that selected credential; a flag or environment refresh returns the
new plaintext without writing local or external secret-manager state. Never
infer a missing token ID from names, list order, timestamps, or token claims.

The ID selector is available on ti fs data-plane/runtime commands and all
`fs-git`, `fs-journal`, and `fs-vault` subcommands. Creation accepts no ID;
description and deletion require an ID. Drain and unmount resolve an existing
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ live-e2e-db: build
$(LIVE_E2E_RUN) -run '^TestLiveDB'

live-e2e-fs: build
$(LIVE_E2E_RUN) -run '^TestLive(FSRemoteInventoryLifecycle|FSCommandSurface|FSConfigurationFreeAccess|FSDataPlaneLifecycle|FSMountRuntime|FSWebDAVMountRuntime)$$'
$(LIVE_E2E_RUN) -run '^TestLive(FSRemoteInventoryLifecycle|FSCommandSurface|FSFileSystemTokenLifecycle|FSConfigurationFreeAccess|FSDataPlaneLifecycle|FSMountRuntime|FSWebDAVMountRuntime)$$'

live-e2e-fs-git: build
$(LIVE_E2E_RUN) -run '^TestLiveFSGit'
Expand Down
41 changes: 41 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,40 @@ ti fs list-files

`create-file-system` does not accept a user-defined name. Drive9 assigns the stable `file_system_id`, and the command returns the owner credential as `fs_token` once in its JSON result. Treat it as a secret. The example above captures both fields from one provisioning request and removes the temporary owner-only JSON file immediately.

One Filesystem can have multiple independently managed tokens for different machines, CI jobs, and sandboxes. Owner tokens authorize the complete Filesystem and can issue path-and-operation-limited `fs_scoped` tokens. The remote service is the source of truth for token inventory, while each local profile stores at most one selected token for each Filesystem. Generate an additional owner token and capture its one-time plaintext response:

```shell
umask 077
ti fs generate-file-system-token \
--file-system-id "$FILE_SYSTEM_ID" \
--token-name ci-deploy \
--ttl 24h > ./ci-token.json
ti fs list-file-system-tokens --file-system-id "$FILE_SYSTEM_ID" --output text
```

Use an owner token to issue a finite scoped token. Repeat `--allow`; supported operations are `read`, `list`, `search`, `write`, and `delete`, and `search` requires `read`:

```shell
export TI_FS_TOKEN="<OWNER_FS_TOKEN>"
ti fs generate-file-system-scoped-token \
--subject sandbox-agent \
--ttl 24h \
--allow /workspace:read,list,write \
--allow /artifacts:read,list
```

`TI_FS_TOKEN` may contain either token kind. Scoped tokens work only for allowed paths and operations and can self-refresh; they cannot generate child tokens or manage token inventory. Explicit `--fs-token` takes precedence over the environment. Token list, enable, disable, and delete use an explicit/environment owner token when present, otherwise they use configured TiDB Cloud API keys. With owner Bearer authentication, enable and disable apply only to scoped targets; TiDB Cloud credentials can manage either token kind. Because the token JWT does not expose its kind or scopes, the FS backend is the final permission authority.

Generation does not modify local credentials by default. Add `--store-locally` to select the new token locally; if a selected token already exists, add `--replace` explicitly. Replacing local selection does not revoke the previous remote token. Use immutable `token_id` values from the list response to disable, enable, or permanently revoke a token:

```shell
ti fs disable-file-system-token --file-system-id "$FILE_SYSTEM_ID" --token-id <token-id>
ti fs enable-file-system-token --file-system-id "$FILE_SYSTEM_ID" --token-id <token-id>
ti fs delete-file-system-token --file-system-id "$FILE_SYSTEM_ID" --token-id <token-id>
```

Rotate a locally selected token with `ti fs refresh-file-system-token --file-system-id "$FILE_SYSTEM_ID"`. To rotate a token supplied by a secret manager, set `TI_FS_TOKEN` and `TI_REGION_CODE`; `ti` returns the replacement plaintext but cannot update the external secret manager. Refresh is not safely retryable if the response is lost. For shared environments, generate and distribute a replacement first, validate it, then disable and delete the old token. Authentication state can take approximately 10 seconds to converge. Before refreshing, disabling, or deleting a token used by a local mount, run `drain-file-system` and `unmount-file-system`.

An agent sandbox can then use that existing file system without running `ti configure` or providing TiDB Cloud API keys:

```shell
Expand Down Expand Up @@ -232,6 +266,13 @@ ti db execute-sql-statement

ti fs create-file-system
ti fs import-file-system-token
ti fs generate-file-system-token
ti fs generate-file-system-scoped-token
ti fs list-file-system-tokens
ti fs enable-file-system-token
ti fs disable-file-system-token
ti fs delete-file-system-token
ti fs refresh-file-system-token
ti fs delete-file-system
ti fs list-file-systems
ti fs describe-file-system
Expand Down
4 changes: 2 additions & 2 deletions docs/spec/done/0012-install-and-update-distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Goal

Make `tdc` installable and updatable through deterministic GitHub Releases artifacts. The MVP channel is GoReleaser plus GitHub Releases, with shell and PowerShell installers. Homebrew and Scoop are intentionally deferred to `docs/spec/0030-homebrew-and-scoop-distribution.md`.
Make `tdc` installable and updatable through deterministic GitHub Releases artifacts. The MVP channel is GoReleaser plus GitHub Releases, with shell and PowerShell installers. Homebrew and Scoop are intentionally deferred to `docs/spec/0031-homebrew-and-scoop-distribution.md`.

## User-facing Commands

Expand Down Expand Up @@ -266,7 +266,7 @@ Installer scripts:
- Silent auto-update.
- Updating TiDB Cloud credentials or DB SQL credentials.
- Config migrations that modify user config during update.
- Homebrew tap and Scoop bucket publishing. See `0030-homebrew-and-scoop-distribution.md`.
- Homebrew tap and Scoop bucket publishing. See `0031-homebrew-and-scoop-distribution.md`.
- Linux apt/yum repositories.
- Winget publishing.
- Notarization or binary signing beyond SHA-256 checksums for MVP.
Loading