Skip to content

feat(cli)!: adopt owner-scoped operator configuration - #1115

Merged
ragnorc merged 8 commits into
ModernRelay:mainfrom
ragnorc:feat/operator-config-v1
Oct 21, 2026
Merged

ragnorc merged 8 commits into
ModernRelay:mainfrom
ragnorc:feat/operator-config-v1

Conversation

@ragnorc

@ragnorc ragnorc commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the legacy operator grammar with strict version-1 connections, resource targets, owner-local authentication profiles, aliases, and explicit selection. This is intentionally breaking; there is no compatibility importer.
  • Implement hosted onboarding, stable credential bindings and renames, explicit credential administration, managed data acquisition, invocation-frozen transport, and recovery-preserving failure handling.
  • Keep OS-backed human/stored-token custody separate from invocation-only env/file/exec credentials. Harden macOS errors and native writes/deletes without changing Keychain permissions.
  • Update the machine-readable schema, user/developer docs, embedded skill, and breaking release note. The RFC remains a draft decision record for review.

Deliberate boundaries

Version 1 supports local file direct targets and named HTTP server/managed targets. Cloud-direct schemas, standalone catalogue targets, Unix sockets, mTLS, arbitrary private identity-provider transport, and workload federation remain outside this revision. Served catalogue reads keep their existing server graph contract.

Custom destination CAs with HTTPS proxies are explicitly rejected; HTTP CONNECT proxies remain supported. Separate control/data routes preserve their own transport and credential boundaries.

Validation

  • CLI library: 309 passed; nine real-deadline/device-flow tests additionally exercised separately.
  • CLI integration: 328 passed; seven nightly deadline/live-apply tests additionally passed. Three opt-in integrations require external release binaries, a GQT binary, or a real Gemini API key and were not run.
  • CLI all-target Clippy with warnings denied, formatting, documentation (363 Markdown files), dependency provenance/advisory/license checks, typo checks, and test-owner/CI-cell checks pass.
  • Both CLI and server binaries build. The macOS linker emits the same large-unwind-table warning seen in the pristine baseline.

No live user Keychain or cloud mutation is used by the new synthetic credential regressions. The control-plane source was inspected at commit 2f61511; this is not a claim of live managed-service or BYOC qualification.

macOS release prerequisite

The native adapter no longer masks denied lookups with add attempts or ignores deletion status, and authentication stops on custody failures. This does not make ad-hoc code signatures stable across upgrades. Consistent Developer ID release signing still requires an approved signing identity and release-secret setup; this PR does not weaken ACLs or claim that deployment prerequisite is complete.

ragnorc commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Why this change matters

Operator configuration becomes one version-1 document. A connection names the control-plane API, the organization, and the owner-local auth profiles. A target names a managed cluster, a standalone server, or a local file graph. An alias names one stored read. config validate parses that document and does no network I/O.

Old documents fail closed. target_config_version is an unknown field. Duplicate YAML keys fail. The changelog, the user guide, and the skill all say there is no importer. That break removes a second on-disk grammar.

I request changes. Three defects let a checked document, a catalogue command, or an alias parameter disagree with the path that actually runs.

Blocking

1. config validate accepts graph and organization ids that execution refuses

The complete-document fixture stores default_graph: exact/graph:id and organization_id: org:issued/123. operator_schema::parse accepts both. That parse is the door config validate uses.

require_graph_segment then refuses that graph id: one path segment of ASCII letters, digits, -, or _. A query that uses the validated default graph stops before any request.

organization_id() accepts only 1–128 ASCII letters, digits, _, or -. select_with and check_account call it before login. The fixture value contains : and /.

The operator RFC says default_graph, alias graph, and --graph carry the graph id the API accepts. The server graph id is ^[a-zA-Z0-9-]{1,64}$. Managed graph_id() also requires a leading letter, forbids _, and reserves policies, healthz, openapi, and graphs. The schema check is nonempty(): any non-empty text without control characters. Alias graph uses that check. cluster_id and cluster_incarnation use it too, while execution uses a 1–256 letter, digit, -, or _ check.

This is one class. Every graph id and organization id has two doors, and the offline door is the wider one. The later command fails closed, so the CLI does not send the bad id. The operator still gets a success from the check the guide tells them to trust.

Fix this in the schema validator, with the functions the request path already calls. Make require_graph_segment, managed graph_id(), and the server GraphId one rule. Change the fixture that locks the split. Add a regression beside complete_document_resolves_paths_and_normalizes_routes_without_io: a document that parses must have graph and organization ids those functions accept. GQT cannot see operator-file validation.

crates/omnigraph-cli/src/operator_schema/validation.rs graph() only calls nonempty(). crates/omnigraph-cli/src/operator_schema/tests.rs locks default_graph: exact/graph:id and organization_id: org:issued/123.

2. A non-object --params value is dropped, and the alias read still runs

alias_params in crates/omnigraph-cli/src/commands/query/prepare.rs copies explicit JSON only when the root is an object. An array, string, number, boolean, or null leaves the parameter map unchanged and returns success.

A temporary call with a JSON array [1] returned Ok and an empty map. The stored read then runs with the alias defaults only. Where those defaults satisfy the query, the command succeeds with a different row set.

The version-1 alias text says caller parameters replace defaults as typed values. Refuse every non-object root in alias_params. Add that case beside alias_parameters_keep_defaults_positionals_and_explicit_precedence. GQT does not parse CLI JSON parameters. The function body is unchanged from main; version 1 now documents this path as the alias contract.

3. The catalogue inventory check still calls the removed selector

On main 50283cef, this command returns 22 @Operator rows and opens no store:

omnigraph query --target og_catalog -e 'query rows() { match { $o: @Operator } return { $o.name as name } order { $o.name } }' --json

This head exits 2:

target_unknown: unknown target 'og_catalog'; register it with omnigraph target add og_catalog, or write ./og_catalog for a relative path

target add registers a managed cluster. ./og_catalog is an ordinary store path. The user guide says version 1 has no standalone catalogue target, and a served read uses --graph og_catalog.

scripts/check-catalogue-docs.py still runs query --target og_catalog. Against this head it exits 1 with refused the catalogue query. CI job Test Workspace stops on that step, so the workspace tests do not run. CI Gate fails because that job fails.

crates/omnigraph-cli/tests/integration/cli_data.rs catalogue_scope_reads_the_binary_registries_without_a_store pins the refusal on purpose. TargetBinding::DirectCatalogue and GraphClient::Catalogue remain in the runtime. The schema rejects kind: catalogue, so that binding has no document form.

Fix the cause in one place. Keep one store-free selector and point the checker at it, or remove the unreachable binding and change the checker to a path this CLI still serves. The checker has to run with no store and no credentials. A relative path does not meet that need.

Optional

YAML anchors copy mappings

Tags and << merge keys fail. A YAML anchor still expands. This document parses, and both servers receive auth.public: anonymous:

version: 1
targets:
  server:
    kind: server
    endpoint: https://server.example
    auth: &auth
      public: {provider: anonymous}
  other:
    kind: server
    endpoint: https://other.example
    auth: *auth

The RFC says this YAML is a serialization format, with no inheritance. Reject anchor and alias tokens in the libyaml scan in crates/omnigraph-cli/src/operator_schema/yaml.rs that already rejects tag tokens. Add this case next to the merge-key test. The expanded form is the same text written twice, so this is a grammar hole.

Legacy errors can name the upgrade section

A target_config_version: 1 document fails with unknown field target_config_version, expected one of version, connections, targets, aliases, defaults. That is fail-closed, and document_grammar_is_strict covers it. One line that points at the upgrade section in docs/user/cli/targets.md would tell the operator what to write next.

Alias help still offers positional args

Version 1 rejects an args field. The alias command help still says positional args bind to that field. A positional value then fails because the alias declares zero args. Align the help with the schema.

Tradeoffs and liability

The new document removes the importer and the second on-disk grammar. That is lower long-term liability. The line count goes up because routing, transport, and credential providers are now explicit modules. The liability to watch is a second copy of each rule. Graph ids already have four checks. Organization ids have two. Catalogue has a runtime binding and no schema form. Five more target kinds on a new checker would fork this door again. Put the id rules and the YAML alias rule in the schema scan.

The RFC is still a draft while this breaking format is the user contract. The user guide matches the code. Accept the RFC, or the next edit will change a shipped format against a draft.

These open CLI changes will rebase onto this surface: #1114, #1113, #1104, and #1078.

This change does not alter Lance storage behavior. Direct targets are absolute local file:// URIs. Cloud-direct, catalogue targets, Unix sockets, and mTLS stay out of version 1. That deferral matches the RFC.

These paths held under test. A percent-encoded prefix stays encoded (https://example.com/caf%C3%A9/ round-trips). login --connection work --target cluster fails in clap. Duplicate keys fail at each mapping depth. Custom CA plus an HTTPS proxy stays rejected by the existing schema tests.

Validation

Reviewed head: 7a752ed9481ce055410ec3e4f65fe2dfd29db1b0. Main: 50283ceff03b8e3a1902d30757fd0808ea277b7d.

Local, this head:

CARGO_TARGET_DIR=/workspace/target cargo test -p omnigraph-cli --lib -- duplicates_are_rejected_at_every_mapping_depth document_grammar_is_strict complete_document_resolves_paths_and_normalizes_routes_without_io alias_parameters_keep_defaults_positionals_and_explicit_precedence connection_and_resource_defaults_are_independent --test-threads=8

Result: 5 passed.

Temporary probes at this head, restored afterward and not pushed:

  • Validated default_graph exact/graph:id fails require_graph_segment.
  • alias_params with JSON [1] returns Ok and an empty map.
  • The two-server YAML anchor parses and copies anonymous auth.
  • https://example.com/caf%C3%A9 is stored as https://example.com/caf%C3%A9/.
  • omnigraph login --connection work --target cluster fails clap parsing.
  • The legacy document produces the unknown-field message above.

Catalogue before, the main binary from 50283cef, same query as above: exit 0, 22 rows.

Catalogue after, this head's omnigraph binary, same query: exit 2, code target_unknown.

python3 scripts/check-catalogue-docs.py --binary <that binary>: exit 1, refused the catalogue query.

CI on this head: rustfmt, clippy, cargo deny, Azure, RustFS S3, and the DST pinned suite passed. Test Workspace failed in "Verify catalogue inventories against the built CLI". CI Gate failed because that job failed. GQT (served) failed gqt_served_smoke::gqt_server_binary_round_trip with error sending request for url (https://127.0.0.1:32845/graphs/smoke/schema). omnigraph-gqt and omnigraph-server sources are unchanged in this diff. I did not reproduce that smoke. GQT ordinary and GQT dst were cancelled before their test steps. The workspace test suite did not run.

GitHub reports mergeable_state: dirty for this head. git merge-tree --write-tree 50283cef 7a752ed9 exits 0. I found no conflict content locally.

Lance: this diff has no storage-path change to compare with the pinned Lance 13.0.0 sources.

Recommendation: request changes.

Inline comments to attach:

  • crates/omnigraph-cli/src/operator_schema/validation.rs:147 — Blocking. graph() only calls nonempty(), so config validate accepts exact/graph:id. require_graph_segment then refuses that id, and the server graph id is ^[a-zA-Z0-9-]{1,64}$. Use the execution grammar here, and use it for alias graph too. organization_id at the connection check has the same split: the fixture value org:issued/123 parses, and organization_id() refuses : and /.
  • crates/omnigraph-cli/src/operator_schema/tests.rs:26 — Blocking. This fixture locks the split. Parse succeeds for default_graph: exact/graph:id and for organization_id: org:issued/123 on line 7. A temporary probe then failed require_graph_segment on that graph id. Change the fixture when the validator and the request path share one grammar.
  • crates/omnigraph-cli/src/operator_schema/yaml.rs:38 — Optional. This scan rejects tag tokens. It lets YAML anchors through. A document with auth: &auth on one anonymous server and auth: *auth on a second server parses, and both targets receive the copied profile. The RFC forbids inheritance. Reject anchor and alias tokens in this same scan.
  • crates/omnigraph-cli/tests/integration/cli_data.rs:5406 — Blocking for the in-repo caller. This test pins removal of --target og_catalog. scripts/check-catalogue-docs.py still runs that command, so Test Workspace exits before the suite. On this head the binary exits 2 with target_unknown and tells the operator to target add og_catalog or write ./og_catalog. target add registers a managed cluster, and ./og_catalog opens a store. DirectCatalogue is still in the runtime and the schema cannot name it. Point the checker at one store-free path, or remove the unreachable binding in the same change.

@ragnorc

ragnorc commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the review in 372ba4b and follow-up 18280ab; merged current main in 400bc3b.

  • Offline and runtime identifier validation now share the server graph grammar and managed identifier limits; JSON Schema and boundary regressions agree.
  • Alias parameters reject every non-object root; unsupported positional arguments and stale help/schema fields are removed.
  • Added explicit store-free omnigraph catalogue and migrated the inventory checker, without restoring magic targets or adding a fourth target kind.
  • YAML anchors/aliases are rejected, alongside tags and duplicate keys.
  • Legacy config errors point to the upgrade guide; no automatic importer or backward compatibility was added.
  • Preserved main's new access command and migrated its fixture to v1 file URIs.

CI investigation: the catalogue inventory failure is fixed. The first attempt to supply the served GQT fixture CA via SSL_CERT_FILE was insufficient: GQT clears its worker environment. The smoke now uses the fixture's original loopback HTTP upstream, as before the shared CLI fixtures acquired TLS. Production transport and CLI TLS verification remain unchanged; no CA trust is disabled and no worker environment allowlist is widened. Updated the existing composed-route regression to expect offline rejection of its intentionally invalid nested graph ID.

Local validation on the merged code: 321 CLI library tests passed (serial), 334 integration tests passed, explicit served GQT smoke passed, catalogue inventories match the built binary, and documentation checks passed (374 files). One parallel library run hit the existing baseline lock-release assertion; its isolated rerun and full serial suite passed. The local GQT build uses workspace rustflags; the flagless Linux qualification remains the CI job's responsibility.

Latest head: 18280ab. CI is rerunning; not claiming it is green yet. RFC remains draft pending design approval.

ragnorc commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Why this change matters

Operator configuration becomes one version-1 document. A connection names the control-plane API, the organization, and the owner-local auth profiles. A target names a managed cluster, a standalone server, or a local file graph. config validate parses that document and does no network I/O.

The follow-up commits close the three blocking gaps from the first review. Identifier checks, alias JSON, and the catalogue command now agree with the path that runs. One required journey still passes a served URL as an inline target. Test Workspace fails on that journey, and CI Gate fails because that job fails.

I request changes.

Earlier findings

  1. Resolved. config validate and the request path share one graph grammar and one managed-id grammar. graph_id calls require_graph_segment for a served target and GraphId for a direct root. Organization, cluster, and incarnation ids call valid_managed_identifier. The fixture values org_issued-123 and 123-exact-graph match those functions. managed_id_boundaries_match_execution_issue_1115 and graph_id_boundaries_and_catalogue_scope_match_execution_issue_1115 passed on this head.

  2. Resolved. alias_params refuses an array, string, number, boolean, or null. Positional alias arguments are gone. alias_parameters_keep_defaults_and_explicit_precedence and alias_rejects_non_object_parameters_and_positional_arguments_without_requests passed.

  3. Resolved. omnigraph catalogue reads the binary registries with no store and no credentials. scripts/check-catalogue-docs.py calls that command. On this head the inventory step prints catalogue inventories match the binary. catalogue_scope_reads_the_binary_registries_without_a_store passed. DirectCatalogue is the internal binding of that command. The document has no catalogue target kind.

  4. Resolved. The libyaml scan rejects anchor and alias tokens. anchors_and_aliases_are_rejected_without_rejecting_literal_text passed.

  5. Resolved. A target_config_version document names docs/user/cli/targets.md#upgrade-from-the-old-format and says there is no importer. legacy_document_error_names_the_upgrade_guide passed.

  6. Resolved. Alias help shows --params. The args field is gone.

Blocking

The v0.9 rebuild journey still addresses the server with an inline URL

spawn_server_process publishes TestServer.base_url as https://127.0.0.1:<port>. scope::resolve refuses an inline target that contains :// and does not start with file://. The message is inline access supports local storage only.

query_command rewrites an https:// target to the registered name test-server. blob get, mutate, and branch merge still pass ["--target", server.base_url, "--graph", "knowledge"]. The reopened server builds that same array. blob get is the first of those commands. Its failure stops the test before mutate and branch merge run. They use the same array.

This is one class. Every current-binary served command in this journey must use the registered target. The 0.12 journey already does that with KEYED_SERVER_NAME, and it keeps legacy_base_url() for the predecessor binary. Copy that pattern for every current-binary command in the v0.9 journey. A rewrite inside query_command leaves the next command on the old path.

The product refusal is the right contract. The journey is the hole.

GQT cannot see CLI target selection. This existing test is the owner. It is not ignored. CI requires the line v0.9 refusal and export/import rebuild completed and forbids the local skip line.

Optional

Cluster admission still uses a second graph-id grammar

The CLI and the server now share GraphId: 1–64 ASCII letters, digits, or hyphens, except policies, healthz, openapi, and graphs. A served target also accepts og_catalog. The user guide states that rule.

omnigraph-cluster validate_id is the grammar cluster apply uses for a graph key. It requires a leading letter or _, then ASCII letters, digits, _, or -, and it has no 64-character limit. cluster apply accepts billing_prod. require_graph_segment and config validate then refuse it, so a direct --graph billing_prod stops before the cluster lookup. config validate accepts 1graph on a direct root. cluster apply refuses that id.

require_graph_segment changed in 372ba4b7, so this applies to every --graph, including a direct root that does not read the new document. The boundary tests compare the CLI with GraphId. They do not compare a direct root with validate_id.

Keep one grammar at the door that admits the id. Either direct selection calls validate_id, or cluster admission calls GraphId. The second choice also matches server startup, which already refuses a non-GraphId before it opens a graph. Name that choice in the changelog if existing direct graphs use _.

Tradeoffs and liability

The version-1 document still removes the importer and the second on-disk grammar. That is lower long-term liability. The review fixes put the id rules and the YAML anchor rule in the schema scan. That is the layer that owns the document.

The new liability is a half-migrated caller. Query uses a registered target. Blob, mutate, and merge still pass a URL. Five more journeys with that split will fail the same CI cell. One helper for current-binary served commands removes the class.

The operator RFC is still a draft. This breaking format is the user contract. Accept the RFC, or the next edit will change a shipped format against a draft.

#1114, #1113, #1104, and #1078 are on main. This head includes them through 092411dc. Open #1118 still rebases onto this CLI surface. git merge-tree --write-tree 13ecdc8b 18280ab6 exits 0.

This diff from 092411dc has no Lance storage-path change. Direct targets remain absolute local file:// URIs.

Validation

Reviewed head: 18280ab6ce3199ac653017f2651ce97cc3c8778b. Main: 13ecdc8b1e1f65f3e4267c2857d5311b58668e65. The review delta, excluding the main merge 400bc3b7, is 372ba4b7 and 18280ab6.

CI on this head, run 39639402228:

  • Passed: rustfmt, clippy, cargo deny, catalogue inventory (catalogue inventories match the binary), GQT ordinary, GQT dst, GQT served, GQT dst-clippy, DST pinned suite, RustFS, Azurite, Azure guards, V5–V10 fence, Storage Upgrade Compatibility, server --features aws.
  • Failed: Test Workspace job 119358759897, then CI Gate job 119362014940.

The workspace failure is one test:

omnigraph-cli::integration crossversion_upgrade::current_v10_refuses_and_rebuilds_genuine_v09_graph_end_to_end

stderr: inline access supports local storage only; configure a server or managed target

The same test on main 092411dc, Test Workspace job 119344123493, passed in 3.532s and printed v0.9 refusal and export/import rebuild completed.

These review regressions passed on this head before the workspace job stopped: managed_id_boundaries_match_execution_issue_1115, graph_id_boundaries_and_catalogue_scope_match_execution_issue_1115, schema_id_grammars_and_scope_references_match_execution_issue_1115, alias_parameters_keep_defaults_and_explicit_precedence, alias_rejects_non_object_parameters_and_positional_arguments_without_requests, anchors_and_aliases_are_rejected_without_rejecting_literal_text, legacy_document_error_names_the_upgrade_guide, catalogue_scope_reads_the_binary_registries_without_a_store.

The author reports 334 local integration tests passed. That run skips this journey when OMNIGRAPH_V09_BIN is unset. CI forbids that skip. I did not rerun the 0.9 binary locally. The before and after evidence is the two CI jobs above, plus the call sites in this head.

GQT served is green on this head. The smoke uses tls_gateway::http_upstream and does not clear a worker CA through SSL_CERT_FILE. That claim matches the code and the job.

Recommendation: request changes.

Inline comments to attach:

  • crates/omnigraph-cli/tests/integration/crossversion_upgrade.rs:1692 — Blocking. This rewrite applies only to query. TestServer.base_url is https://127.0.0.1:<port>. blob get (about line 1766), mutate (about line 1785), and branch merge (about line 1802) still pass ["--target", server.base_url, "--graph", "knowledge"]. The reopened server builds that same array near line 1830. scope::resolve refuses that inline URL with inline access supports local storage only. CI job 119358759897 fails this test on that message. On main 092411dc the same test passed. Use the registered test-server target for every current-binary served command, as the 0.12 journey does with KEYED_SERVER_NAME. Keep legacy_base_url() for the 0.9 binary.
  • crates/omnigraph-cli/src/operator_schema/validation.rs:169 — Optional. Served ids use require_graph_segment (GraphId, plus og_catalog). Direct ids use GraphId. Cluster validate_id in crates/omnigraph-cluster/src/config.rs is a third grammar: a leading letter or _, then letters, digits, _, or -, with no 64-character limit. cluster apply accepts billing_prod. This function and --graph then refuse it. This function accepts 1graph for a direct root. cluster apply refuses that id. require_graph_segment changed in 372ba4b7, so the refusal applies to every --graph, including a direct root. Point both doors at one grammar. The boundary tests compare the CLI with itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant