Repository navigation
Conversation
A ranked read's JSON envelope gains two additive arrays derived from the accepted plan: retrievals (one per ranked scan, per rrf() fusion and per fusion arm, ids in plan preorder with a fusion before its arms and the arms in argument order) and metrics (one per returned column whose value is a score a retrieval of the plan writes, a return distinct group key included). The planner derives them; execution attaches them to the query result before the plan runs, so every read door and a replay report the same ids for one plan. No parameter value is included, and a stored property named like a score column on an older schema is no metric.
RecommendationDo not merge this head yet. The read-descriptor behavior matches RFC 0047. Test Workspace on this commit fails one test, This review is a comment. It is not an approval. What this change doesA ranked read now reports the retrievals that produced its order, and the returned columns that hold a score. The planner derives that report from the accepted physical plan (
Legal queries keep the same rows. This change does not write Lance data, and it does not change index use. Lance ANN search stays approximate. A flat scan stays an execution choice. The declared recall follows the kind, so a client does not depend on a guarantee that disappears when an index is built. Blocking
Add one inventory row per occurrence in
The Graph Vocabulary Guard job is skipped on pull requests, and Optional
Tradeoff and liabilityNearest recall stays approximate on the exact overfetch pass. A client must not treat a flat pass as a promise. That promise would disappear when an index is built. This matches RFC 0047 and the Lance alignment fence. The report is one derived view of the plan. There is no second stored description. Query semantics stay in the plan. Index layout stays derived state. The new cost is a stable wire surface. Each later envelope array should use the same door. Derive it from the plan, attach it in A future score carrier that is not a bare property access would omit a metric. A cast, or an alias of a group key, is that shape. It is not a legal return today. T9 refuses a rank expression beside an aggregate. T56 refuses Other open workThis head conflicts with current
On Open PR #1012 also edits ValidationReviewed head: Local, checkout
Result: 1 passed, Local, same checkout,
Result: Exact-commit CI, run 39746533611, Test Workspace job
GQT ordinary, GQT dst, GQT served, and the DST pinned suite passed on this commit. rustfmt and clippy passed. RustFS S3, default and failpoints, passed. I did not finish a local rerun of the engine, server, or CLI integration tests. Those passed on CI for this commit. I did not measure performance. This change does not alter storage traffic, so the DST cost golden is unchanged. I did not run S3 or Azure locally. Regression proofNo GQT before-and-after pair applies. This is a new envelope contract. GQT records rows, counts, result column types, and errors. It has no expectation for Inline:
|
# Conflicts: # openapi.json
RecommendationDo not merge this head yet. The read-descriptor behavior still matches RFC 0047. One OpenAPI naming rule from current main fails on this commit. Rename the four string-enum schemas, regenerate the spec, move the inventory rows, then merge. This review is a comment. It is not an approval. What this change doesA ranked read reports the retrievals that produced its order, and the returned columns that hold a score. The planner derives that report from the accepted physical plan (
Legal queries keep the same rows. This change does not write Lance data. Lance ANN search stays approximate. A scan without an index can score every vector, and this report still says Earlier findings
Blocking
Give each string enum a schema name with no transport suffix. Regenerate The JSON values stay Object schemas keep Merge with mainThe second parent is current
No conflict markers remain. Other open pull requests#1137 and #1139 merge cleanly onto this head. Both use #1012 still conflicts with this head in Tradeoff and liabilityThe report stays one derived view of the accepted plan. There is no second stored description. Query semantics stay in the plan. Index layout stays derived state. Nearest recall stays approximate on a full scan, so a client does not depend on a guarantee that disappears when an index is built. The new cost is a stable wire surface plus the OpenAPI naming rule main now enforces. Each later envelope array should use the same door: derive it from the plan, attach it in This merge closes the vocabulary gap and the two coverage gaps from the previous review. It leaves the enum-name rule that main added after the feature was written. Setting the names before merge avoids a later client rename. Shipping the ValidationReviewed head: Exact-commit CI, run 40101672643:
Local, checkout
Result:
Result: 1 passed. I did not finish a second local run of the api-types, engine, server, or CLI tests. Those descriptor tests passed on CI for this commit. I did not measure performance. This change does not alter storage traffic, so the DST cost golden is unchanged. I did not run S3 or Azure locally. I did not rerun Regression proofNo GQT before-and-after pair applies. This is a new envelope contract. GQT records rows, counts, result column types, errors, and plan facts. It has no expectation for Inline:
|
The four string enums of the read descriptors become RetrievalKind, Recall, MetricKind and CoverageUnknownReason, as the OpenAPI conventions require; their JSON values are unchanged. The vocabulary inventory rows for their descriptions move to the new schema paths.
RecommendationMerge this head. The OpenAPI suffix blocker is resolved. This review finds no new blocker. This review is a comment. It is not a GitHub approval. What this change doesA ranked read reports the retrievals that set its order. It also reports each returned column that holds a score. The planner derives both arrays from the accepted plan. This commit only renames four string-enum schemas. The names are Earlier findings
BlockingNone. OptionalNone. Other open pull requests#1137 and #1139 merge cleanly onto this head. #1139 returns the query-route JSON from the #1150 merges cleanly. It changes the description of #1151 merges cleanly. The merged lock keeps #1012 still conflicts in Tradeoff and liabilityThe report stays one derived view of the accepted plan. This commit adds no second description. It sets the string-enum names before HTTP contract 0.14 ships, so a later client does not rename these schemas. The JSON contract is unchanged. Nearest recall stays Five similar arrays would add wire fields, schema names, and inventory rows. They add no second source of truth while they stay derived from the plan. Each new string enum needs a name with no transport suffix. ValidationReviewed head: The delta is four files: Local, checkout
Result:
Result: 106 passed, 0 failed. This includes A direct read of Exact-commit CI, run 40123494256:
I did not rerun the engine, route, or CLI descriptor tests locally. They passed in the Test Workspace report for this commit. I did not measure performance. This commit does not change storage traffic, so the DST cost golden is unchanged. I did not run S3 or Azure locally. Those CI jobs passed. Regression proofNo GQT before-and-after pair applies. This commit renames schemas. It does not change rows, counts, result column types, errors, or JSON values. GQT records those facts and plan facts. It has no expectation for |
Summary
RFC 0047 rollout step 6: a read says which retrievals produced its order and what each returned score column measures.
The read envelope (
POST /query, stored-query calls,query --json, and the--format jsonlmetadata line) gains two additive arrays, absent when a read ranks nothing:omnigraph_planner::read_descriptorsderives both arrays from the accepted physical plan: one retrieval per rankedScan, perRankFuseand per fusion arm, ids in plan preorder with a fusion before its arms and the arms in argument order; one metric per return column of the rootProjection(areturn distinct'sAggregate) that reads a_distance,_scoreor_rrfcolumn a retrieval of the plan writes. Nothing is stored beside the plan.engine::executederives them before the plan runs and attaches them to theQueryResult, so the ordinary, inspected and replay doors report the same ids for one plan.read_outputrenders them; a metric naming a retrieval the read does not describe is refused rather than rendered half.recallis the contract, not what one run did:nearestis approximate,bm25exact,rrfapproximate when either arm is. No parameter value (query text or vector) appears.Three readings of the RFC text, recorded in its decision log: a column computed from a score is not a metric, nor is a stored property of a score column's name on a schema older than the reservation (such a read keeps working); no run establishes embedding counts yet, so
nearestreportsembedding_coverageas unknown (not_computed); and the Arrow IPC response, which has no JSON envelope, carries neither array.The RFC's rollout table now records steps 3 (#1066) and 4 (#1069) as merged.
Tests
engine_v2_plan_replay.rs:a_read_describes_its_retrievals_and_score_columns(twobm25arms over one property with two columns returning one fused score, anearestwith its distance projected, a mixed fusion, areturn distinctscore key, an unranked read; inspected, ordinary and replayed runs agree). The sharedreplayedhelper now requires every replay to reproduce its run's descriptors.bound_plan.rs: a fused plan read back from its serialized form describes itself identically; a fusion arm that lost its ranking is an internal error. In-sourcedescriptors.rs: a stored property named_scoreis no metric and does not fail the read.data_routes.rs:query_envelope_describes_retrievals_and_score_columns(the exact wire shape, ordered arm references, shared score projections, no parameter values, absence on an unranked read); the remotenearesttest also checks its retrieval.stored_queries.rs::invoke_stored_read_returns_rowscovers the stored-query door.cli_data.rs:ranked_read_describes_its_retrieval_in_json_and_jsonl;parity_matrix.rs::parity_ranked_query_descriptorscompares the embedded and remote envelopes; the in-source jsonl test covers both presence and absence.read_descriptors_render_and_round_trip(wire shape, decode/encode identity for the remote CLI, old payloads, orphan metric refusal).openapi.jsonregenerated.Docs
Search guide "What a ranked read reports", the CLI reference's machine-readable positions, the execution guide, the RFC decision log, and a changelog note.