Skip to content

fix: respect namespace ordering in multi-shard Get and List - #434

Draft
void-ptr974 wants to merge 1 commit into
oxia-db:mainfrom
void-ptr974:fix/namespace-key-sorting
Draft

void-ptr974 wants to merge 1 commit into
oxia-db:mainfrom
void-ptr974:fix/namespace-key-sorting

Conversation

@void-ptr974

@void-ptr974 void-ptr974 commented Oct 6, 2026 •

Copy link
Copy Markdown

Cross-shard primary-key Get and List must merge results in the namespace's key order. The Java schema omitted the server's key_sorting field, so these merges always used the legacy CompareWithSlash order, producing incorrect Get results and List ordering.

Examples

Each example contains exactly three stored keys on different shards. Names omit the common random-probe: prefix. In the order lines below, brackets mark query keys that are not stored.

Natural order compares unsigned UTF-8 bytes:

da//z < da1 < [da10] < db

Hierarchical order compares slash depth first, then encoded bytes. The p0/... records have depth one; da//z has depth two:

p0/k05 < [p0/k06] < p0/k07 < da//z < [xx//a]

For these absent queries, FLOOR/LOWER selects the immediately preceding stored key; CEILING/HIGHER selects the immediately following one. List should follow the stored-key order.

Operation Java 0.9.5 Expected / fixed
Natural FLOOR/LOWER(da10) da//z da1
Hierarchical CEILING/HIGHER(p0/k06) da//z p0/k07
Hierarchical FLOOR/LOWER(xx//a) p0/k07 da//z
Natural List [da1, db, da//z] [da//z, da1, db]
Hierarchical List [da//z, p0/k05, p0/k07] [p0/k05, p0/k07, da//z]

Fix and Go alignment

Add key_sorting to the Java schema and use comparators matching the server's key encodings. Metadata updates refresh the comparator, including when shard placement is unchanged. Each asynchronous merge captures one comparator.

Absent/unknown metadata and secondary-index queries continue using the legacy comparator. Its supplementary-Unicode comparison is corrected to use UTF-8 bytes instead of UTF-16 code units, matching Go.

Primary-key comparison Get now follows the tested Go client's ordering. Java List agrees with Go RangeScan, used by the Go CLI's client list. Go SDK List does not globally sort shard results. Full secondary-index nearest-key parity is outside this PR.

Validation

  • 97 targeted tests, client:spotlessCheck and client:spotbugsMain passed.
  • On the same servers, Java 0.9.5 failed 12/24 Get checks and all four List checks. Both 0.9.5 with this fix and the PR build passed 24/24 Get and 4/4 List checks through sync/async APIs.
  • Full integration tests and performance benchmarks were not run. Key encoding during comparisons may increase allocations in large List merges.

Read the namespace key_sorting metadata and use one ordering snapshot
for primary-key nearest reads and global lists. Match the server's
UTF-8 storage encodings while retaining the legacy order for old servers
and secondary-index merges.

Cover metadata-only assignment updates, encoded-key boundaries,
Unicode ordering, async shard merges and secondary-index isolation.
@void-ptr974 void-ptr974 changed the title fix: respect namespace key sorting when merging shard results fix: respect namespace ordering in multi-shard Get and List Oct 6, 2026
@void-ptr974
void-ptr974 marked this pull request as draft October 6, 2026 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant