[LeeroyHannigan]

fix(query): check Query values against the key schema and the number rules

What changes

A Query with a KeyConditionExpression did not check a key condition value's type against the key attribute's AttributeDefinitions type, and did not check that an N value in ExpressionAttributeValues is a number. The storage layer compared the value's inner text with the stored key and ignored the type tag: on a table whose partition key is an S attribute, pk = :u with :u = {"N": "bob"} read the bob partition, :u = {"N": "5"} read the 5 partition, and a sort key value of another type answered InternalServerError on SQLite. The service refuses these before it reads a row.

Two checks are added on the expression path, in crates/engine/src/query_key_values.rs:

1. An N value in ExpressionAttributeValues that is not a number answers The parameter cannot be converted to a numeric value: <value> (the overflow, underflow and 38-digit texts for those cases; an empty value answers the message with the colon and nothing after it; no ExpressionAttributeValues contains invalid value wrapper, which the service adds on UpdateItem and not on Query). The check covers the whole map, including NS members and values nested in L and M, whether the key condition, the FilterExpression or nothing reads them; the walk covers the same values PutItem and UpdateItem check, and accepts a Unicode decimal digit (general category Nd, 68 runs of ten digits in Unicode 15.0, among them ASCII, fullwidth, Arabic-Indic and Devanagari) as the digit it stands for, as the service does (see the fold round below). The check runs at request time on the raw body, before it is deserialized and before the table lookup, in the service's order: the Limit check, the expression and non-expression parameter mix check, the ReturnConsumedCapacity enum check, the numbers, then the Select enum check. This moves Limit and the mix check ahead of the table lookup and of the two enum checks, and the Select check after the numbers; because the checks read the raw body, a field that does not deserialize does not stop the ones before it. When a check fails, the body is deserialized as the typed input and a SerializationException from that (a field of the wrong JSON type anywhere in the body) is returned instead, which is the service's order; a body that passes every check is deserialized once, by the handler. The server runs the same request-time checks ahead of the IAM decision, because authorization binds no dynamodb:LeadingKeys value to a number it cannot parse: without it a principal whose Allow carries ForAllValues:StringEquals dynamodb:LeadingKeys ["alice"] was authorized with no key bound and served the bob partition, and a principal under a ForAnyValue, Null or plain-operator fence answered AccessDeniedException where the request's error is ValidationException.
2. A key condition value whose type is not the key attribute's type answers One or more parameter values were invalid: Condition parameter type does not match schema type. The values checked are the partition key value, the sort key value under =, <, <=, >, >= and the reversed operand order, both BETWEEN bounds, the begins_with prefix, and the extra conditions of a multi-attribute index key. The key schema is the queried table's or index's, so a GSI partition or range key and an LSI range key are checked against their own definitions. The check runs after the key schema, FilterExpression, ProjectionExpression, unused value and Select checks and before the ExclusiveStartKey check. When a compared placeholder is undefined the check returns without a verdict and the undefined placeholder is reported, which is the service's order.

The legacy KeyConditions path is not changed here; its checks land with #507. Scan is not changed here (see below).

One line in crates/storage-cassandra/src/data/index.rs passes the bare index name in IndexNotFound, so a Query naming an unknown index on Cassandra answers The table does not have the specified index: <name> as the other backends do, where it read ... index: Index <name> not found. The new test file asserts that text. The same commit is on #481 and #507, so the three merge without a conflict on that line.

Measured on the service

us-east-1, admin principal, client retries off, each request sent twice, on tables with pk S and sk S (with an LSI and a GSI), pk S and sk N (with a GSI), and bpk B and bsk B.

2026-10-24 15:10 UTC, the reported shapes:

request	answer
pk = :u, :u = {"N": "bob"}	ValidationException The parameter cannot be converted to a numeric value: bob
pk = :u, :u = {"N": "5"}	ValidationException One or more parameter values were invalid: Condition parameter type does not match schema type
pk = :u, :u a B value	the schema type message
pk = :u, :u = {"BOOL": true}	the schema type message
pk = :u AND sk = :s, :s = {"N": "5"}	the schema type message
pk = :u AND pk > :v, :u = {"N": "bob"}, :v = {"S": "alice"}	The parameter cannot be converted to a numeric value: bob
pk = :u AND pk > :v, :u = {"N": "5"}, :v = {"S": "alice"}	Invalid KeyConditionExpression: KeyConditionExpressions must only contain one condition per key

2026-10-24 21:00 UTC, 77 requests, where the two checks fall against the other Query checks:

• The numeric message is reported ahead of a missing table, an unknown IndexName, an undefined placeholder, an unused value, a syntax error, ConsistentRead on a GSI, a nested key path, a non-equality partition operator, an invalid ExclusiveStartKey, the Select and ProjectionExpression checks and a reserved word in the filter; and for an N value that only the FilterExpression reads or that nothing reads. Limit 0 and the expression and non-expression parameter mix are reported ahead of it. An empty N answers The parameter cannot be converted to a numeric value: ; 1e200 answers the overflow message; 39 digits answer the 38-digit message.
• The schema type message is reported after Query condition missed key schema element, a reserved word in the key condition, the unknown index, Limit 0, the unused value, the undefined placeholder, the missing table, ConsistentRead on a GSI, a begins_with operand error and a reserved word in the filter, a ProjectionExpression overlap and the Select checks; and before an invalid ExclusiveStartKey. It applies to the partition and the sort key, on the table, a GSI and an LSI, for the six comparison operators, BETWEEN and begins_with, for NULL, SS, L, M, BOOL and B values on an S key, S and N values on a B key, and S values on an N key.

2026-10-24 21:06 UTC, 29 requests: an N value that is not a number inside L, M, NS or M in L in M, read by the filter or by nothing, answers the numeric message; a nested empty N answers the message with the colon; the numeric message is reported ahead of a missing table for a value nothing reads, ahead of the missing key condition refusal, and for a value beside legacy KeyConditions; Limit 0 is reported ahead of a missing table, an unknown index, ConsistentRead on a GSI, the parameter mix and the numeric check; the parameter mix is reported ahead of a missing table.

2026-10-24 23:42 UTC, the request-time order: an invalid Select beside {"N": "bob"} answers the numeric message; an invalid ReturnConsumedCapacity beside it answers 1 validation error detected: Value 'bad' at 'returnConsumedCapacity' failed to satisfy constraint: Member must satisfy enum value set: [INDEXES, TOTAL, NONE]; Limit 0 is reported ahead of an invalid Select and an invalid ReturnConsumedCapacity, and the parameter mix ahead of an invalid Select; Limit -1 is reported ahead of the numeric message; Limit 0 ahead of a type mismatch and a number that is not a number in one request.

Not measured

Whether the service answers a principal fenced away from the value with ValidationException or AccessDeniedException for the schema type case cannot be measured on this host (no restricted principal can be created); the test file does not encode it. The order of Limit 0 against the Select ALL_ATTRIBUTES index projection check was not measured; the Limit check now runs first.

Differences that remain

• When two or more N values are invalid, the service reports one of them in an order that is neither the request's nor sorted (it matches Java HashMap iteration order for the measured keys); ExtendDB reports the first in sorted placeholder order.
• pk > :u with :u = {"N": "5"} on an S key: the service answers the schema type message; ExtendDB's parser refuses the shape (partition key condition must use equality (=)) before the key schema is known, as on main. fix(query): refuse a key condition with no HASH equality the way the service does #479 and fix(auth): bind LeadingKeys for a refused Query's partition operand #511 own that shape.
• The service checks the BETWEEN bounds order, the begins_with operand type and one condition per key in a KeyConditionExpression before the schema type; main does not make those checks on the expression path (Count: 0, InternalServerError, Count: 0), so a request that carries both faults answers the schema type message here. fix(core): refuse a second condition on a key in KeyConditionExpression #484 adds the one-condition-per-key check. The service's Invalid KeyConditionExpression: prefix on the undefined placeholder message, its begins_with text for an N prefix on an N sort key, and its empty key value message (The AttributeValue for a key attribute cannot contain an empty string value. Key: pk) are also absent on main and unchanged here.
• Scan on the service applies the same numeric check to its ExpressionAttributeValues (top-level and nested, overflow, 39 digits, empty with the colon) before the table lookup, with Limit 0 reported first; main's Scan does not, and its Limit 0 message reads Value '0' at 'limit' where the service's reads Value at 'Limit'. Not changed in this branch.
• pk = :u AND other = :s with a non-reserved name outside the key schema is applied to the sort key column on SQLite on main and here.
• The service reads a Unicode decimal digit ({"N": "５"}) as a number when it compares and stores the ASCII value. Here the number check accepts it, and everything after the check reads the value as the request wrote it, as on main: a key condition or filter written with a Unicode digit matches no item stored with the ASCII digit, and a numeric sort key or ExclusiveStartKey written that way answers InternalServerError on SQLite and PostgreSQL. The write paths (PutItem, UpdateItem, the batch and transaction writes) refuse such a value on main and are not changed here. Reading the digit as a number has to change the server's dynamodb:LeadingKeys binding and the engine together, and is left to a separate issue. A principal under a dynamodb:LeadingKeys fence sending such a key value binds no number at the server; the engine then matches nothing, which a test pins. The service's answer for a fenced principal is not measured (no restricted principal on this host).

Fold round (2026-10-25)

Two regressions the request-time design introduced, found on the Scan branch stacked on this one and fixed here first:

1. Unicode decimal digits. The number check applied the write paths' validator, which accepts ASCII digits only, so {"N": "\uff11"} (fullwidth one) in a key condition, a filter or an unused value answered The parameter cannot be converted to a numeric value: １ where the service and the previous release return rows. Measured 2026-10-25 11:31 UTC on an S/N table holding an item with sk N 1 and a N 1, each request twice: a fullwidth, Arabic-Indic or Devanagari one as the sort key value returns that item (Count 1), as a filter value matches a (Count 1), and as an unused value answers the unused-value text (the digit passed the number check); thirty nine Devanagari ones answer Attempting to store more than 38 significant digits in a Number in all three positions; 1\uff12 and 1e\uff12 are accepted (Count 0); a fullwidth digit in an NS passes the check; a superscript two, a vulgar half, a fullwidth minus and a fullwidth full stop are refused with the numeric message and the value echoed as sent. PutItem with sk N \uff11 is accepted and GetItem by the fullwidth key and by the ASCII key both return the item with sk N 1: the service stores the ASCII value. The check now folds every Nd digit to ASCII before the number rules run (crates/engine/src/decimal_digits.rs: the 68 run zeros of Unicode 15.0, every run being ten consecutive code points, with a binary search) and echoes the value as written when the rules refuse it. The fold applies to the check only; the handler and the server's key binding read the value as written (see Differences that remain). No new dependency. The write paths are unchanged (their refusal is on main; a separate issue covers it).
2. Shape faults. The checks ran on the raw body before deserialization, so a field of the wrong JSON type in another field answered the check's ValidationException where the service and the previous release answer SerializationException. Measured 2026-10-25 11:31 UTC: Limit: 0 with ExpressionAttributeValues: "x" and ExpressionAttributeNames: "x" with {"N": "bob"} answer SerializationException: Unexpected field type; Limit: "1" with {"N": "bob"} answers STRING_VALUE cannot be converted to Integer; AttributesToGet: {"a": 1} with ProjectionExpression answers Start of structure or map found where not expected; QueryFilter: [1] with FilterExpression answers SerializationException with no message; Select: 5 with {"N": "bob"} and Limit: 0 with Select: 5 answer NUMBER_VALUE cannot be converted to String; Limit: 0 with {"N": "bob"} and no shape fault answers the Limit text. When a check fails, validate_query_request_body now deserializes the body as QueryInput and returns the deserializer's SerializationException if there is one; a ValidationException the deserializer raises (an empty set, a duplicate, an invalid AttributeValue) does not preempt the check, since those orders are not measured. Neither does an AttributeValue with an unknown type descriptor, which the deserializer reports as a SerializationException: Limit: 0 beside {"X": "1"} answers the Limit ValidationException on the service (measured 2026-10-25 12:56 UTC), in its older Value '0' at 'limit' form; ExtendDB answers its Value at 'Limit' text, the form the service uses when no AttributeValue is malformed. The server's pre-dispatch call goes through the same function. The message texts of the SerializationException differ between the service and ExtendDB (pre-existing); the tests assert the error type.

Tests

• crates/engine/src/query_key_values/tests.rs: 18 unit tests at the checks (the six comparison operators and the reversed operand order, both BETWEEN bounds, S, N and B keys, a GSI range key, a multi-attribute hash schema, #name references, the undefined placeholder stand-aside, the numeric messages, nested NS, L and M values, values the key condition does not name, sorted order, Limit then the parameter mix then the numbers at request time, and the two enum checks on either side of the numbers).
• Fold round: crates/engine/src/decimal_digits.rs has 4 unit tests (the measured digits and controls, run boundaries, folding of text) and query_key_values/tests.rs gains 3 (digits accepted, non-digits refused with the text as sent, shape faults first with the string-enum, deserializer-ValidationException and unknown type descriptor controls). tests/test_query_key_value_types.py gains 21 rows (g1 to g8: the 38-digit message and the four refused characters in the key condition, filter and unused positions, five accepted forms and an NS as unused values), TestUnicodeDecimalDigitsPassTheCheck (6 tests: a Unicode digit, a mixed form and an NS member accepted in the filter), TestShapeFaultsAreReportedFirst (7 raw signed rows plus a control) and, in the LeadingKeys class, a test that a partition key written with Unicode digits returns no item of a partition a ForAllValues fence keeps the caller from. At the previous head of this branch (4a4e92f) 22 of the new tests fail on SQLite; all 133 tests in the file pass at the new head on SQLite.
• tests/test_query_key_value_types.py (dual target): 94 parametrized rows, one per measured request above whose answer the branch produces, two controls, and a LeadingKeys class (extenddb only) sending four shapes (pk = :u with :u = {"N": "bob"}; the same with an invalid Select; pk = :u with an S value and {"N": "bob"} only in the filter; pk = :u with an S value and {"N": "bob"} unused) under ForAllValues, ForAnyValue, Null false, StringEquals, StringLike and StringNotEquals fences and as the owner, expecting ValidationException and no items. On main (0.1.14, SQLite) 74 of the 97 tests fail and 23 pass: the two controls and the 21 rows whose service answer is a check main already makes ahead of the new ones (Limit, the parameter mix, the key schema, reserved words, the unknown index, the missing table, ConsistentRead on a GSI, the filter, projection and Select checks), which pin the order of the new checks against those. At the head all 97 pass on SQLite.

Gates

At the head:

• cargo +1.97.0 fmt --all -- --check: clean.
• cargo +1.97.0 clippy -p extenddb-engine -p extenddb-server --all-targets -- -D warnings: 0 warnings.
• cargo test -p extenddb-engine -p extenddb-server --lib: 194 and 202 passed, 0 failed.
• Wire tests: devtools/run-tests --extenddb --pytest --filter "test_query_key_value_types or test_query or test_fgac_leading_keys or test_key_condition or test_legacy" against a release binary built with the sqlite feature only: 275 passed, 0 failed.

On the first commit, before the last change moved the request-time checks onto the raw body (that change touches crates/engine and crates/server only):

• cargo +1.97.0 clippy --all-targets -- -D warnings: 0 warnings.
• cargo test --workspace --exclude extenddb-storage-cassandra on stable: 1782 passed, 0 failed (34 test binaries); cargo test -p extenddb-storage-cassandra --lib: 61 passed, 0 failed. cargo +1.88.0 test --workspace --exclude extenddb-storage-cassandra: 1782 passed, 0 failed.
• The same wire filter against the full release binary on PostgreSQL 16: 268 passed, 0 failed.

For the Cassandra line: cargo test -p extenddb-storage-cassandra --lib 61 passed, and cargo +1.97.0 clippy -p extenddb-storage-cassandra --all-targets -- -D warnings 0 warnings.

Cargo.lock unchanged.

Fold round, at the new head (SQLite release binary with the sqlite feature, port 18666):

• cargo +1.97.0 fmt --all -- --check: clean.
• cargo +1.97.0 clippy -p extenddb-engine -p extenddb-server --all-targets -- -D warnings: 0 warnings.
• cargo test -p extenddb-engine -p extenddb-server --lib: 201 and 202 passed, 0 failed.
• tests/test_query_key_value_types.py: 133 passed on the new head on SQLite and on PostgreSQL; 22 failed and 111 passed on the previous head 4a4e92f on SQLite.
• devtools/run-tests --extenddb --pytest --filter "test_query or test_fgac_leading_keys or test_key_condition or test_legacy": 311 passed, 0 failed.
• The workspace suite, 1.88.0, PostgreSQL, MongoDB and Cassandra run in CI for the fold round (disk on the build host did not allow them locally). The fold touches crates/engine and the test file only.

Merge checks

git merge-tree --write-tree against the open PRs that touch the same files: #511, #484, #483, #479 and #429 merge clean. #481 conflicts in crates/engine/src/lib.rs (both branches add a module line next to mod query;; keep both) and in crates/engine/src/query.rs where both insert a check between the filter operand checks and the ExclusiveStartKey check: #481's comment places the type check above its key size check, so keep this branch's block first and #481's after it. #507 conflicts on the doc comment of validate_attribute_number in crates/core/src/validation/mod.rs (both branches make it public; either comment) and in crates/engine/src/query.rs, as does #399, where #399 moves the request-time checks ahead of the table lookup and #507 builds on it; this branch's request-time call lands directly before serde_json::from_value in that layout (replacing the two enum calls ahead of it and their Limit and mix checks after it) and the type check lands directly before the ExclusiveStartKey check. The legacy desugar_key_conditions call that #507 extends is not touched here.