Repository navigation
fix(query): refuse key condition values over the key size limits - #481
Open
LeeroyHannigan wants to merge 3 commits into
Open
LeeroyHannigan wants to merge 3 commits into
LeeroyHannigan wants to merge 3 commits into
Conversation
Closes #435. The service refuses a Query whose key condition values exceed the key size limits before it reads anything; ExtendDB ran the Query. Measured on 2026-10-23, every row repeated with identical answers: - A HASH value over 2048 bytes: "One or more parameter values were invalid: Size of hashkey has exceeded the maximum size limit of2048 bytes" (no space after "of", as the service writes it). 2048 bytes is accepted. - HASH values that each fit but add up to more than 2048 bytes (an index with two HASH attributes): "One or more parameter values were invalid: Aggregated size of all hash keys has exceeded the size limit of 2048 bytes". A sum of exactly 2048 is accepted. A single oversized value gets the single sentence even when the sum is also over. - RANGE values adding up to more than 1024 bytes, one clause or several: "One or more parameter values were invalid: Aggregated size of all range keys has exceeded the size limit of 1024 bytes". 1024 is accepted. Every operator is sized (=, <, <=, >, >=, begins_with); BETWEEN contributes its larger bound, not the sum of both. - The HASH checks are reported before the RANGE check. - Strings are sized by UTF-8 bytes (683 U+20AC characters, 2049 bytes, refused; 682 plus "xx", 2048 bytes, accepted) and binary values by byte length. Base tables, LSIs, GSIs and legacy KeyConditions behave the same. - An undefined #name or :value in the key condition is reported before the size check. Ordering, measured with a 2049-byte value: filter syntax, unused names and values, index name, ConsistentRead on a GSI, missing table, Limit, projection syntax and overlap, the Select conflicts and the filter operand type are all reported first; the ExclusiveStartKey checks after. The check is crates/core/src/validation/key_size.rs, called from handle_query after the expression validations and before the start key checks. The limits come from the configuration and the sentences name them. Tests: unit tests in validation/key_size/tests.rs and the engine; tests/python/test_query_key_size.py, test_query_key_size_values.py and test_query_key_size_order.py (64 rows) pass against the service. On ExtendDB alone, five rows with two or more RANGE clauses on one index skip until the multi-attribute range change in #429 merges (the handler refuses the shape earlier today); they pass on the merged tree. The undefined-name row stays unit level because the undefined-name refusal for a sort key clause arrives with #479. Pre-existing, measured, not changed here: a BETWEEN with the lower bound above the upper bound is refused by the service with a bound-order sentence; two conditions on one key are refused with "KeyConditionExpressions must only contain one condition per key"; a Number with 39 or more digits is refused with "Attempting to store more than 38 significant digits in a Number"; a FilterExpression on a key attribute is refused with "Filter Expression can only contain non-primary key attributes". ExtendDB accepts all four.
LeeroyHannigan
requested review from
amrith,
c33howard,
jcshepherd,
pdf-amzn,
robinnsc and
yesyayen
as code owners
October 24, 2026 00:52
…y sentence matches the other backends
… multi-RANGE index The Cassandra backend answers InternalServerError to any Query on an index with several RANGE attributes: its ORDER BY lists sk_* then base_pk and skips the sk2_* to skN_* clustering columns, which Cassandra refuses with "Order by currently only supports the ordering of columns following their declared order in the PRIMARY KEY". require_multi_range now also skips on InternalServerError, so test_one_600_byte_range_value_is_accepted carries the probe like the other idx rows.
This was referenced Oct 24, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #435.
Summary
The service refuses a Query whose key condition values exceed the key size limits before it reads anything; ExtendDB ran the Query. This change adds the check with the service's sentences and ordering.
Measured on the service on 2026-10-23 (every row repeated, identical answers):
One or more parameter values were invalid: Size of hashkey has exceeded the maximum size limit of2048 bytes(no space afterof)One or more parameter values were invalid: Aggregated size of all hash keys has exceeded the size limit of 2048 bytesOne or more parameter values were invalid: Aggregated size of all range keys has exceeded the size limit of 1024 bytesxxand 341 plusxaccepted#nameor:valuein the key condition with an oversized valueOrdering, measured with a 2049-byte value: filter syntax, unused names and values, index name, ConsistentRead on a GSI, missing table, Limit, projection syntax and overlap, the Select conflicts and the filter operand type are reported before the size check; the ExclusiveStartKey checks after it.
Changes
crates/core/src/validation/key_size.rs: the check, with the limits from the configuration and the sentences naming them. Unit tests invalidation/key_size/tests.rs.handle_querycalls it after the expression validations and before the start key checks.docs/dynamodb-limits.md: the partition key cell names the HASH sum.Tests
tests/python/test_query_key_size.py,test_query_key_size_values.py,test_query_key_size_order.py(shared fixtures inquery_key_size_support.py): 64 rows, each asserting the exact sentence. 64 passed against the service.Query condition missed key schema element: h; the skip is a capability probe that lifts itself once the multi-attribute range change in fix(query): classify and order key conditions on a multi-attribute GSI the way the service does #429 merges. On a tree with fix(query): classify and order key conditions on a multi-attribute GSI the way the service does #429 merged all rows pass.git merge-treeagainst fix(query): classify and order key conditions on a multi-attribute GSI the way the service does #429 and against the Query accepts an ExclusiveStartKey from another partition and returns an empty page #440 and Query: a lone non-equality condition on the partition key gets the parser's message, not the service's #427 branches: clean.Related
pk = :v AND #n = :w,#nundefined) is unit level only: ExtendDB has no undefined-name refusal for a sort key clause today; fix(query): refuse a key condition with no HASH equality the way the service does #479 adds it, and with both merged the row matches the service.Index nope not foundinsideIndexNotFound, so the Query sentence readThe table does not have the specified index: Index nope not foundon Cassandra only. Corrected to pass the bare name like the other backends (one line instorage-cassandra/src/data/index.rs).Order by currently only supports the ordering of columns following their declared order in the PRIMARY KEY, because the data table's clustering key issk_s, sk2_s, ..., base_pkand the Query emitsORDER BY sk_s, base_pk) and surfaces asInternalServerError, with or without a RANGE clause, for any value size. Not a one-line change (the single sort key model also drives the pagination restrictions and the start key read). The idx accept rows skip on that backend through the same capability probe, with the reason named; the defect will be filed on its own.Pre-existing, measured, not changed here
The BETWEEN condition was provided a range where the lower bound is greater than the upper bound), ExtendDB runs it.KeyConditionExpressions must only contain one condition per keyon the service, 200 here.Attempting to store more than 38 significant digits in a Numberon the service, 200 here.Filter Expression can only contain non-primary key attributes: Primary key attribute: pkon the service, 200 here.These will be filed as one issue together with the ones from #479.