Repository navigation
fix(limits): attribute-name size limit on every write path - #433
Open
LeeroyHannigan wants to merge 2 commits into
Open
LeeroyHannigan wants to merge 2 commits into
LeeroyHannigan wants to merge 2 commits into
Conversation
…-form wording An attribute name is at most 65,535 bytes of UTF-8, at every depth: a top-level name, a map key inside any value, and a name supplied through ExpressionAttributeNames. Measured against Amazon DynamoDB (us-east-1, 2026-10-21) at 65,535 and 65,536 bytes, the service refuses a longer name on PutItem, UpdateItem, DeleteItem, TransactWriteItems and BatchWriteItem with a ValidationException whose wording follows the request form: - PutItem for a name in the item, and UpdateItem for a map key inside an ExpressionAttributeValues value or an AttributeUpdates name or value: "1 validation error detected: Attribute name is too large, must be less than 65536 bytes". PutItem and DeleteItem use the same message for a map key inside a condition value. - PutItem, UpdateItem and DeleteItem for an ExpressionAttributeNames value: the same message followed by " for key #n", naming the placeholder. - Any member of TransactWriteItems (Put, Update, Delete, ConditionCheck) for an ExpressionAttributeNames value: "ExpressionAttributeNames contains invalid value: Attribute name is too large, must be less than 65536 bytes for key #n"; for a map key inside an ExpressionAttributeValues value: "ExpressionAttributeValues contains invalid value: Attribute name is too large, must be less than 65536 bytes for key :v". The whole request fails with no CancellationReasons. - The Put member of TransactWriteItems and a BatchWriteItem PutRequest, for a name in the item: "Attribute name is too large, must be less than 65536 bytes". The limit counts bytes: 32,768 two-byte characters are refused, and 32,767 of them plus one ASCII character are accepted. The service checks every ExpressionAttributeNames value, including one no expression uses. The name check runs ahead of the table lookup on every one of these requests. In a transaction it runs for every member before any member's table lookup, so a request whose first member names a missing table answers with a later member's name refusal; in a batch it runs for every PutRequest before the table lookup and the key check, so an item that is also missing its key answers with the name refusal. UpdateItem and a transacted Update also report the name ahead of an UpdateExpression syntax error, a failing condition and a secondary-index key type mismatch. With a long name in two places at once, a standalone UpdateItem reports the ExpressionAttributeNames placeholder before a map key in a value, and a standalone PutItem reports the item before the ExpressionAttributeNames placeholder; a transaction member reports a map key in a value first, then the ExpressionAttributeNames placeholder, then (for Put) the item. ExtendDB checked names only on the put paths (PutItem, BatchWriteItem PutRequest, TransactWriteItems Put and ImportTable), after the table lookup, with its own wording: "One or more parameter values were invalid: Size of attribute name '...' has exceeded the maximum size limit of 65535 bytes". UpdateItem and the Update member of TransactWriteItems stored a 65,536-byte name with HTTP 200, and a long name that arrived only through a condition's ExpressionAttributeNames was accepted on PutItem, DeleteItem and every transaction member. validate_attribute_name_sizes and the new validate_expression_attribute_name_sizes take an AttributeNameRequestForm (SingleItem, TransactionMember or BatchRequest), which selects the wording and the order of the two expression maps. validate_attribute_updates_name_sizes covers the legacy AttributeUpdates map. Each map is visited in sorted placeholder order, and the figure in the message is max_attribute_name_bytes plus one. PutItem, UpdateItem and DeleteItem run the checks before the table lookup. TransactWriteItems checks every member in a pass before prepare_write_op; BatchWriteItem checks every PutRequest in a pass before the per-table loop. The checks run in the engine, which all four storage backends share, so no backend code changes. The attribute name size row in docs/dynamodb-limits.md moves from Partial to Enforced and gives the wording and order for each form. tests/test_attribute_name_limit_on_update.py has 49 dual-target cases. They pass on servers built from this change on SQLite, PostgreSQL and MongoDB. The first 35 cases pass on the service; the 14 added for the condition maps, the missing-table and missing-key positions and the two-map orders have not yet been run on the service, and each one encodes a probe row the service answered. A 91-row probe of the same shapes gives the same status, code, message and stored item on the service and on each of the three servers, with one exception outside this change: the control row for an index-key type mismatch alone on UpdateItem, which PostgreSQL does not refuse on main. Removing, switching, swapping or moving each engine call site, and each wording and ordering branch in core, makes the cases that cover it fail. Core unit tests pin every message form, both map orders, the byte count of multi-byte names and the sorted placeholder order. Fixes #365.
LeeroyHannigan
requested review from
amrith,
c33howard,
jcshepherd,
pdf-amzn,
robinnsc and
yesyayen
as code owners
October 22, 2026 01:08
Conflicted file and resolution: - docs/dynamodb-limits.md, Enforcement Coverage summary table: this branch moved the attribute name size row from Partial to Enforced (Items row 6/0/0/0); main recounted Secondary Indexes (5/0/1/0) and Import/Export/Backup (2/2/0/7). Kept both category rows and recomputed the Total row from the merged category rows: 32 enforced, 2 partial, 16 not enforced, 13 not applicable.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #365. An attribute name is at most 65,535 bytes of UTF-8, at every depth. ExtendDB enforced the limit only on the put paths, after the table lookup, with its own wording; UpdateItem and the Update member of TransactWriteItems stored a 65,536-byte name with HTTP 200, and a long name that arrived only through a condition's ExpressionAttributeNames was accepted on PutItem, DeleteItem and every transaction member.
What the service does
Measured on Amazon DynamoDB (us-east-1) on 2026-10-21 at 65,535 and 65,536 bytes: 35 wire cases at 23:40 UTC and a 91-row probe at 23:52 UTC.
1 validation error detected: Attribute name is too large, must be less than 65536 bytes. PutItem, UpdateItem and DeleteItem for an ExpressionAttributeNames value: the same message followed byfor key #n. Any TransactWriteItems member for an ExpressionAttributeNames value:ExpressionAttributeNames contains invalid value: Attribute name is too large, must be less than 65536 bytes for key #n; for a map key inside an ExpressionAttributeValues value:ExpressionAttributeValues contains invalid value: ... for key :v. A TransactWriteItems Put or a BatchWriteItem PutRequest for a name in the item:Attribute name is too large, must be less than 65536 bytes. A transaction fails as a whole with no CancellationReasons.#n] answers the Update member's name refusal. In a batch it runs before the table lookup and the key check: a PutRequest item with a long name and no key answers the name refusal. UpdateItem and a transacted Update report the name ahead of an UpdateExpression syntax error, a failing condition and a secondary-index key type mismatch.Changes
crates/core/src/validation/mod.rs:AttributeNameRequestForm { SingleItem, TransactionMember, BatchRequest }selects the wording and the order of the two expression maps.validate_attribute_name_sizestakes it; newvalidate_expression_attribute_name_sizes(names half and values half, ordered by form) andvalidate_attribute_updates_name_sizes.validate_put_itemno longer checks names (the handler does, earlier). Unit tests for every form and both orders.crates/engine/src/put_item.rs: item names, then the expression maps, right after deserialization and before the table lookup.crates/engine/src/update_item.rs: expression maps and AttributeUpdates checked aftervalidate_table_name, before the table lookup.crates/engine/src/delete_item.rs: expression maps checked aftervalidate_table_name, before the table lookup.crates/engine/src/transact_write_items.rs:validate_member_attribute_name_sizesruns for every member in request order beforeprepare_write_op(expression maps, then a Put member's item); the per-arm checks are removed.crates/engine/src/batch_write_item.rs: every PutRequest item checked in a pass before the per-table loop; the per-request check is removed.crates/engine/src/import_export.rs: passesSingleItem; the error is only logged.docs/dynamodb-limits.md: Attribute name size row Partial to Enforced, with the wording, position and order per form; summary counts updated.tests/test_attribute_name_limit_on_update.py: 49 dual-target cases.No storage crate changes: every path reaches the checks through the engine, so Cassandra receives them too.
Verification
Negative controls
Each mutation applied to the committed tree with an anchor occurring once, run, then restored with
git checkout; the tree was clean after each.SingleItemarm ofvalidate_expression_attribute_name_sizesreverted to names-then-values for every formtransaction_expression_values_are_checked_before_namesfails; wireTestTransactUpdate::test_values_map_reported_before_the_names_mapfailsTestPutItem::test_placeholder_name_only_in_the_conditionfails (DID NOT RAISE)TestPutItem::test_reported_ahead_of_a_missing_tablefailsTestDeleteItem::test_placeholder_name_only_in_the_conditionfails (DID NOT RAISE)prepare_write_oploopTestTransactPut::test_name_over_limit[2]andtest_reported_ahead_of_a_missing_tablefailTestTransactPut::test_names_map_reported_before_the_itemfails (bare sentence instead of... for key #n)TransactionMemberchanged toSingleItemin the Put arm; in the Update armBatchRequestchanged toSingleItemGates
cargo fmt --all -- --check: clean.cargo +1.97.0 clippy -p extenddb-core -p extenddb-engine --all-targets -- -D warnings: clean.cargo +1.88.0 build -p extenddb-core -p extenddb-engine: clean.cargo test -p extenddb-core: 524 passed.cargo test -p extenddb-engine: 103 passed.Not in this change