Repository navigation
fix: answer TTL and tag calls during a table's DELETING window - #599
Open
LeeroyHannigan wants to merge 1 commit into
Open
LeeroyHannigan wants to merge 1 commit into
LeeroyHannigan wants to merge 1 commit into
Conversation
… service does DescribeTimeToLive on a DELETING table answers 400 ValidationException "Cannot describe time to live while table is in DELETING state: Current table state is DELETING", and on a CREATING table the same sentence naming CREATING; it is served on ACTIVE and UPDATING tables. UpdateTimeToLive on a DELETING table runs the same rules as on an ACTIVE one. Amazon DynamoDB answers these calls this way (measured 2026-10-25 and 2026-10-26). On ExtendDB the UpdateTimeToLive write goes to the catalog row, which the delete removes. ExtendDB answered UpdateTimeToLive on a DELETING table with 400 ResourceInUseException "Table <name> is not in ACTIVE state", because every backend's update_ttl admitted ACTIVE rows only, and DescribeTimeToLive with 200 on CREATING and DELETING tables, because nothing read the status. The engine reads the status with describe_table, since TableKeyInfo carries no status and MongoDB and Cassandra report a DELETING row as not found from the key-info read. The read is four catalog queries on PostgreSQL and SQLite and two on MongoDB and Cassandra, once per call. update_ttl admits DELETING beside ACTIVE in the conditional write on PostgreSQL, SQLite and MongoDB, and in the status check and both lightweight transactions on Cassandra. CREATING and UPDATING keep TableNotActive, and a missing table keeps TableNotFound. UpdateTimeToLive sends no index DDL for a table it read as DELETING, since the delete removes the data table or collection with its index. On PostgreSQL a disable's DROP INDEX waited with no lock timeout behind whatever held the data table and queued the table's data-plane requests behind it. On MongoDB an enable's create_index could create the collection again after the delete dropped it, and the collection outlived the table. MongoDB create_ttl_index also reads an ACTIVE catalog document only. A delete that commits between the status read and the write meets the index work, which the base refused at the write; on MongoDB the ACTIVE filter covers that order. TagResource and UntagResource on the ARN of a DELETING table answer 400 ResourceInUseException "Attempt to change a resource which is still in use: Table is being deleted: <name>", and ListTagsOfResource is served with the tags. On the table's stream ARN both writes and the read are served. Amazon DynamoDB answers this way (measured 2026-10-25 and 2026-10-26). ExtendDB answered ResourceNotFoundException for all three on MongoDB and Cassandra and accepted both writes on PostgreSQL and SQLite. An accepted TagResource on the table ARN writes an unconditioned tag row; on PostgreSQL the delete removes the ARN's tag rows before the catalog row, so a tag written between the two statements could outlive the table and be inherited by the next table of that name. The tag path reads the table's status before the key-info read, so all four backends reach the window's answers, and refuses the two writes when the ARN names the table. The key-info read still follows for every status but DELETING, so a tag call on an ACTIVE table sends seven catalog queries before its tag statements on PostgreSQL and SQLite, where it sent three, and four on MongoDB and Cassandra, where it sent two. The refusal removes the writes admitted during the window. A write whose status read came before the delete can still be written after the delete removed the tag rows, as before. ListTagsOfResource returns the tags for the whole window on PostgreSQL and SQLite. On MongoDB it returns no tags once the delete call has removed them; the service returns them for the whole window. A MongoDB storage test drives the tag and TTL handlers on a DELETING document, and devtools/run-mongodb-tests runs the crate's ignored deleting_window tests in CI. Closes #552 Closes #551
LeeroyHannigan
requested review from
amrith,
c33howard,
jcshepherd,
pdf-amzn,
robinnsc and
yesyayen
as code owners
October 26, 2026 10:11
This was referenced Oct 26, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #552 and #551. The branch is based on #550 (fix/505-deleting-window-answers), which adds the Table is being deleted sentence; it targets main once #550 is merged.
While a table is DELETING, Amazon DynamoDB answers these calls as follows, on the table and on the Amazon Resource Name (ARN) of its stream, measured 2026-10-25 and 2026-10-26:
DescribeTimeToLive on a CREATING table answers the same ValidationException with CREATING in the sentence, and is served on an UPDATING table.
ExtendDB now answers these calls this way on PostgreSQL, SQLite, MongoDB and Cassandra. Before, UpdateTimeToLive answered "Table is not in ACTIVE state", DescribeTimeToLive answered 200, and the tag calls answered ResourceNotFoundException on MongoDB and Cassandra while PostgreSQL and SQLite accepted both writes.
How it works:
Catalog cost: the status read is four catalog queries on PostgreSQL and SQLite and two on MongoDB and Cassandra, once per DescribeTimeToLive, UpdateTimeToLive, TagResource, UntagResource and ListTagsOfResource call. The tag path keeps the key-info read after it for every status but DELETING, so a tag call on an ACTIVE table sends seven queries before its tag statements on PostgreSQL and SQLite (three before this change) and four on MongoDB and Cassandra (two before). These are control-plane calls; the item and batch paths are unchanged.
Differences from the service that remain:
Tests:
Merge notes
With #553 (fix/553-cassandra-concurrent-delete-table): one conflict in tests/python/test_deleting_window_control_plane.py, where both branches append test classes at the same point; keep both blocks, this branch's two classes first, then TestConcurrentDeleteTable. crates/storage-cassandra/tests/direct/deleting_window.rs merges on its own.
With #543 (fix/438-get-records-round-trips): devtools/run-mongodb-tests, where both add a live-test step at the same place; the resolution is one step with
--ignored records_live deleting_window.