Skip to content

Cassandra: CreateBackup runs inside the request future #528

Open

Description

@LeeroyHannigan

Summary

On the Cassandra backend CreateBackup copies the source table inside the request future. A client that disconnects or times out while the copy runs drops the future and the copy stops; the backup record is left as the request left it. Measured on main 744ea29: a CreateBackup of a 200,000-item table whose client gave up before the response completed left no backup record (no row in backups_by_arn, nothing for ListBackups to show), so the client learns nothing about what happened and a retry starts over.

RestoreTableFromBackup on the same backend runs its copy in a task detached from the request with a heartbeat (#384), so a disconnect does not cancel it and an abandoned target is swept. CreateBackup has no such task.

Fix shape

Run the copy in a detached task the request awaits, as the restore does: write the backup record as CREATING first, copy, then mark it AVAILABLE; a heartbeat on the record so a copy whose process dies is swept rather than left CREATING. The service answers CreateBackup with BackupStatus CREATING and the backup becomes AVAILABLE later, which is the shape the detached form produces.

Found while reviewing #527.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backend:cassandraRelated to the Cassandra backend specificallybugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions