Skip to content

fix: propagate SupportsBulkOperations through the FileIO wrappers - #5378

Open
rinzool wants to merge 2 commits into
apache:mainfrom
rinzool:fix/bulk-delete-wrapper-capability
Open

fix: propagate SupportsBulkOperations through the FileIO wrappers#5378
rinzool wants to merge 2 commits into
apache:mainfrom
rinzool:fix/bulk-delete-wrapper-capability

Conversation

@rinzool

@rinzool rinzool commented Aug 25, 2026

Copy link
Copy Markdown

Background

#4850 added batched deletion to the file cleanup tasks: BatchFileCleanupTaskHandler hands the
whole batch to FileCleanupTaskHandler.tryDelete(..., Iterable<String> files, type, isConcurrent, ...), which calls CatalogUtil.deleteFiles.

CatalogUtil.deleteFiles batches only when the FileIO satisfies
instanceof SupportsBulkOperations, and falls back to one deleteFile per path otherwise.

The FileIO that reaches the cleanup tasks is never the configured one. It is wrapped by
DefaultFileIOFactory in ExceptionMappingFileIO, and on Azure wrapped again by
WasbTranslatingFileIOFactory in WasbTranslatingFileIO. Both declared only FileIO, and a
capability held by the wrapped object is not visible through the wrapper: instanceof tests the
wrapper's own type. The check was therefore always false and the per-file branch always ran, so the
batching added by #4850 never executed.

This affects every storage backend, since each FileIO Polaris selects by storage type implements
DelegateFileIO, which extends SupportsBulkOperations: S3FileIO, GCSFileIO, ADLSFileIO and
HadoopFileIO.

Fixes #5377

Change

Both wrappers gain a wrap(FileIO) factory that returns a subclass implementing
SupportsBulkOperations, delegating deleteFiles to the wrapped FileIO but only when the wrapped
FileIO actually supports bulk operations. The constructors become private so callers go through
the factory and cannot silently reintroduce the problem. Both factories now call wrap().

WasbTranslatingFileIO translates each path on the bulk path, as it already does for deleteFile.
It passes a re-iterable Iterable rather than a one-shot iterator, since an implementation may
traverse the paths more than once (for example to size the batch before deleting).

Advertising the capability unconditionally was deliberately avoided: it would send callers down a
bulk path the wrapped FileIO cannot serve. Implementing DelegateFileIO was also avoided, because
it would additionally claim SupportsPrefixOperations, which these wrappers do not delegate.

Effect

Measured on a downstream deployment carrying the same wrappers, dropping a table with 660 data
files across 66 manifests (859 files deleted in total):

requests to object storage
wrappers as they were 859 DeleteObject + 859 HeadObject
with the capability propagated 859 files deleted in 86 batched calls

Notes

Two follow-ups were left out to keep this change focused, and can be filed separately if useful:

  • ManifestFileCleanupTaskHandler still deletes manifest data files one per file; it never adopted
    the batch method from Support bulk deletion in batch file cleanup task #4850.
  • TableCleanupTaskHandler builds each cleanup task's name by concatenating the batch's file list.
    Since the name is part of a unique index, raising TABLE_METADATA_CLEANUP_BATCH_SIZE makes the
    insert exceed the PostgreSQL btree limit, the parent cleanup task fails, and the dropped table's
    files are never removed. This currently caps metadata batches at roughly 26 paths.

AI assistance was used in preparing this change (investigation, drafting and tests); the
implementation and its rationale have been reviewed and are my responsibility.

@github-project-automation github-project-automation Bot moved this to PRs In Progress in Basic Kanban Board Aug 25, 2026
@rinzool
rinzool force-pushed the fix/bulk-delete-wrapper-capability branch 2 times, most recently from b823d16 to 105cf10 Compare August 25, 2026 15:52
@rinzool
rinzool marked this pull request as ready for review August 25, 2026 15:54

@vigneshio vigneshio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.. Note that BatchFileCleanupTaskHandler still does a HEAD per file via TaskUtils.exists before delete. This pr batches the deletes, but the exists check remains... Worth a follow-up?

Thanks for contribution @rinzool

@rinzool

rinzool commented Aug 28, 2026

Copy link
Copy Markdown
Author

LGTM.. Note that BatchFileCleanupTaskHandler still does a HEAD per file via TaskUtils.exists before delete. This pr batches the deletes, but the exists check remains... Worth a follow-up?

Thanks for contribution @rinzool

Hi @vigneshio
Good catch
I checked all provider, they should not raise any error in case of missing file when calling the bulk delete. Therefore we could totally drop the check of existing files on classes that support SupportsBulkOperations and therefore skip those head object.

I opened a Draft PR on top on this one to see what we could do. If you want, I can include that in this PR or open a PR after this one is merged for this fix.
rinzool#2

@nandorKollar

Copy link
Copy Markdown
Contributor

Thanks for looking into this. The fix looks good to me.

nandorKollar
nandorKollar previously approved these changes Sep 3, 2026
@github-project-automation github-project-automation Bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Sep 3, 2026

@ayushtkn ayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanx @rinzool for the fix. Minor suggestion

Comment on lines +119 to +125
public void deleteFiles(Iterable<String> paths) throws BulkDeletionFailureException {
bulkIo.deleteFiles(
() ->
StreamSupport.stream(paths.spliterator(), false)
.map(WasbTranslatingFileIO::translate)
.iterator());
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could do this in one line

bulkIo.deleteFiles(Iterables.transform(paths, WasbTranslatingFileIO::translate));

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, thanks! I updated

Quentin FLEURENT NAMBOT added 2 commits September 4, 2026 17:00
CatalogUtil.deleteFiles batches only when the FileIO satisfies
`instanceof SupportsBulkOperations`, and falls back to one deleteFile per path
otherwise.

The FileIO reaching the cleanup tasks is always wrapped: DefaultFileIOFactory
wraps it in ExceptionMappingFileIO, and on Azure WasbTranslatingFileIOFactory
wraps it again. Both declared only FileIO, and a capability held by the wrapped
object is not visible through the wrapper, since instanceof tests the wrapper's
own type. The check was therefore always false and the per-file branch always
ran. Every storage backend is affected: S3FileIO, GCSFileIO, ADLSFileIO and
HadoopFileIO all implement DelegateFileIO, which extends SupportsBulkOperations.

Both wrappers now expose a wrap(FileIO) factory returning a subclass that
implements SupportsBulkOperations and delegates deleteFiles, but only when the
wrapped FileIO actually supports bulk operations; advertising it unconditionally
would send callers down a bulk path the wrapped FileIO cannot serve. The
constructors become protected so callers go through the factory.
WasbTranslatingFileIO translates each path on the bulk path, and passes a
re-iterable Iterable since an implementation may traverse the paths more than
once.

Fixes apache#5377
Replaces the stream-plus-iterator lambda with a lazy Guava view, which keeps the
argument re-iterable while reading as a single expression.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bulk file deletion never engages because the FileIO wrappers hide SupportsBulkOperations

4 participants