fix: propagate SupportsBulkOperations through the FileIO wrappers - #5378
fix: propagate SupportsBulkOperations through the FileIO wrappers#5378rinzool wants to merge 2 commits into
Conversation
b823d16 to
105cf10
Compare
Hi @vigneshio I opened a Draft PR on top on this one to see what we could do. If you want, I can include that in this PR or open a PR after this one is merged for this fix. |
|
Thanks for looking into this. The fix looks good to me. |
| public void deleteFiles(Iterable<String> paths) throws BulkDeletionFailureException { | ||
| bulkIo.deleteFiles( | ||
| () -> | ||
| StreamSupport.stream(paths.spliterator(), false) | ||
| .map(WasbTranslatingFileIO::translate) | ||
| .iterator()); | ||
| } |
There was a problem hiding this comment.
maybe we could do this in one line
bulkIo.deleteFiles(Iterables.transform(paths, WasbTranslatingFileIO::translate));
CatalogUtil.deleteFiles batches only when the FileIO satisfies `instanceof SupportsBulkOperations`, and falls back to one deleteFile per path otherwise. The FileIO reaching the cleanup tasks is always wrapped: DefaultFileIOFactory wraps it in ExceptionMappingFileIO, and on Azure WasbTranslatingFileIOFactory wraps it again. Both declared only FileIO, and a capability held by the wrapped object is not visible through the wrapper, since instanceof tests the wrapper's own type. The check was therefore always false and the per-file branch always ran. Every storage backend is affected: S3FileIO, GCSFileIO, ADLSFileIO and HadoopFileIO all implement DelegateFileIO, which extends SupportsBulkOperations. Both wrappers now expose a wrap(FileIO) factory returning a subclass that implements SupportsBulkOperations and delegates deleteFiles, but only when the wrapped FileIO actually supports bulk operations; advertising it unconditionally would send callers down a bulk path the wrapped FileIO cannot serve. The constructors become protected so callers go through the factory. WasbTranslatingFileIO translates each path on the bulk path, and passes a re-iterable Iterable since an implementation may traverse the paths more than once. Fixes apache#5377
Replaces the stream-plus-iterator lambda with a lazy Guava view, which keeps the argument re-iterable while reading as a single expression.
105cf10 to
45f3ba8
Compare
Background
#4850 added batched deletion to the file cleanup tasks:
BatchFileCleanupTaskHandlerhands thewhole batch to
FileCleanupTaskHandler.tryDelete(..., Iterable<String> files, type, isConcurrent, ...), which callsCatalogUtil.deleteFiles.CatalogUtil.deleteFilesbatches only when the FileIO satisfiesinstanceof SupportsBulkOperations, and falls back to onedeleteFileper path otherwise.The FileIO that reaches the cleanup tasks is never the configured one. It is wrapped by
DefaultFileIOFactoryinExceptionMappingFileIO, and on Azure wrapped again byWasbTranslatingFileIOFactoryinWasbTranslatingFileIO. Both declared onlyFileIO, and acapability held by the wrapped object is not visible through the wrapper:
instanceoftests thewrapper's own type. The check was therefore always false and the per-file branch always ran, so the
batching added by #4850 never executed.
This affects every storage backend, since each FileIO Polaris selects by storage type implements
DelegateFileIO, which extendsSupportsBulkOperations:S3FileIO,GCSFileIO,ADLSFileIOandHadoopFileIO.Fixes #5377
Change
Both wrappers gain a
wrap(FileIO)factory that returns a subclass implementingSupportsBulkOperations, delegatingdeleteFilesto the wrapped FileIO but only when the wrappedFileIO actually supports bulk operations. The constructors become
privateso callers go throughthe factory and cannot silently reintroduce the problem. Both factories now call
wrap().WasbTranslatingFileIOtranslates each path on the bulk path, as it already does fordeleteFile.It passes a re-iterable
Iterablerather than a one-shot iterator, since an implementation maytraverse the paths more than once (for example to size the batch before deleting).
Advertising the capability unconditionally was deliberately avoided: it would send callers down a
bulk path the wrapped FileIO cannot serve. Implementing
DelegateFileIOwas also avoided, becauseit would additionally claim
SupportsPrefixOperations, which these wrappers do not delegate.Effect
Measured on a downstream deployment carrying the same wrappers, dropping a table with 660 data
files across 66 manifests (859 files deleted in total):
DeleteObject+ 859HeadObjectNotes
Two follow-ups were left out to keep this change focused, and can be filed separately if useful:
ManifestFileCleanupTaskHandlerstill deletes manifest data files one per file; it never adoptedthe batch method from Support bulk deletion in batch file cleanup task #4850.
TableCleanupTaskHandlerbuilds each cleanup task's name by concatenating the batch's file list.Since the name is part of a unique index, raising
TABLE_METADATA_CLEANUP_BATCH_SIZEmakes theinsert exceed the PostgreSQL btree limit, the parent cleanup task fails, and the dropped table's
files are never removed. This currently caps metadata batches at roughly 26 paths.
AI assistance was used in preparing this change (investigation, drafting and tests); the
implementation and its rationale have been reviewed and are my responsibility.