Fix get_files(directory) on S3 and local storage drivers - #222
Open
tmgbedu wants to merge 1 commit into
Open
Conversation
Fixes #218. get_files(directory) was broken differently on each driver, and the two disagreed about what the method means: - S3: the nested-file filter tested "/" against the whole key instead of the part below the prefix, so any non-empty directory returned [] while root-level keys leaked through. Filter now uses a slash-terminated Prefix and tests only the key's relative part, which also stops 'backups' from matching 'backups-old/'. - Local: os.listdir yields bare names but contents were fetched via get(name), which resolves against the disk root — a miss that get() swallows, returning File objects with None content. Contents are now read from directory/name, and a nonexistent directory returns []. - Fake: inherits the fixed LocalDriver.get_files; tests lock the semantics in. All three drivers now share one contract: non-recursive listing of the files directly under the given directory, bare filenames, no directory entries, [] for empty or nonexistent directories. Adds mocked S3 unit tests plus live integration tests against a real S3 API (MinIO, skipped when unreachable), and boto3 as a dev dependency to support them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KVFQTBaxKHnCZfjmtx2HYg
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #218
Problem
get_files(directory)was broken on both drivers, in two different ways, and the drivers disagreed about what the method means — making it unusable in disk-agnostic code:"/" not in keyagainst the whole key instead of the part below the prefix. Any key matched by a non-emptyPrefixnecessarily containsprefix + '/', so every match was discarded and root-level keys leaked through instead.get_files("backups")always returned[].os.listdiryields bare names, but contents were fetched withself.get(f), which resolves against the disk root rather thanroot/<directory>. The lookup missed,get()swallowed theFileNotFoundError, and callers silently receivedFileobjects withNonecontent.FakeDriverextendsLocalDriver, so it inherited the same broken listing (its ownassert_count(n, directory)relied on it).Fix
One converged, documented contract across all three drivers: non-recursive listing of the files directly under the given directory, bare filenames, no directory entries,
[]for empty or nonexistent directories.Prefix="backups/") and test only the key's relative part. This also stopsget_files("backups")from matching abackups-old/sibling and skips directory-marker keys.join(directory, f); return[]for a nonexistent directory (matching S3's behavior for an unknown prefix).LocalDriver.get_files; dedicated tests lock the semantics in.Testing
MINIO_ENDPOINT/MINIO_ACCESS_KEY/MINIO_SECRET_KEY. All 8 pass against the live server.boto3added to the dev dependency group to support the S3 tests.Full suite:
uv run pytest --ignore=tests/masoniteorm/postgres --cov→ 2213 passed, 7 skipped, coverage 84.50% (threshold met).🤖 Generated with Claude Code
https://claude.ai/code/session_01KVFQTBaxKHnCZfjmtx2HYg