fix: sign use_http_get reads so private S3/MinIO buckets work MAPCO-11391 - #95
Open
razbroc wants to merge 1 commit into
Open
fix: sign use_http_get reads so private S3/MinIO buckets work MAPCO-11391#95razbroc wants to merge 1 commit into
razbroc wants to merge 1 commit into
Conversation
…1391 Mirrors the upstream fork's feat/sign-s3-http-get-reads fix into the vendored s3.py patch. is_cached/load_tile now sign HTTP-GET reads with a boto3 presigned URL addressed via Bucket=self.bucket_name (matching the write path), instead of issuing unsigned requests that a private bucket rejects with 403 and the old code treated as a cache miss. 403 now raises S3ConnectionError; 404 remains a normal miss. Debug logs print the object key instead of the full (now signature-bearing) URL. Only the pre-existing, documented config-surface differences (env vars, ImageSource) remain between this file and the upstream fork. Verified the Docker build's patch-apply/import sanity check still passes with PATCH_FILES=true.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Mirrors the upstream MapProxy fork's signed-read fix (mapproxy/mapproxy#1477) into this repo's vendored
config/patch/s3.py, applied at Docker build time whenPATCH_FILES=true.is_cached/load_tilepreviously issued completely unauthenticatedurllib3HEAD/GET requests whenuse_http_getis enabled. A private bucket returns403to any unsigned request, and the old code treated403identically to404("not cached"). Writes remain authenticated via boto3 and always succeed, so tiles get written to a private bucket but could never be read back through this path — any deployment using an S3-compatible cache (e.g. MinIO) withuse_http_get: trueagainst a private bucket effectively never served tiles from cache.Changes
is_cached/load_tilenow sign their HEAD/GET requests using a boto3-generated presigned URL, built locally from the same session already used for writes (no extra network round-trip).Bucket=self.bucket_name, matching the write path — the{endpoint}/{username}:{bucket}/{key}URL convention is retired from this path.403on a signed request now raisesS3ConnectionErrorinstead of being treated as a cache miss;404remains a normal cache miss, unchanged.ImageSource) remain between this file and the upstream fork.No changes to mapproxy-api/mapproxinator or this repo's Helm chart.
Test plan
PATCH_FILES=truecompletes and passes the existing patch-apply/import sanity check.GetObjectdenied surfacesS3ConnectionErrorwith a clear traceback rather than a silent, permanent cache miss; existing public-bucket +use_http_getbehavior is unchanged.MAPCO-11391