Skip to content

Client layer: rebase advertised links, refuse redirects, add calls the filesystems need - #60

Merged
tilorspace merged 4 commits into
mainfrom
fs-stack/1-client-layer
Oct 12, 2026
Merged

tilorspace merged 4 commits into
mainfrom
fs-stack/1-client-layer

Conversation

@tilorspace

@tilorspace tilorspace commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Changes to the two REST clients that the filesystem work needs, and fixes that stand on their own.

  • URLs the server advertises in _links are rebased onto the configured base URL, path prefix included. Next-page and download links then work behind a port-forward, in a container, or behind a proxy that serves RSpace under a sub-path such as /rspace.
  • The shared session never follows an HTTP redirect. requests keeps custom headers such as apiKey across a redirect to another host, which would hand the key to that host.
  • doDelete tolerates a leading slash, which fixes delete_document.
  • ELNClient.upload_file and InventoryClient.upload_attachment_by_global_id take an optional filename=; list_folder_tree takes page_size=.
  • New InventoryClient.get_workbench_by_id (the containers endpoint answers 422 for a bench) and attach_gallery_file_by_global_id (the "link from Gallery" the web interface offers).
  • Inventory CSV import goes through the shared session, so it gets the same timeout, retries and error handling as every other request.
  • A stray debug print is removed from upload_attachment_by_global_id.

The existing filesystem tests are updated to expect allow_redirects=False on every request.

Review follow-up

  • Link rebasing now keeps the configured path prefix. Before, a client set to https://proxy/rspace followed links to https://proxy/api/v1/... and lost /rspace.
  • client_methods_test.py covers each changed or new client method: doDelete URLs, the multipart filename=, page_size=, get_workbench_by_id, attach_gallery_file_by_global_id, and CSV import through the shared session.
  • Checked end to end against a docker RSpace 2.27 behind an nginx proxy at /rspace: before the fix every followed link returned 404, after it ELN paging, Gallery downloads and Inventory paging all work. The Galaxy 26.1 file source works through the same proxy.

Stack

This started as part 1 of a five-part stack towards a unified PyFilesystem (#58). On 2026-10-06 the filesystem moved to fsspec instead, so parts 3 to 5 (#62, #63, #64) were closed without merging. This client layer and the offline mock server (#61) don't depend on the filesystem framework and are kept.

  1. This PR, client layer (merged)
  2. Add an offline mock of the RSpace ELN and Inventory APIs for the test suite #61, offline mock of the RSpace APIs for the test suite
  3. Add rspace_client.fs: the Gallery, Inventory and ELN Workspace as one fsspec filesystem #68, rspace_client.fs as an fsspec filesystem, built on Add an offline mock of the RSpace ELN and Inventory APIs for the test suite #61; replaces Add rspace_client.fs: the Gallery, Inventory and ELN Workspace as one PyFilesystem #62 to Pin what Galaxy's RSpace file source relies on, and add a plugin check tool #64

🤖 Generated with Claude Code

…s the filesystems need

URLs the server advertises in `_links` are rebased onto the configured host so paging and
download links work behind a proxy or port-forward. The session never follows a redirect,
because `requests` would carry the `apiKey` header to the new host. `doDelete` tolerates a
leading slash. `upload_file` and `upload_attachment_by_global_id` take `filename=`,
`list_folder_tree` takes `page_size=`, and the Inventory client gains `get_workbench_by_id`
and `attach_gallery_file_by_global_id`. Inventory CSV import goes through the shared session.
The existing filesystem tests now expect the redirect flag on every request.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@tilorspace
tilorspace added this pull request to stack #65 September 21, 2026 12:29
@tilorspace
tilorspace marked this pull request as ready for review September 21, 2026 12:43
tilorspace and others added 2 commits October 12, 2026 08:17
_rebase_url swapped scheme and host but kept the server's path, so a client
configured with a path prefix (https://proxy/rspace) followed next-page and
download links to https://proxy/api/v1/... and lost /rspace. Re-anchor the
part from /api/ on under our own base path, the same base relative requests
use. Links without /api/ keep their path. Raised in review of PR #60.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Covers the review's test gaps: doDelete single-slash URLs (regression for
delete_document), upload_file and upload_attachment_by_global_id filename=,
list_folder_tree page_size=, get_workbench_by_id id forms and includeContent,
attach_gallery_file_by_global_id body, and CSV parse through the shared
session with redirects off and the client timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@nebay-abraha nebay-abraha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍

@tilorspace
tilorspace merged commit 057d1fc into main Oct 12, 2026
7 checks passed
@tilorspace
tilorspace deleted the fs-stack/1-client-layer branch October 12, 2026 07:43
tilorspace added a commit that referenced this pull request Oct 12, 2026
fs-stack/2 was rebased onto main after #60 was squash-merged; its content is unchanged, so the feature branch keeps its own tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tilorspace added a commit that referenced this pull request Oct 12, 2026
… suite (#61)

A standard-library mock of the ELN (/api/v1) and Inventory
(/api/inventory/v1) endpoints the filesystem classes use, under
rspace_client/tests/mock_rspace. Nothing here is installed with the
package.

- server.py serves an in-memory dataset from fixtures.py and counts
  requests, so a test can assert what a listing costs.
- The dataset carries the cases a filesystem has to get right: duplicate
  names, names that look like global IDs, system folders in Home,
  sections large enough to paginate, a document with text and non-text
  fields, a signed document, a bench with attachments.
- conftest.py starts one server for the whole pytest session;
  MockServerTestCase resets its fixtures before every test.

No test uses it yet; #68 adds the fsspec filesystem tests, the Galaxy
contract test and the plugin check tool that run against it.

Stack: part 2 of the original PyFilesystem stack (#58). Parts 3 to 5
(#62, #63, #64) were closed when the filesystem moved to fsspec; the
client layer (#60) and this mock are framework-neutral and were kept.
#68, the fsspec filesystem, is now based on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tilorspace added a commit that referenced this pull request Oct 29, 2026
… fsspec filesystem (#68)

**Stacked PR. Merge after #60 (client layer) and #61 (offline mock
server); this one targets `fs-stack/2-mock-server`.**

Replaces the PyFilesystem2 version of the unified filesystem (#62 to
#64, closed). PyFilesystem2 has had no release since 2022, and Galaxy,
the main consumer, is moving every file source to fsspec
(galaxyproject/galaxy#21832, where `rspace.py` is on the list). Tracked
in RSDEV-1519.

## What it adds

- **`rspace_client.fs`**, an fsspec filesystem. `RSpaceFilesystem`
mounts `gallery/`, `inventory/` and `workspace/`, sharing one ELN and
one Inventory client. Each branch is also usable on its own
(`GalleryFilesystem`, `InventoryFilesystem`, `WorkspaceFilesystem`).
- Paths are record names (`gallery/Images/microscope.png`). Duplicates,
and names that read as addresses, carry their global ID (`data
[GL112].csv`). `path_style="labelled"` or `"id"` give paths that survive
a rename.
- Read-only by default (`writable=True`, `allow_delete=True`). Uploads
into Inventory records and ELN fields go through the Gallery and are
linked. `cp_file` from a Gallery-backed file links instead of copying
bytes. Transactions defer uploads.
- `rspace://host/...` URLs via `fsspec.open`,
`fsspec.filesystem("rspace")` and `UPath`, registered as an
`fsspec.specs` entry point. The key comes from `RSPACE_API_KEY` only,
never from the URL.
- RSpace errors surface as builtin `OSError` subclasses, so `exists`,
`walk`, `copy` and Galaxy behave.
- **Deprecation** of `rspace_client.eln.fs.GalleryFilesystem` and
`rspace_client.inv.attachment_fs.InventoryAttachmentFilesystem`. Their
behaviour is unchanged apart from the two Galaxy fixes below, they warn
on construction, and they're removed in 3.0 together with `fs`, after
Galaxy's shipped plugin has moved.
- **Fix for Galaxy 26.1 exports to RSpace.** requests 2.34 (pinned by
Galaxy 26.1) refuses a file part that forwards `read` through
`__getattr__`. That's how Galaxy's `FakedNameIO` wrapper works, so
exports through Galaxy's shipped `rspace` source fail on every released
rspace-client. `_post_multipart` now wraps such objects. Verified inside
a Galaxy 26.1.2 container: 2.7.4 raises the `TypeError`, this branch
uploads.
- **Fix for Galaxy on Python 3.10.** Galaxy's shipped `rspace` source
parses a file's `created` date with `datetime.fromisoformat`, which
rejects RSpace's trailing `Z` before Python 3.11, so listing any Gallery
folder with files failed on Python 3.10 with every released
rspace-client. `GalleryFilesystem` now gives the same UTC instant as
`+00:00`. Verified with Galaxy's real plugin (galaxy-files 26.1.1) on
Python 3.10.
- **`galaxy_plugin_contract_test.py`**, which pins what Galaxy's shipped
plugin relies on. A failure there is a release blocker.
- **`tools/galaxy/rspace_fsspec_source.py`**, the reference Galaxy file
source (type `rspace_unified`) to propose upstream once this is
released.
- **Docs:** `docs/usage-guide.md` and the design record
`docs/fsspec-filesystem-design.md`, which explains why the package is
shaped this way and what must not move.

## Review order

1. `docs/fsspec-filesystem-design.md` (the decisions)
2. `rspace_client/fs/paths.py`, `rspace_client/fs/base.py` (shared
behaviour)
3. `gallery.py`, `inventory.py`, `workspace.py`, then `rspace.py` (the
mount)
4. Shims: `rspace_client/eln/fs.py`,
`rspace_client/inv/attachment_fs.py`, and the contract test
5. Tests (`fs_*_test.py`), `tools/galaxy/`

## Testing

- Offline suite: 305 passed (`poetry run pytest -m "not integration"`),
using the mock server from #61.
- `fs_integration_test.py` against a docker RSpace 2.27, plus a
record-type matrix covering Gallery, documents, samples, subsamples,
containers, instruments, and the bench and template refusals.
- Galaxy 25.1 and 26.1 containers with the reference plugin: listing,
search (including bracketed names, CJK and `%`), pagination, import from
Gallery, Inventory and Workspace, and export into a sample, a field and
the Gallery.

New dependency: `fsspec>=2024.3.0`. `fs` and the `setuptools<82` pin
stay until 3.0.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants