Repository navigation
Add an offline mock of the RSpace ELN and Inventory APIs for the test suite - #61
Merged
Merged
Conversation
tilorspace
marked this pull request as ready for review
September 21, 2026 12:44
This was referenced Oct 9, 2026
tilorspace
added a commit
that referenced
this pull request
Oct 12, 2026
…e filesystems need (#60) Client layer: rebase advertised links, refuse redirects, add calls the filesystems need (#60) Changes to the ELN and Inventory REST clients that the filesystem work needs, plus fixes that stand on their own: - Links the server advertises in `_links` are rebased onto the configured base URL, path prefix included, so next-page and download links work behind a port-forward, in a container, or behind a proxy that serves RSpace under a sub-path such as /rspace. - The shared session never follows an HTTP redirect, because requests would carry the apiKey header to the new host. - doDelete tolerates a leading slash, which fixes delete_document. - ELNClient.upload_file and InventoryClient.upload_attachment_by_global_id take an optional filename=; list_folder_tree takes page_size=. - New InventoryClient.get_workbench_by_id (/containers answers 422 for a bench) and attach_gallery_file_by_global_id (link an existing Gallery file to an Inventory record). - Inventory CSV import goes through the shared session, so it gets the same timeout, retries and error handling as every other request. - A stray debug print is removed from upload_attachment_by_global_id. Review follow-up: the path-prefix handling in link rebasing, and unit tests for each changed or new client method (client_methods_test.py). Stack: this was part 1 of five PRs towards a unified PyFilesystem (#58). On 2026-10-06 the filesystem was moved to fsspec instead, so parts 3 to 5 (#62, #63, #64) were closed without merging. This client layer and the offline mock server (#61) are independent of the filesystem framework and are kept. The fsspec filesystem itself is #68, which builds on #61. Merge #61 next, then retarget #68 to main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… suite A standard-library HTTP server covering the endpoints the PyFilesystem classes use, backed by an in-memory dataset with the awkward cases a filesystem has to handle (duplicate names, system folders, paginated sections, documents with mixed field types). It counts requests so tests can assert what an operation costs. One server runs for the whole pytest session and `MockServerTestCase` resets its fixtures before each test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
tilorspace
force-pushed
the
fs-stack/2-mock-server
branch
from
October 12, 2026 07:43
638a685 to
631051a
Compare
tilorspace
added a commit
that referenced
this pull request
Oct 12, 2026
#61 was squash-merged into main; main now holds the same content this branch already had, so the branch keeps its own tree (conftest.py stays deleted, the feature's mock fixtures stay). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tilorspace
added a commit
that referenced
this pull request
Oct 29, 2026
… fsspec filesystem (#68) **Stacked PR. Merge after #60 (client layer) and #61 (offline mock server); this one targets `fs-stack/2-mock-server`.** Replaces the PyFilesystem2 version of the unified filesystem (#62 to #64, closed). PyFilesystem2 has had no release since 2022, and Galaxy, the main consumer, is moving every file source to fsspec (galaxyproject/galaxy#21832, where `rspace.py` is on the list). Tracked in RSDEV-1519. ## What it adds - **`rspace_client.fs`**, an fsspec filesystem. `RSpaceFilesystem` mounts `gallery/`, `inventory/` and `workspace/`, sharing one ELN and one Inventory client. Each branch is also usable on its own (`GalleryFilesystem`, `InventoryFilesystem`, `WorkspaceFilesystem`). - Paths are record names (`gallery/Images/microscope.png`). Duplicates, and names that read as addresses, carry their global ID (`data [GL112].csv`). `path_style="labelled"` or `"id"` give paths that survive a rename. - Read-only by default (`writable=True`, `allow_delete=True`). Uploads into Inventory records and ELN fields go through the Gallery and are linked. `cp_file` from a Gallery-backed file links instead of copying bytes. Transactions defer uploads. - `rspace://host/...` URLs via `fsspec.open`, `fsspec.filesystem("rspace")` and `UPath`, registered as an `fsspec.specs` entry point. The key comes from `RSPACE_API_KEY` only, never from the URL. - RSpace errors surface as builtin `OSError` subclasses, so `exists`, `walk`, `copy` and Galaxy behave. - **Deprecation** of `rspace_client.eln.fs.GalleryFilesystem` and `rspace_client.inv.attachment_fs.InventoryAttachmentFilesystem`. Their behaviour is unchanged apart from the two Galaxy fixes below, they warn on construction, and they're removed in 3.0 together with `fs`, after Galaxy's shipped plugin has moved. - **Fix for Galaxy 26.1 exports to RSpace.** requests 2.34 (pinned by Galaxy 26.1) refuses a file part that forwards `read` through `__getattr__`. That's how Galaxy's `FakedNameIO` wrapper works, so exports through Galaxy's shipped `rspace` source fail on every released rspace-client. `_post_multipart` now wraps such objects. Verified inside a Galaxy 26.1.2 container: 2.7.4 raises the `TypeError`, this branch uploads. - **Fix for Galaxy on Python 3.10.** Galaxy's shipped `rspace` source parses a file's `created` date with `datetime.fromisoformat`, which rejects RSpace's trailing `Z` before Python 3.11, so listing any Gallery folder with files failed on Python 3.10 with every released rspace-client. `GalleryFilesystem` now gives the same UTC instant as `+00:00`. Verified with Galaxy's real plugin (galaxy-files 26.1.1) on Python 3.10. - **`galaxy_plugin_contract_test.py`**, which pins what Galaxy's shipped plugin relies on. A failure there is a release blocker. - **`tools/galaxy/rspace_fsspec_source.py`**, the reference Galaxy file source (type `rspace_unified`) to propose upstream once this is released. - **Docs:** `docs/usage-guide.md` and the design record `docs/fsspec-filesystem-design.md`, which explains why the package is shaped this way and what must not move. ## Review order 1. `docs/fsspec-filesystem-design.md` (the decisions) 2. `rspace_client/fs/paths.py`, `rspace_client/fs/base.py` (shared behaviour) 3. `gallery.py`, `inventory.py`, `workspace.py`, then `rspace.py` (the mount) 4. Shims: `rspace_client/eln/fs.py`, `rspace_client/inv/attachment_fs.py`, and the contract test 5. Tests (`fs_*_test.py`), `tools/galaxy/` ## Testing - Offline suite: 305 passed (`poetry run pytest -m "not integration"`), using the mock server from #61. - `fs_integration_test.py` against a docker RSpace 2.27, plus a record-type matrix covering Gallery, documents, samples, subsamples, containers, instruments, and the bench and template refusals. - Galaxy 25.1 and 26.1 containers with the reference plugin: listing, search (including bracketed names, CJK and `%`), pagination, import from Gallery, Inventory and Workspace, and export into a sample, a field and the Gallery. New dependency: `fsspec>=2024.3.0`. `fs` and the `setuptools<82` pin stay until 3.0. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A standard-library mock of the ELN (
/api/v1) and Inventory (/api/inventory/v1) endpoints the filesystem classes use, underrspace_client/tests/mock_rspace. Nothing here is installed with the package.server.pyserves an in-memory dataset fromfixtures.pyand counts requests, so a test can assert what a listing costs.conftest.pystarts one server for the whole pytest session;MockServerTestCaseresets its fixtures before every test.The mock doesn't depend on a filesystem framework. No test uses it yet in this PR. #68 adds the fsspec filesystem tests, the Galaxy contract test and the plugin check tool that run against it, and adjusts a few fixtures.
Stack
This started as part 2 of a five-part stack towards a unified PyFilesystem (#58). On 2026-10-06 the filesystem moved to fsspec instead, so parts 3 to 5 (#62, #63, #64) were closed without merging.
main.rspace_client.fsas an fsspec filesystem, built on this PR; replaces Add rspace_client.fs: the Gallery, Inventory and ELN Workspace as one PyFilesystem #62 to Pin what Galaxy's RSpace file source relies on, and add a plugin check tool #64. Once this is merged, Add rspace_client.fs: the Gallery, Inventory and ELN Workspace as one fsspec filesystem #68 is retargeted tomain.🤖 Generated with Claude Code