Skip to content

Add an offline mock of the RSpace ELN and Inventory APIs for the test suite - #61

Merged
tilorspace merged 1 commit into
mainfrom
fs-stack/2-mock-server
Oct 12, 2026
Merged

tilorspace merged 1 commit into
mainfrom
fs-stack/2-mock-server

Conversation

@tilorspace

@tilorspace tilorspace commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

A standard-library mock of the ELN (/api/v1) and Inventory (/api/inventory/v1) endpoints the filesystem classes use, under rspace_client/tests/mock_rspace. Nothing here is installed with the package.

  • server.py serves an in-memory dataset from fixtures.py and counts requests, so a test can assert what a listing costs.
  • The dataset carries the cases a filesystem has to get right: two files with the same name, names that look like global IDs, system folders in Home, sections large enough to paginate, a document with text and non-text fields, a signed document, a bench with attachments.
  • conftest.py starts one server for the whole pytest session; MockServerTestCase resets its fixtures before every test.

The mock doesn't depend on a filesystem framework. No test uses it yet in this PR. #68 adds the fsspec filesystem tests, the Galaxy contract test and the plugin check tool that run against it, and adjusts a few fixtures.

Stack

This started as part 2 of a five-part stack towards a unified PyFilesystem (#58). On 2026-10-06 the filesystem moved to fsspec instead, so parts 3 to 5 (#62, #63, #64) were closed without merging.

  1. Client layer: rebase advertised links, refuse redirects, add calls the filesystems need #60, client layer (merged)
  2. This PR, offline mock server. Now based on main.
  3. Add rspace_client.fs: the Gallery, Inventory and ELN Workspace as one fsspec filesystem #68, rspace_client.fs as an fsspec filesystem, built on this PR; replaces Add rspace_client.fs: the Gallery, Inventory and ELN Workspace as one PyFilesystem #62 to Pin what Galaxy's RSpace file source relies on, and add a plugin check tool #64. Once this is merged, Add rspace_client.fs: the Gallery, Inventory and ELN Workspace as one fsspec filesystem #68 is retargeted to main.

🤖 Generated with Claude Code

@tilorspace
tilorspace added this pull request to stack #65 September 21, 2026 12:29
@tilorspace
tilorspace marked this pull request as ready for review September 21, 2026 12:44
Base automatically changed from fs-stack/1-client-layer to main October 12, 2026 07:43
tilorspace added a commit that referenced this pull request Oct 12, 2026
…e filesystems need (#60)

Client layer: rebase advertised links, refuse redirects, add calls the filesystems need (#60)

Changes to the ELN and Inventory REST clients that the filesystem work
needs, plus fixes that stand on their own:

- Links the server advertises in `_links` are rebased onto the configured
  base URL, path prefix included, so next-page and download links work
  behind a port-forward, in a container, or behind a proxy that serves
  RSpace under a sub-path such as /rspace.
- The shared session never follows an HTTP redirect, because requests
  would carry the apiKey header to the new host.
- doDelete tolerates a leading slash, which fixes delete_document.
- ELNClient.upload_file and InventoryClient.upload_attachment_by_global_id
  take an optional filename=; list_folder_tree takes page_size=.
- New InventoryClient.get_workbench_by_id (/containers answers 422 for a
  bench) and attach_gallery_file_by_global_id (link an existing Gallery
  file to an Inventory record).
- Inventory CSV import goes through the shared session, so it gets the
  same timeout, retries and error handling as every other request.
- A stray debug print is removed from upload_attachment_by_global_id.

Review follow-up: the path-prefix handling in link rebasing, and unit
tests for each changed or new client method (client_methods_test.py).

Stack: this was part 1 of five PRs towards a unified PyFilesystem (#58).
On 2026-10-06 the filesystem was moved to fsspec instead, so parts 3 to 5
(#62, #63, #64) were closed without merging. This client layer and the
offline mock server (#61) are independent of the filesystem framework and
are kept. The fsspec filesystem itself is #68, which builds on #61. Merge
#61 next, then retarget #68 to main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… suite

A standard-library HTTP server covering the endpoints the PyFilesystem classes use, backed
by an in-memory dataset with the awkward cases a filesystem has to handle (duplicate names,
system folders, paginated sections, documents with mixed field types). It counts requests so
tests can assert what an operation costs. One server runs for the whole pytest session and
`MockServerTestCase` resets its fixtures before each test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@tilorspace
tilorspace force-pushed the fs-stack/2-mock-server branch from 638a685 to 631051a Compare October 12, 2026 07:43
@tilorspace
tilorspace merged commit 5e5674e into main Oct 12, 2026
7 checks passed
@tilorspace
tilorspace deleted the fs-stack/2-mock-server branch October 12, 2026 08:22
tilorspace added a commit that referenced this pull request Oct 12, 2026
#61 was squash-merged into main; main now holds the same content this branch already had, so the branch keeps its own tree (conftest.py stays deleted, the feature's mock fixtures stay).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tilorspace added a commit that referenced this pull request Oct 29, 2026
… fsspec filesystem (#68)

**Stacked PR. Merge after #60 (client layer) and #61 (offline mock
server); this one targets `fs-stack/2-mock-server`.**

Replaces the PyFilesystem2 version of the unified filesystem (#62 to
#64, closed). PyFilesystem2 has had no release since 2022, and Galaxy,
the main consumer, is moving every file source to fsspec
(galaxyproject/galaxy#21832, where `rspace.py` is on the list). Tracked
in RSDEV-1519.

## What it adds

- **`rspace_client.fs`**, an fsspec filesystem. `RSpaceFilesystem`
mounts `gallery/`, `inventory/` and `workspace/`, sharing one ELN and
one Inventory client. Each branch is also usable on its own
(`GalleryFilesystem`, `InventoryFilesystem`, `WorkspaceFilesystem`).
- Paths are record names (`gallery/Images/microscope.png`). Duplicates,
and names that read as addresses, carry their global ID (`data
[GL112].csv`). `path_style="labelled"` or `"id"` give paths that survive
a rename.
- Read-only by default (`writable=True`, `allow_delete=True`). Uploads
into Inventory records and ELN fields go through the Gallery and are
linked. `cp_file` from a Gallery-backed file links instead of copying
bytes. Transactions defer uploads.
- `rspace://host/...` URLs via `fsspec.open`,
`fsspec.filesystem("rspace")` and `UPath`, registered as an
`fsspec.specs` entry point. The key comes from `RSPACE_API_KEY` only,
never from the URL.
- RSpace errors surface as builtin `OSError` subclasses, so `exists`,
`walk`, `copy` and Galaxy behave.
- **Deprecation** of `rspace_client.eln.fs.GalleryFilesystem` and
`rspace_client.inv.attachment_fs.InventoryAttachmentFilesystem`. Their
behaviour is unchanged apart from the two Galaxy fixes below, they warn
on construction, and they're removed in 3.0 together with `fs`, after
Galaxy's shipped plugin has moved.
- **Fix for Galaxy 26.1 exports to RSpace.** requests 2.34 (pinned by
Galaxy 26.1) refuses a file part that forwards `read` through
`__getattr__`. That's how Galaxy's `FakedNameIO` wrapper works, so
exports through Galaxy's shipped `rspace` source fail on every released
rspace-client. `_post_multipart` now wraps such objects. Verified inside
a Galaxy 26.1.2 container: 2.7.4 raises the `TypeError`, this branch
uploads.
- **Fix for Galaxy on Python 3.10.** Galaxy's shipped `rspace` source
parses a file's `created` date with `datetime.fromisoformat`, which
rejects RSpace's trailing `Z` before Python 3.11, so listing any Gallery
folder with files failed on Python 3.10 with every released
rspace-client. `GalleryFilesystem` now gives the same UTC instant as
`+00:00`. Verified with Galaxy's real plugin (galaxy-files 26.1.1) on
Python 3.10.
- **`galaxy_plugin_contract_test.py`**, which pins what Galaxy's shipped
plugin relies on. A failure there is a release blocker.
- **`tools/galaxy/rspace_fsspec_source.py`**, the reference Galaxy file
source (type `rspace_unified`) to propose upstream once this is
released.
- **Docs:** `docs/usage-guide.md` and the design record
`docs/fsspec-filesystem-design.md`, which explains why the package is
shaped this way and what must not move.

## Review order

1. `docs/fsspec-filesystem-design.md` (the decisions)
2. `rspace_client/fs/paths.py`, `rspace_client/fs/base.py` (shared
behaviour)
3. `gallery.py`, `inventory.py`, `workspace.py`, then `rspace.py` (the
mount)
4. Shims: `rspace_client/eln/fs.py`,
`rspace_client/inv/attachment_fs.py`, and the contract test
5. Tests (`fs_*_test.py`), `tools/galaxy/`

## Testing

- Offline suite: 305 passed (`poetry run pytest -m "not integration"`),
using the mock server from #61.
- `fs_integration_test.py` against a docker RSpace 2.27, plus a
record-type matrix covering Gallery, documents, samples, subsamples,
containers, instruments, and the bench and template refusals.
- Galaxy 25.1 and 26.1 containers with the reference plugin: listing,
search (including bracketed names, CJK and `%`), pagination, import from
Gallery, Inventory and Workspace, and export into a sample, a field and
the Gallery.

New dependency: `fsspec>=2024.3.0`. `fs` and the `setuptools<82` pin
stay until 3.0.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant