Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
0358362
build: migrate pyproject to PEP 621 / Poetry 2, require Python 3.10
nebay-abraha Sep 28, 2026
05c9696
chore: add ruff, mypy and pre-commit config
nebay-abraha Sep 28, 2026
dbd387f
fix: avoid shared mutable default arguments in inv and eln
nebay-abraha Sep 28, 2026
f863a30
fix(inv)!: Sample.wherep() returns a list as documented
nebay-abraha Sep 28, 2026
598073d
fix(inv)!: GridContainerPost sends cType GRID
nebay-abraha Sep 28, 2026
6cfdc59
fix(inv): ListContainer.is_list() missing self
nebay-abraha Sep 28, 2026
f581c5b
fix: URL validator error message and bare except
nebay-abraha Sep 28, 2026
8323263
fix(eln): include server error detail when an export job fails
nebay-abraha Sep 28, 2026
8c991e4
style: apply ruff format and ruff check --fix
nebay-abraha Sep 28, 2026
bc82e3a
refactor: resolve remaining ruff findings and mypy errors
nebay-abraha Sep 28, 2026
83036f8
ci: add lint job and unit test coverage gate
nebay-abraha Sep 28, 2026
6326a72
ci: PyPI Trusted Publishing release workflow, Dependabot, simpler rel…
nebay-abraha Sep 28, 2026
1213f8e
test(inv): stop asserting DataCite password is echoed back
nebay-abraha Oct 31, 2026
45e1535
build: refresh poetry.lock for fsspec and universal-pathlib from main
nebay-abraha Oct 31, 2026
32a8f3c
test(inv): skip DataCite tests when the server has real DataCite sett…
nebay-abraha Oct 31, 2026
ad8ce4f
test(eln): cover FAILED export-job errors; bump conda env to 3.12
nebay-abraha Oct 31, 2026
3588123
docs: tighten changelog, developing guide and pyproject comments
nebay-abraha Oct 31, 2026
2d54618
build: restore Python version classifiers
nebay-abraha Nov 1, 2026
3173991
test(inv): cover filter leaking into later list/stream calls
nebay-abraha Nov 2, 2026
d51a87f
fix: address review comments on PR #69
nebay-abraha Nov 7, 2026
0730ab5
test(inv): tighten DataCite test guard against real settings
nebay-abraha Nov 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
version: 2
updates:
- package-ecosystem: "pip"
Comment thread
nebay-abraha marked this conversation as resolved.
Resolved
directory: "/"
schedule:
interval: "weekly"
groups:
python-dependencies:
patterns:
- "*"
# ruff and mypy get their own PRs: a new ruff can change formatter
# output, which needs a reformat that shouldn't hide in a grouped bump.
# .pre-commit-config.yaml runs these same pinned versions via Poetry.
exclude-patterns:
- "ruff"
- "mypy"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
groups:
github-actions:
patterns:
- "*"
1 change: 1 addition & 0 deletions .github/scripts/warmup_sysadmin.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
SYSADMIN_USERNAME = "sysadmin1"
SYSADMIN_PASSWORD = "sysWisc23!"


# A user account that is only ever authenticated via its API key (never a real
# login) has its home folder created lazily on first write - a code path that
# throws on that first attempt (though it leaves the account usable afterward).
Expand Down
52 changes: 45 additions & 7 deletions .github/workflows/codeql-and-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,16 +39,51 @@ jobs:
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4

lint:
name: lint
runs-on: ubuntu-24.04
timeout-minutes: 15
permissions:
contents: read

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 2.4.1
virtualenvs-create: true
virtualenvs-in-project: true

- name: Install dependencies
run: poetry install --no-interaction

- name: Ruff check
run: poetry run ruff check rspace_client examples

- name: Ruff format check
run: poetry run ruff format --check rspace_client examples

- name: mypy
run: poetry run mypy rspace_client

test:
name: unit test
runs-on: ubuntu-24.04
timeout-minutes: 30
needs: analyze
needs: [analyze, lint]
permissions:
contents: read
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
python-version: ["3.10", "3.11", "3.12", "3.13"]

steps:
- name: Checkout repository
Expand All @@ -62,15 +97,18 @@ jobs:
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 1.8.4
version: 2.4.1
virtualenvs-create: true
virtualenvs-in-project: true

- name: Install dependencies
run: poetry install --no-interaction

- name: Run unit tests
run: poetry run pytest -m "not integration"
run: |
poetry run pytest -m "not integration" \
--cov=rspace_client --cov-report=term-missing --cov-report=xml \
--cov-fail-under=70

integration-test:
name: integration test
Expand Down Expand Up @@ -109,15 +147,15 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python 3.12
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.12"
python-version: "3.13"

- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 1.8.4
version: 2.4.1
virtualenvs-create: true
virtualenvs-in-project: true

Expand Down
93 changes: 93 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: Release to PyPI

on:
push:
tags:
Comment thread
nebay-abraha marked this conversation as resolved.
Resolved
- "v*"

permissions:
contents: read

jobs:
build:
name: Build and verify
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # full history, to check the tag is on main

- name: Check the tagged commit is on main
run: |
set -euo pipefail
git fetch --no-tags origin main
if ! git merge-base --is-ancestor "$GITHUB_SHA" origin/main; then
echo "Tag $GITHUB_REF_NAME points at $GITHUB_SHA, which is not on main"
exit 1
fi

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 2.4.1
virtualenvs-create: true
virtualenvs-in-project: true

- name: Install dependencies
run: poetry install --no-interaction

- name: Ruff check
run: poetry run ruff check rspace_client examples

- name: Ruff format check
run: poetry run ruff format --check rspace_client examples

- name: mypy
run: poetry run mypy rspace_client

- name: Run unit tests
run: poetry run pytest -m "not integration"

- name: Check tag matches package version
run: |
set -euo pipefail
tag_version="${GITHUB_REF_NAME#v}"
pkg_version="$(poetry version --short)"
if [ "$tag_version" != "$pkg_version" ]; then
echo "Tag v$tag_version does not match pyproject.toml version $pkg_version"
exit 1
fi

- name: Build sdist and wheel
run: poetry build

- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/

publish:
name: Publish to PyPI
needs: build
runs-on: ubuntu-24.04
timeout-minutes: 10
environment: pypi
permissions:
id-token: write # required for PyPI Trusted Publishing (OIDC)
steps:
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: dist
path: dist/

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
24 changes: 24 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# The hooks run the ruff and mypy pinned in pyproject.toml (via Poetry), so the
# versions here, in CI and in Dependabot's updates are always the same.
repos:
- repo: local
hooks:
- id: ruff
name: ruff check
entry: poetry run ruff check --fix --force-exclude
language: system
types: [python]
files: ^(rspace_client|examples)/
- id: ruff-format
name: ruff format
entry: poetry run ruff format --force-exclude
language: system
types: [python]
files: ^(rspace_client|examples)/
- id: mypy
name: mypy
entry: poetry run mypy rspace_client
language: system
# also run when only the mypy settings in pyproject.toml change
files: ^(rspace_client/.*\.py|pyproject\.toml)$
pass_filenames: false
67 changes: 65 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ All notable changes to this project will be documented in this file
- replace naming of classes and methods using 'Workbench' to 'Bench'
- replace create_sample, create_container long argument lists with new XXXPost objects

## 2.8.0 (unreleased)
## Unreleased

- New `rspace_client.fs`: RSpace as an [fsspec](https://filesystem-spec.readthedocs.io)
filesystem. `RSpaceFilesystem` mounts `gallery/`, `inventory/` and `workspace/` (Gallery
Expand Down Expand Up @@ -65,7 +65,70 @@ All notable changes to this project will be documented in this file
the Gallery root is matched by its `GF` id as well as its name, and `removedir` refuses a
malformed listing instead of treating it as empty.

## Unreleased
- Moved `pyproject.toml` to the standard PEP 621 `[project]` table, built
with Poetry 2.x.

Potentially breaking:
- Python 3.9 is no longer supported. The minimum is now 3.10.

- Added `ruff` for linting and formatting (it replaces `black`), plus
configuration for `mypy`, `pre-commit` and `pytest-cov`.

- Fixed shared mutable default arguments (`Pagination()`, `tags=[]`,
`extra_fields=[]`, `TopLevelTargetLocation()`, `item_ids=[]`,
`typesToInclude=[]`) in several `inv` and `eln` methods. A call that used
the default could see state left behind by an earlier call. Passing
`extra_fields=None` to `SamplePost`, `ContainerPost` or `InstrumentPost` now
sends `"extraFields": []`, the same as leaving it out. It used to leave the
key out of the request.

- Fixed a search filter carrying over into later calls. The `inv` list and
stream methods that take a `SearchFilter` (`list_samples`, `stream_samples`,
`list_instruments`, `list_sample_templates` and others) wrote the filter into
the `Pagination` they were given, so a later call reusing that `Pagination`
sent the old filter. They now leave it unchanged.

- Fixed the `ByRow` / `ByColumn` out-of-range error message, which printed the
literal `{total_columns}, {total_rows}` placeholders and had unbalanced
parentheses.

- `Workbench.capacity()` returns `sys.maxsize` (unlimited, like
`ListContainer`). It used to return `None`.

- Fixed `Sample.wherep()` returning a `set` instead of the documented `list`.

Potentially breaking: code that calls `.add()` on the result, uses set
operators on it, or compares it with `== {...}` needs updating.

- Fixed `GridContainerPost` sending `cType: "LIST"` instead of `"GRID"`.

Potentially breaking: the server now creates these as grid containers, not
list containers.

- Fixed `ListContainer.is_list()` raising `TypeError` when called on an
instance. The method was missing `self`.

- Fixed the `URL` field validator's error message. It was missing its
f-string prefix, so it printed the literal placeholders. The validator also
no longer catches unrelated exceptions with a bare `except`.

- Fixed the error raised when an ELN export job fails. It dropped the
server's error detail and said only "Export job failed: ". An abandoned
export job now raises `ApiError` instead of `AttributeError`, and is written
to the progress log like a failed one. A null or malformed error body, or a
running job with no `percentComplete`, no longer crashes the export.

- Added a CI lint job that runs `ruff check`, `ruff format --check` and
`mypy`. Unit tests in CI now report coverage and fail below 70%.

- Pushing a `v*` tag now builds the package and publishes it to PyPI from a
GitHub Actions workflow that uses Trusted Publishing. The workflow refuses a
tag that is not on `main` and runs the lint checks and unit tests first.
`release.sh` bumps the version with `poetry version`, no longer publishes,
and only runs on `main` when it is level with `origin/main`.

- Added Dependabot for pip and GitHub Actions updates. The pre-commit hooks
run the ruff and mypy pinned in `pyproject.toml`, so they always match CI.

- Gallery upload routing and section-mismatch handling (PR #56): clearer
`GallerySectionMismatch` exception, optional `on_mismatch="reroute"`
Expand Down
45 changes: 36 additions & 9 deletions DEVELOPING.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,39 @@
## Development

Python 3.9 or later is required. We aim to support only active versions of Python.
Python 3.10 or later is required. We aim to support only active versions of Python.

### Setup

Create a virtual environment with python 3.7 installed. If you use `conda`, you can do this
Create a virtual environment with Python 3.10 or later. With `conda`, this creates one with Python 3.12:

```
conda env create -f environment.yaml
Comment thread
nebay-abraha marked this conversation as resolved.
Resolved
conda activate rspace-client
```

We use `poetry` for dependency management. [Install Poetry] (https://python-poetry.org/docs/#installation)
We use `poetry` (2.x) for dependency management. [Install Poetry](https://python-poetry.org/docs/#installation)

From this directory, run

`poetry install`

to install all project dependencies into your virtual environment.
to install all project dependencies, including the dev tools below, into your virtual environment.

### Linting, formatting and type checking

We use [ruff](https://docs.astral.sh/ruff/) for linting and formatting, and [mypy](https://mypy-lang.org/) for type checking.

```
poetry run ruff check rspace_client examples
poetry run ruff format rspace_client examples
poetry run mypy rspace_client
```

To run these checks on every commit, install the [pre-commit](https://pre-commit.com/) hooks:

```
poetry run pre-commit install
```

### Running tests

Expand Down Expand Up @@ -65,8 +81,19 @@ Galaxy ships an `rspace` file source built on this library's deprecated `rspace_

### Making a release

- Get a clean test run
- Update version in README.md and pyproject.toml
- update changelog to include the new version and required RSpace version
- tag with syntax like 'v2.2.0'
- Build and submit to Pypi using `poetry publish`
Releases are made on `main`. PRs are squash-merged, so a commit tagged on a PR branch never
reaches `main`, and the release workflow refuses any tag that isn't on `main`.

- Merge everything for the release into `main`, then check out `main` and pull.
- Get a clean test run.
- Make sure `CHANGELOG.md` has exactly one `## Unreleased` section, and that it names the required
RSpace version.
- Run `./release.sh <new-version>`, for example `./release.sh 2.8.0`. It refuses to run unless you
are on `main`, with no uncommitted changes and level with `origin/main`. It renames
`## Unreleased` to the version and date, sets the version with `poetry version` (`pyproject.toml`
is the only file that holds it), commits, and creates the tag `v<new-version>`.
- Push the commit and the tag: `git push origin main && git push origin v<new-version>`.
- The tag push starts `.github/workflows/release.yml`. It stops if the tag isn't on `main` or doesn't
match the `pyproject.toml` version, runs ruff, mypy and the unit tests, builds the sdist and wheel,
and publishes them to PyPI with [Trusted Publishing](https://docs.pypi.org/trusted-publishers/),
so you don't need an API token.
Loading
Loading