Skip to content

ci: add CI, release workflow and e2e tests; fix stale dependency pins - #70

Open
soroushdeimi wants to merge 47 commits into
cnrl:mainfrom
soroushdeimi:ci/ci-release-e2e
Open

soroushdeimi wants to merge 47 commits into
cnrl:mainfrom
soroushdeimi:ci/ci-release-e2e

Conversation

@soroushdeimi

@soroushdeimi soroushdeimi commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds continuous integration, a tag-driven release workflow and an end-to-end test suite, and fixes the dependency pins that made the existing test suite fail.

Why the pins changed

requirements.txt pinned pymonntorch==0.1.0, but tests/conftest.py builds Network(dtype=...), which that version rejects. With the old pins the existing suite gave 47 failed, 87 passed, 100 errors. I probed pymonntorch 0.1.0 to 0.1.4 with torch==1.13.1:

pymonntorch result
0.1.0, 0.1.1 47 failed, 87 passed, 100 errors
0.1.2 41 failed, 93 passed, 100 errors
0.1.3 224 passed, 10 errors
0.1.4 234 passed

sympy is imported by pymonntorch but declared nowhere, so it is added. numpy<2 is kept in requirements.txt only: torch 1.13.1 was built against the numpy 1.x C API, so Tensor.numpy() fails under numpy 2. setup.py leaves numpy uncapped.

What is in this PR

  • CI (.github/workflows/ci.yml): flake8, test matrix (ubuntu and macOS, Python 3.8 to 3.10), a separate e2e job, wheel build with twine check, and one CI OK job to require in branch protection. Jobs run in parallel and superseded runs are cancelled.
  • Release (.github/workflows/release.yml, scripts/check_release.py, docs/RELEASING.md): on a v* tag it checks the tag against setup.py and HISTORY.rst, runs the tests and the build in parallel, creates the GitHub release, and publishes to PyPI with trusted publishing (no stored token). A manual run can publish to TestPyPI. This replaces python-publish.yml. The pypa action is pinned by commit SHA.
  • Lint: the 12 unused-variable and redefinition findings (F841, F811) are fixed with minimal edits. The remaining pre-existing style codes are listed in setup.cfg with a note to tighten them as files are touched.
  • E2E tests (tests/e2e/): per-module tests for neurons, synapses, layer and network behaviors and for the HTM, predictive coding, grid cell, voting, nn and helper code; scenario tests (competitive STDP from an unbiased init, reward-modulated STDP, dataset to readout, seeded reproducibility); pipeline tests (encoder to SDR to pooler to temporal memory to hierarchy, voting columns, grid cells with sequence memory, structure save/load round trip); and golden values generated by tests/e2e/golden/generate.py from the library itself.

Verification

Run on torch==1.13.1, pymonntorch==0.1.4, numpy 1.26.4, Python 3.10:

  • pytest tests: 718 passed (234 existing, 12 release script, 472 e2e); the e2e suite gives identical results on repeated runs
  • flake8 conex tests: no output
  • actionlint on both workflows: clean
  • python -m build and twine check: passed for the sdist and the wheel
  • Key tests were shown to fail when the code under test is broken (for example flipping the sign in SimpleSTDP.compute_dw, or making TemporalMemory never predict)

Line coverage of conex/ is 91% with the e2e tests.

Not verified

  • The workflows have not run on GitHub yet; they were checked with actionlint and by running each step locally.
  • Ubuntu cells were not run locally; torch 1.13.1 wheel availability was checked from PyPI metadata.
  • The release workflow needs a pypi and a testpypi environment and a trusted publisher set up once (see docs/RELEASING.md).
  • HISTORY.rst has no version entries, so check_release.py v0.1.5 fails its history check until one is added.

Existing library bugs found by the new tests (not fixed here)

  • OutputLayer calls NeuronGroup with positional arguments in an order the current pymonntorch does not accept.
  • One2OneRSTDP with the default 2D weight broadcasts the one-to-one update over the rows; with a 1D weight it works. This is a usage question, not a crash.
  • Sparse*STDP rules need a COO tensor, but WeightInitializer produces CSC; on torch 1.x the sparse init raises "Expected result Tensor to be of format CSR". The e2e test asserts the real behaviour per torch major version.
  • GridEraseMask and GridCropMask import torchvision, which does not import with torch 1.13.1.
  • save_ports fails for a port whose behavior is None, and build_ports is not the inverse of save_ports.

Probe results (pytest -q on tests/, macOS arm64, uv venvs):

pymonntorch 0.1.0 + torch 1.13.1: 47 failed, 87 passed, 100 errors
pymonntorch 0.1.1 + torch 1.13.1: 47 failed, 87 passed, 100 errors
pymonntorch 0.1.2 + torch 1.13.1: 41 failed, 93 passed, 100 errors
pymonntorch 0.1.3 + torch 1.13.1: 224 passed, 10 errors
pymonntorch 0.1.4 + torch 1.13.1: 234 passed
pymonntorch 0.1.4 + torch 2.14.1: 234 passed

Root cause: tests/conftest.py passes dtype= to Network, which only
pymonntorch >= 0.1.4 accepts. 0.1.4 is the smallest fully-green version.

sympy: pymonntorch imports it but does not declare it; torch 1.13.1
does not pull it in (torch 2.x does), so it must be listed explicitly.

numpy<2: torch 1.13.1 with numpy 2.2.6 fails
tests/test_voting.py::TestColumnVoting::test_partial_match_still_votes
(1 failed, 233 passed); numpy 1.26.4 gives 234 passed.

torch pin kept at 1.13.1: 0.1.4 passes with it on py3.8/3.9/3.10.
Runs on push to main, pull_request and workflow_dispatch.
- lint: flake8 on conex and tests with the project config
- test: matrix ubuntu-latest and macos-latest x python 3.8/3.9/3.10,
  fail-fast false, pip cache, pytest on tests/
- build: python -m build, twine check, install wheel in a fresh venv
- ci-ok: single check that fails unless lint, test and build all succeed

All six matrix cells have torch 1.13.1 wheels (checked on PyPI:
cp38/39/310 x manylinux1_x86_64, macosx_10_9_x86_64, macosx_11_0_arm64).
Remove F841 (unused local) and F811 (redefinition) from the flake8
extend-ignore and fix each site with the smallest change:
- dataset.py: drop duplicate `import torch` (F811)
- active_dendrites.py: drop unused device/dtype (x2) and predicted_activation
- grid_cells.py: drop unused n_cells in forward
- predictive_coding.py: drop unused pred_dim (keep the parameter() call,
  which registers the key in used_attr_keys), unused dt/tau/alpha, and
  two unused cfg reads
- sequence_memory.py: drop unused device

Full suite still 234 passed; flake8 clean.
- test job now runs `pytest tests/ --ignore=tests/e2e -n auto`
- new e2e job runs `pytest tests/e2e` on the same OS/python matrix;
  skips cleanly (exit 0) when tests/e2e does not exist yet
- pytest-xdist installed in the workflow only (not requirements.txt);
  kept because the suite is 234 passed and identical across 3 local runs
- ci-ok now needs lint, test, e2e and build

actionlint clean.
On torch 1.13.1 (the CI pin) WeightInitializer crashes: it converts the
weights to CSC (specs.py:189) then multiplies by scale (specs.py:198), and
torch 1.x cannot multiply a CSC sparse tensor by a scalar. Branch on
torch.__version__ inside the test: assert the RuntimeError on <2.0 and the
sparse density on >=2.0. No skip/xfail.
Remove unused imports (TimeResolution, seed, Conv2dSTDP, Local2dSTDP),
drop the unused 'n' binding, and wrap long lines / reword docstrings so
'flake8 tests/e2e' prints nothing. The library attribute 'I' (pymonntorch
API name) gets a targeted noqa: E741.
Resolve E127/E128 continuation-line indents and an F841 unused variable,
drop unused imports, and add a create_structure_from_dict test for the
NeuronGroup branch.
Replace the two-line note about the unmerged shared conftest with a single
line stating each file carries its own small network builder.
Add 4 end-to-end learning scenarios built from the public API that assert
on what the networks learn, not a single function:
- competitive STDP: Poisson input -> LIF+KWTA+homeostasis, per-pattern
  selectivity and weight-level preference, weights clipped to [0,1]
- reward-modulated: Payoff + Dopamine + SimpleRSTDP, rewarded pattern's
  weights grow more than the unrewarded one's
- dataset to network: Poisson-encoded SpikeNdDataset batches -> InputLayer
  -> readout, label-wise spike statistics and readout weight movement
- reproducibility matrix: same seed bit-identical (torch.equal), different
  seed differs, each scenario run twice in-process
Plus per-scenario failure/edge cases (zero input, saturated weights,
wrong shapes). Inputs are Poisson-encoded so networks are stochastic and
seed-dependent.
The selectivity was baked into a hand-biased 0.8/0.2 init, so the test
stayed green when SimpleSTDP.compute_dw was broken. Start from an unbiased
init (0.5 base + tiny seeded noise) and assert learning emerges as both LTP
and LTD being active (some weights potentiate, some depress). This fails for
a_plus=0 (no LTP) and for the LTD->LTP sign mutation (no LTD).
Raise a_plus and run more steps so the readout weight change is well above
the 0.5 -> 0.52 noise floor (max |dw| 0.116-0.500 across seeds 0-4), and
assert the direction per label: each label's active input channels
potentiate. Fails when a_plus=0 (no potentiation).
Record the min/max observed across seeds 0-4 for the RSTDP rewarded growth
(gA 0.219-0.257, gB 0.0) and the dataset readout change (max |dw| 0.116-0.500),
so the thresholds are justified by a sweep, not a single seed.
Add a scenario using the Intensity2Latency encoder (the task asked for it):
two intensity patterns drive a LIF readout over a time window. Assert the
latency-coding property (a stronger intensity spikes sooner, one spike per
channel) and that the readout weights learn from the encoded input.
@saeedark

Copy link
Copy Markdown
Collaborator

Thank you for your interest in the Conex. This PR is huge in size. If I find the time I would review it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants