Skip to content

CI test convention: support pytest / test/ / *_test.py, or require addons to conform #53

Description

@eduralph

Summary

The CI test runner supports exactly one test convention: a tests/ (plural)
directory, test_*.py (prefix) filenames, and Python's unittest loader.
Issue #52 made discovery recursive and added a per-addon import root so
nested-package addons that follow this convention (e.g. NameSuite, upstream
PR 941 — 20 modules / 234 tests now run) work. But a second in-flight
submission uses a different convention, which #52 deliberately did not
broaden to (narrow-scope decision). This issue records that gap so we can decide
it on its own.

The divergence (dsblank PR 1)

dsblank/addons-source PR 1 ("Add FastFiltersLib, SQLiteFastFilters,
PostgreSQLFastFilters") ships its tests as:

  • test/ (singular) directory — not tests/.
  • *_test.py (suffix) filenames — e.g. filter_overrides_test.py,
    sql_compat_test.py — not test_*.py.
  • pytest, via conftest.py that does the sys.path setup, including
    cross-addon sharing (SQLite/PostgreSQL conftest.py add FastFiltersLib/
    and FastFiltersLib/test/ to the path to reuse shared base classes), and a
    sibling import from filter_overrides_base import ….

Consequences for our runner:

  • Discovery (*/tests/**/test_*.py) does not match test/ + *_test.py,
    so these suites are never found.
  • Even if found, the unittest loader does not execute conftest.py, so
    the sys.path setup (and the cross-addon sharing) would not run — from fastfilterslib… import / from filter_overrides_base import would fail.

Decision needed

Pick one (or a mix):

  1. Require addons to conform to the documented convention (tests/ +
    test_*.py + unittest, with per-addon import root and relative imports for
    private helpers). Lowest CI cost; pushes the work to each addon.
  2. Broaden discovery to also accept test/ and *_test.py (still unittest).
    Cheap glob change, but does not solve pytest conftest.py / cross-addon
    sharing — partial at best.
  3. Add pytest as a CI runner so conftest.py (fixtures, cross-addon path
    sharing) works. Largest change — effectively a second test-execution mode.

This needs a maintainer convention call (it touches every future addon and the
two core maintainers' differing styles), so it is intentionally not bundled
into #52 / PR 820.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions